Anthropic says it has identified and disrupted attempts to use its AI model for “malicious activity” which could support the development of biological weapons.
The findings, made in the firm’s recent threat intelligence report, is the latest in a growing series of warnings from AI researchers and industry insiders about the technology’s potential risks to humanity.
The warnings have prompted calls to action, with US Senator Bernie Sanders demanding a pause on advanced AI development and a ban on artificial superintelligence.
President Donald Trump has so far rejected such fears, saying on Thursday he was concerned “if we don’t win AI, we’re going to be put in a very bad position”.
AI firms report attempts to misuse models
The company published its latest threat intelligence report on Thursday.
Such reports have become a regular feature across the AI industry as firms seek to demonstrate how they identify and disrupt attempts to misuse their models.
This includes Google, which wrote on Tuesday that a person had attempted to use its AI tool, Gemini, to obtain a “complete, step-by-step technical guide for synthesizing weaponised biological agents”.
Claude, Anthropic’s AI model was also used by actors linked to a Russia-based cyber espionage campaign and by an Iranian propaganda institution, according to its report.
Anthropic says it has identified and disrupted attempts to use its AI model for “malicious activity” which could support the development of biological weapons.
The findings, made in the firm’s recent threat intelligence report, is the latest in a growing series of warnings from AI researchers and industry insiders about the technology’s potential risks to humanity.
The warnings have prompted calls to action, with US Senator Bernie Sanders demanding a pause on advanced AI development and a ban on artificial superintelligence.
President Donald Trump has so far rejected such fears, saying on Thursday he was concerned “if we don’t win AI, we’re going to be put in a very bad position”.
The company published its latest threat intelligence report on Thursday.
Such reports have become a regular feature across the AI industry as firms seek to demonstrate how they identify and disrupt attempts to misuse their models.
This includes Google, which wrote on Tuesday that a person had attempted to use its AI tool, Gemini, to obtain a “complete, step-by-step technical guide for synthesizing weaponised biological agents”.
Claude, Anthropic’s AI model was also used by actors linked to a Russia-based cyber espionage campaign and by an Iranian propaganda institution, according to its report.
Information can have legitimate and malicious uses
“The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease,” Anthropic said.
Jacob Klein, the head of threat intelligence at Anthropic, told the New York Times it was “an incredibly nuanced situation”.
“You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,'” he said.
The report also noted six cases where Claude was used “to develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them”.
Hackers increasingly using AI
The report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations.
Hacking group ShinyHunters, as well as China-based labs, were among those named in the report.
The report also said that a hacking group whose work is consistent with the Russia-based Midnight Blizzard allegedly used AI to build a system that automatically detected when its malware was flagged by security defences and rewrote code until it evaded detection.
The California-based company said it had incorporated its findings into its processes “to better prevent, detect, and disrupt these activities in the future”.
Anthropic said it had shared intelligence with authorities and industry partners where appropriate.
Calls to slow advanced AI development
The report – the company’s first this year – comes after a top safety researcher at Anthropic warned AI is advancing so quickly he believes there is a greater than 10% chance it “could kill all humans” within the next decade.
In an article published on 6 September, OpenAI chief scientist Jakub Pachocki called for the industry to implement “voluntary slowdowns” until safeguards are set.
“I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” Pachocki said, adding that OpenAI, which makes ChatGPT, will continue its work on building safeguards, though broader interventions are required.
The warning prompted an open letter to UK Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI and a call for governments to collaborate about what a treaty based on the development of superintelligence should look like.
In the US, Democratic lawmaker Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development.
“When scientists tell you there is a chance, a chance that it could have a cataclysmic impact on humanity, you’ve got be a moron not to say, slow it down,” he said on Thursday on BBC’s Newsnight.
Source: BBC
Also read: Ayia Napa: Two arrested after attempting to pay with fake €200 notes
For more videos and updates, check out our YouTube channel
