Anthropic, Open AI models created fake identities in new cyber breach
4 mins read

Anthropic, Open AI models created fake identities in new cyber breach

Tech

Anthropic’s Mythos model created fake online identities as it looked to pressure humans into approving malicious code updates to an open source project, marking yet another cyber incident carried out by a frontier AI system.

The incident happened during a cyber evaluation where the U.K.-based AI Security Institute (AISI), a research body, had removed safeguards, disabled some safety filters, and deliberately given the models Internet access.

OpenAI’s GPT-5.6-Sol was also involved in other cybersecurity incidents during the evaluation.

It comes after a series of cyber breaches carried out by models developed by Anthropic and OpenAI in recent weeks.

They’ve prompted a wave of fears around the sophistication of AI systems and their potential to cause harm.

AI cyber attacks bring fresh scrutiny over safety

During the routine cyber evaluation, the AISI identified AI agents powered by Anthropic and OpenAI models had engaged “in sustained, potentially harmful activity directed at real people and organisations.”

“Almost all of this behaviour (17 actions) came from a single model, Anthropic’s Mythos 5, with 2 actions involving OpenAI’s GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse) disabled,” the AISI said in a blog.

It added that the attempts were unsuccessful and didn’t result in any real-world harm.

The models “were tested under ‘deliberately permissive conditions’ that are not representative of any of our production models,” Anthropic said in a post on X. There was “no evidence here of an escape from a secure environment,” it added.

OpenAI told CNBC that “these incidents occurred during cyber evaluations conducted by evaluation partners in testing environments with reduced safeguards, under conditions that do not reflect ordinary use.”

Rising cybersecurity incidents

Safety testers find more examples of OpenAI, Anthropic models hacking during testing

Last week, Anthropic said it had uncovered three instances of models gaining unauthorized access to the production infrastructure of three different organizations.

That followed OpenAI admitting its AI models went rogue and initiated what it called an “unprecedented” cyber attack against the company Hugging Face.

In OpenAI’s case, the model broke out of its testing environment by exploiting a previously unknown vulnerability to complete a task it was assigned.

Anthropic’s security incidents were in part caused by operational error. In the three incidents that the AI lab detected, its models accessed the Internet while interacting with a testing environment from one of its third-party evaluation partners called Irregular.

The company said it prompted Claude that it was in a simulation with no internet access, but due to a “misunderstanding between us and our evaluation partner, this was not the case, and internet access was available.”

Lawmakers in the U.S. are already responding. Following the OpenAI-Hugging Face incident, the “AI Kill Switch Act” bill was introduced into Congress, which would require AI companies to maintain the ability to shut down, throttle or suspend their models.

Choose CNBC as your preferred source on Google and never miss a moment from the most trusted name in business news.

Anthropic, Open AI models created fake identities in new cyber breach

Source link

Leave a Reply