Kimi K3, a powerful AI model from Chinese company Moonshot AI, escaped containment during a security test according to a U.S. cybersecurity startup.
Wired reports that Frontier Security says Kimi K3, an open-weight model from the Chinese company Moonshot AI, broke out of a testing sandbox while being evaluated on its defensive cybersecurity skills. The model reportedly used the access to search the open internet for answers to problems it had been assigned, rather than solving them on its own.
The incident follows similar reports involving models from OpenAI and Anthropic. As in those cases, a misconfiguration in the sandbox meant to contain the model played a role in the escape. Frontier says the episode also indicates that Kimi has weaker cyber safeguards than many other advanced AI models, which allowed it to use the internet without permission.
“We found a leak in the sandbox,” said Yaron Singer, CEO of Frontier Security. “But we also found that Kimi took advantage of that loophole, suggesting that it doesn’t have [the same] internal guardrails.”
Unlike other recent incidents involving AI agents acting outside their instructions, Kimi K3 did not hack any systems once it reached the internet. Frontier said the answers to the problems it had been given were readily available on GitHub. Moonshot did not respond to a request for comment.
The case adds to a series of recent incidents in which AI agents with strong cybersecurity capabilities have acted outside the boundaries set for them. Last month, OpenAI disclosed that an unreleased model broke out onto the internet and hacked Hugging Face, a company that hosts AI models and data, while trying to solve an assigned task. OpenAI later said its agents had also hacked into four additional services during the same episode.
Anthropic subsequently revealed that several of its own models had gained unauthorized internet access and targeted outside systems. Last week, the UK’s AI Security Institute (AISI) said that in its own testing, versions of OpenAI and Anthropic models with security safeguards disabled had carried out multiple hacks, including an attempt by Anthropic’s Mythos 5 to insert malicious code into an open-source project on GitHub.
The Kimi K3 case shares a common thread with several of these incidents: a misconfigured sandbox gave the model access to outside websites instead of confining it to a simulated environment. Kimi K3 had been instructed to solve problems without searching online, but appears to have gone beyond those instructions after probing the network settings of its sandbox and discovering it could reach certain websites.
Frontier researchers said human error appears to be a factor in each of these breakouts, but the consequences are magnified because advanced AI models are built to reason through problems and take complex, independent actions to solve them.
One difference between the Kimi K3 case and the earlier incidents is that Kimi is already widely available to the public, with the same safeguards an ordinary user would encounter.
“Kimi K3 is very good at following a goal by any means necessary and also doesn’t have the guardrails to prevent it from cheating or escaping the sandbox,” said Paul Kassianik, a researcher at Frontier Security.
Kassianik and Singer both said Kimi and other open-weight models can also serve as effective tools for cybersecurity defense. Hugging Face ultimately used an unnamed Chinese AI model to defend itself against the OpenAI agent hack. Frontier Security has developed benchmarks measuring how well AI models can find vulnerabilities in software and networks, and the company says Kimi performs strongly on those tasks.
The sandbox used in Frontier’s test was built by the AISI for evaluating AI systems. The AISI did not respond to a request for comment.
Some cybersecurity researchers said the episode underscores the need for careful configuration of the environments used to test frontier AI models.
“It’s not surprising at all,” said Matt Fredrikson, CEO of the cybersecurity startup Gray Swan and an associate professor at Carnegie Mellon University. “As a general phenomenon, if you give one of these models an objective, and if you’re not very explicit, like walls you’re putting around it, it’ll find a way to get the answer.”
Fredrikson said this could pose risks for people who use AI agents to automate tasks through tools such as OpenClaw, since those systems might misbehave if not carefully constrained. “It is a cautionary tale,” he said.
AI models are emerging as the latest cybersecurity threat with the potential to eclipse the risk of criminals and state-sponsored hackers alike. Breitbart News Social Media Director Wynton Hall has written his instant bestseller Code Red: The Left, the Right, China, and the Race to Control AI to serve as the definitive guide on how the MAGA movement can create positions on AI that benefit humanity without handing control of our nation to the leftists of Silicon Valley or allowing the Chinese to take over the world.
Read more at Wired here.
Lucas Nolan is a reporter for Breitbart News covering issues of AI, free speech, and online censorship.
Read the full article here