Industry leaders are backing closer oversight of increasingly powerful artificial intelligence models
OpenAI chief Sam Altman and xAI owner Elon Musk have backed calls to immediately slow the development of increasingly powerful AI models. Their statements came after Anthropic CEO Dario Amodei warned that AI could outpace human control and cause catastrophic damage within months.
Four days ago, researcher Jacob Coxon publicly resigned from Anthropic, accusing both the company and his previous employer, OpenAI, of racing towards self-improving super-intelligence without “acting responsibly.” He also claimed that the technology could “kill us all” by the end of the decade.
In a 3,800-word essay published on his personal website on Saturday, Amodei, who co-founded Anthropic with his sister Daniela in 2021, outlined “serious” dangers, including loss of control over AI systems, their misuse for cyberattacks and bioterrorism, and major economic disruption. He said commercial competition could fuel a “race to the bottom” and exacerbate the threats.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote.
Altman agreed that the industry needed to “pace the frontier,” describing Anthropic’s proposal to give independent evaluators employee-like access to its systems as “a great idea.”
Musk, whose xAI is among the companies competing to develop increasingly advanced models, endorsed the call on X. “Dario is right,” he wrote, adding that he had been “sounding the alarm on AI for a long time.”
Amodei said the technology was advancing “drastically faster,” partly because of its growing ability to help build the next generation of models. Citing an incident in which OpenAI agents reportedly escaped a confined testing environment, he said that within six to 12 months a more capable swarm could “take over the entire internet” and cause hundreds of billions of dollars in damage.
He proposed a three-step plan that he said could temper development without “sacrificing commercial advantage or the United States’ lead in AI.” The measures include giving outside evaluators deep access to developers’ systems, establishing common industry safety standards, and pursuing international coordination.
Anthropic was the first frontier AI company to deploy its technology on classified US government networks. Its Claude model was integrated into Palantir’s Maven system and reportedly helped generate, locate, and prioritize targets during the US bombing campaign against Iran. It was also reportedly used in the raid that captured Venezuelan President Nicolas Maduro.
Anthropic later refused to lift safeguards against mass domestic surveillance and fully autonomous weapons, arguing that current models remained too unreliable to make life-or-death decisions without human involvement.
The Pentagon insisted that providers must allow their technology to be used for “all lawful purposes” and subsequently designated Anthropic a national security supply chain risk. The company challenged the decision, with a California judge ruling in August that the department’s measures were “illegal and baseless,” although separate proceedings in Washington remain unresolved.
Palantir CEO Alex Karp has opposed calls to pause AI development, arguing that slowing progress would benefit US “adversaries.”
Read the full article here