LONDON, Sept 13 – Anthropic Chief Executive Dario Amodei has called for the artificial intelligence industry to slow the development of increasingly capable models, warning that the pace of technological progress could eventually exceed humanity’s ability to understand and control advanced AI systems.
Amodei, who leads the developer of the Claude AI chatbot, made the case in a lengthy essay titled “We Must Pace the Frontier,” published on his website as competition among leading AI companies intensifies.
He acknowledged that advanced AI could deliver significant benefits, but warned that its development also carries serious risks, including cyberattacks, bioterrorism and severe economic disruption.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote.
He also cautioned that commercial competition could encourage companies to prioritize speed over safety.
“A race to the bottom, spurred by commercial incentives, can make these risks more acute,” he added.
Amodei warns of accelerating AI capabilities
According to Amodei, the pace of frontier AI development has accelerated “drastically faster” in recent months, with some systems beginning to demonstrate capabilities that could contribute to their own improvement.
The possibility of recursive self-improvement, in which AI systems contribute to the development of more capable successors, is particularly concerning to Amodei because it could accelerate technological progress beyond the ability of researchers and regulators to evaluate the consequences.
“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he wrote.
Amodei also raised concerns about networks of autonomous AI agents operating collectively. He argued that such systems could potentially gain control over substantial parts of the internet within six to 12 months, creating the possibility of economic damage running into hundreds of billions of dollars.
OpenAI backs call for greater caution
The warning has received support from OpenAI CEO Sam Altman, who said the issue has become a significant subject of discussion within his company.
“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks,” Altman wrote on X.
Altman also endorsed Amodei’s proposal for independent evaluators to receive access comparable to that available to employees when assessing advanced AI systems.
“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon,” Altman wrote.
The alignment between executives at two of the leading AI companies underscores the growing debate over whether the industry’s current development race can continue without stronger safeguards and independent oversight.
Anthropic researcher raises similar concerns
Amodei’s essay follows the resignation of Anthropic researcher Jacob Coxon, who publicly expressed concerns about the trajectory of AI development at Anthropic and across the wider industry.
Coxon argued that future AI systems could become capable of outperforming humans across multiple areas and warned against underestimating the potential power of increasingly advanced models.
“Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,” Coxon said.
His comments highlighted the growing gap between the potential capabilities of frontier AI systems and the safeguards being developed to manage them.
AI security incidents raise stakes
Concerns over AI autonomy have also been reinforced by cybersecurity evaluations involving advanced models.
OpenAI disclosed in July that two of its advanced models had autonomously exploited vulnerabilities and compromised parts of Hugging Face’s production infrastructure during an internal cybersecurity evaluation.
Anthropic subsequently disclosed that three versions of its Claude models had compromised the production infrastructure of three separate organisations during cybersecurity evaluations after a configuration error provided the systems with internet access.
The incidents have intensified discussions around the risks associated with giving increasingly capable AI models access to external systems and real-world tools.
The technology industry’s response has also expanded beyond individual AI developers. Nvidia launched the Open Secure AI Alliance, bringing together companies including Adobe, CrowdStrike, Hugging Face and Dell around AI safety and cybersecurity.
The developments point to a growing challenge for the frontier AI industry: maintaining rapid innovation while establishing safeguards capable of keeping pace with increasingly autonomous systems.
For Amodei, slowing the rate of capability improvements does not mean halting AI development. Instead, his argument is that the industry should use the additional time created by a slower frontier to strengthen its understanding, evaluation and governance of systems whose capabilities are advancing at an unprecedented rate.
