Departing Anthropic researcher calls for stronger controls on advanced AI
Ars Technica reports that AI researcher Jacob Coxon used his departure from Anthropic to call for stronger controls on the development of self-improving AI. His public warning concerns possible future systems with capabilities beyond those of today's models.
Reporting by a news outlet — not a first-party document.
What we could not confirm independently: This claim is attributed to the named outlet. We did not verify it against a first-party document. (arstechnica.com)
Replaced an automatically assembled version that included publisher navigation and generic filler with a source-attributed summary. Clarified that predictions are assessments, not established outcomes.
What happened
Ars Technica reports that AI researcher Jacob Coxon used his departure from Anthropic to call for stronger controls on the development of self-improving AI. His public warning concerns possible future systems with capabilities beyond those of today's models.
The report also describes comments from Anthropic alignment researcher Evan Hubinger, who expressed his own concern about severe future risks. These statements are researchers' assessments, not evidence that a catastrophic outcome is certain or has already occurred.
The distinction matters because a warning about future capabilities is different from a demonstrated failure of a deployed product. The article discusses both concerns about accelerating development and uncertainty about whether current technical progress will continue at the same pace.
Interactive Mechanism
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
3Self-correction: Backtrack and refute subtle edge cases
4Exhaustive consistency check & final output synthesis
Core takeaway: Test-time compute fundamentally changes AI economics. Instead of only scaling during pre-training, giving reasoning models more tokens at inference time allows them to systematically solve PhD-level STEM problems.
Interactive Concept Check+10 Points
What is AI? Quiz
A route planner searches possible journeys using explicit rules. What does this illustrate about AI?
What to watch next
Coxon urged greater coordination between laboratories and more rigorous understanding of advanced systems before further scaling. Assessing those proposals requires specific evidence about capabilities, safeguards and enforcement, rather than treating a headline prediction as an established probability.
This canonical story is updated in place when the developing event materially changes. Its URL and original publication date never change.
Replaced an automatically assembled version that included publisher navigation and generic filler with a source-attributed summary. Clarified that predictions are assessments, not established outcomes.