(Image credit: Getty / Bloomberg)
Anthropic has published a report warning that the development path it’s on could eventually leave humans unable to control AI systems, even as it disclosed that Claude now writes more than 80% of the code merged into its own codebase. The Anthropic Institute, the company's research arm, said AI has already started to speed up AI development and that the trend could lead to recursive self-improvement, the point at which a model designs and builds its own successor with little human input. The report argued that the world should keep open the option to slow or pause frontier development, and cautioned that the occasional misalignment seen in current models could grow more common and harder to understand as those models build the next generation.Go deeper with TH Premium: AI and data centersThe company set out three pretty dire ways the next few years could unfold, reserving its most severe warnings for the scenario in which models become capable of fully improving themselves. In that case, Anthropic said, the pace of progress would be set almost entirely by available compute, with humans pushed toward oversight and verification roles and a self-improving model dominating as its abilities outstrip those of the people who built it.The firm described this potential issue with alignment and the task of keeping a system's behavior tied to human intent as part of the future it’s least sure about. Misalignment that’s rare and survivable today could compound generation over generation until control slips, it said, though it allowed that a sufficiently capable and well-aligned model might instead choose to halt its own development. Anthropic wrote that this misalignment could keep "growing more frequent but less understood until we lose control of them."Anthopric is backing these warnings with a bunch of internal figures that we’ve not seen before. More than 80% of the code merged into its production codebase as of last month was authored by Claude, up from low single digits before Claude Code reached research preview in February last year. Anthropic says the typical engineer is now “merging 8x as much code per quarter as they did from 2021-2025.”On the hardest, least-specified coding tasks, Anthropic said Claude succeeded 76% of the time in May 2026, a rise of 50 percentage points in six months. A recurring internal test that asks each new model to make training code run faster saw results climb from roughly triple the original speed with Claude Opus 4 in May 2025 to about 52 times with the unreleased Mythos Preview model in April.Anthropic said it’d slow or pause only if rival labs at or near the frontier did the same in a verifiable way, and that a halt by one company would change who leads without achieving anything wider. That’s obviously not going to happen.All the figures cited by Anthropic are self-reported and unaudited, and come days after the company filed to go public. The company issued a similar self-assessment in April, when it said Mythos Preview had found thousands of severe software vulnerabilities, a claim that later drew scrutiny over how much of it rested on a small manual sample.Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
