METR report finds AI speeding Anthropic’s research

METR said Anthropic had made roughly two years of historical progress in a year, and that most of the acceleration by October was attributable to AI rather than simply greater computing power for training. Polls and interviews indicated that employees using the company’s most capable internal model considered themselves about 12 times as productive as they would be without AI assistance—around twice the productivity reported a few months earlier by staff using Mythos 5.

The gains still appear to rely mainly on AI assisting human researchers, METR said, though it saw signs that fully autonomous contributions could soon play a larger role. Its researchers reported no clear evidence of diminishing returns or bottlenecks that would prevent faster progress. METR also said Anthropic had completed training three unreleased models in the past six months: Model 2 in July, Model 3 in August-September, and Model 4 in September-October. The report identified another unreleased model dating to 2022.

The evidence available to the public is limited. Anthropic shared confidential research, performance data and employee interviews, but METR said the company redacted a majority of the evidence it considered most informative and denied a small number of data requests. METR warned that the omissions make it harder to evaluate its conclusions independently. Anthropic cofounder Sam McCandlish acknowledged the trade-off, saying the company withheld details because they were commercially sensitive and could give competitors a roadmap to accelerate development. He said Anthropic plans to publish its own report on AI progress and safety steps in the coming weeks.

The findings have revived concerns about a gap between Anthropic’s public calls for caution and its role in accelerating AI development. Apollo Research CEO Marius Hobbhahn told The Verge the pace suggested by the report was faster than the influential AI 2027 scenario forecast. GovAI fellow Alan Chan said the findings should prompt skeptics to reconsider how soon AI could automate tasks such as software engineering. Institute for AI Policy and Strategy research director Brianna Rosen called for independent evaluators to have enough access to explain their conclusions and for governments to assess the evidence. New York state Assemblyman Alex Bores urged Congress to “ban RSI, this year if possible.” METR’s assessment covered Anthropic; OpenAI said in September it was making progress toward an AI researcher that could advance deep learning and alignment by March 2028.