The METR Anthropic report says Anthropic’s AI models have been getting better about 1.9 times faster since October 2025, and that AI doing Anthropic’s own research explains most of it. If that holds, one of the biggest AI labs is already using its models to speed up the next models.
METR, an independent AI evaluation group, published its findings on November 5. Anthropic blacked out much of the strongest evidence, so the numbers come with real uncertainty.
Here are the facts worth knowing first:
- METR estimates Anthropic’s benchmark progress runs about 1.9x faster than before October 2025, with a 95% range of 1.5x to 2.8x.
- It guesses about 80% of the speedup comes from AI helping with research, not from more training compute.
- Anthropic redacted most of the evidence METR rated as most informative.
- METR still rates the near-term catastrophic risk as low.
That list is the short version. The rest of this post explains what the numbers mean and what they do not.
What the METR Anthropic report actually measured
METR scored Anthropic’s models, including some that aren’t public, on a benchmark index called AECI. It is Anthropic’s version of the Epoch Capabilities Index. Scores climbed about 15 points per year before October 2025 and about 29 points per year after. That jump is the 1.9x figure.
METR also tried other methods as a sanity check. Those gave a wider spread of 1.1x to 2.9x. METR says it would be surprised if the real answer were no speedup at all, or more than 3x.
One more detail: the gain shows up on well-defined tasks that can be graded automatically. METR notes the index could overstate real-world progress if that is where most of the improvement sits.
| Claim | Number | How solid it is |
|---|---|---|
| Speedup in benchmark progress | About 1.9x (range 1.5x to 2.8x) | Main estimate, with a stated confidence range |
| Share of speedup caused by AI | About 80% | METR’s own judgment, called highly uncertain |
| Staff-reported productivity gains | Very large (about 12x in some self-reports) | METR calls self-reports “very loose evidence” |
| Industry-wide speedup (all labs) | About 1.4x | Lower, possibly because other labs are catching up |
Why the 80% figure is the one to watch
Faster progress alone isn’t surprising. Labs have been adding compute for years. What stands out is the cause. METR says it is confident the speedup was not driven by faster growth in training compute.
So the likely driver is AI tools helping researchers write code, run experiments and analyze results. That is the early shape of a feedback loop: better models help build better models. This is the same trend behind agents that remember and improve over time, which we covered in our look at Claude’s agent memory and “dreaming”.
Not everyone inside the company agrees on the size of the effect. According to The Decoder, some internal researchers put AI’s share of the gains at only 15 to 20%.
Did Anthropic cross its own warning line in July?
Anthropic’s Responsible Scaling Policy treats AI R&D acceleration as a key risk signal. A doubling of progress from automated research is the kind of change that triggers extra caution.
According to The Decoder, METR’s analysis suggests that line may have been crossed around July 15. Anthropic’s public August risk report said the acceleration was “less than a factor of 2.” The unredacted internal version reportedly implies about 1.97, and Anthropic called a crossing “plausible.”
METR itself said it did not focus on the formal thresholds. It aimed for a broader picture, so treat the July date as one outlet’s reading, not a confirmed finding.
What METR could not see
This is the biggest caveat. Anthropic redacted a majority of the evidence METR considered most informative. It says the redactions protect confidential details that could help rival labs move faster.
METR also lacked staff-level access and could not query unreleased models. The public therefore has to lean heavily on METR’s judgment. METR itself says better access to evidence is needed so outsiders can check these risks independently.
According to The Decoder, Anthropic also cancelled one planned piece of the project, an audit of how it splits compute between capability work and safety work, saying it wanted to focus on risk assessment first. For context on how the company handles other trust questions, see our post on Anthropic FlashProve.
What this means if you just use AI tools
Nothing changes in your apps today. This report is about the pace of model development, not a bug or a security hole. You do not need to change settings or switch tools.
What you may notice is the release rhythm. If progress really is running near twice the old speed, expect new model versions to arrive faster and benchmark rankings to shift more often. Our explainer on why AI model rankings are harder to game is a useful companion when new scores appear.
METR says it cannot rule out another sharp acceleration within six months. Still, it rates the acute catastrophic risk as low, in line with Anthropic.
Frequently Asked Questions
Who is METR?
METR is an independent group that evaluates advanced AI systems for risky capabilities. It reviewed Anthropic’s internal data and interviewed staff over about 11 weeks.
Does 1.9x mean Anthropic’s AI is twice as smart?
No. It means benchmark scores are rising about 1.9 times faster than the older trend. Those benchmarks are a proxy, and METR warns they may not match real-world impact.
Is the 80% AI-driven claim confirmed?
No. It is METR’s subjective estimate, given with substantial uncertainty, and some Anthropic researchers think the share is much lower.
Does this affect my Claude account or data?
No. The report is about research speed. If you manage a team account, our guide to Claude Enterprise security controls covers the settings that do matter day to day.
Our take
The headline number matters less than the missing evidence. A 1.9x estimate from a respected outside group is useful, but it rests on data the public can’t inspect. Read it as a strong signal, not a settled fact, and watch for whether Anthropic and other labs let outsiders see more. If you rely on AI tools for work, plan for faster model changes and keep your workflows easy to switch.
Leave a Reply