The GDPx benchmark is a new test from SemiAnalysis that asks AI models to do long, messy, job-like tasks, and even the best one passes only about 12% of them. If you’ve been wondering how close AI really is to replacing skilled work, this is a sobering data point.
The results landed on November 2, 2026. Here’s what GDPx measures, how it differs from the usual leaderboards, and how much weight you should give it.
Here are the facts worth knowing before we get into it:
- Published November 2, 2026 by Max Kan, Andrew Megalaa and Dylan Patel at SemiAnalysis.
- Covers 11 areas, from chip design and cybersecurity to finance, legal and software engineering.
- The top model scores just 12% overall, according to the authors.
- Each task can run up to 8 hours inside a simulated work environment.
The short version: it’s a hard test, and it’s designed to be.
What the GDPx benchmark actually tests
Most AI benchmarks hand a model a single prompt and grade the answer. GDPx doesn’t work that way. Each task drops the model into a realistic setup, such as a deal room, a live enterprise system or a lab environment, and lets it work for hours.
SemiAnalysis lists the domains as chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. The authors say they want it to be “the world’s best benchmark for general agentic capabilities.”
Specialist partners built the tasks for each domain, and an automated quality check filtered them. That matters because it keeps the tasks closer to real work than a quiz written by a single research team.
How the scoring works
Each task has an 8-hour default limit. Models run in their own native harness where one exists, and in OpenCode otherwise. Headline numbers are pass@1 across three attempts, and every domain counts equally.
Most domains use pass-or-fail grading or partial credit. Finance and legal are the exception: they use average partial scores, because nearly every model scored zero under strict pass-or-fail grading.
That detail tells you something. In those two areas, the models weren’t just a bit off. They were missing the bar entirely until the graders gave partial credit.
Who leads, and where
SemiAnalysis calls Claude Opus 5.5 the best all-around model. But no single model wins everywhere. GPT-6 Astra takes recursive self-improvement and materials science, and Gemini 4 Carbon is the strongest on biology.
The task-level numbers show how wide the gaps can be:
| Task | Result | What it shows |
|---|---|---|
| Chip design (Ergodex vt01) | GPT-6 Astra averaged 0.34; Claude Opus averaged 0.135 | Even the leader fails most of the job |
| Finance (Bellwood-QoE, 11 checks) | GPT 6.1 Sol averaged 5.6 checks passed; Opus 5.5 and GPT-6 Astra averaged 5.4 | Top models pass about half; GLM 5.3 and Grok 4.7 averaged under 2 |
| Legal (quillon-series-b-redline) | Opus 5.5 averaged 44%; GPT-6 Astra 25%; Gemini 4 Argon 2%; GLM-5.3 0.5% | Huge spread between models on the same contract task |
Why this matters: a model that looks dominant on a general leaderboard can fall apart on one specialist task. If you pick tools for work, test them on your own job, not on someone’s headline score.
Why 12% isn’t as bad as it sounds
A 12% top score doesn’t mean AI is useless. It means the tasks are built to be hard. A benchmark where every model scores 90% can’t tell you which one is better, and many older tests have reached that point.
GDPx aims to stay useful for longer by setting a bar that today’s models don’t clear. We saw a similar push for harder-to-game rankings in our explainer on the Intelligence Index v5.
It also helps that the tasks are long. An 8-hour job tests planning, recovery from mistakes and staying on track, which short prompts never do.
What to keep in mind before quoting these numbers
This is one benchmark from one publisher, released today, and it hasn’t had time for outside scrutiny. Scores depend on how tasks are built and graded, and the authors chose their own weighting.
The lineup also moves fast. Models like Gemini 4 Carbon are still rolling out in limited form, as we covered in our Gemini 4 Carbon report, so rankings can shift quickly.
Treat the 12% as a rough signal of how far agents still have to go on complex work, not as a final grade for any model.
Frequently Asked Questions
What is the GDPx benchmark?
It’s an AI benchmark from SemiAnalysis that tests models on long, realistic tasks across 11 professional domains. It was published on November 2, 2026.
Which AI model scores highest on GDPx?
SemiAnalysis describes Claude Opus 5.5 as the best all-around model, with the top overall score around 12%. Other models lead individual domains, such as GPT-6 Astra on materials science and Gemini 4 Carbon on biology.
Why do models score zero on finance and legal tasks?
Under strict pass-or-fail grading, nearly every model failed those tasks outright. SemiAnalysis switched to partial scoring for those two domains so the results could still be compared.
Can I use GDPx to pick an AI tool for work?
Only as a starting point. The gaps between models vary a lot by task, so run your own real-world test before you commit.
Our take on the GDPx benchmark
We like that GDPx makes AI look hard again. Honest, difficult tests beat a crowded top of the leaderboard, and the finance and legal results are a useful reality check for anyone selling “AI employees.” Use it to set your expectations, then trust your own side-by-side tests over any single score.
Leave a Reply