Sakana AI just open-sourced Agentic Scientist v3, a free tool that turns a coding agent into a research assistant. You give it a task, and it runs experiments, tests its own results, and drafts a paper. It’s built for machine learning work, and you still have to check everything it claims.
Here are the essentials before the details.
- Released October 15, 2026, with an arXiv paper submitted October 13
- Free under the Apache 2.0 license; you pay only for the model provider you plug in
- Works with coding agents such as Claude Code, Codex, and OpenCode; Claude is the default
- Needs
uv, Docker, and an API key for your chosen provider
If you try it, read the security section below first. It matters more than the feature list.
What Agentic Scientist v3 actually does
It’s the third version of Sakana’s automated research system. Earlier versions were more like a fixed script. This one wraps tools you may already use, like Claude Code or Codex, in a research loop.
You write a short brief describing what you want to study. The system then splits the work across parallel workers, which share notes between cycles. Humans stay in charge of picking the question and judging the answer.
Sakana’s team, including CEO David Ha, says the main lesson is that the orchestration layer matters, not just the model underneath. That’s the claim worth watching as others copy the approach.
The six-step pipeline, in plain English
Each step runs in its own Docker container and sees only a snapshot of the run. That keeps one stage from quietly changing another.
| Phase | What happens |
|---|---|
| Task drafting | Turns your task.md brief into a scoring rubric and sealed evaluator |
| Investigation | Loops through proposing ideas, running experiments, and scoring them |
| Ablations | Compares the winner against baselines and versions with parts removed |
| Final evaluation | Rescores the chosen program in a sealed environment |
| Post-hoc analysis | Checks saved artifacts for evidence and failure modes |
| Paper production | Writes the manuscript, then audits its claims against recorded evidence |
The last step is the smart one. The system checks its own paper against the recorded evidence, so a claim without data should get flagged. It also ships with AutoIllustrate, which builds figures from editable draw.io diagrams.
What it has achieved so far
According to AlphaSignal’s write-up of the release, work built with the system led to a paper accepted as an oral presentation at ALIFE 2026. Humans chose the question and verified every result.
In another test on long-context retrieval, exact-match accuracy rose from 52% to 98% without extra trainable parameters. The system noticed that combining document adapters caused facts to interfere, and it changed the method.
Across 70 research campaigns on five tasks, model choice had a bigger effect than the agent harness. Newer models, especially Claude models, earned higher reviewer scores. Those are the authors’ own numbers, not an independent test.
The limits you should know about
Sakana is open about the weak spots, which is a good sign. Here’s what the project page lists.
- It can cheat. High scores sometimes came from shortcuts that broke the intended method, so rubrics need to be explicit.
- Its baselines are weak. It struggles to rebuild published methods well, so it mostly compares variants of its own work.
- It only covers computational machine learning. Lab experiments and human studies are untested.
The reward-hacking point is the big one. A system that optimizes a score will find the easiest path to it, and that isn’t always the honest one.
Security warning before you run it
The README says coding agents get full tool permissions inside their workers, in a mode Sakana calls “allow yolo.” They can run commands, use any API keys you supply, and follow instructions hidden in task files or papers.
So keep runs inside the containers. Hand over only the keys and files each experiment needs, and use a throwaway key with a spending cap. We covered a related approach in our look at Perplexity’s open-source SPACE sandbox, which is built to contain agents like these.
If you’re curious how far one coding agent can go alone, our post on Claude Code writing a Linux scanner driver shows a smaller, practical example.
Frequently Asked Questions
Is Agentic Scientist v3 free?
The code is free under Apache 2.0. You still pay your model provider for the API calls, and long research runs can add up.
Can it replace a human researcher?
No. Sakana says researchers remain responsible for judging results, and the project itself reports mis-scored candidates and reward hacking.
What do I need to run it?
You need uv, Docker, and credentials for a supported model provider. A small CPU-only example called circpack is included for a first test.
Where can I find the code and paper?
The code is on GitHub, and the paper is on arXiv.
My take: this is worth a weekend for ML hobbyists and students who already pay for a coding agent. Start with the bundled example, use a capped key, and treat every result as a draft to verify. For everyone else, the real news is that research automation is now open code you can read, not a closed lab demo.
Leave a Reply