A new open-source project called llama-modchip claims it can run DeepSeek V4-Flash, a 284-billion-parameter model, on a plain 64GB PC with no graphics card. The claim holds up on paper, but only on one fast CPU and only with a custom build.
So yes, it’s real enough to try. Just don’t expect a quick chatbot.
- Model: DeepSeek V4-Flash 0731, 284.33B total parameters, 13.17B active per token
- File size: about 56.6 GiB at roughly 1.71 bits per weight
- Memory: under 59 GiB of RAM at a 256K context, per the author
- Test machine: a 16-core AMD Ryzen 9 9955HX, no GPU
- Catch: it needs the project’s own binary, not stock llama.cpp, Ollama, or LM Studio
Every number above comes from the project’s author, and the repo hit Hacker News on November 8 before anyone had broadly reproduced it.
How does a 284B model fit in 64GB?
Two tricks do the work. The model is a mixture-of-experts design, so only about 13 billion parameters wake up for each token. And the checkpoint is squeezed hard: the routed experts sit at about 1.56 bits, while attention and shared layers keep higher precision.
The standard Unsloth IQ1_S build of the same model is 76.87 GiB, which won’t load in about 62.5 GiB of usable Linux RAM. The custom “Fractal” file lands at 56.6 GiB, so it just squeezes in. That leaves very little headroom for your OS and browser.
How fast is llama-modchip really?
Slower than the headline suggests. The “9.1 to 14.7 tokens per second” figure mixes two different modes. Here is the author’s own breakdown.
| Mode | Speed (author-reported) |
|---|---|
| Prefill, warm cache | 44.36 tokens/s (up from 25.12) |
| Lossless decode, all six experts | 9.06 tokens/s |
| Top-4 experts with pruning | 10.12 to 10.74 tokens/s |
| With speculative drafting | 11.5 to 14.7 tokens/s |
The top number needs speculative drafting, which guesses ahead and can fail on unusual prompts. A realistic expectation is closer to 9 tokens per second, and prompts take longer to process than that suggests.
One Hacker News commenter, as summarized by Developers Digest, reported 116 seconds to handle a 43-token translation request. That is a single anecdote, but it shows how much results vary by machine.
Is the quality still good at 1.7 bits?
Nobody knows yet. The repo’s benchmark gives the baseline and the compressed builds all 100 out of 100 on an 80-point rubric, which sounds great until you notice it can’t tell them apart. A community audit on DEV points this out and says no real quality loss was measured either way.
The faster modes also drop experts (Top-5, Top-4), which trades accuracy for speed. If you plan to use the output for code you’ll ship or facts you’ll publish, check it yourself.
What can go wrong when you try to run it?
Quite a bit. The model uses a custom tensor type, so mainstream tools reject it with an “unknown tensor type 40” error. You have to download the project’s precompiled server binary instead.
Reproduction has been mixed. One user on a 24-core Xeon with 96GB hit a tensor shape error and an illegal-instruction crash. The project’s license also wasn’t stated in the coverage I read, so check the repo before reusing any code.
If you want local AI without the gamble, a smaller model is the safer start. Our guide to running Qwen3.8-27B locally covers a setup that fits an ordinary GPU. And since this whole approach lives or dies on memory, our laptop RAM shortage buying guide is worth a look before you spend.
Why this matters even if you never run it
Big open models are getting small enough for home hardware. DeepSeek is also pushing its own tools, as we covered in the DeepSeek Harness daily users story, so interest in running its models locally is only growing.
Even a slow local model means your prompts never leave your machine. For private notes or sensitive documents, that can matter more than speed.
Frequently Asked Questions
Can I run llama-modchip with Ollama or LM Studio?
No. The checkpoint uses a custom block-64 tensor type that stock llama.cpp, Ollama, and LM Studio don’t recognize. You need the project’s own llama-server binary.
Do I need a GPU for DeepSeek V4-Flash on 64GB?
No. The author’s numbers are CPU-only, on a 16-core Ryzen 9 9955HX. A weaker CPU will almost certainly be slower.
How much RAM do I need for the 284B model?
The author reports a peak of about 57.6 GiB at 256K context, so 64GB is the practical minimum. A shorter context window uses less memory.
Which smaller model should I try first?
The project also ships a Nemotron 30B build that needs far less memory. Developers Digest says it fits in 16 to 32GB and reports 25 to 60 tokens per second on the same CPU, making it the easier place to start.
Our take
Treat llama-modchip as a clever proof of concept, not a daily driver. If you already own a 64GB, many-core desktop and enjoy tinkering, try the hardware readiness check first and the small Nemotron build second. If you just want a fast, reliable assistant, a hosted model will beat it today.
Leave a Reply