ds4, the repo that now presents itself as DwarfStar, is a C inference engine by Salvatore Sanfilippo (antirez, the creator of Redis). It runs DeepSeek V4 Flash and a short list of other open-weight models natively on Metal, CUDA and ROCm. It is not a general GGUF runner: it only loads the GGUF files the project itself produces.
I read the README and the docs/ folder on 2026-10-14 and did not run any of it. Every speed in this post is reported by the author and attributed to the page it comes from.
1. What ds4 Is, and Which Model It Really Runs
github.com/antirez/ds4 is MIT-licensed and written in C. The README opens under the DwarfStar name, while ds4 is the repo name and the prefix of the binaries (./ds4, ./ds4-agent, ./ds4-server).
Repo snapshot (GitHub API, 2026-10-14)
- • Stars and forks: 23,853 and 2,303
- • License: MIT; the LICENSE file also keeps the ggml authors’ and DeepSeek’s copyright lines
- • Language and age: C, created 2026-05-06, last push 2026-10-08
- • Releases: GitHub shows none (the releases and tags endpoints returned empty lists), so you build from main
- • Status, in the README’s words: “very fast changing. Consider it beta quality.”
Which model, exactly?
Older write-ups that say “DeepSeek V4 Flash only” are out of date. The README header now lists a family: DeepSeek V4 Flash (including an experimental vision model) first, DeepSeek V4.1 Flash on Metal with text inference on CUDA, GLM 5.2, 5.3 and 5.3 Flash, DeepSeek V4 PRO, and Qwen3.8 Flash Next on Metal and CUDA. The GitHub description still says “DeepSeek 4 Flash and PRO”.
The default first-run download, ds4f-q2, is DeepSeek V4 Flash (the checkpoint the docs call “Flash 0731”), not V4.1. V4.1 Flash needs its own GGUF, tokenizer and inference graph (ds41f-q2), and V4 Flash weights are not interchangeable with it. For the model itself, see my V4.1 Flash migration guide.
2. Why antirez Built It, in His Own Words
Two primary sources: the README and his post “A few words on DS4” (May 2026; the page only shows a relative age, so I can’t give you the exact day). The quotes below are verbatim and short.
“…aims to be the best way to run a few excellent large language models on consumer hardware (that is, hardware that people can actually own).”
README, opening paragraph
“…there was a need for single-model integration focused local AI experience…”
antirez, A few words on DS4
“…an extremely asymmetric quants recipe of 2/8 bit, so that 96 or 128GB of RAM are enough to run it.”
antirez, A few words on DS4
“…using a local model for serious stuff that I would normally ask to Claude / GPT.”
antirez, A few words on DS4
Read plainly, it is a bet that one strong open model, a recipe that squeezes it into a laptop-class memory budget and an engine tuned to that one model beat a general runner at that one job. He also says the slot is not permanent (“the model can change over time”) and the README agrees: “A model may be removed when a better replacement arrives.”
Disclosure: AI-assisted code and a debt to llama.cpp
The README says the software is developed with “strong assistance from AI coding agents”, with humans leading ideas, testing and debugging, and adds that if you are not happy with AI-developed code, “this software is not for you.” antirez’s post credits GPT 5.5. The README is just as explicit that ds4.c does not link against GGML but “exists thanks to the path opened by the llama.cpp project”, and keeps some MIT-licensed pieces adapted from it: GGUF quant layouts, CPU quant logic and certain kernels. It also asks readers who have coding agents to use them to adapt the code to their own hardware. That is the author’s workflow, not a requirement; if you do it, I would check any agent-made change against his regression checks before trusting the output.
3. Architecture in 4 Steps
ds4 is deliberately a vertical slice, not a framework. The README says model loading, prompt rendering, tool calls, KV state, the HTTP server and the coding agent are “built and tested together”. I read it as four layers:
From download to answer
- 1. Its own GGUFs. download_model.sh pulls files from Hugging Face repos, mostly the author’s own. The Q2 recipe squeezes the routed experts (IQ2_XXS gate/up, Q2_K down) and keeps Q8 projections, shared experts and output at higher precision; an imatrix guides the routed quantization.
- 2. One C engine, three native backends. ds4.c does not link GGML. Metal is the primary target, CUDA targets the DGX Spark first (plus Ada cards such as the L40S), and ROCm targets Strix Halo. The backend is picked at build time with make targets.
- 3. A memory plan, not just a loader. Weights stay resident, or SSD streaming keeps a bounded cache of routed experts and reads the rest from the GGUF. KV state can persist to disk, and two-machine tensor parallelism or multi-machine pipelines pool RAM.
- 4. Three front ends over the same state. ./ds4 is an interactive CLI, ./ds4-agent is a native coding agent that runs inference directly without an HTTP server, and ./ds4-server exposes OpenAI-style chat, responses and completions endpoints plus Anthropic-style messages.
Why a vertical slice pays off
A concrete example from docs/SERVER.md: for DeepSeek models the server preserves the sampled DSML tool-call blocks and replays them by ID, which avoids retokenizing a differently formatted JSON history when a client resends the conversation. That only works because rendering, parsing and KV handling are written for one model family.
4. Hardware and Memory: What You Actually Need
ds4 is a big-memory engine. The README names a 96 GB Mac as the floor for resident inference; below that you stream from SSD and accept the speed cost.
Metal
Primary target: Macs with 96 GB or more. Smaller machines use SSD streaming. The same build covers M3 and M5 (docs/METAL.md).
Needs: Apple’s command-line developer tools.
make
CUDA
DGX Spark (GB10) is the main goal. Multi-GPU and Ada Lovelace cards, including 48 GB L40S, are supported for the Flash kernels.
Needs: NVIDIA driver, CUDA toolkit with nvcc and cuBLAS.
make cuda-spark make cuda-generic
ROCm
Strix Halo, such as a 128 GB Framework Desktop (Radeon 8060S, gfx1151). Some 128 GB setups expose only about 62 GB to the GPU, which is too little for resident Flash Q2.
Needs: HIP, hipBLAS, rocBLAS and rocWMMA development files.
make strix-halo
Where to start by Mac memory (docs/METAL.md)
- 64 GBFlash Q2 with --ssd-streaming
- 96 GBFlash Q2; leave room for context and other apps
- 128 GBFlash Q2, or GLM 5.3 Flash Q2 with modest context
- 256 GBFlash Q4 / MXFP4 or GLM 5.3 Flash Q4
- 512 GBLarger models, including PRO Q2
The docs call these “starting points, not guarantees” that a given context or session count will fit.
DeepSeek V4 Flash downloads (docs/MODELS.md)
- ds4f-q2Flash 0731, about 81 GiB; starting point for 96/128 GB systems
- ds4f-q2-q4Mostly Q2 with the last six routed-expert layers at Q4; needs more memory
- ds4f-q4Flash 0731 Q4; larger-memory or distributed systems
- ds4f-mxfp4DeepSeek’s native MXFP4 routed experts, not requantized; larger-memory or distributed
- pro-q2-imatrixPRO 0813; 512 GB resident target, or SSD streaming
V4.1 Flash is a different size class
Per docs/MODELS.md, ds41f-q2 is a 341 GiB file (152 GiB of main weights plus 189 GiB of Engram tables that are always read from disk) and ds41f-q4 is 483 GiB. On one 128 GB Mac or Spark the docs say to use SSD streaming, and Q4 needs streaming or a 512 GB Mac. Budget a fast local SSD and the disk space before you start the download.
5. Quickstart
Commands below are copied from the README (and docs/METAL.md, DGX_SPARK.md, STRIX_HALO.md, SSD_STREAMING.md and SERVER.md for the variants) and verified against the README on 2026-10-14. I did not execute them. Read download_model.sh before you run it: the first model is about 81 GiB.
Clone and build (Apple Silicon, Metal)
git clone https://github.com/antirez/ds4.git cd ds4 make
First model for a 96 or 128 GB machine
./download_model.sh ds4f-q2 # lands in gguf/, rerun the command to resume an interrupted download
Ask once, chat, or serve
./ds4 -p "Explain Redis streams in one paragraph." ./ds4 ./ds4-server --ctx 32768
Other builds (DGX Spark and Strix Halo)
make cuda-spark && ./ds4 --cuda make strix-halo && ./ds4 --rocm
Smaller Mac: stream experts from SSD
./ds4 --ssd-streaming --ctx 32768 --nothink
Call the local server (docs/SERVER.md)
curl http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Explain Redis streams."}],"stream":true}'Good to know, from the docs
- • The default model is ds4flash.gguf, a link updated by main-model downloads; pass -m FILE to be explicit.
- • Thinking is on by default; --nothink (or /nothink in the CLI) gives direct answers.
- • The server listens on 127.0.0.1:8000. --host 0.0.0.0 opens it to the network and there is no built-in authentication: the dsv4-local strings in client configs are placeholders.
- • docs/CLIENTS.md has setups for Pi, OpenCode, Codex CLI and Claude Code against ds4-server.
6. Speed: Author-Reported Numbers Only
I did not benchmark anything. These figures come from docs/PERFORMANCE.md and docs/DGX_SPARK.md. The project calls the first table a recorded baseline for DeepSeek V4 Flash Q2, not a fresh benchmark of every commit.
Flash Q2 baseline, 128 GB machines (tokens per second)
Dated single-case numbers from the docs
- • DSpark speculative decoding on one DGX Spark (2026-09-06, median of three runs, 256 tokens): a C hash-table prompt went from 19.72 to 31.41 t/s at temperature 0, but unpredictable prose at temperature 1 went from 19.53 to 18.81 t/s, which is slower. The docs say poor draft acceptance can make it slower.
- • Two DGX Sparks over a direct RoCE link (2026-10-02): 26.0 t/s decode on Flash 0731 Q2, described as a single measurement and not a release median.
- • Eight L40S in the Flash tensor-parallel setup: roughly 126 t/s aggregate with 16 decode sessions, which is total throughput and not the speed of one user.
- • SSD streaming on a 128 GB M5 Max (2026-09-06): GLM 5.3 Flash Q4_K at 177.77 GiB generated 11.9 and 14.9 t/s after the initial and continued prefill; DeepSeek Flash Vision Exp MXFP4 generated 11.9 and 19.3 t/s from a single run.
My reading, not a measurement
On the author’s numbers a top-end Mac holds 34 to 39 t/s up to 32K of context, and the Spark’s decode is less than half of that while its prefill is higher. The docs themselves warn that a streamed model that starts can still be too slow for interactive work, and thinking is on by default, so the visible answer is not the whole token bill.
7. Quantization Quality and Beta Status
Two separate caveats: what 2-bit costs you, and how settled the code is.
“Q2” does not mean the whole model is 2-bit
Per the docs the compression is spent on routed experts, while projections, shared experts and the output stay at Q8 and F16/F32 tensors remain, so the file is “not all untouched source weights”. The README’s own motivation list says V4 Flash, PRO and GLM 5.2 “tolerate aggressive routed-expert quantization”. That is the maintainer’s claim, and the evidence he publishes is internal.
No leaderboard scores
The README says ds4-eval results are DwarfStar integration checks, “not official leaderboard scores”. Treat them as regression tests, not as a quality ranking.
Quality is scored against hosted-model continuations
QA_BEFORE_RELEASES.md lists a 100-case reference next to the Flash 0731 Q2 ROCm speed notes: NLL 0.398 and 56/100 first-token matches. No full-precision baseline sits beside it, so I can’t tell you what 2-bit costs against the original weights. The only Q2 vs Q4 pair in that file is GLM 5.3 Flash: Q2 0.458 and 89/100 versus Q4 0.300 and 90/100 (lower NLL is better).
Not bit-exact across paths
docs/MODELS.md says scalar, batched and tensor-parallel execution are “not numerically identical”. Speculative decoding in its default mode is “deliberately more deterministic than ordinary sampling”; --mtp-exact-sampling preserves the ordinary distribution.
Beta, no tagged releases
The README warns that “instabilities and regressions are definitely possible”. GitHub counts 779 open issues and pull requests (it counts both; I did not triage them).
Do not use the CPU backend on a Mac
CONTRIBUTING.md calls the CPU backend a reference/debug path and warns that running it on Metal “can crash the system because of a kernel bug in macOS”.
8. Use Cases That Fit
A private coding agent on one Mac
Use ds4-agent directly, or run ds4-server and point Pi, OpenCode, Codex CLI or Claude Code at it. Set the client context no higher than the server’s, and expect a slow first prefill with large agent prompts (docs/CLIENTS.md).
A multi-user box on CUDA cards
The documented eight-L40S setup serves 16 sessions through --batched-session. Ada support matters when another implementation of the same model asks for newer GPUs.
Two machines, one bigger model
Two 128 GB Macs over Thunderbolt RDMA, or two Sparks over RoCE, run larger quants resident. The docs warn the network protocols have no authentication or encryption and that every peer must run the same commit.
Studying a purpose-built engine
The repo is a worked example of one-model inference: ds4_test logprob vectors, ds4-bench context sweeps and a long QA checklist. The author says software should be “a working template for the biggest use cases”.
9. When NOT to Use ds4
The README is unusually frank about scope. This is where I would pick something else.
✅ Use ds4 if...
- • You own a 96 to 128 GB+ Mac, a DGX Spark, a Strix Halo or CUDA cards
- • One of the supported models is exactly what you want to run
- • You want a native agent and server for that model without a framework stack
- • You can live with a beta engine you rebuild from main
- • You have tested 2-bit routed experts on your own prompts
⚠️ Pick something else if...
- • You need arbitrary GGUFs or the long tail of models: use llama.cpp, which the ds4 README itself credits
- • You have 16 to 32 GB of memory or want LoRA on small models: use MLX
- • You need high-concurrency serving with SLAs: use vLLM or SGLang
- • You are on Windows or CPU only: the README lists only Metal, CUDA and ROCm
- • You need a pinned, tagged release for production
- • You only need the model’s answers: the hosted API is far less work
For the neighbours, see my mlx-lm guide and the vLLM vs SGLang comparison. For serving, docs/SERVER.md lists ordered fallback (concurrency without native batching) on single-GPU CUDA, with a documented exception for V4.1 Q2 SSD streaming on the Spark.
My perspective
The one-model, one-engine bet is the most interesting idea in local inference this year, and it is also the risk: when the next checkpoint changes the layout, a purpose-built engine has to be adapted, as the author himself says. Treat it as a very good tool for a narrow job.
What I did not verify
- • I did not build or run ds4, so no speed, memory or quality claim here is first-hand.
- • I found no independent third-party benchmark that I could verify; all numbers are the author’s.
- • The blog post only shows a relative age, so “May 2026” is approximate.
- • The README does not expand labels like “0731” or “0813”; I treat them as checkpoint tags.
Conclusion
ds4 is the clearest example I have seen of the “one model, one engine” approach: a C codebase, its own GGUFs, native kernels for three GPU families and a tested agent and server on top. If you own a 96 to 128 GB Mac, a DGX Spark or a Strix Halo and you want DeepSeek V4 Flash running locally, it is the first thing I would try, with the caveats above.
If you don’t, or you need stability, a general runner or a hosted API is the better call. Whatever you pick, measure on your own prompts: every number here comes from the author.
Resources
- • GitHub: github.com/antirez/ds4
- • Model guide: docs/MODELS.md
- • Author’s post: antirez.com/news/165
- • Weights: huggingface.co/antirez/deepseek-v4-gguf
- • llama.cpp: github.com/ggml-org/llama.cpp
