Stripping antirez's ds4 to 45k lines makes Qwen3.8 Flash Next ~10% faster, bit-exact
Chida82 · reddit · 2026-10-05
Developer Chida82 forked antirez's ds4 (DwarfStar) inference engine, keeping only Qwen3.8 Flash Next and the Metal backend, cutting ds4.c from 85k to 45k lines so the code tree fits in one context window.
Key results:
- Rationale: in the coding-agent era, a patch's real cost includes how many tokens the agent must read; removing noise makes optimization cheaper and safer
- Q2: decode +9–13%, MTP from 75.8 to 86.7 tok/s; Q4: prefill +2–11%, MTP 77.8 → 85.9 tok/s
- Output is bit-exact vs stock ds4 via GGUF/greedy parity checks and interleaved A/B benchmarks; no KV cache quantization or approximate kernels
- Triaged upstream PRs: adopted 20 commits, dropped 30
- Added SSD streaming for expert weights: 27 tok/s (Q2) on a simulated 48GB machine, 35 tok/s with MTP
- Keeps syncing with upstream via git merge + rerere; StarForge repo is model/backend-agnostic
A showcase of the new engineering paradigm: slimming codebases for agent maintainability.
More from coding & agent
- TanStack AI adds experimental text-to-GLB 3D model and prompt-to-motion animation generation — bennash · 2026-11-20
- Dev builds typing survival game with troops and skill trees in ~24 hours using agents and kaigen3d — hullabaloo22 · 2026-11-20
- Harmonic's Aristotle writes proven-correct fusion plasma controllers in C and Verilog in 4 hours — satnam6502 · 2026-11-20
- R3 Code: A Native Rust Mac Client That Runs All Your Coding Agents — jasonkneen · 2026-11-20
- zlib: 2.4K-star open-source CLI turns Z-Library into a skill for AI agents — udmrzn · 2026-11-20
- Andriy Burkov's True Positive Weekly #186: what to know when agents do the work — burkov · 2026-11-20