Defer: a fully deterministic LLM inference engine with bit-for-bit reproducible output
iyioioio · reddit · 2026-11-03
A developer released Defer, an experimental deterministic LLM inference engine (pure Rust, code mostly written by Claude Opus 5.5 via the Pi harness): same prompt, model and parameters always yield bit-identical output.
- Runs every op in fixed sequential order on CPU — no threads, no BLAS, no GPU kernel floating-point reordering, making output a pure function of inputs
- Slow and single-threaded: 17 tokens/s for SmolLM2-135M on a laptop
- Sessions need no KV cache storage: a 4KB write-ahead log of token IDs per 1,000 tokens (vs 46MB KV cache), with replay re-sampling every token to catch tampering or wrong models
- Enables zero-noise leave-one-out token attribution
The author calls it a fun experiment, not a llama.cpp/vLLM replacement.
More from coding & agent
- xAI launches page of 56 one-click Grok bot agents for everyday tasks — victor_explore · 2026-11-03
- Dev's side project ships: agents pull HTS codes and customs paperwork live — aarthir · 2026-11-03
- Matt Pocock: Widely panned Opus 4.7-5 was 'SOTA enough' for me on £180/mo CC Ultra — mattpocockuk · 2026-11-03
- I built a LinkedIn MCP because the most popular one can get your account banned — velvetotter8 · 2026-11-03
- Mistral Large 4 builds real-time 3D tracker for 19,000 satellites — MistralAI · 2026-11-03
- If an agent can edit its own spending cap, was it ever a cap? — quiet_flare · 2026-11-03