Ollama's prompt cache changes LLM outputs even at temperature 0 with fixed seed
Full-Travel857 · reddit · 2026-11-07
A developer running Polish grammar benchmarks noticed different answers at temperature 0 with a fixed seed depending on whether Ollama reused the prompt cache. He ran a controlled experiment — 3 models, 10 model-prompt pairs, 4 conditions (streaming on/off × model loaded/unloaded), 5 reps, 200 responses:
- Streaming made no difference; prompt cache state did
- Bielik-Minitron 7B answered a Polish historical-language question correctly 5/5 after unloading the model, incorrectly 5/5 when cached
His takeaway: benchmarking with fixed seeds may not measure what you think unless you control server state. He shipped AI Bóbr v0.8.1 with a controlled testing mode (unload + verify before each attempt, cache stats logged) and is rerunning his 11-model benchmark, asking if others have seen the same.
More from Infra
- Epoch AI: Huawei's largest China AI data center holds just 7% of Colossus 2's compute — Jsevillamol · 2026-11-07
- Musk says SpaceX will launch StarMind orbital supercomputers from Q3 2027, co-designed with NVIDIA — XFreeze · 2026-11-07
- Netflix builds ontology-driven observability: telemetry, incidents and services as one knowledge graph — JeremyCMorgan · 2026-11-07
- Google buys Crusoe's Wyoming data center site as hyperscaler takeover of neoclouds begins — SumitGup · 2026-11-07
- Oracle Down 66% From Peak, 301 Days After Michael Burry Went Short — kingslyj · 2026-11-07
- AI supplier multiples fall ~15% post-selloff, below analyst growth forecasts — BenBajarin · 2026-11-07