DeepSeek V4.1 Flash ships 512K context (1M optional) with on-demand KV cache allocation
gnukeith · x · 2026-11-08
Developer gnukeith describes keeping his custom inference engine slim—updating it only when needed or when switching models, and skipping hyper-niche scenarios.
Quoted context adds specifics: 3GB is hard-reserved for the system to avoid crashes under load; DeepSeek V4.1 Flash defaults to 512K context and can be set to 1M in the harness, which the server honors; KV cache is allocated per instance on demand rather than upfront, with current max capacity set at 7 instances.
More from Infra
- Running a local model on a Surface laptop to orchestrate Codex and Claude agents — draginol · 2026-11-08
- Ben Bajarin: Compute analysts don't grasp enterprise software, let alone enterprise AI workloads — BenBajarin · 2026-11-08
- SimCity veteran: data centers should be every city's dream, so why the backlash? — signulll · 2026-11-08
- SF Systems Reading Group on 11/10: multi-silicon systems and chip design for AI workloads — _shreya_s · 2026-11-08
- Underdog's Woof voice model runs offline in a browser tab at just 31.7MB — dr_cintas · 2026-11-08
- RTX 5060 Ti jumps from £400 to £650: local AI users debate GPU upgrades — OllieLearnsCode · 2026-11-08