Patched Strata cuts DeepSeek V4.1 agent session startup 20× on a gaming GPU
HearMeOut-13 · reddit · 2026-11-02
Responding to Sam Altman's claim that home DRAM is "quite terrible" allocation, a developer shows an offline setup (RX 9070 XT 16GB, 64GB DDR4) running DeepSeek V4.1 Flash at 1M context via Strata, experts split across VRAM/RAM/SSD. The pain point: each new agent session re-read a 37K-token system prompt, waiting 100s for the first token. Two engine patches:
- KV splice: aligns new prompts against saved conversations, cuts stale lines from cached state as whole lines (recording a RoPE gap), reading only changed lines. Session startup drops from 100s to 5s (36,949 tokens reused, 513 read) — 20× faster.
- Resident-only routing: during prompt reads, tokens route only to experts already in VRAM/RAM (a 1.2K-token prompt previously streamed 58GB from SSD); fresh prompts drop from 22.8s/58GB to 8.2s/1.3GB.
Hard coding suite stays 12/13; HIP-only (gfx1201); splice is approximate with no formal eval; on by default, toggleable via env vars. Build commands and upstream issues included; author seeks CUDA test data for merging.
More from coding & agent
- Dev builds a MetaHuman alternative for three.js in a weekend with Claude — majidmanzarpour · 2026-11-02
- AI agent operates FileMaker from plain English, narrating every click in a cloned voice — petrusenko_max · 2026-11-02
- DeepSeek says it's betting on scaling across agent harnesses, pledging per-generation support — teortaxesTex · 2026-11-02
- What actually happens in the 3 seconds before your chatbot replies: tokens to RAG — Ok_Translator_227 · 2026-11-02
- Video-to-Game Workflow: Video Model Drafts the Gameplay, Claude Builds It in Three.js — OdinLovis · 2026-11-02
- "Someone must solve the cost of serving coworking agents" — hyhieu226 · 2026-11-02