Strata on a 4090 48GB: conversation parking buys 35x, n-gram table in RAM just 1%
Shot-Ad-4147 · reddit · 2026-10-04
A detailed single-machine benchmark of Strata 0.1.38 on i9-13900K + RTX 4090 48GB + 128GB DDR5, using a real 91,836-token prompt.
Four-arm test of n-gram table placement
- Keeping the full 28.8 GB table in RAM (+28.4 GiB) buys only +0.65% warm prefill / +1.2% decode — the SSD-resident table isn't a bottleneck here.
- --ple-io mmap is worse: first long prompt takes 72.8s vs 19.5s (3.7x slower) and silently disables the row cache.
- The default 90 MB row cache already absorbs 92% of reusable traffic (3,114 MB → 242 MB on second pass).
Methodology lesson: the author's earlier +4.7% claim came from a wrong baseline — session-to-session spread on identical config was 11.5% (149.1 → 166.3 tok/s). A/B runs must happen back-to-back in the same session.
Biggest win — conversation parking (35x): with --conversation-cache-mib 8192 and 4 slots, returning to a 91,836-token conversation drops from 19.5s to 537 ms for just 3.1 GB of RAM.
Other wins: --prefill auto:32768 gives +9.7% prefill / +13.6% decode at 78.7K context; --calibrate found another +3.2% by changing CPU workers 23 → 12. Baseline throughput: 128-151 tok/s decode, 4,249-5,154 tok/s prefill.
More from Infra
- Ex-Waymo exec Abe Murray: CUDA and ISAs won't stay moats, huge chip innovation incoming — IgorCarron · 2026-11-20
- MIT and NVIDIA's QDA uses agents to auto-design LLM quantization recipes, beating top public ones by up to 2.2 points — omarsar0 · 2026-11-20
- Caixin: US remote-access cutoff and Nvidia CEO exit leave China's tech giants halting overseas compute deals — Gregory_C_Allen · 2026-11-20
- QuixiInfer: Rust Engine Runs 125B MoE Model on a Single RTX 5090 — QuixiAI · 2026-11-20
- QuixiInfer: from-scratch Rust engine runs DeepSeek with 256k context on a single RTX 5090 at ~30 tok/s — QuixiAI · 2026-11-20
- AMD's Lisa Su: data center to hit ~80% of revenue by 2027, sees $220B server CPU market — AccBalanced · 2026-11-20