Dense 32B open model runs 239K tokens in 16GB VRAM with only 18 of 72 layers keeping KV cache
solyarisoftware · x · 2026-10-19
Blockway released Agens Volundr 32B v1, an Apache 2.0 open model (with an MIT-licensed Agens Engine and official GGUFs) that runs a 239K-token conversation within a 16GB GPU-memory budget.
The key architectural trick: of 72 layers, 54 keep a fixed-size recurrent state instead of a growing KV cache, with only 18 layers maintaining the traditional growing cache — combined with 2.82-bit quantization for a 262K configured context.
Under a hard 16 GiB GPU cap and a 219K-token document, reported speeds at T=0.6:
- Chat: 57 tps
- Code: 79 tps
- Thinking: 71 tps
- Tool calls: 67 tps
Benchmark scores (LiveCodeBench v6 67.9, AIME 2025 80.0, etc.) compare favorably with Qwen3.8-27B.
Related event: Blockway Open-Sources Agens Volundr 32B Model and Agens Engine(2 posts)→
More from Infra
- Alibaba Cloud shows Omarchy running on Wuying AI cloud PCs for enterprise agents — AIFlow_ML · 2026-10-19
- NVIDIA paper shows cross-model KV cache transfer skipping prefill, 2.7–25x faster than recomputing — solyarisoftware · 2026-10-19
- pi-durable v1.3.0 adds SQLite table prefixes, fixes 140 MB memory spike on DO — solyarisoftware · 2026-10-19
- Open-source SwiftUI dashboard brings a Docker Desktop-like UI to Apple's native container runtime — AIFlow_ML · 2026-10-19
- Strata runs on Windows with dual AMD GPUs despite no official support: ~1350pp on a 9070XT — Ok-Importance-3529 · 2026-10-19
- A frozen 16MB vocabulary replaces learned embeddings in small and ternary LMs — Sai Kiran Bathula · 2026-10-19