15M-param decision model bake-off: linear recurrence ties attention on accuracy, wins 2.4x on latency
AlarmingTrouble5261 · reddit · 2026-10-15
Inspired by TypeSafe's Jev (a non-generative model that only outputs bounded decisions), the author trained two budget-matched 15.09M-param decision models end-to-end — a linear recurrent backbone (Gated Segmented State Space) vs standard Transformer attention — on identical tasks, with everything reproducible in open notebooks on a Colab T4.
Main results (V4, long-context retrieval-decision task, 2k–8k chars):
- Accuracy is noise: 26.8% vs 28.2% on test (chance = 25%); neither substrate learns the hard task at this scale, and the negative result is reported straight.
- Latency crossover is real: Transformer is faster below 1550 tokens; at T=8192 the linear model is 2.37x faster (108.6 vs 257.7 ms) — O(T) vs O(T²) showing up exactly where theory predicts.
- Training speed is the biggest gap: 6 finetune epochs took 30 min for the linear model vs 40–45 min per epoch for the Transformer at 2k–8k context, a 6–7x gap that worsens with scale.
Earlier rounds (V2/V3, shorter decision tasks) had the linear model winning outright on accuracy (50.4 vs 46.2, 66.2 vs 61.0). Takeaway: linear recurrence ties or wins on short-context decisions and dominates long-context cost, but 15M-param decision models are accuracy-starved regardless of substrate. A bidirectional V5 is in progress.
More from Infra
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- Jensen Huang details Vera Rubin agentic stack: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23