Blogger builds asymmetric SLM pipeline inspired by DeepSeek V4.1 Flash, rebutting Yoav Goldberg on inference gains
adi_shik · x · 2026-09-25
In a debate with Yoav Goldberg, who argued that inference optimizations beyond standard tools are negligible once a model fits on a single GPU, adishik published a blog post showing significant gains. The post, Designing an Asymmetric SLM Pipeline, borrows three ideas from DeepSeek V4.1 Flash — encoder/decoder separation, asymmetric compute allocation between reading and generating, and Engram-style memory lookup — and applies them to a medical-record use case where a small model must read long documents, extract relevant info, and draw conclusions. Without pre-training compute, the author composes existing models rather than modifying architectures.
More from Infra
- Epoch AI: cost of SOTA AI performance falls 66%/quarter, slowing to 32% after two years — FinanceYF5 · 2026-09-26
- AI costs fall ~47% per quarter at constant performance, outpacing every past tech — FinanceYF5 · 2026-09-26
- Comparing Ghent and imec silicon photonics PDs: 320 Gb/s links but 5 dB grating coupler loss — jwt0625 · 2026-09-26
- House votes 417-3 on bill making data centers pay added grid costs — VraserX · 2026-09-26
- Railway launches free no-account VMs (59 min) and OpenCode cloud agents — jasonkneen · 2026-09-26
- DistribAI v2 pools free Colab/Kaggle GPUs into ~2x RTX 5090 training power — Enderchef · 2026-09-26