Blogger builds asymmetric SLM pipeline inspired by DeepSeek V4.1 Flash, rebutting Yoav Goldberg on inference gains
adi_shik · x · 2026-09-25
In a debate with Yoav Goldberg, who argued that inference optimizations beyond standard tools are negligible once a model fits on a single GPU, adishik published a blog post showing significant gains. The post, Designing an Asymmetric SLM Pipeline, borrows three ideas from DeepSeek V4.1 Flash — encoder/decoder separation, asymmetric compute allocation between reading and generating, and Engram-style memory lookup — and applies them to a medical-record use case where a small model must read long documents, extract relevant info, and draw conclusions. Without pre-training compute, the author composes existing models rather than modifying architectures.
More from Infra
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Nvidia's supply commitments jump from $119B to $279B in one quarter, locking up 2027-28 memory — tengyanAI · 2026-11-11
- Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight — oguzeldereli · 2026-11-11