Solo devs are outperforming vLLM and SGLang by reworking KV cache and MoE routing themselves
max_paperclips · x · 2026-11-13
A viral call-to-action in the open-source community: a handful of people independently took on open-source inference engineering and, through research, hillclimbing and mad-scientist experiments, drastically improved KV cache quality, MoE routing and performance — to the degree that several one-person teams now outperform larger projects like vLLM, llama.cpp and SGLang.
One example: Sekai-1, a 100M-parameter model scoring #2 in its weight class, was trained by a single person on one 16GB RTX 5070 Ti. The author urges others to join: read The Annotated Transformer over a weekend, learn attention mechanism differences, and start competing.
More from Infra
- Dukoi Turns Your Old Android Phone Into a Free App Server in 2 Minutes — Scobleizer · 2026-11-13
- DeepSeek Has ~20,000 H-Class GPUs and Wants More: 'Cash Into Nvidia Cards Beats Bank' — teortaxesTex · 2026-11-13
- Over 1,600 comments on internal Nvidia post urging board to bring back Jensen Huang — rohanpaul_ai · 2026-11-13
- Texas A&M Builds One of the Strongest US Academic Supercomputers, Hiring 10 — yuz9yuz · 2026-11-13
- Tencent Reportedly DeepSeek's Biggest Backer as China Compute Centralizes — teortaxesTex · 2026-11-13
- Data Architecture 101: one infographic covering terms, models and patterns — nevrekaraishwa2 · 2026-11-13