Team builds custom inference engine in 4 days for $500 in tokens, beats SGLang by 15% with 6 prompts
IgorCarron · x · 2026-11-15
The waterloointern team built a custom inference engine from scratch (literally starting at git init) in 4 days with $500 in token costs, claiming a 15% performance win over SGLang.
The entire build took just 6 prompts — three of which were "carry on," "proceed," and "push to github." The authors note that building a bespoke inference engine per open-source model has become a clear industry trend, driven by a wave of papers earlier this year like hazy research's megakernels and MetaInfer. The core argument against traditional engines like vLLM/SGLang: their abstractions (Triton vs CUDA, Python, etc.) inherently add performance overhead.
More from coding & agent
- Dev builds Dark Souls-inspired browser game in 3 days with Claude, shares full workflow — Eric520CC · 2026-11-15
- Microsoft's 15-hour outage: why a senior engineer says AI won't replace programmers — amaarora · 2026-11-15
- Dynalist MCP Server lets Claude search, edit, and bulk-import Dynalist outlines — modelcontextprotocol · 2026-11-15
- PayPal MCP connector offers read-only AI access to transactions and invoices — modelcontextprotocol · 2026-11-15
- rea + mitmproxy: a 60-second workflow that makes Claude Code expose what your apps are phoning home — Kyrannio · 2026-11-15
- Are We in a Vibecoding Trough? Unreviewed PRs Blamed for Constantly Breaking Infra — nptacek · 2026-11-15