Launch HN: Magnitude (YC S25) ships self-optimizing inference engine, up to 2x faster than llama.cpp
anerli · hn · 2026-10-01
Magnitude (YC S25), from two engineers who previously built an open-source browser agent with 4k+ GitHub stars, launches a self-optimizing inference engine for local agents, claiming up to 2x speedups over llama.cpp on any Mac/Linux/Windows hardware.
- The gap: existing engines trade off—vLLM/SGLang optimize batched datacenter inference at the cost of single-session speed; llama.cpp/Ollama prioritize compatibility over hardware-specific performance; specialized engines lack completeness. None is designed for agent workloads (long, concurrent sessions that must coexist with normal desktop use).
- Approach: kernels are written with tunable parameters and compiled/tuned on the user's actual device, combining broad compatibility with hardware-specific performance ceilings; efficient kernels target only the most popular open-weight model families; dynamic memory allocation reserves only weight memory up front and grows/frees the heap as agent sessions start and end.
More from coding & agent
- shadcn boosts @uisdk: one-shot UI rebuild that's instantly E2E-testable by AI — shadcn · 2026-11-16
- AI agent deletes 43,599 lines of OpenClaw test code in one PR — vincent_koc · 2026-11-16
- JPMorgan shows Jarvis, its internal agent assembly line, at Interrupt NYC — LangChain · 2026-11-16
- Dev clarifies: Apple doesn't ban feature flags, only fooling App Review — talkaboutdesign · 2026-11-16
- Antigravity Matches Claude Cowork in Tests, While Gemini Tasks' Purpose Remains Unclear — sebajun2 · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16