llama.cpp keeps closing the gap: months-long benchmarks show it now beats a custom vibe-coded engine
Danmoreng · reddit · 2026-11-16
Reddit user Danmoreng shares months of informal benchmarks on an RTX 5080 16GB laptop: months ago he easily beat llama.cpp with his own vibe-coded inference engine, but re-testing shows llama.cpp has caught up and surpassed it.
Methodology: fixed 16k prefill / 1k output on Wikipedia text, optimal settings per engine. On Gemma 4 12B (same checkpoint), vLLM leads prefill at 6,937 tok/s vs llama.cpp 4,754; on Gemma 4 26B A4B (different quants, not apples-to-apples), llama.cpp wins decode and D2 decode at 202.9 tok/s. Takeaway: llama.cpp ships steady performance gains over time, and the author prefers the stable engine over vibe-coded alternatives.
More from Infra
- KubeCon's big theme is inference: a full-stack breakdown from DRA GPU slicing to vLLM and llm-d — AccBalanced · 2026-11-16
- Is bad news for Jensen Huang also bad news for the AI industry? — TechCrunch AI · 2026-11-16
- Taiwan Prosecutes Nvidia Manager Over Chip Smuggling; Nvidia Disputes Charges — GarrisonLovely · 2026-11-16
- Decentralized HuggingFace rival Hippius Hub launches with 11,646 models indexed — markjeffrey · 2026-11-16
- Nvidia Down ~24% Drags S&P 500, but Rest of Index Has Fully Recovered — Gregory_C_Allen · 2026-11-16
- Google ships two new TPUs, Gemini Robotics 2.5 and more in a single week — haider1 · 2026-11-16