Foundry Local beats llama.cpp by 35-135% on long-context decoding for on-device agentic coding
pmttyji · reddit · 2026-11-06
A Reddit user benchmarks Microsoft's Foundry Local for on-device agentic coding on an RTX 5090 and DGX Spark, reporting 35-135% decode/tg improvements over llama.cpp. The chart shows generation speed gaps widening with context: 1.77x at 4K tokens (265.82 vs 150.60 tok/s), 1.94x at 131K, and 1.95x at 260K tokens. The poster asks whether these optimizations could be ported into llama.cpp.
More from coding & agent
- 7 techniques for building cost-efficient AI pipelines, from model routing to cost-per-task — goyalshaliniuk · 2026-11-06
- Claude Code leaked users' emails via User-Agent header during web searches — NousResearch · 2026-11-06
- Harness-S rescues failed RL rollouts with sibling harnesses for generalizable agents — SJUT1 · 2026-11-06
- First personal agent installed, running on Gemini Flash 3.8 with end-to-end encryption — allenainie · 2026-11-06
- Sprite Fusion Animates Pixel Art From a Prompt, Directly Via Claude and Codex — HugoDuprez · 2026-11-06
- Nous Research's Hermes Agent v1.0.0 ships with 21 built-in tools, mobile and web apps — alexcovo_eth · 2026-11-06