Local LLM lesson: slower tokens finish agent tasks faster than aggressive speed optimizations
AssiduousLayabout · reddit · 2026-11-05
Running coding models locally (Qwen 3.8 27B, Flash-Next) on a 5090 with 64GB RAM, the author found that aggressive quantization plus experimental speed optimizations (IQ3XXS) produced tokens fast but the agent overthought and self-doubted on real tasks. Switching to IQ3S without the optimizations yielded fewer but more meaningful tokens, less spinning, faster real-world completion, and slower context burn. They're now comparing DeepSeek 4.1-Flash IQ1 Coder — much slower tok/sec, but successful-tasks-per-hour is still TBD.
More from coding & agent
- Dev finds pi coding tool defaults to Opus 4.8 on a fresh install — lucasmeijer · 2026-11-05
- Kylon puts whole teams and agents in shared rooms, with a Neon branch per app draft — jefrankle · 2026-11-05
- "Evals > vibes": agent workflow debates are just guessing without eval stats — annetgriffin · 2026-11-05
- OpenAI hackathon winner joins YC, launches Raven, an agent-first phone — ycombinator · 2026-11-05
- Open-source Cicada brings real-time translation and AI meeting notes to any Mac call, free 50h/month — dotey · 2026-11-05
- IBM launches AI coding assistant named "Bob", devs can't stop laughing — zeeg · 2026-11-05