The hardest local LLM failures are the ones that look like success
FulcraDynamics · reddit · 2026-11-19
A Reddit thread on the sneakiest failure mode of local LLMs: a prompt that works five times in a row starts drifting at scale—missing fields, format drift, instructions followed in spirit but not in letter, with nothing crashing.
Key trouble spots: structured output and tool calls, where the real culprit may be chat template, sampler, or parsing mismatches rather than the model; stacked instructions, where longer rule lists get randomly dropped per run; quantization effects, where two quants with similar perplexity can behave differently on format/tool reliability.
The better question than "can this model do X?" is "how often does it do X across a hundred runs, and how would I notice when it doesn't?"—test on your own data and hardware.
More from coding & agent
- Tensor launches as an ADE for doing real work with AI, live on Mac, Windows and web — saheedniyi_02 · 2026-11-19
- Netlify launches agent runners; a developer shares his favorite workflow — DavidWells · 2026-11-19
- Fable 5.2 one-shots a full Blender classroom scene from a single prompt — WatercressBorn4400 · 2026-11-19
- CodeGraphy v2.2 fixes AST blind spots, hits 2.6x the file recall of naive BM25 — GeologistNo4811 · 2026-11-19
- Ultrafast mode review: great for math proofs, not recommended for coding — gnukeith · 2026-11-19
- Speed up CI for AI-written code: caching, faster runners, test sharding — rseroter · 2026-11-19