The hardest local LLM failures are the ones that look like success

FulcraDynamics · reddit · 2026-11-19

A Reddit thread on the sneakiest failure mode of local LLMs: a prompt that works five times in a row starts drifting at scale—missing fields, format drift, instructions followed in spirit but not in letter, with nothing crashing.

Key trouble spots: structured output and tool calls, where the real culprit may be chat template, sampler, or parsing mismatches rather than the model; stacked instructions, where longer rule lists get randomly dropped per run; quantization effects, where two quants with similar perplexity can behave differently on format/tool reliability.

The better question than "can this model do X?" is "how often does it do X across a hundred runs, and how would I notice when it doesn't?"—test on your own data and hardware.

Original post →

More from coding & agent

coding & agent channel →