A grammar-based approach stops local models from breaking JSON in production agents
paulqq · reddit · 2026-07-29
The post explains how to make small local models stop breaking JSON by using grammar constraints instead of hoping the model behaves.
Key points:
- Hosted APIs like OpenAI enforce structured output server-side, but local models do not.
- llama.cpp supports GBNF grammars, which can constrain token generation so malformed JSON is impossible.
- The author wrote a compiler that converts each tool schema into grammar rules.
- The grammar is narrowed each turn so the model only sees the 3–5 relevant tools instead of the full set of 50.
The article is presented as a deep dive into Eris, a local Rust agent that uses Markdown notes as memory and runs entirely on the user’s machine, with code and a public repo included.
More from coding & agent
- Reddit user compares fixed-price AI coding plans after months of real use — krisurbas · 2026-07-29
- Claude Opus 5 tops DeepSWE with a 74% score and a claimed 28% cost edge — daniel_mac8 · 2026-07-29
- A Claude-powered folder turns raw files into a self-updating second brain — wschroll · 2026-07-29
- Five days after Opus 5’s launch, users are already generating full games and websites — heypearlai · 2026-07-29
- Opus 5 is already producing full Minecraft builds with no outside assets — heypearlai · 2026-07-29
- Five days after Opus 5’s release, users are already showing full game demos — heypearlai · 2026-07-29