Constraining Agents to Write Docs with Schema

gimalay · reddit · 2026-07-13

While evaluating an agent-maintained markdown knowledge base, the author noticed that rules in prompts gradually fail, but mechanical validation does not. Relying on multi-round prompt iterations often leads to frontmatter drift, length overflow, excessive nesting, and unstable link rules. A validator triggered on every write, however, can directly pinpoint specific violations and suggest fixes.

The author extracted "structural validation" into a standalone tool called document-schema / schematter: it defines page shapes similarly to JSON Schema, covering frontmatter fields, required sections, heading formats, allowed heading depth, and token-based page/section budgets. Examples show how error outputs use schema descriptions as hints to help agents self-correct in a single turn.

The core value here is transforming context budgets and document shapes from prompt constraints into hard rules at write time. The author notes this proved more effective than three rounds of prompt tuning in their experiments, bringing runaway pages back within budget and preventing context bloat before retrieval. It can be integrated into CI/git hooks, agent write paths, and manual/batch validation workflows.

Original post →

Constraining Agents to Write Docs with Schema

More from coding & agent