Per-sentence citations cost more than the prose: lessons from a grounded doc system
alatmaca · reddit · 2026-11-02
A team building source-grounded application documents (where a wrong figure sinks the application) details two architectures and four sticking points. Attempt 1 (schema-bound slots per sentence) was safe but unpersuasive. Today's approach: code fills tables; a writer model returns JSON with per-sentence footnotes and declared figures; a code guard verifies each figure against its own footnote; an editor model flags semantic faults; a jury model scores and a final model compares against sources. Sticking points: 48% of stored characters are prose vs 35% figure declarations and 17% footnotes, with the model footnoting 285/295 sentences; "only from the dossier" vs "reasonable inference" rules fight each other and dry out the text; the model's internal missing-list leaks into prose; and measurement is slow/noisy (25-45 min runs, jury scores 72 vs 76 on identical text, rule changes moving scores 66→72). The author suspects writing should not do verification's job, citing Anthropic's separate citation agent and Abridge's post-hoc unsupported-claim detection, and asks how others attribute figures and draw the inference line.
More from coding & agent
- 312 MCP tools, only 5% loaded: how modern clients tame context bloat with tool search — tristanbob · 2026-11-02
- Together launches post-training platform aligning RL training with serving — togethercompute · 2026-11-02
- Five controls that keep LangGraph agents alive in production: 15% of chats routed to humans — LangChain · 2026-11-02
- Building a Company OS on Claude Code: skills run in 21-85s at $0.08-0.32 per task — aakashgupta · 2026-11-02
- Building a local tool to give agents access to ReMarkable notes, skipping third-party MCPs — annetgriffin · 2026-11-02
- Developer shares hard-won lessons on publishing Android apps to Chinese app stores — dotey · 2026-11-02