Long-form LLM system with per-sentence citations: metadata hit 52% of stored characters
alatmaca · reddit · 2026-11-02
A developer building a long-form LLM generation system (financial applications, where a wrong figure can sink the deal) asks whether their architecture is fundamentally wrong.
Two attempts:
- Strict templates: every sentence assembled from predefined prose + fact references. Figures safe, writing mechanical and unconvincing.
- Free writing + structured citations (current): Python extracts facts into tables; the LLM writes narrative but returns JSON with sentence-level footnotes; a validator checks figures against declared sources; separate agents review and repair.
Problems: in a 295-sentence test, 285 had footnotes (98 with no figures); internal metadata was 52% of stored characters; the editor rejects reasonable inferences ("exporting since 2019" rejected because continuous activity isn't explicitly documented); the model leaks its missing-info lists into the output; runs take 25–45 minutes with an inconsistent LLM jury.
Under consideration: separate writing from verification — plain text first, then extract atomic claims, link evidence, deterministically verify numbers, and classify direct facts / derived facts / reasonable inferences / unsupported claims in an independent evidence ledger. The author asks for real production experience on citation timing, numerical guarantees, and reliable long-form evaluation.
More from coding & agent
- Install a Gmail connector on ChatGPT to avoid output-misreading mistakes — jxnlco · 2026-11-02
- DeepSeek Harness Super-agent design: the bottleneck isn't smarter models, it's your attention — sujingshen · 2026-11-02
- tito: the hidden tokenizer bugs breaking RL training, and a Lean-verified fix — VoidAsuka · 2026-11-02
- Build an Agentic Knowledge Graph from Scratch — No Graph DB, No Extraction Model — adnan_hashmi · 2026-11-02
- One Vault, Two Sides: An Obsidian + Claude Agent Workflow for Personal Knowledge — adnan_hashmi · 2026-11-02
- Vercel engineer open-sources Security Audit skill you trigger via @bot reply — brandon_galang · 2026-11-02