Lessons from a cited paper-writing LangGraph agent: token blowups, fake sources, and four fixes
Altruistic-Video-849 · reddit · 2026-09-20
What broke
The author built a seven-step LangGraph workflow that writes cited research papers, wired to Groq, OpenAlex and Crossref. Fake-data tests passed; real runs surfaced four issues:
- Token blowout: a thesis prompt hit 8,779 tokens vs an 8,000/min cap. Fix: cap sources at 15, trim abstracts, show the critic only the body plus cited sources.
- Fake "verified" sources: 42 "verified" sources included an unrelated edge-computing paper. Fix: rank by relevance first, then verify in rank order.
- Length drift: first draft was 2,086 words vs a 1,500 limit — the count included a 369-word reference list.
- Unsupported numbers: the critic caught a "12% fuel reduction" no source backed. Fix: the writer may not state numbers absent from abstracts.
Remaining gaps
Citations are checked in code against Crossref, and a human approves topic and thesis before drafting. But the author hasn't measured paper quality or per-sentence faithfulness to sources — currently judged only by an LLM reviewer, which isn't proof. He's considering sentence-level entailment checks and asks the community how to verify claim-source consistency without just asking another LLM.
More from coding & agent
- Claude Skill turns an image's buildings into resizable Three.js models — jasonkneen · 2026-09-20
- Cursor Pro users report credits draining fast and degraded Auto mode, weighing $25/month alternatives — ReporterUnusual9295 · 2026-09-20
- Fuzzing vs property-based testing: the terms are splitting — exploration vs assertion — blaizedsouza · 2026-09-20
- Software factory take: LLMs may generate, but never specify or verify — blaizedsouza · 2026-09-20
- Jev as an NSFW prompt filter: 93% on CSAM evals, sub-cent cost, and where thresholds bite — Murky_Ad8671 · 2026-09-20
- "A heuristic cannot certify a heuristic": why software-factory orchestration thinking is wrong — blaizedsouza · 2026-09-20