Jerry Liu names "Just-in-Time OCR": the two-pass document pattern agent harnesses converge on
solyarisoftware · x · 2026-09-15
LlamaIndex founder Jerry Liu coins "Just-in-Time OCR" for a pattern now common across agent harnesses (Cowork, Codex, Grok) handling user-uploaded document rooms:
- Fast first pass: a free/OSS parser (pypdf, or LiteParse — Rust-based, 50+ formats) does layout-aware parsing over all pages — spatial text, bounding boxes, headings, tables, plus per-page complexity flags. LlamaIndex reports a full data room parsed in 32 seconds.
- Just-in-time second pass: a dedicated OCR tool (LlamaParse) transcribes only the pages the agent actually reads, returning cell-level tables, bounding boxes, and confidence scores.
- Works well for small-to-medium batches (10–100 docs), trading cost and latency for accuracy; not a substitute for large-scale offline indexing.
- He argues the "default" frontier-harness approach (pypdf + native VLM) is both expensive and inaccurate, and that pypdf/pdftotext can't even do the layout-aware first pass well.
- Most pipelines parse every page up front — slow, costly, and most pages never get read.
Related event: LlamaIndex Proposes Just-in-Time OCR for Agent Document Processing(2 posts)→
More from coding & agent
- General Reasoning explains how to train agent swarms with RL and latency pressure — rosstaylor90 · 2026-11-02
- "Coding isn't the bottleneck anymore — usage windows are," says founder Kyle Gawley — kylegawley · 2026-11-02
- Demo passed ≠ production-ready: 5 questions to answer before letting agents run your business — _kellhus · 2026-11-02
- An AI agent passed 100 test chats—still not ready to run the business — djgoel123 · 2026-11-02
- Agent given total creative freedom produces a 1:44 code-drawn animated short film — ivegotnoidea1 · 2026-11-02
- Meme: Wake Up, Use Claude Code, Repeat All Day — tekbog · 2026-11-02