Jerry Liu names "Just-in-Time OCR": the two-pass document pattern agent harnesses converge on
solyarisoftware · x · 2026-09-15
LlamaIndex founder Jerry Liu coins "Just-in-Time OCR" for a pattern now common across agent harnesses (Cowork, Codex, Grok) handling user-uploaded document rooms:
- Fast first pass: a free/OSS parser (pypdf, or LiteParse — Rust-based, 50+ formats) does layout-aware parsing over all pages — spatial text, bounding boxes, headings, tables, plus per-page complexity flags. LlamaIndex reports a full data room parsed in 32 seconds.
- Just-in-time second pass: a dedicated OCR tool (LlamaParse) transcribes only the pages the agent actually reads, returning cell-level tables, bounding boxes, and confidence scores.
- Works well for small-to-medium batches (10–100 docs), trading cost and latency for accuracy; not a substitute for large-scale offline indexing.
- He argues the "default" frontier-harness approach (pypdf + native VLM) is both expensive and inaccurate, and that pypdf/pdftotext can't even do the layout-aware first pass well.
- Most pipelines parse every page up front — slow, costly, and most pages never get read.
Related event: LlamaIndex Proposes Just-in-Time OCR for Agent Document Processing(2 posts)→
More from coding & agent
- The 5 tool functions every coding agent needs, from file access to search — Al_Grigor · 2026-09-15
- I Banned My Agents From Writing Tests for Their Own Code After Two Months — it_beaver · 2026-09-15
- Dev reworks Chain Lightning skill with Astra: animation-aligned damage and better lighting — Dimillian · 2026-09-15
- FlockMTL DuckDB extension brings LLM and RAG functions directly into SQL — _reachsumit · 2026-09-15
- Feishu and Doubao launch team AI agent 'Doubao Work Partner', China's first Claude Tags-style product — APPSO · 2026-09-15
- Question's Gambit Boosts Agentic Search Accuracy From 83.1% to 90.5% on BrowseComp-Plus — _reachsumit · 2026-09-15