Why temperature 0 no longer fixes PDF extraction: Apple's 10-K off by 6 orders of magnitude
Zachly · x · 2026-11-18
Analytics engineering instructor Jai Bhagat shares lab notes on the long tail of PDF extraction (Snowflake Document AI experiment).
Key points:
- The dangerous result is the one that looks right: Apple's 2023 10-K prints net sales as 383,285; an LLM can return that exact number as valid JSON in USD and still be wrong — the "in millions" scale note is missed, and the true value is $383.285B, off by six orders of magnitude.
- Temperature 0 is no longer the fix: the bootcamp's takeaway is that setting temperature to 0 no longer guarantees reliable PDF extraction.
- The fix is an evidence contract: don't ship a single plausible row into production; validate against independent reference data (SEC accession numbers, period, unit) matched by filing identity — not by resemblance to the model answer.
He proposes a "60-second trust test": decide whether a row should ship before inspecting the source detail.
More from coding & agent
- Microsoft's Autopilot Agent Is Built on OpenClaw, Now in Preview for Frontier Customers — pswider · 2026-11-18
- SimplePost Gives ChatGPT and Claude a Publishing Button for 10 Social Networks — haltakov · 2026-11-18
- One set of Obsidian notes feeds Claude Code, Codex and Gemini alike — dSebastien · 2026-11-18
- Obsidian Starter Kit gives Claude Code, Codex and Gemini the same persistent context — dSebastien · 2026-11-18
- AI tools start from zero at every prompt — the missing piece is persistent context — dSebastien · 2026-11-18
- 27 vibe-coded Finder clones appeared in a month, most already dead — KaKi_87 · 2026-11-18