Unsloth Q8 crushes default Q3 in a self-hosted test over 331 financial PDFs
No_Cartographer3953 · reddit · 2026-11-16
The poster benchmarked several small local models on a realistic privacy-sensitive task: 331 bank/credit-card/mortgage/brokerage/retirement PDF statements, requiring data extraction, accounting reconciliation, and an insightful frontend, using the PI harness.
Results:
- Qwen FN Q3: nice frontend but PDF extraction completely wrong, basic analytics.
- Orca nano-1: finished in 2 hours; decent for its speed/size, mostly correct data with clear errors, shallow frontend.
- Maya-S24: failed — stuck in PDF parsing/validation for 10+ hours before being abandoned.
- Qwen FN Unsloth Q8: 8 hours overnight, all data accurate, final frontend both good-looking and genuinely insightful.
The author argues this is the perfect local-model test: financial data you don't want in the cloud, nightmarish PDF layouts (duplicative summaries vs transactions), and real accounting logic (a credit in one account is a debit in another). Models chose different approaches (JSON vs sqlite) and mostly used pdfplumber. The takeaway: quantization level (Q8 vs Q3) massively affects complex agent tasks.
More from coding & agent
- Generic API-catalog MCP server vs one tool per job: a context-bloat experiment — MandeepKaur9407 · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- Effective HTML: 4.2k-star agent skills turn wireframes into self-contained interactive prototypes — tom_doerr · 2026-11-16
- skills-gate: one gate skill fronts 148 agent skills at just 389 tokens — Distinct-Pie2389 · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- shadcn boosts @uisdk: one-shot UI rebuild that's instantly E2E-testable by AI — shadcn · 2026-11-16