Unsloth Q8 crushes default Q3 in a self-hosted test over 331 financial PDFs

No_Cartographer3953 · reddit · 2026-11-16

The poster benchmarked several small local models on a realistic privacy-sensitive task: 331 bank/credit-card/mortgage/brokerage/retirement PDF statements, requiring data extraction, accounting reconciliation, and an insightful frontend, using the PI harness.

Results:

The author argues this is the perfect local-model test: financial data you don't want in the cloud, nightmarish PDF layouts (duplicative summaries vs transactions), and real accounting logic (a credit in one account is a debit in another). Models chose different approaches (JSON vs sqlite) and mostly used pdfplumber. The takeaway: quantization level (Q8 vs Q3) massively affects complex agent tasks.

Original post →

More from coding & agent

coding & agent channel →