LlamaExtract Agentic Plus still leads benchmark, but overfitting or scoring errors possible
VikParuchuri · x · 2026-08-15
VikParuchuri notes that LlamaExtract Agentic Plus still leads the benchmark, but possible reasons include its strength on this document mix, benchmark overfitting (vendors often lead their own benchmarks), or significant scoring errors. He previously mentioned that open-source model lift scored higher than API (77% vs 65%), suggesting scoring issues.
Related event: LlamaIndex Benchmark Scoring Bug Fixed, Datalab Jumps from 65% to 93.6%(6 posts)→
More from Models
- 200 chess games suggest in-context learning doesn't work yet, experimenter says — akbirthko · 2026-10-16
- OpenAI's internal model shows real math "research taste," experts say its proofs go beyond prior literature — cong_ml · 2026-10-16
- User catches Opus spinning up throwaway zz- debug scripts, echoing the METR OAI-HuggingFace incident — holdenmatt · 2026-10-16
- OpenAI drops 722 math papers tied to unreleased model, mathematicians demand receipts — thursdai_pod · 2026-10-16
- Fable 5/5.1 models show a striking drive toward record keeping, observers report — repligate · 2026-10-16
- Google's Gemini 4 Argon performs noticeably worse in AI Mode than in AI Studio — gaganghotra_ · 2026-10-16