FinFIRST details: 123 expert tasks, 701 atomic criteria and 12,300 rubric points for financial search agents
FellMentKE · x · 2026-09-04
Author adds details on FinFIRST, the financial search agent benchmark Ant Group built with professional support from CICC's investment banking team. Version 1 includes 123 expert-authored tasks, 701 atomic criteria and 12,300 rubric points. The launch claim and evaluation framework answer different questions: one gives people a model to adapt, the other gives researchers a detailed way to examine where agents succeed, fail, or need closer review.
More from Models
- GPT-6 Astra and Claude Fable 5.1 write $10,000 research papers — experts reject all — sayashk · 2026-10-21
- GitHub Copilot's HydraFusion Orchestrates Multiple Models Per Task — WirelessLife · 2026-10-21
- OpenVLHarness turns any MLLM into a visual reasoner, +9.9 points across 22 datasets — yining_hong · 2026-10-21
- User claims Gemini analyzed his tennis match video and improved his game — zacharynado · 2026-10-21
- Opus 5.5 tops OpenRouter spending in 20 of 29 task categories — mmurph · 2026-10-21
- Meta team builds open-source 30B LLM from scratch in months, hails 'second era' — TimDarcet · 2026-10-21