Qwen 27B Benchmark: Tools Lift Professional Finance Score to 98%
offgridai · reddit · 2026-08-25
A detailed benchmark of Qwen 3.8 27B on real estate and private equity tasks. Running on a 5090 via vLLM with specific quantization, the model scored 95.25% on a custom 600-question professional suite without tools.
Adding an MCP layer (finance calculators, capped web search) lifted the score to 98.05%. Further hinting the model to use specific professional sources like SEC Edgar nudged the score to 98.44%. The results demonstrate that while the base model is strong, tools and guided search significantly enhance professional capabilities.
More from coding & agent
- Dev pushes back on "AI coding means insecure apps": humans always shipped bugs — dreamwieber · 2026-10-15
- Eight non-negotiable principles for building AI that actually learns from experience — tokenbender · 2026-10-15
- Animated teardown of XBOW's Chrome full-chain exploit that won Google's first $250K VRP bounty — moyix · 2026-10-15
- pstack's durable core: 10-20 line skills that distill agent best practices into principles — HankYeomans · 2026-10-15
- 'Children of Elon' forum lets AI agents post and teach each other while humans watch — AiJohnAllen · 2026-10-15
- VoidZero open-sources Void: SDK, CLI and platform to deploy Vite apps on Cloudflare — cnakazawa · 2026-10-15