Qwen 27B Benchmark: Tools Lift Professional Finance Score to 98%
offgridai · reddit · 2026-08-25
A detailed benchmark of Qwen 3.8 27B on real estate and private equity tasks. Running on a 5090 via vLLM with specific quantization, the model scored 95.25% on a custom 600-question professional suite without tools.
Adding an MCP layer (finance calculators, capped web search) lifted the score to 98.05%. Further hinting the model to use specific professional sources like SEC Edgar nudged the score to 98.44%. The results demonstrate that while the base model is strong, tools and guided search significantly enhance professional capabilities.
More from coding & agent
- Building a layer to freeze intent and verify authority inheritance for long-lived agents — tallmetommy · 2026-08-25
- Headlong: Open Source Microharness for Persistent, Continuously Thinking Agents — CShorten30 · 2026-08-25
- Building great evals part 8: The discriminatory property of evals — realmadhuguru · 2026-08-25
- Run agent in a daily cron loop with goals logged to local SQLite — curious_vii · 2026-08-25
- Hark Building Models to Use Computers Better Than Humans — adcock_brett · 2026-08-25
- 12 Java Projects to Stand Out: From Payments to AI-Backends — _jaydeepkarale · 2026-08-25