Recursive's 40B Flint shines as an agentic model — unmerged mlx-vlm PR adds up to 30% speed
jeffclune · x · 2026-10-24
jtdavies took a deeper look at Recursive's 40B Flint model, focusing on third-party inference support he'd missed before: mlx-vlm was doing the model an injustice until he re-ran tests with unmerged PR #2569 (fused Metal kernels), a stark difference worth up to 30%.
- Long context: On Mac Studio M5 Ultra, 2,746 tok/s prompt processing at 256k (4-bit + PR #2569); generation 114 tok/s at 256k.
- Concurrent agents: With 64k context and prefix cache on, Flint 4-bit scales to 32 agents at 180 total tok/s, far ahead of Flash-Next FP4-Local (19) and Ornith-1.5-122B (8).
- Verdict: as an agentic model Flint is a beast, holding its own with the leaders. Local-deployment users should watch for the PR merge.
More from Infra
- Security teams increasingly run open models via Ollama for red teaming — mchiang0610 · 2026-10-24
- PCIe x1 pushed to the limit: concurrently serving 27B, 9B, 2B vision LLMs on Frankenstein hardware — FieldMouse-AI · 2026-10-24
- humans& makes NVFP4 decode 10% faster by using more bits, not fewer — gharik · 2026-10-24
- GEPA author's team open-sources a kernel boosting accuracy and speed at once — gharik · 2026-10-24
- Crowdsourced solvers make NVIDIA's Nemotron 3.5 Lightning 34.6% faster on Mac — gajesh · 2026-10-24
- Video DiT runs on 12GB VRAM at same speed via int8, model weighs only 9GB — cocktailpeanut · 2026-10-24