Flint int4 hits 122.6 tok/s on Mac mini matching Qwen3.6-35B quality at 3.5x speed
_rockt · x · 2026-10-22
A user benchmarked Flint (int4) against his daily-driver Qwen3.6-35B-A3B (4bit) on a Mac mini and called it a winner. The shared chart shows Flint decoding at 122.6 tok/s at 1K context vs 91.5 for Qwen, staying 25-34% faster up to 20K (90.9 vs 73.9). On a 52-task local quality screen, Flint scored 50/52 (math 8/8, median 2.8s/task), matching Qwen3.6 with thinking on (50/52, 10.1s/task) at roughly 3.5x the speed, while thinking-off Qwen dropped to 46/52 with math collapsing to 2/8. You.com founder Richard Socher replied inviting feedback, suggesting this is his team's new inference stack.
More from Infra
- Even an RTX PRO 6000 can't replace cloud LLMs, argues Reddit work-usage test — C1rc1es · 2026-10-22
- NVIDIA unveils OpenShell reference architecture for AI agent safety — VectorWireAI · 2026-10-22
- llama.cpp PR shows R9700 beating strata decode performance — keyboardhack · 2026-10-22
- EU should stop worrying about models and fund GPU clusters, grids, nuclear — AI builder — DionysianAgent · 2026-10-22
- Oct 21 attacks on Railway and LiteLLM show AI defense now needs agentic ops — alvelda · 2026-10-22
- Smaller Quants May Be Slower in Practice: Longer Thinking Cancels Speed Gains — Banana-Negra · 2026-10-22