FreeToken benchmarks: a single RTX PRO 6000 runs a 753B model at 14.9 tok/s

jasonkneen · x · 2026-08-23

Developer jasonkneen replied with his PR adding Apple Silicon / Metal / MLX support to FreeToken. The quoted tweet shared benchmarks:

• A single RTX PRO 6000 runs the 753B GLM-5.2 at 14.9 tok/s
• An $1,000 8GB RTX 4060 laptop runs Qwen3.6-35B at 39.3 tok/s
• 2–4x faster than Ollama across consumer GPUs

FreeToken was open-sourced by UC Berkeley's Sky Lab; macOS was previously unsupported, which this PR fills in.

Related event: Berkeley and MIT Open-Source FreeToken, Enabling Consumer GPUs to Run Massive MoE Models Locally(11 posts)→

Original post →

More from Infra

• Underwater data centers? A new solution for AI infrastructure placement. — MickeySteamboat · 2026-08-23
• US compute dominance threatened by deindustrialization and energy stagnation — beffjezos · 2026-08-23
• Designing Backend for Agent Systems: Architecture and AWS Deployment — blaizedsouza · 2026-08-23
• Building Serverless AI Agents with Azure Functions — adnan_hashmi · 2026-08-23
• Rise of Private Compute Rights: Decentralized Garage Datacenters Trend — beffjezos · 2026-08-23
• GB300 memory bandwidth hits 7.4TB/s, driving global HBM shortage — JosephJacks_ · 2026-08-23

Infra channel →