MLX over Vulkan hits 124 tok/s on RTX 5060 Ti, beating llama.cpp CUDA by 25%
SumitGup · x · 2026-11-22
Joshua Warren shipped an Omarchy-mlx update where Apple's MLX framework, running over Vulkan on Linux, decodes Qwen3-4B 4-bit at 124 tok/s on an RTX 5060 Ti eGPU — beating llama.cpp CUDA's 101 tok/s on the same card, challenging the idea that CUDA is NVIDIA's moat.
Key technical detail:
- One change: replacing full GPU pipeline barriers between dependent compute shaders with narrow compute-to-compute barriers
- Median decode jumped from 99 to 124 tok/s (+25.8%) across five interleaved A/B pairs with bit-identical outputs
- Per-token latency dropped 2ms; speed from token 6 reached 131 tok/s (+27.4%)
- Setup: RTX 5060 Ti 16GB over USB4 attached to an M1 Max MacBook Pro, Omarchy Linux
Takeaway: framework-level scheduling optimization still has big headroom, and MLX-Linux-Vulkan now outruns CUDA on NVIDIA's own silicon.
More from Infra
- Intruder swapped Prime Intellect's hosted DeepSeek for an abliterated copy for 53 hours — mawcopolow · 2026-11-22
- Token economics model projects $1.14T global API spend by 2030, but only 1% of users pay beyond coding — AccBalanced · 2026-11-22
- Zero-degradation Strata tuning: fixing layer_split lifts decode ~10-15% and halves CPU expert misses — PathfinderTactician · 2026-11-22
- llama.cpp Vulkan SCALE fix PR #32735 delivers 2.3-3.2x speedup on Radeon R9700 — pmttyji · 2026-11-22
- $1400 decade-old V100 32GB runs Qwen 27B: Q4_K_M +48% gen speed, Q8_0 +16% prefill — PuzzleheadedArm3732 · 2026-11-22
- Inference Stack Launches One-Command Self-Hosted LLM Serving with vLLM, LiteLLM and llm-d — AccBalanced · 2026-11-22