Apple Silicon GEMM benchmarks: M6 does fp8 at ~2x fp16, M5 doesn't
Brilliant-Hall1387 · reddit · 2026-10-15
Precisit open-sourced apple-matrix-capabilities, a benchmark project measuring GEMM performance across matrix data formats (fp8/fp4/int8) on Apple Silicon GPUs.
- Key finding: M6's matrix unit supports fp8 at 2x fp16 speed; M5 does not
- GEMM rates matter most for AI prefill; the data guides format choice — fp8 on M6, int8 for 4-bit quant on M5
- Tested M5/M5 Pro/M5 Max/M6 so far; M5 Ultra and other variants still needed — community contributions welcome (Xcode 27 + Metal toolchain, calm hardware required)
- The tooling was built with AI; the author suggests contributors just have their AI agent run the tests and submit PRs
Results table and contribution guide are on GitHub.
More from Infra
- Selling B300 inference nets $40/hr vs $6/hr rental — but the math is disputed — TheZachMueller · 2026-10-15
- ASML CFO says US Match Act restricting DUV sales to China would backfire — pstAsiatech · 2026-10-15
- ASML sold out through 2028 but stock dips 2%; TL20 index up 57% YTD — TiernanRayTech · 2026-10-15
- Bloomberg breakdown: a 1GW Texas AI data center may need nearly $3B for overruns alone — rohanpaul_ai · 2026-10-15
- 7 System Design Algorithms to Know: Merkle Trees to CAP/PACELC — goyalshaliniuk · 2026-10-15
- MiniCPM-V 4.7 runs a 35B MoE multimodal model on phones with just 3GB memory — nevrekaraishwa2 · 2026-10-15