EdgeRock-4B runs locally on MacBook: first token in 4.6s, ~9 tok/s on M5 Pro
testingcatalog · x · 2026-10-31
EdgeRock-4B can run locally on a laptop: on a MacBook Pro (M5 Pro, 28GB RAM) using Rock Matrix's PyTorch runtime on the Mac's GPU, it solved 17 × 23 step by step, started answering after 4.6 seconds, and generated about 9 tokens per second.
Per official benchmarks, it also hits 12,124 output tokens/s on a single GPU (32K in + 32K out) — 6.42× its base Qwen3.5-4B (1,889) and well above peers like Gemma 4 E4B (2,196) and MiniCPM5-2B (2,117).
Related event: Rock Matrix Open-Sources EdgeRock-4B with Constant-Memory KV Cache(3 posts)→
More from Infra
- The AI race is now an infrastructure race: Blackwell packs 208B transistors and 1.8TB/s NVLink — Olivier__OG · 2026-10-31
- Inference optimization race heats up: practitioners now want fast, cheap 'good enough' models, not the absolute best — bendee983 · 2026-10-31
- NVIDIA Releases Nemotron 3.5 Bolt E4B: 4GB On-Device Model with 80% Fewer Output Tokens — heatOverflower · 2026-10-31
- RTX 6000 Pro workstation GPU spotted at $13,999, down from ~$17K — HankYeomans · 2026-10-31
- Musk envisions lunar mass drivers launching compute to deep space for a petawatt-scale future — XFreeze · 2026-10-31
- DOE approves first commercial nuclear microreactor safety review; Radiant cleared for full-power testing — McDonaghMatthew · 2026-10-31