Mistral Large 4 (1T params) runs locally on two M3 Ultras at 20 tok/s via MLX-VLM
andrejusb · x · 2026-11-03
Prince Canuma demoed running Mistral Large 4 (1T parameters, A52B active) locally on two M3 Ultra Macs using MLX-VLM distributed inference, hitting 20 tok/s with a batch of 3. The support was just merged to main, with a new release and updates coming the next day — another step for trillion-parameter models on consumer hardware.
More from Infra
- Microsoft Cloud hits $63B quarter at 30% growth as AI buildout keeps scaling — thedealdirector · 2026-11-03
- Open-source sparse attention: Qwen 27B with no accuracy loss, 10M-token context — Immediate_Sky_6566 · 2026-11-03
- A year of renting an H100 costs $37.6K as warnings grow about tightening compute supply — teortaxesTex · 2026-11-03
- AEHR jumps 16% and LSCC 9% after hours as both chip stocks report earnings — BenBajarin · 2026-11-03
- RidgeFill: training-free prefill skipping cuts TTFT 1.6x, 64K request 82.6s to 47.5s — Public_Umpire_1099 · 2026-11-03
- Stanford physicist argues space compute cooling is feasible with a black-sphere thought experiment — shaunmmaguire · 2026-11-03