Running 2.4T Params on Mac Ultra: 1-bit Quantization Real-world Test
Ok_Technology_5962 · reddit · 2026-08-14
A developer successfully ran a massive 2.4-trillion-parameter model on a 512GB Mac Ultra using Unsloth's 1-bit quantization (iQ1S), consuming around 508GB of memory.
In medium reasoning mode, the quantized model achieved 50 tokens/s generation speed, dropping to 5 t/s at 50k tokens. The author conducted several generation tests:
- Game Code: Successfully generated a flight simulator with gravity and physics, requiring up to 96k tokens to fully output the code.
- SVG Generation: Showed impressive results in creating complex SVGs (e.g., a panda picnic, capybara in an onsen). The model autonomously used Python for noise and texture generation, maintaining high structural coherence.
Related event: Developers Test Run 2.4T Parameter Qwen Model on Consumer Hardware(3 posts)→
More from Infra
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Q4-quantized coder model on 8-core CPU builds interactive solar system demo in one shot — IntrepidMindExplorer · 2026-10-16
- TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode — zhyncs42 · 2026-10-16