124B model in 22GB: Cactus Quants hits 1.418 bits/weight, 47 tok/s on an M5 Max
pbaylies · x · 2026-11-13
Cactus Compute compressed Nemotron-3.5-SuperVL-120B to 1.418 effective bits per weight using Cactus Quants, the algorithm behind their foundation models. They note "1-bit" labels often describe weight codes alone, while their figure covers the entire file.
- The full 124.45B-parameter checkpoint (vision tower, projector, MTP head included) fits in 22.06 GB
- On a 36GB M5 Max MacBook Pro: 47 tokens/sec median warmed decode with verified MTP, 21.0 GiB peak memory
- Mean relative score reduction across 18 benchmarks is 3.6%
- Clear weakness on long agentic coding runs: BF16 solves 24 tasks vs 7 for CQ
More from Infra
- Notion indexes 10B+ docs on turbopuffer, cuts search costs 80% — DevinLewtan · 2026-11-13
- Even cutting all China GPU purchases leaves massive unmet compute demand — peterjliu · 2026-11-13
- Snowflake and Databricks are converging into the same platform, feature by feature — Zachly · 2026-11-13
- Inference chip startup Fractile raises $720M Series C six months after its $220M Series B — AjdDavison · 2026-11-13
- Another Maine town extends data center moratorium as local opposition spreads to 12+ communities — jonippolito · 2026-11-13
- Together launches Post-Training service: bring your own RL loop, they handle the GPU infra — togethercompute · 2026-11-13