Iterative sensitivity probing: how bitsandbytes2 finds each layer's compression limit
Tim_Dettmers · x · 2026-09-22
The core challenge of dynamic compression is estimating each layer's sensitivity, which shifts as other layers compress — a combinatorial problem. Iterative sensitivity probing exploits a simple intuition: the closer a model is to its instability edge, the easier it is to estimate multi-layer interactions from single-layer compression. Start uncompressed, measure sensitivities, identify sensitive layers, compress slightly while reallocating bits to sensitive layers, and iterate until the (bit, expert removal) allocation stops changing.
Related event: Dettmers Explains Iterative Sensitivity Probing and Dynamic REAP(2 posts)→
More from Infra
- H Company trains computer-use agents on SkyPilot: thousands of sub-second sandboxes — skypilot_org · 2026-09-23
- MiniMax H3 video gen runs locally on M5 Ultra: 768p in ~2m22s with optimizations — bakawolf123 · 2026-09-22
- Cisco: Agentic AI to Drive 9X Enterprise Traffic Growth by 2035 vs 2.5X Without — Beth_Kindig · 2026-09-22
- Cloudflare ships Vary support in Cache Rules to tame HTTP's 'ugliest' header — threepointone · 2026-09-22
- Fluidstack breaks ground on $4B Texas data center campus planned for up to 1.5 GW — MxMnr · 2026-09-22
- Right-size GenAI endpoints: SageMaker concurrency sweeps with Nemotron-3 Nano 30B — AWS ML Blog · 2026-09-22