Iterative sensitivity probing: how bitsandbytes2 finds each layer's compression limit
Tim_Dettmers · x · 2026-09-22
The core challenge of dynamic compression is estimating each layer's sensitivity, which shifts as other layers compress — a combinatorial problem. Iterative sensitivity probing exploits a simple intuition: the closer a model is to its instability edge, the easier it is to estimate multi-layer interactions from single-layer compression. Start uncompressed, measure sensitivities, identify sensitive layers, compress slightly while reallocating bits to sensitive layers, and iterate until the (bit, expert removal) allocation stops changing.
Related event: Dettmers Explains Iterative Sensitivity Probing and Dynamic REAP(2 posts)→
More from Infra
- Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
- a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
- AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
- Tsinghua's BaGuaLu-2 tames MoE expert-load spikes at runtime, accepted to SC26 — jiqizhixin · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12