Dettmers Explains Iterative Sensitivity Probing and Dynamic REAP

Dettmers detailed how iterative sensitivity probing tackles per-layer compression estimation, and how Dynamic REAP keeps only about 25% of experts per layer while asynchronously swapping experts in from CPU RAM/NVMe every 256 tokens without hurting inference.

2026-09-22 ~ 2026-09-22 · 2 related posts

• Iterative sensitivity probing: how bitsandbytes2 finds each layer's compression limit — Tim_Dettmers · 2026-09-22
• Dynamic REAP swaps 4 experts every 256 tokens, keeping only 25% in GPU memory — Tim_Dettmers · 2026-09-22