bitsandbytes2 slightly delayed: zero-config lazy compression plus dynamic expert swaps for near-infinite KV cache
Tim_Dettmers · x · 2026-09-22
bitsandbytes2 was set to release this week but is slightly delayed. It offers "lazy compression" — no user configuration, always a perfect memory-quality-speed trade-off, even as the KV cache grows. Combined with another technique releasing tomorrow, it enables virtually infinite KV cache. bnb2 targets small companies and individuals deploying large open-weight models on limited-memory hardware; private beta signups are open.
Related event: bitsandbytes2 Slightly Delayed, Promising Near-Infinite KV Cache(2 posts)→
More from Infra
- Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
- a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
- AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
- Tsinghua's BaGuaLu-2 tames MoE expert-load spikes at runtime, accepted to SC26 — jiqizhixin · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12