Gemini 4 Carbon self-calibrated 4-bit quantization plus suffix-tree speculative decoding
maksym_andr · x · 2026-11-17
New detail from InferenceBench: Gemini 4 Carbon calibrated its own 4-bit quantization, picking settings that keep vLLM's fastest kernel usable, then layered suffix-tree speculative decoding on top—landing #3 on the leaderboard at 10.48x aggregate.
Related event: Gemini 4 Carbon Breaks Into InferenceBench Top Three(2 posts)→
More from Infra
- Baseten hits ~2,500 tok/s on DeepSeek v4.1 Flash with Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-17
- Microsoft outage hits airlines and banks, sparking global cyber incident fears — mallow610 · 2026-11-17
- Texas data center freeze faces first test: Core Scientific's $1.3B AMD build — edgarpavlovsky · 2026-11-17
- vLLM claims 90x throughput on Rubin as analysts ignore software-driven gains — bookwormengr · 2026-11-17
- Runway CTO Kamil Sindi to break down training and real-time serving of video foundation models at AI Dev NYC — DeepLearningAI · 2026-11-17
- Microsoft's 30-hour Outlook outage reignites debate over lost senior engineers — 新智元 · 2026-11-17