Pipeshift claims 40% faster first token than Sarvam serving Gemma 4 31B at 150B tokens/day
NirantK · x · 2026-11-17
Sarvam AI says its Token Factory serves Gemma 4 31B for real-time conversations with 5× faster first token and 1.6× more tokens/s than the only other provider, already serving 150 billion tokens daily. Pipeshift then published benchmarks pushing back: on identical hardware (1× B300, 11.2K prompt, single stream, P50), Pipeshift hits 0.24s TTFT vs Sarvam's 0.40s (40% lower) and 359 vs 308 tokens/s (20% higher).
More from Infra
- LoopFill: training-free speculative decoding speeds up looped Transformers — UIUC-CS · 2026-11-17
- Tencent's DSpine speculative decoding ships GGUFs and a llama.cpp PR: 4.5x speedup on 27B — pmttyji · 2026-11-17
- Founder lands 1000 GPUs (40% GB300, 60% B300) on short-term contracts, dismissing the GPU shortage narrative — SumitGup · 2026-11-17
- Agent Substrate: open-source zero-trust runtime suspends idle agents, fits 10x more per hardware — Saboo_Shubham_ · 2026-11-17
- Practical tips to cut agent memory costs on OpenAI: cache TTL, Flex, fewer calls — gabrielchua · 2026-11-17
- xAI affiliate buys 96 acres near Memphis Colossus 2 for $5.5M as compute footprint expands — XFreeze · 2026-11-17