OVH's Olala Mini 7A1B: 360 tok/s decode on one RTX 5090, 96K context, v2.0 to exceed 100B
JiliJeanlouis · x · 2026-11-21
- OVH founder demos Olala Mini 7A1B (1B activated params per token), hitting 220–380 tok/s on B300; live testing, comparison, download and docs are public.
- RTX 5090 numbers: 360 tok/s single-stream decode (2.6–2.8ms TPOT), 7,280 tok/s at batch 128; decode only drops to 285 tok/s at 96K context; prefill of a 1K prompt takes 34ms, 50K tok/s at 64 concurrency.
- Roadmap: v1.0 has 32K context, v1.1 in 6 weeks adds 128K and better math; v2.0 will exceed 100B targeting agentic, devops and cyber workloads. Tiny activation lets v1.0 run agentic tasks without a GPU; models are destined for OVH's cloud/local encrypted offerings.
More from Infra
- OpenRouter accused of sending users' BYOK private keys to China despite earlier denials — 0xkarasy · 2026-11-21
- If you bet the century on AI, you need the financial and physical supply chain to back it — we don't — ZeroStateReflex · 2026-11-21
- NInfer-all adds Inkling 2 Mini, nearly 2x llama.cpp decode speed on RTX 5090 — WaveCut · 2026-11-21
- Obscura: a 70MB Rust headless browser with zero Chromium that drops into Puppeteer and Playwright — thisguyknowsai · 2026-11-21
- Airbnb reportedly runs 80,000 daily chats on Alibaba's open-weight Qwen as Chinese models undercut US APIs up to 30x — mawcopolow · 2026-11-21
- Local AI field manual shows how to query PDFs offline on your own machine — alifcoder · 2026-11-21