Nebius Token Factory launches cached tokens with automatic prefix reuse and discounted billing
HowDevelop · x · 2026-10-24
Nebius Token Factory now serves cached tokens automatically: repeated prompt prefixes reuse stored computation with no config, cache keys, or opt-in, working on both Serverless and Dedicated endpoints.
Key billing details:
- Serverless requests are billed across three token types: uncached input, discounted cached input, and output; every response reports cached token counts.
- Dedicated endpoints bill per GPU hour, so caching doesn't change pricing there; caching is best-effort.
Long, stable prefixes benefit most: repeated system prompts, shared tool/function definitions, few-shot examples, and multi-turn agent loops.
More from Infra
- Cutting KServe cold starts from 134s to 9s with warmctl — kbesplv · 2026-10-24
- Five inference engines on M3 Max 36GB: Kapitsa hits 41 tok/s dense, oMLX ties on MoE — Hyungsun · 2026-10-24
- Ramjet beats nginx by up to 61% more agent turns for open-source domyn swarm — DoggoProfessor959 · 2026-10-24
- Micron's G11 NAND doubles density vs G9 with 400+ layers, volume production in 2H 2027 — rohanpaul_ai · 2026-10-24
- PSA: Streaming n-grams from an SSD causes real wear via read disturbance and reclaim — Chromix_ · 2026-10-24
- ISTA quantizes DeepSeek-V4.1-Flash to 2 bits with zero benchmark loss, runs on one 12GB GPU — Loginhe · 2026-10-24