AI Studio's 1M-token counter is an illusion: how cold cache and rereads really burn your quota
Former-Aerie6530 · reddit · 2026-10-25
A heavy Google AI Studio user breaks down what long chats actually cost, surfacing things most people miss:
- The counter illusion (snowball effect): the number at the top is just the current context's 'backpack weight'. Asking 10 small questions in a 100k-token chat doesn't cost 10k — the server rereads the whole conversation 10 times, processing 1M+ tokens.
- Feeding chats 'a bit per day' hits cold cache: same-hour messages get 95% implicit cache discount, but returning next day to a 500k–1M chat means the cache died; the first question bills 100% raw and can eat most of the daily quota. He calculates his six 1M-token chats would cost thousands of reais via the cache-less API.
- Why consumer apps feel 'shallower': Gemini's app cuts thinking budget, caps output length, and applies RAG/sliding-window truncation; AI Studio (and Claude up to its limits) injects the full raw text every turn. Claude even locks the chat at max context instead of silently truncating — 'the only consumer app that doesn't lie to you'.
He asks the community how they handle mobile workflows (third-party API clients?), whether to compress 50k+ chats into memory summaries, and whether the $10/month developer credits in Google AI Pro are worth using for Tier 1 access to Argon 4.
More from Infra
- CMU lab open-sources Quail, a SQL inference engine 1.8x faster than vLLM and up to 14x on AI joins — sh_reya · 2026-10-25
- AI compute is Bitcoin mining redux: middle-layer operators eat the depreciation — sebpaquet · 2026-10-25
- Musk: SpaceX Super Intelligence to bring ~200,000 GB300s online per month, shifting to Vera Rubin next year — Beth_Kindig · 2026-10-25
- SGLang matches DeepSeek's DeepInfer: up to 1.04x prefill throughput on GB300/B300 — ying11231 · 2026-10-25
- Epoch AI: Memory Chips Shipped Through 2027 Could Run 33–171 Million Frontier AI Agents — PeterDiamandis · 2026-10-25
- Is a €250 modded Dell T5810 + RTX 3090 a smart local-LLM host in 2026? — SlechteConcentratie · 2026-10-25