Agents are sessions, not requests: Google explains sizing GKE node pools for agent workloads
rseroter · x · 2026-11-21
A Google Developers Blog post argues that moving coding agents onto shared GKE clusters requires a different capacity model than web apps.
Core idea: agents are sessions, not requests
- Web requests finish in milliseconds and keep nothing, so Kubernetes scales on traffic
- A coding agent's workspace stays open all day, keeps memory and files between prompts, but only works when a prompt arrives
- A July 2026 study of agent traces found agents wait for the user 80% of the time in typical multi-prompt sessions; one turn averages 6.5 minutes of agent work
- At 2 prompts/hour, most of the node bill pays for waiting
Two concepts (from the diagram):
- Agent Substrate (where/how it runs): holds a worker until suspended; state is a snapshot in object storage; many sessions per warm worker; own sandbox with egress denied
- AX (what the agent is): Workspace of repos and files; task defines image, command, limits; observable via ax get/describe/ssh; ax suspend keeps /workspace
The article teaches why agent sessions need a different runtime than web requests and how to spot cost-optimization opportunities in GKE node pools.
More from coding & agent
- Agent-scale night: 21-person company hits $100M ARR with zero support teams — LukeW · 2026-11-21
- "Diagram Driven Development": make the AI-drawn logic diagram the single source of truth — max__drake · 2026-11-21
- shadcn launches UI SDK: reusable component behaviors designed for agent-built interfaces — shadcn · 2026-11-21
- Claude Code dual-model setup: one model plans and reviews, subagents write the code — PrajwalTomar_ · 2026-11-21
- Dev argues agent approval flows should let you approve actions by voice — NYCounihan · 2026-11-21
- ENZO v1.7.0: one-click self-hosted AI workspace with evidence-citing research mode — anonymous20256 · 2026-11-21