LLMs retry by default and can hammer MCP servers to death — a rate-limiting walkthrough
pjausovec · x · 2026-08-18
A Cloud Native Deep Dive post explains why rate limiting is mandatory for MCP servers: LLMs retry failed requests by default in their drive to return some answer, so an uncontrolled MCP call loop can behave like a memory leak — crashing the server, eating memory, blowing up API bills, and effectively self-DOSing your system.
The fix demonstrated uses agentgateway on Kubernetes: the GitHub Copilot MCP Server is capped at 10 requests per minute, with the 11th call receiving a clean 429. Prerequisites for following along are minimal — a local k8s cluster (Kind or Minikube), agentgateway installed, and a GitHub account. Bottom line: add rate limiting before shipping MCP to production.
More from coding & agent
- Agent coding tips: run /simplify, then fresh-context review of the diff — lucasmeijer · 2026-08-18
- Developer claims Go is miles ahead for AI coding agents — dosco · 2026-08-18
- Claude Code CLI cuts p99 CPU usage by 50% via GC tweak — dsp_ · 2026-08-18
- Enterprise AI fails on messy data and context, not on the model — Rajxai · 2026-08-18
- DeepTeam: Open-Source Framework for Red Teaming LLMs and AI Agents — tom_doerr · 2026-08-18
- Builder replaces value model with an LLM call, routes estimator outputs to the trainer — cephaloform · 2026-08-18