Agentpause Suspends LLM Agents Before Rate Limits Hit

Maleficent_Pain2722 · reddit · 2026-07-20

The author built **agentpause**, a scheduler for long-running LLM agents that suspends work before a rate limit is hit and resumes later without losing progress. - It checks each next call against the provider’s real remaining budget using rate-limit headers plus a safety margin. - If the step would exceed the budget, it either waits just long enough for refill or checkpoints and exits cleanly. - On the next run, it resumes from the exact step instead of restarting the whole task. - For hosted APIs like OpenAI, Anthropic, and Groq, this is a logical warm start: the context is resent, but no work is redone. - On **llama.cpp**, the checkpoint can also save/restore the **KV cache** via `/slots save/restore`, which makes it a true warm start. - The author measured a ~9k-token Qwen3-8B resume on an M1 Pro: **46.9s** for cold re-prefill vs **0.5s** for warm restore, about **93x** faster. - An additional experiment showed that blindly truncating context can destroy recall quality, while a cheap summary preserved all planted facts. The repo is MIT-licensed, dependency-light, and works with providers directly or via LiteLLM; it also plugs into LangGraph in two lines.

Original post →

Agentpause Suspends LLM Agents Before Rate Limits Hit

More from coding & agent