Caveman Optimizes Agent Context Representation, Slashing Input Tokens by 33%
VeryVexxy · reddit · 2026-08-13
A developer shared progress and benchmark data for their project Caveman, focusing on optimizing AI agent context.
Caveman initially served as a terse-output intervention for coding agents, reporting a 65% reduction in output tokens. However, independent testing by JetBrains on long-horizon coding tasks measured only an 8.5% reduction. The developer discovered the discrepancy stemmed from benchmark scope: in complex tasks, model output was only a small fraction of total traffic, while agent requests repeatedly carried massive context like conversation history, source files, and tool results.
Consequently, the developer shifted focus to input context representation. By deploying a local Caveman proxy between the agent and the provider to compress or restructure eligible context, provider-reported input tokens dropped by 33.2% across a 54-run benchmark, with all 18 exact-answer checks passing. Furthermore, representing existing skills as multimodal image context reduced their footprint by about 70% while maintaining roughly 99% retrieval accuracy.
The developer concluded that agent efficiency depends heavily on context representation, not just prompt engineering.
More from coding & agent
- Developer Builds 8 MCP Servers to Fight Health Insurance Denials with AI — aidenhartxxx · 2026-08-13
- Coding agent implements SHA-256 on its own in slow interpreted language, researcher laments — matthew_d_green · 2026-08-13
- AI Agent Governance: Build in a Day, Govern for as Long as They're in Production — uxmag · 2026-08-13
- Complex AI Agent Architecture: Graph Engineering is Missing the Graph — mattturck · 2026-08-13
- Developer builds 8 MCP servers with 35 tools to fight US health-insurance denials — aidenhartxxx · 2026-08-13
- Inside Pi's Engineering Approach to LLM Context Compaction — viksit · 2026-08-13