Put per-turn action schemas in the last user message to preserve prompt caching
Low_Bad_6585 · reddit · 2026-09-09
A developer building an AI life-sim where characters choose actions in a shared world hit a hard problem: available actions (tools) change every turn, and tool definitions early in the prefix keep invalidating the cached history.
The fix: put current action schemas (choices + parameter constraints) in the last, dynamic user message, after the history. Request layout: stable system instructions → conversation history → latest user message with current state + schemas. The model returns JSON parsed by the app. For continuously deciding characters this matters for cost and latency.
He compares with Anthropic's Tool Search in Claude Code: lightweight tool stubs in the prefix, full definitions loaded on demand — same concern of introducing changing info without rewriting the front of the conversation.
He tested 17 model/output-mode combos (GPT, Claude, Gemini, Qwen, Doubao): JSON schema mode, JSON object mode, plain text with JSON, generic doaction/dochoice tools, and a separate tool generated per available action. Notes favored Gemini Flash + JSON schema for stability; Doubao Seed and Sonnet produced interesting action chains via plain-text JSON. Parsing, choosing sensible actions, and coherent behavior proved to be different problems. He stresses these were exploratory runs, not a controlled benchmark.
He asks the community: regenerate native schemas, deferred loading, or schemas-in-messages? Anyone measured cache reuse and latency alongside action quality? How to preserve historical calls when tools or allowed arguments change?
More from coding & agent
- Stop using random multi-agent topologies: how to audit wasted subagent delegation — keyanzhang · 2026-09-09
- opencode v1.18.30 Adds Astra System Prompt for GPT-6, Fixes Bedrock DeepSeek IDs — opencode-agent[bot] · 2026-09-09
- Abacus.AI teases near-free LLM targeting long-running personal agentic loops, launching Thursday — bindureddy · 2026-09-09
- huashu-chrome: Open-Source MCP + Chrome Extension Lets Agents Drive Your Logged-in Browser — AlchainHust · 2026-09-09
- GPT-6 Astra instantly finished browser agent tasks Claude Code couldn't, user reports — AlchainHust · 2026-09-09
- How Do You Verify an Agent Fix When It Only Fails One Run in Twenty? — Such-Process5697 · 2026-09-09