Benchmark: Persistent Bash Breaks AI Agent Spatial Awareness
_philschmid · x · 2026-07-16
Phil Schmid shares an engineering evaluation of AI coding agent toolchains, comparing two Bash execution modes: persistent (preserving directory and environment variables) vs stateless reset.
- Pitfall of persistence: Although persistence sounds natural, when the agent switches directories in Bash (e.g., cd app/data) and later calls tools like Write that require absolute paths, path confusion arises and causes errors.
- System prompt adjustment: Removing the working directory from the system prompt alleviates path confusion but causes the agent to completely lose awareness of its current location, resulting in a drop in overall benchmark performance.
The conclusion is that under current agent frameworks, simple Bash persistence tends to break spatial consistency across multi-tool coordination.
More from coding & agent
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27
- Non-coder vibe-codes a Godot JRPG with Claude Code in two weeks, music and art synced — ADHDavid · 2026-10-27
- StackChan robot performs a drama skit via Hermes agent and GPT-Luna-6 — alexcovo_eth · 2026-10-27