Running AI agents on a legacy IBM WMS: enforce tool order, script checks, verify evidence
ksrijith · reddit · 2026-10-27
The author spent months building agents for a customized IBM Sterling WMS, where behavior spans code, per-installation DB config, and unchangeable vendor code — with no test suite and human deploys. Key lessons:
- Agents follow what's enforced, not what's asked: tool ordering rules in config (requires:) worked where prompt instructions failed.
- Rules on tools don't work for full coding agents like Claude Code that have built-in file tools — check outputs instead.
- Require evidence: every written file must name the existing file it modeled, flagged by script.
- Never accept "done" as proof: designs go through JSON schema check → scripted fact checks → LLM reviewer, with human approval by a non-author; tests are written from the approved spec.
- Pick executors per step: Kimi K2.5 handled triage; a 22-field design step succeeded 2/7 runs and moved to Claude Code — but savings came mostly from scripts, skipping steps, cheap checks first, and hash-based re-approval skips.
- Knowledge only counts if delivered: one run spent $3.63 rebuilding a known-wrong approach while the correct fact sat unused; delivery dropped cost to $1.75.
- Don't let agents write their own memory; only humans record facts. Move human decisions earlier to triage/spec. The open problem is scaling human approvals.
More from coding & agent
- DeepSeek Harness plugin ecosystem is booming as cheap, mid-tier-but-solid model draws tinkerers — op7418 · 2026-10-27
- Open-source auto-research agent hits Top 2 on GPU kernel challenge in 7 hours — sguada · 2026-10-27
- 'Codex decided': devs admit making up reasons in code review for agent-written code — FlatStill2540 · 2026-10-27
- 365 public MCP servers tested: 51% lack tool input descriptions, the most common bug — abhiramreddykalluru · 2026-10-27
- 300 Agents, 60 People: Runlayer Found Engineer Details Real-World Agent Security Failures — AI Engineer · 2026-10-27
- CoreWeave ARIA agent ships four new live in-chat panels for ML run analysis — _ScottCondron · 2026-10-27