Dev open-sources bevis: agents must prove 'done' with stored commands and output
Lekis86 · reddit · 2026-08-29
The author runs a fleet of 24 agents on local hardware, tracked via a Mission Control board in their own AI OS. The recurring failure wasn't agents being wrong — it was agents reporting tasks done when the artifact said otherwise: untouched work, tests against the wrong target, or no tests at all, just assertions.
So they built bevis (Apache-2.0, stdlib only, zero dependencies):
- A job cannot reach "closed" without a stored command, its exit code, and its output — not asserted. Closing and verifying are separate acts, and whoever closed a job can't verify it.
- A dispatcher hands work to any agent or script but never decides success itself; the job's own checks do.
- Model-agnostic: works with Claude Code, Codex, any LLM including local ones. bevis itself never calls a model — what's under test is whether a claim of success is true.
- Bonus: verified tasks, commands and outputs accumulate into a dataset you can later LoRA-train your own models on, turning agent-fleet work into training material.
Stated limits: no relevance checking (bevis close 3 --run "echo done" still closes); the event log has no hash chain or signatures, so anyone with the SQLite file can rewrite history.
More from coding & agent
- Perplexity open-sources SPACE, the Firecracker microVM sandbox behind Perplexity Computer — perplexity_ai · 2026-10-16
- Blackstone goes all-in on Factory for autonomous software development — matanSF · 2026-10-16
- AI software factories hinge on the Factory Loop, not faster code generation — Pavan_Belagatti · 2026-10-16
- 'Dropping a 3,000-line PR twenty minutes after ticket assignment' — AI coding meme goes viral — facontidavide · 2026-10-16
- Self Labs launches Agent Vault to let AI agents log in and pay with credit cards — EXM7777 · 2026-10-16
- Plane Agents: triggers, not prompts, are what turn chatbots into workflow agents — JosephJacks_ · 2026-10-16