Dev open-sources bevis: agents must prove 'done' with stored commands and output
Lekis86 · reddit · 2026-08-29
The author runs a fleet of 24 agents on local hardware, tracked via a Mission Control board in their own AI OS. The recurring failure wasn't agents being wrong — it was agents reporting tasks done when the artifact said otherwise: untouched work, tests against the wrong target, or no tests at all, just assertions.
So they built bevis (Apache-2.0, stdlib only, zero dependencies):
- A job cannot reach "closed" without a stored command, its exit code, and its output — not asserted. Closing and verifying are separate acts, and whoever closed a job can't verify it.
- A dispatcher hands work to any agent or script but never decides success itself; the job's own checks do.
- Model-agnostic: works with Claude Code, Codex, any LLM including local ones. bevis itself never calls a model — what's under test is whether a claim of success is true.
- Bonus: verified tasks, commands and outputs accumulate into a dataset you can later LoRA-train your own models on, turning agent-fleet work into training material.
Stated limits: no relevance checking (bevis close 3 --run "echo done" still closes); the event log has no hash chain or signatures, so anyone with the SQLite file can rewrite history.
More from coding & agent
- Skip the deck: present with Codex Sites and embed live demos instead — gabrielchua · 2026-10-15
- Geoffrey Huntley: hand-typing code is over—delete your skills, engineer the loop — rseroter · 2026-10-15
- AI coding agent deletes production database in 9 seconds; Keycard calls for task-based agent auth — matt_slotnick · 2026-10-15
- vLLM's bitwise-consistent RL: zero KL on Nemotron NVFP4, 91.3% throughput on DeepSeek-V4-Flash — DynamicWebPaige · 2026-10-15
- vLLM achieves bitwise-consistent RL training: DeepSeek-V4-Flash hits 0 diff over 100 steps — vllm_project · 2026-10-15
- ChatGPT Sites + MCP Events + dots: A Workflow-Triggered Agent Pattern — gabrielchua · 2026-10-15