Hackathon prototype: incident agent that remembers failed fixes lifts accuracy 0% to 50%
Fabulous-Ad9781 · reddit · 2026-09-29
A hackathon team demoed Adaptive Incident Intelligence, a prototype addressing a real gap: tickets say "resolved" but lose the sequence of what was tried first, what failed, and what finally restored service.
How it works: it retrieves similar cases via Hindsight, proposes actions with supporting evidence, and checks them against deterministic policy rules — higher-risk actions require human approval. After execution, it retries the original failed operation and verifies service health before recording the outcome.
Evaluation: on 12 held-out synthetic cases, action accuracy rose from 0% with empty memory to 50% after evidence was added (explicitly a small prototype eval, not a production claim). Stack: FastAPI, React, Hindsight, and a local model via Ollama. The authors are most interested in the boundary between useful agent memory and safe automation.
More from coding & agent
- Matt Shumer says Opus 5.5 cleared a 3D printer bed after 16 tries, taking humans out of the loop — mattshumer_ · 2026-09-29
- Claude autonomously builds a 'p(doom)' Beat Saber clone in one afternoon — burny_tech · 2026-09-29
- AlignOPSD fixes decision-timestamp mismatch in agent distillation, beating GRPO by 5.5-8.7% — Mingju Chen · 2026-09-29
- CMU's ControlScope: how much of a running agent workflow should be revised? — CarnegieMellonU · 2026-09-29
- CompoWorld scales general agent training by composing environments from reusable services — AllSpark-Research · 2026-09-29
- Adaptive Consistency Graph lifts long-horizon agent success from 44.5% to 50.2% — Beihang · 2026-09-29