Harvard-MIT's FINSKILLOPS turns LLM failures into regression-tested skill patches
rohanpaul_ai · x · 2026-09-22
A Harvard + MIT paper introduces FINSKILLOPS, a self-evolving multi-agent system for SEC filing QA.
- Core idea: frame post-deployment improvement as controlled behavioral maintenance—recurring failures become scoped skill patches, each earning deployment via validation.
- Governance: evidence-grounded, typed failure diagnoses produce reusable skills, governed by targeted validation, protected-case regression checks, negative controls, and versioned replacement/retirement; any fix that breaks prior correct answers is rejected.
- Results: across six financial QA benchmarks, a single frozen skill registry achieves the highest verdict-weighted correctness and reference consistency among evaluated systems.
The upshot: turn a static RAG system into one that continuously learns from failures and safely updates how it answers.
More from coding & agent
- Category-aware expert RL framework hits 59% on SWE-bench Multilingual — Logics-MLLM · 2026-09-22
- QM v0.1.12 ships beta agent swarms, hosted Slack installs, app management — ycombinator · 2026-09-22
- Josh Rosen says the job is to build System 1.5: connect fast models to frontier reasoning via software — iamrobotbear · 2026-09-22
- Agent Substrate roadmap: sub-second suspend/resume runtime for dense agent deployments — rakyll · 2026-09-22
- Hot take: Fable beats Astra at coding, but Astra wins at computer use — PratikKadam_ · 2026-09-22
- Building a (deliberately unsafe) restricted shell MCP server for local coding agents — ag789 · 2026-09-22