Decision model leads, LLM reviews: SRE diagnosis pass rate jumps to 89.5%
tianyin_xu · x · 2026-10-21
The SREGym team (Yukun He, Saad Mohammad Rafid Pial, Tianyin Xu et al.) published a new experiment where a decision model leads incident investigation and the LLM plays a supporting review role.
Architecture: a programmatic collector gathers cluster evidence (logs, metrics, workload state) → TypeSafe AI's Jev decision model picks which tool to run and proposes a root cause → the LLM, with no tool access, confirms or replaces the cause against collected evidence → the decision model selects supporting observations → the LLM writes the report.
Results: on 21 SREGym-Lite faults, LLM review lifted Jev's pass rate from 78.1% to 89.5%, averaging 38 seconds per diagnosis. A decision-model-only pipeline (no LLM) had passed 76.2%. The team also compared Jev with OpenAI's GPT-6 Luna Decisions.
More from coding & agent
- GPT-6.1 Agent With Full Access Nukes Windows Bloatware, Sparks 'This Is AGI' Hype — keyanzhang · 2026-10-21
- MMEnterprise-Bench from KAIST Tests Enterprise Agents on Conflicting, Evolving Multimodal Records — Seohyun Lee · 2026-10-21
- NTU and Alibaba's FocusGUI: a lightweight adapter filters multimodal evidence to stop GUI agent error compounding — NanyangTechnologicalUniversity · 2026-10-21
- Snowflake's HybridResearcher: a 4B agent trained on synthetic cross-source trajectories beats much larger models — Snowflake · 2026-10-21
- LangChain's Open-SWE (11k stars) offers a forkable open-source software factory — BraceSproul · 2026-10-21
- PyTorch accelerator integration group details progress and priorities at PyTorchCon — PyTorch · 2026-10-21