MedRSI paper shows self-improving medical agents need guardrails to hold 94.4% accuracy
rohanpaul_ai · x · 2026-09-23
A new Stanford+Oxford paper, MedRSI, shows medical agents can improve themselves from their own mistakes — but only if new capabilities are validated on fresh patient cohorts before becoming permanent. Adding every promising tool immediately degrades accuracy to 76.9% by round 30 with 57 tools; with slow registration, the agent keeps just 18 tools and holds 94.4%. Two mechanisms drive clinically aligned self-evolution: clinical-cost-aware failure prioritization (fixing potentially harmful errors over common ones) and fast discovery with slow registration (rapid invention, conservative adoption). Takeaway: let agents invent aggressively, but permanent self-changes must earn their place through repeated independent evaluation.
More from Research
- Stanford's DyCraft Uses Agentic Decomposition to Cut Physical Errors in Long-Horizon 4D Scene Generation — StanfordUniversity · 2026-11-11
- Bert Chan on His Unconventional Path from Side-Projects to the ALife Institute — BertChakovsky · 2026-11-11
- CGM Study of 62,000 People: Average Glucose Beats HgbA1C at Detecting High Insulin — arjunrajlab · 2026-11-11
- First open dictation benchmark: 761 recordings across 10 apps, Wispr Flow 79% vs Apple 35% — rohanpaul_ai · 2026-11-11
- Boltz models showcased at MSK Cancer Center for accelerating early-stage drug discovery — charlieharris01 · 2026-11-11
- Carlman: a Pac-Man-style game where all characters are Lenia cellular-automata solitons — BertChakovsky · 2026-11-11