MLCR-AA revamps methodology: models prompted as medical reviewers, 3-judge panel agrees on 69%
ArtificialAnlys · x · 2026-10-21
Artificial Analysis details its updated MLCR-AA methodology:
- Models get a system prompt framing the task as medical record review, with questions placed after full 25k–64k token records.
- Each question part is answered in one paragraph with a conclusion plus supporting evidence; with answer lengths now consistent, the concision test was dropped.
- Grading uses a three-model judge panel (GPT-6.1 Sol, Claude Opus 5.5, Grok 4.7 at medium effort) that flags contradictions and missed key points, decided by majority; judges agree on 69% of answers, up from 48% under the previous panel.
More from Models
- Recursive AI ships Flint, a 40B-A2.5B MoE claimed to match Qwen3.6-27B, weights on Hugging Face — ChengleiSi · 2026-10-22
- DeepSeek ships V4.1-Flash-1017, now the #1 open model on RSI-Inference Bench — scaling01 · 2026-10-22
- OpenAI's Cybersecurity Guardrails Block a Dev From Auditing His Own Patched Server — Next_Technology6361 · 2026-10-22
- TPN-006 hits Hugging Face: 7.7GB compressed model beats its 17GB Mistral teacher — markjeffrey · 2026-10-22
- Kimi K3.1's latest checkpoint draws 49k-token SVG portrait zero-shot in 22 minutes — lyraxana · 2026-10-22
- MiniMax launches M3.1 Flash at 1/3 the cost and 3x the speed of M3, weights out Oct 23 — MiniMax_AI · 2026-10-22