Medical AI benchmark builder: his first ECG dataset had answers printed on images
MaziyarPanahi · x · 2026-10-08
A medical AI developer shares benchmark hygiene lessons: in medicine the Git history is the chart — diagnoses live in discharge summaries, ICD codes in billing footers, and his first ECG benchmark had the answer printed right on the image. He now never includes labels in questions and hunts every leak before models see the data.
More from Research
- Perplexity launches asymmetric embedding models: 0.6B queries a 9B index at zero latency cost — antoine_chaffin · 2026-10-08
- LeCun: automated formal proofs open a new era for math; critics say question taste can't automate — AryHHAry · 2026-10-08
- RSIGym treats recursive self-improvement as systems engineering; Opus 5 leads at 0.4809 — teortaxesTex · 2026-10-08
- DIRU: dendrite-inspired recurrent units improve chaotic forecasting and neonatal epilepsy detection — Dr_Alex_Crimi · 2026-10-08
- Formalization agents auto-generate errata for papers, says mathematician Armstrong — burny_tech · 2026-10-08
- New arXiv Paper Shows Lean Formalization Agents Deviate from Original Proofs — burny_tech · 2026-10-08