2026-10-01
Nature: generative-AI tools that inform clinical decisions need real-world tests like drugs. Only 23% of ~4,600 clinical-AI papers used real patient data; 19 were prospective RCTs.
Generative AI reached clinics faster than the rules that govern medical products. ChatGPT shipped in November 2022. In less than four years, the clinical-AI literature grew to about three peer-reviewed papers a day. More than 230 million people a week now ask ChatGPT health questions. Systems already handle admin work, order lab tests, help with prescribing, read X-rays, MRI and CT, and assist with rare-disease diagnosis.
The open question is who regulates this class of product, and how anyone outside the vendor can tell that a device does what the label says. In August 2026 the US FDA posted a discussion paper on generative-AI medical devices and asked for comment by 19 October. This Nature editorial is the journal's public answer.
The piece is an unsigned editorial in Nature volume 658 (1 October 2026), not a new study. The argument follows existing risk-tiered device law. Every product must meet statutory quality and safety standards; how hard it is tested scales with harm if it fails. Bandage makers can self-certify. Stethoscopes and similar assessment tools need third-party authorization, which can be done in a lab. Diagnostic products that change clinical decisions have to be tested in the real world, and sometimes in trials that look like drug trials.
The editorial's call is that most generative-AI medical products should get that fuller look. Tools that record and summarize doctor-patient talk are not admin software once their output feeds decisions. Some systems already help with diagnosis. They should be tested thoroughly and in the open before deployment. A Nature Medicine Comment in September argued that pre-registered clinical trials should become standard for AI used in health, as they are for drugs and vaccines.
The comparison the editorial wants is self-driving cars. Cities already put those through lab hours, simulated scenarios, and supervised street tests, on principles closer to drug approval than to gadget refresh, because a miss is catastrophic.
The numbers are cited, not measured here.
| Source | Metric | Result |
| March 2026 Nature Medicine systematic review | Clinical-AI papers using real patient data | 23% of 4,600 |
| Same review | Prospective randomized trials | 19 studies |
| Estimate quoted in the editorial | Clinical-AI peer-reviewed output | 3 papers/day |
| Editorial | Weekly ChatGPT health queries | >230 million people |
A new stethoscope or bandage can be scored against decades of real-world data. Generative-AI devices mostly cannot, yet some are already being certified without real-world assessment. Vendors often run one-off simulated scenes and score the model against physicians in the same scene. Head-to-heads of specialized medical models versus general-purpose ones disagree on whether the specialist is more accurate. Even a high score skips the patient: companies usually do not ask people whether they want these tools in their care.
The editorial does not demand a full RCT for every device. It does treat data transparency, open data, and real-world testing as non-negotiable.
For anyone shipping medical AI, beating a doctor on a lab benchmark is not an authorization argument. The regulatory analogue is a drug or a robotaxi, not a stethoscope revision. If a visit summary lands in the chart and changes a decision, it should be tested as a diagnostic product. For evaluation teams, simulated accuracy does not fix missing real-world data or missing patient consent. The FDA paper is still a request for comment; Nature is putting the journal on the side of stricter review in most cases.
An unsigned editorial is a stance, not new evidence. The 23% and 19-trial figures come from Chen et al.; the editorial does not re-do the sample. It allows some products to skip full RCTs, then never draws the line: visit summaries, lab-order bots, imaging readers, and rare-disease diagnosticians are not mapped onto tiers. The specialist-versus-general conflict is pointed at two papers and left unresolved. The robotaxi analogy stays at the level of principle; it does not say how many hours of real-world exposure a medical model needs. Other countries' proposals are mentioned in one sentence, with no clause-level comparison to the FDA.