This is a Physician in the Loop special topic.

Wait for the Evidence

Calling it malpractice not to consult AI assumes doing so improves diagnoses, treatment and outcomes. Outside a few tasks the evidence does not yet show that; the standard of care should follow the trials.

At a summit in Washington on Sept. 29, the health secretary, Robert F. Kennedy Jr., said artificial intelligence can give a patient a second opinion "much better informed than any doctor in the country." He told of a visit from OpenAI's chief executive, Sam Altman, who, by Mr. Kennedy's account, had said that "it would be malpractice for a doctor to make a diagnosis or to make a prescription without at least checking AI." Earlier at the summit an OpenAI executive, Felipe Millon, told the audience, "It is medical malpractice not to get a second opinion from AI today," Medical Economics reported.

Mr. Kennedy's reasons were time and volume: a patient's record can run to a thousand pages, he said, and a doctor with six minutes for a visit cannot review it, while software can. That is an argument about reading. A duty to consult AI before every diagnosis and prescription rests on a larger claim: that doing so makes diagnoses more accurate, treatment better and patients better off. The transcript of his remarks cites no study for it, and the report of Mr. Millon's remark does not show him citing one. Outside a few specific tasks, the evidence does not yet support it. The standard of care has moved toward what a competent physician would do in light of available knowledge and resources, and until trials show, task by task, that patients do better when their doctors consult AI, not consulting it is no lapse from that standard.

Each part of that claim can be tested, and the kind of test matters. Most of what is known about language models, the software behind chatbots such as ChatGPT, comes from exams, benchmarks and written cases. Of the studies in clinical medicine published from 2022 to September 2025, a review in Nature Medicine found that most tested the models on simulated cases or exam-style questions; of the roughly 1,000 that used real patients' data, only 19 were prospective randomized trials. Tests on paper show what a model knows, not whether patients do better when doctors use it.

On diagnosis, the best models now beat doctors on paper, often under rules that handicap the doctors. In a randomized, blinded comparison published in Nature last year, Google's diagnostic chatbot, AMIE, made more accurate diagnoses than primary care physicians when each held text consultations with actors playing patients, a format the authors call unfamiliar in clinical practice. In a preprint, Microsoft reported that its own diagnostic system solved more than four in five hard cases from the New England Journal of Medicine, against one in five for generalist physicians working without colleagues, textbooks or other AI. And in a randomized trial on written cases, run by researchers at Stanford, Beth Israel Deaconess and elsewhere, an exploratory analysis found that GPT-4 working alone outscored physicians using their usual resources by 16 points.

Giving a model to doctors has helped less. In that trial, physicians given GPT-4 reasoned no better about diagnosis than those with their usual resources, and in a randomized study of hospital clinicians reading case vignettes, a deliberately biased chest X-ray model cut their diagnostic accuracy by 11 points. Real clinics have been studied far less. In a quality improvement study with financial support from OpenAI, Penda Health, which runs clinics in Kenya, gave an AI assistant to a randomly chosen half of its clinicians, and reviewing physicians judged that those with the assistant made fewer diagnostic errors. In a randomized trial in Penda's clinics, published in June, clinicians with the assistant recorded an appropriate diagnosis more often. Google's AMIE has been tried with 98 patients at Beth Israel Deaconess, in a study published in The Lancet this month; its authors said it was designed to assess feasibility, not whether AI improves health outcomes.

On treatment, the evidence on paper is somewhat more favorable. OpenAI says the version of ChatGPT it built for clinicians outperformed physicians who had unlimited time and web access, on its own benchmark, drawn from doctors' chats about patient care, documentation and research. The benchmark's authors note that it was weighted toward cases OpenAI's models find hard, so its absolute scores are not directly comparable to real-world performance rates. In a second trial by the Stanford and Beth Israel group, physicians given GPT-4 scored six and a half points higher on written cases about managing patients' care. In Penda's clinics, reviewing physicians judged that clinicians with the assistant made fewer treatment errors, yet in the randomized trial, treatment failure within two weeks, its main measure, was about as common with the assistant as without it, in roughly one case in fifty.

On outcomes, which matter most, the assistant in Kenya has yet to show a gain; in the earlier study, too, patients' own reports of whether they felt better did not differ significantly. Narrower tools have done better. In one randomized trial in Taiwan, software read hospital patients' electrocardiograms and alerted their doctors to those at high risk; deaths within 90 days fell by about a sixth, though its authors cannot yet fully explain why. In Sweden, a randomized trial of breast screening found that mammograms read with AI support caught more cancers and cut radiologists' reading by more than two-fifths, while no more cancers surfaced between screenings than under the usual reading by two radiologists.

Even where AI plainly finds more, the gain for patients can stay unproven. In randomized trials pooled for the gastroenterologists' guideline, software that flags polyps during colonoscopy found precancerous growths in about a fifth more patients, mostly tiny ones with little potential to become cancer. Yet the American Gastroenterological Association made no recommendation for or against it last year, saying its impact on preventing colorectal cancer "remains unclear." The companies' own accounts concede the gap. Microsoft's preprint says its results "do not yet establish the clinical efficacy" of its system, and Google's account of the clinic study says "larger clinical trials are needed to assess patient-facing AI at scale." OpenAI wrote last year of the assistant in Kenya: "Follow-up studies are needed to further study how these copilots affect patient outcomes."

Waiting has its own toll, and that is the best case for expecting doctors to consult AI now. By one estimate, diagnostic errors kill or permanently disable nearly 800,000 Americans a year. The models already beat doctors on hard cases, the argument runs, and every year spent on trials means preventable harm. The toll is real, and a reason to run trials quickly rather than to skip them. Beating doctors on paper is not the same as reducing that toll: in Kenya, better-recorded diagnoses did not bring fewer treatment failures, and a model that outscored physicians on written cases did not make them better diagnosticians. A duty to consult would spread a model's errors as widely as its insights, and clinicians handed a biased model became markedly less accurate. Nor are such trials exotic: Taiwan's and Sweden's ran within ordinary hospital care and routine breast screening.

The companies that make these models are best placed to test the claim. Before anyone tells doctors they must consult a model, its makers should show, in randomized trials in ordinary care, that patients do better when doctors do: fewer serious diagnoses missed, fewer treatment failures, fewer deaths. Those trials should be registered before they start and published whatever they find. Health systems can add evidence of their own by giving a new tool first to a randomly chosen half of their clinicians, as Penda did, and counting what happens to patients.

Specialty societies should then weigh those trials task by task, as the gastroenterologists did for polyp software, and say plainly where consulting AI is expected, where it is a reasonable choice and where the evidence is too thin to say. When trials show that patients do better, the standard of care should move, and physicians with it. Until then, a model's answer is one more opinion for the physician to weigh, and the question to put to anyone selling one is what happened to the patients.

By Mr. Kennedy's account, Mr. Altman also said that two years ago he would not have advised anybody to take medical advice alone from AI. Opinions about these models have changed quickly. The evidence that patients do better when their doctors consult AI has moved more slowly, and it is the evidence that should set the standard.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.