Microsoft diagnostic AI beats generalist physicians on hard cases, study says
Microsoft diagnostic AI beats generalist physicians on hard cases, study says
The voices on this episode are generated, and so is the script, from the day's post at Physician in the Loop. Every fact in the post links to its source.
Microsoft's multi-agent diagnostic system reached the correct diagnosis in 83.2% of complex New England Journal of Medicine cases, against 36.1% for generalist physicians, in a benchmark study Science published online Oct. 22.
The physicians in the comparison were primary care physicians and general internists working alone with reference materials, and no specialists or physician teams took part, TechCrunch reported. The authors described the results as coming from a controlled benchmark, and Microsoft said it plans a prospective study at Beth Israel Deaconess Medical Center in Boston in which patients interact with the system directly.
Elsewhere, the director of the Food and Drug Administration's Digital Health Center of Excellence said Oct. 20 that draft guidance on generative AI devices will come in the coming months, MedTech Dive reported. Sutter Health said more than 6,000 of its clinicians have drafted more than 5 million notes with Abridge's ambient AI, and the FDA approved Edwards Lifesciences' Pascal Precision system for transcatheter repair of the tricuspid valve.
In this episode
How technological innovation is changing medicine for patients and the people who practice on the front lines.
Voice 1: This is Physician in the Loop, the daily briefing on AI in medicine. It's Friday, October 23rd, 2026.
Voice 1: Microsoft says its diagnostic AI reached the correct diagnosis in about 83% of complex hospital cases, compared with about 36% for generalist physicians working alone, in a benchmark study Science published online yesterday.
Voice 2: Alone meaning no specialist, no team backup at all?
Voice 1: Right. The physicians were primary care doctors and general internists, working only with reference materials. No specialists and no physician teams took part in the comparison.
Voice 2: How does the AI side actually work, is it one model doing this?
Voice 1: No, it's called MAI-DxO, and it coordinates several AI agents acting like a panel of physicians. They question a simulated patient, order tests, and revise a differential diagnosis while weighing what each test costs, according to the 15 Microsoft authors. On emergency department cases from the MIMIC-IV-ED database, the system scored about 80%, against about 47% for the same physicians.
Voice 2: Microsoft had another number buried in there, physicians did worse without any references at all.
Voice 1: Yes, Microsoft said physicians working without reference resources scored only 22% on the NEJM cases. Most of the system's advantage seems to come from the underlying models themselves, not the orchestrator layer. On the emergency department cases, the orchestrator added little or no accuracy on top of what the newest models already managed alone.
Voice 2: So the panel structure matters less than the models underneath it.
Voice 1: That's what the comparison suggests. A 2025 preprint of this same work had reported 80% for the system against 20% for generalist physicians, so both sides have moved since then.
Voice 2: Is this heading toward real patients anytime soon?
Voice 1: Microsoft says a prospective study at Beth Israel Deaconess Medical Center in Boston will compare the system against physicians' independent assessments, with patients interacting with it directly. It gave no size or timeline for that study.
Voice 2: A controlled benchmark is one thing. An actual hospital is another.
Voice 1: The FDA has its own update this week, about how fast rules will actually come.
Voice 2: Rick Abramson, who directs the FDA's Digital Health Center of Excellence, said on October 20th that the agency has received more than 500 submissions on its generative AI discussion paper. He said, “we intend to draft policy, and it'll come soon,” and called it “a priority at the highest levels of the administration.”
Voice 1: Is there any sense of which devices get drafted first?
Voice 2: A draft on generative AI devices for mental disorders is on the agency's top-tier list for fiscal 2027, and an umbrella draft on risk assessment and evidence generation for these devices is under construction too.
Voice 1: Does the center writing all this actually have the staff to do it?
Voice 2: Not fully. Dr. Michelle Tarver, who directs the Center for Devices and Radiological Health, said the center had about 22% attrition over the past year and a half, including up to 30% of its clinical reviewers. It's since hired more than 65 people.
Voice 1: Soon is doing a lot of work in that sentence.
Voice 2: Sutter Health, meanwhile, put out real usage numbers for its ambient AI.
Voice 1: More than 6,000 of its clinicians have drafted more than 5 million notes using Abridge's ambient AI. Sutter says that's cut documentation time by up to 30%, and that average start-of-care documentation in home health fell from 97 minutes to 55.
Voice 2: Any numbers outside of documentation?
Voice 1: Yes. Ten of its Aidoc radiology and stroke tools had analyzed more than 583,000 studies and flagged more than 43,000 cases for clinician review as of August. About 158,000 patients had used its Emmie patient assistant as of September.
Voice 2: What about remote monitoring, is that showing results too?
Voice 1: Sutter says 90% of patients with stage 2 hypertension in its remote monitoring program, which has more than 14,000 enrollees, reached blood pressure control within six months. All of these are Sutter's own figures, not published in a peer-reviewed study.
Voice 2: Good to know those numbers come from the health system itself, not an outside review.
Voice 1: The FDA approved a new heart valve device this week too.
Voice 2: The Pascal Precision system, from Edwards Lifesciences, now covers transcatheter repair for severe tricuspid regurgitation. It's for patients on optimal medical therapy who are at intermediate or greater surgical risk, and whom a multidisciplinary heart team judges appropriate for the procedure.
Voice 1: What's it actually meant to improve?
Voice 2: Edwards says the aim is better quality of life and functional status, but the release itself gave no outcome figures. One-year results from the CLASP IITR pivotal trial are coming in a late-breaking session at the TCT 2026 meeting.
Voice 1: So the real evidence is still a few weeks away.
Voice 2: Before we go, something to watch.
Voice 1: Comments close October 26th on Colorado's automated-decision and chatbot safety rules.
Voice 2: Is there anything in there specific to health care chatbots?
Voice 1: No, the proposal doesn't call out health care specifically. The watch note just flags that it's worth seeing whether the final rules reach into health care chatbots at all.
Voice 2: A rule written for chatbots in general could end up touching clinical intake bots without ever saying so directly.
Voice 1: Sources and corrections are at physicianintheloop.org, where you can sign up for the free email briefing. This briefing was read by AI voices.