Wrong automated advice cuts clinicians' accuracy, meta-analysis finds
Wrong automated advice cuts clinicians' accuracy, meta-analysis finds
The voices on this episode are generated, and so is the script, from the day's post at Physician in the Loop. Every fact in the post links to its source.
Clinicians had about 10 times the odds of a correct decision when automated advice was right than when it was wrong, a meta-analysis of 46 reports published Oct. 23 in npj Digital Medicine found.
In four real-world studies of computerized electrocardiogram readings, 26.6% of erroneous machine interpretations went uncorrected and unrecognized, the authors reported. Studies of current AI and large language models documented harmful reliance on wrong output but were too varied to pool, and only one study reported harm at the patient level. Human oversight is protective "only if clinicians recognize and resist erroneous output," wrote the authors, from West China Hospital at Sichuan University.
Elsewhere, Quest Diagnostics told regulators an unauthorized party reached patient records through the Quanum system clinicians use to order lab tests, and the Department of Health and Human Services moved to keep its workforce on Anthropic's Claude without a competitive bid.
In this episode
How technological innovation is changing medicine for patients and the people who practice on the front lines.
Voice 1: This is Physician in the Loop, the daily briefing on AI in medicine. It's Saturday, October 24th, 2026.
Voice 1: Clinicians had about 10 times the odds of making the correct decision when automated advice was right than when it was wrong, according to a meta-analysis of 46 reports published Thursday in npj Digital Medicine.
Voice 2: Forty-six reports is a lot to pool. What did they actually cover?
Voice 1: They covered 41 independent groups of clinicians. In three adjusted cohorts, the odds of a correct decision were about 10 times higher with correct advice than with incorrect advice, and a sensitivity analysis limited to studies from 2020 onward found a similar but less precise result.
Voice 2: Did they look at whether clinicians actually catch the machine when it's wrong?
Voice 1: In four real-world studies of computerized electrocardiogram readings, about 27% of erroneous machine interpretations went neither corrected nor recognized, with wide variation among the studies.
Voice 2: And how about current AI and large language models, not just older ECG software?
Voice 1: Studies there documented harmful reliance on wrong output, but they were too varied to pool. Only one study reported harm at the patient level.
Voice 2: The authors didn't pull their punches on what that means for oversight.
Voice 1: No. They wrote that human oversight is protective “only if clinicians recognize and resist erroneous output,” and that it should be evaluated as an error-interception capability rather than assumed to ensure safety. The authors are from West China Hospital at Sichuan University.
Voice 2: Quest Diagnostics had a security problem this week, not a research one.
Voice 1: What happened there?
Voice 2: Quest told the Securities and Exchange Commission an unauthorized party reached patient records through Quanum, the system certain clients use to place lab orders and look up results. It learned of the access on October 21st and filed the disclosure the next day.
Voice 1: Do they know how many records were touched?
Voice 2: They haven't said. Quest paused access to the system, notified federal law enforcement, and hired outside cybersecurity experts. The company says the incident hasn't materially disrupted its operations or its ability to serve patients.
Voice 1: A pause on the ordering system is still a real disruption for whoever was trying to use it that day, even if the company says the overall operation wasn't hit.
Voice 2: The federal government made its own AI move this week, with Anthropic.
Voice 1: What's HHS doing exactly?
Voice 2: The Department of Health and Human Services posted a notice of intent on October 19th for what it calls a limited continuity of services award, to keep its workforce on Anthropic's Claude without a competitive bid. It says the capability “is specific to one provider.”
Voice 1: Is there a dollar figure attached?
Voice 2: No value is listed. The notice covers chat, drafting, summarization, data analysis, and administrative controls, and says HHS data is excluded from model training by default and stays in the continental United States. HHS rolled out Claude across the department in December 2025, and Anthropic's discounted government-wide deal ends October 31st.
Voice 1: So the timing is the deadline, not a new evaluation.
Voice 2: That's the question Jessica Tillipman is raising. She's a procurement law scholar at George Washington University, and she's questioned whether the notice actually justifies skipping competition.
Voice 1: A regulator overseas took a harder look at AI scribes than anyone here has so far.
Voice 2: What did it find?
Voice 1: Australia's Therapeutic Goods Administration surveyed about 100 digital scribe suppliers, got 40 responses, and reviewed 12 products, including those with the largest market shares. It found eight of the 12 likely meet the definition of a medical device, and none of their makers had said so.
Voice 2: Did they name which eight?
Voice 1: No, and the agency didn't test safety or performance either. The 12 products it reviewed included Heidi, Lyrebird Health, and Microsoft's scribe.
Voice 2: What's the agency's actual position on the disclaimers these companies put in their marketing?
Voice 1: It said disclaimers are “not sufficient” to settle whether a product is a device, and it plans targeted compliance activity.
Voice 2: So a scribe calling itself a note-taking convenience doesn't get to skip the device question just by saying so in the fine print.
Voice 1: Before we go, something to watch. Palmetto GBA holds an open meeting on October 26th on proposed noncoverage of AI brain MRI quantification, a web conference on the proposed local coverage determination for Alabama, Georgia, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia.
Voice 2: Noncoverage, meaning Medicare wouldn't pay for it in those states?
Voice 1: That's what the proposal is weighing, for that specific set of states. The meeting itself is a hearing, not a final decision.
Voice 2: Worth watching whether that reasoning stays contained to brain MRI quantification, or starts shaping how other AI reads get paid for.
Voice 1: Sources and corrections are at physicianintheloop.org, where you can sign up for the free email briefing. This briefing was read by AI voices.