Special Topic: The patient's AI got there first

Show Notes

A chatbot's differential or a smartwatch alert belongs in the history, as a record of what worried the patient; physicians should treat it neither as a diagnosis nor as a threat.

Read the special topic and its sources: https://physicianintheloop.org/specials/patients-ai-got-there-first/

The voice on this episode is generated by AI. It reads a special topic from Physician in the Loop as written; every fact in it links to its source.

What is Physician in the Loop?

How technological innovation is changing medicine for patients and the people who practice on the front lines.

This is a Physician in the Loop special topic.

The patient's AI got there first

A chatbot's differential or a smartwatch alert belongs in the history, as a record of what worried the patient; physicians should treat it neither as a diagnosis nor as a threat.

On Oct. 5 a randomized trial from Guangzhou, China, reported that a language model agent took better histories than ophthalmology residents. Before their consultations at a tertiary eye hospital, 172 patients with non-emergency appointments were interviewed either by the agent, which was guided by question trees built around symptoms, or by a resident. The agent's histories scored an average of 17.7 points higher, after adjustment, on a 100-point quality scale, and its interviews ran a median of 11.2 minutes against the residents' 3.1. The authors propose a division of labor in which software gathers the history and clinicians concentrate on the examination and on putting the findings together.

The AI that patients bring to the clinic is different. It is a general chatbot that the patient questioned alone, or a watch worn day and night, and it arrives as a list of possible diagnoses or a screen of numbers. About a third of American adults turned to an AI chatbot for health information in the past year, and a quarter say they use one to work out what is causing their symptoms. The physician has a short visit in which to respond, and the right response is to take what the patient brings as history: a record of what the patient noticed, asked and fears. It is not a diagnosis to be confirmed or refuted, and it is not a challenge to be fended off.

It is not a diagnosis, because a chatbot's answer depends on what the patient told it. In a preregistered Oxford study of 1,298 people, language models working alone through ten written medical scenarios named a relevant condition in 94.9% of cases. Participants randomly assigned to use the same models named one in fewer than 34.5% of cases, often because they gave the models too little information, and those left to use whatever they usually would, mostly an internet search, did better. Urgency is harder still. When researchers at Mount Sinai tested ChatGPT Health with clinical vignettes in January, it sent textbook emergencies such as stroke to the emergency department, but in 33 of 64 responses about diabetic ketoacidosis and an asthma attack with impending respiratory failure it advised being seen within 24 to 48 hours instead. Its advice in borderline cases also shifted, mostly toward less urgent care, when a relative or friend in the vignette played down the symptoms. OpenAI said the test did not reflect typical use; at launch it had said the product is not intended for diagnosis or treatment.

Nor is it a threat. Patients brought printouts from the internet long before chatbots. In a national survey of 1,050 American physicians conducted in 2000 and 2001, 85% had seen a patient bring internet information to a visit; in their judgment it helped care when it was accurate and relevant and harmed it when it was not. Most physicians accept the newer version: in an AMA survey this year, 70% called patients' use of general-purpose AI for health information positive or neutral when used appropriately. Patients' reasons are ordinary: in KFF's poll two-thirds of users wanted quick information, and about one in five cited the cost of seeing a clinician or the difficulty of getting an appointment. About four in ten who used AI for physical health did not follow up with a doctor or other provider. The patient who brings a chatbot's list to the visit is one who did.

Treating it as history puts it where physicians already know what to do with it. A patient's account of symptoms is incomplete, colored by worry and sometimes wrong, and physicians still take it seriously, test it against the examination and write it down. A chatbot's differential is the same kind of evidence. It shows what the patient described, which possibilities are now on the patient's mind and which of them the patient fears. Many patients will not offer it unprompted: 29% of physicians in the AMA survey said they had not had a single patient disclose using AI, although about a third of adults use it for health.

Time is the practical objection. In 21 million primary care visits in 2017, the physician's time with the patient averaged 18 minutes, and in an older study of videotaped visits by elderly patients, the longest topic got about five minutes and each of the others about a minute. Taking the patient's research as history should cost less time than rebutting it item by item. Asking what the patient read and what worries them is also a way of drawing out the patient's agenda, which clinicians did in 36% of recorded encounters in one study; in two-thirds of those, they interrupted the patient after a median of 11 seconds.

Wearable data need one adjustment. They are measurements rather than a story, and they should be read like the result of a screening test. In 2024, 41% of American adults used a wearable device. Sleep scores, heart rate variability and similar outputs can be sold as wellness features, which the FDA, under guidance revised in January, does not intend to examine as medical devices when they are low risk and meant only for wellness. The few features that are cleared have modest numbers. Apple's hypertension notification, cleared in September 2025, detected 41.2% of people whose home blood pressure averaged 130/80 or higher in the company's validation study and stayed silent for 92.3% of those below it; with a negative likelihood ratio of 0.64, a silent watch only modestly lowers the chance of hypertension. Abnormal readings also generate work. Of 264 patients whom Mayo Clinic evaluated for an abnormal pulse detected by an Apple Watch, 30, or 11.4%, received a clinically actionable cardiovascular diagnosis.

The strongest objection is that calling this history undersells it, because the software is sometimes the better diagnostician. In a secondary analysis of a randomized trial published in 2024, GPT-4 working alone scored 16 percentage points higher on diagnostic reasoning than physicians using conventional resources, and in the trial itself physicians given the model did no better than those without it. In Guangzhou, in an analysis the authors call exploratory, the agent's own diagnoses from the history alone beat the residents'. But those models had complete written cases or asked their own structured questions. A patient's chatbot has only what the patient typed, which is one reason the Oxford participants did poorly with models that did well alone; the other was that they often failed to carry the models' correct suggestions into their own answers. And in Guangzhou, adding the eye findings helped the residents more than the agent, significantly narrowing the gap. Taking the output as history does not mean discounting it. A diagnosis on a patient's printout deserves the same consideration as any other possibility the history raises, and sometimes it will be the right one.

None of this takes long. Early in the visit the physician should ask every patient whether they have looked the problem up, asked a chatbot or brought readings from a device, since many will not say so otherwise. What the patient typed shows what the software was told, and the possibility that worries the patient most is what the visit has to answer. Red flags come before the list, because a printout that reassures is no reason to stop looking. Device data should be read like a screening result, with two questions: which feature produced the reading, and whether it is cleared for that use. Anything that matters should be confirmed with a validated measurement such as a home blood pressure cuff, an ECG or a sleep study. A wellness score diagnoses nothing, and a silent watch rules nothing out.

Then the list deserves an answer, however brief: why each serious possibility does or does not fit, and why a test will not be ordered. Copic, a liability insurer, suggests telling patients that AI tools "do not account for downstream harms of unnecessary testing." The reasoning belongs in the chart in neutral terms, such as "patient inquired about." And the patient should leave able to say the plan back, the teach-back method, knowing what would warrant coming back sooner and invited to bring the whole chatbot conversation next time rather than just its conclusion.

In Guangzhou the authors would have software ask the questions and leave the examination and the synthesis to clinicians. A patient's chatbot attempts the synthesis with nothing but what the patient typed. Physicians should take its work as part of the history, examine the patient and do the synthesis themselves.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.