Special Topic: Count the cases

Show Notes

Chatbot-linked psychosis appears rare, but hospital counts depend on what clinicians happened to write down. Physicians should ask about chatbots in every new psychosis or mania, and record the answer the same way.

Read the special topic and its sources: https://physicianintheloop.org/specials/count-the-cases/

The voice on this episode is generated by AI. It reads a special topic from Physician in the Loop as written; every fact in it links to its source.

What is Physician in the Loop?

How technological innovation is changing medicine for patients and the people who practice on the front lines.

This is a Physician in the Loop special topic.

Count the cases

Chatbot-linked psychosis appears rare, but hospital counts depend on what clinicians happened to write down. Physicians should ask about chatbots in every new psychosis or mania, and record the answer the same way.

Researchers at Vanderbilt University Medical Center have searched their own records for psychosis linked to chatbots, and found few cases. In a study published in JAMA Psychiatry on Oct. 7, they classified 28 of more than 215,000 patients in mental health care as cases of what has come to be called AI psychosis, about one in 7,700. Most of them were in their first episode of psychosis.

The figure is softer than it looks. The team found its cases by searching clinicians' notes for words related to artificial intelligence, so a patient could be counted only if someone had written a chatbot into the record, and only patients with a recorded psychotic disorder were kept. The authors list ascertainment bias, a single site and a rating system that is not clinically validated among the study's limitations, and they state that their design "cannot establish causation." The remedy is to count on purpose. Every patient who presents with a new psychosis or mania should be asked about chatbot use, and the answer recorded the same way in every hospital. Psychiatry should agree on what counts as a case, so that regulators can work from clinical counts rather than from anecdotes and company estimates.

The detail of this single-center review says more than its headline figure. Raters assigned the chatbot's part in each case to one of four roles set in advance, or to none. In nearly two-thirds of the cases they judged that it had amplified symptoms the patient already had, by validating distorted ideas; in most of the rest they judged it the object of the delusion. Patients classified with AI psychosis were far more often in a first episode than those whose notes recorded only neutral chatbot use, such as looking up recipes. All but four of the cases were documented after OpenAI released its GPT-4o model in May 2024, and the time since then makes up more than half of the study period. The authors note, with the same caution about cause, that the high number of cases after the release coincides with documented sycophancy in that model. In April 2025 OpenAI withdrew an update to the model that it said was overly flattering or agreeable.

A second record study points the same way, with the same weakness. A Danish study published in February screened the notes of nearly 54,000 patients in one region's psychiatric services for the words chatbot or ChatGPT, and found 38 whose notes were compatible with potentially harmful consequences, most often involving delusions. Its authors note that patients were not systematically questioned about chatbot use, that the results do not speak to incidence, and that the notes are "by no means" evidence of a causal effect. An experiment by researchers at Columbia University, reported in JAMA Psychiatry in March, found high rates of inappropriate or partly appropriate responses when three versions of ChatGPT were given prompts with psychotic content in August 2025, though the paid version did better than the free one most users interact with. That makes a mechanism plausible, though test prompts are not patients.

OpenAI has published an estimate of its own. Its initial analysis, released in October 2025, estimated that about seven in 10,000 users active in a given week show possible signs of a mental health emergency related to psychosis or mania. The company warns that such conversations are "difficult to detect and measure given how rare they are," and that small differences in how they are measured can change its numbers significantly. The Vanderbilt authors write that their estimate converges with the company's. Yet the two measure different things: one counts patients whose notes tie a chatbot to a psychotic episode, the other counts users whose conversations the company's own analysis flags. Neither says how many people a chatbot has made ill. The company said at the time that an update to its default model gave undesired responses in conversations about mental health less often, by its own measures.

Regulators are acting in the meantime. Among the draft guidance it proposes for fiscal 2027, the Food and Drug Administration lists recommendations on evidence for conversational AI devices for mental disorders, which would apply to chatbots sold as medical devices. The Federal Trade Commission has ordered seven companies, OpenAI among them, to explain how they measure, test and monitor the potential harms of chatbots that act as companions, with a focus on children and teenagers. Both agencies will depend largely on evidence the companies produce, which clinical counts, made the same way across hospitals, could check.

Advice to ask about chatbots is spreading. The Vanderbilt authors write that routine assessment of AI use during psychiatric encounters "appears warranted," and the authors of the Columbia study write in their manuscript that clinicians should routinely screen patients, particularly those with psychotic disorders, for chatbot use. A paper in JAMA Psychiatry in April argued that mental health clinicians should ask about it as they ask about sleep, diet, exercise and alcohol. But advice to ask produces impressions, not counts, unless the same question is put to every patient with new psychosis or mania and the answer is recorded in the same form.

Skeptics will say this treats a coincidence as an exposure. People who are becoming psychotic may turn to chatbots, and use them intensely, because they are already ill. Karthik Sarma, a psychiatry resident at UC San Francisco, calls the relationship a "chicken and egg" problem: heavy use could be a symptom of psychosis, as a patient becoming manic may take many showers without the showers causing the mania. On this view the numbers are too small to act on, and one more question will lengthen assessments that are already crowded. A paper posted in August, weighing whether AI psychosis should count as a distinct condition, warns that a formal label risks turning anecdotes into a syndrome too soon.

The skeptics describe the evidence fairly, and their point is the best argument for counting. Vanderbilt's data cut both ways. The raters judged that a chatbot had set off new symptoms in someone previously healthy in only three cases, and another 28 patients had delusions about AI without any documented chatbot use. Yet the role the raters assigned most often, amplifying symptoms already present, would matter if confirmed, even if chatbots never caused a psychosis. Asking every patient with new psychosis or mania, whatever the delusions are about, would show how often patients report heavy use before their first symptoms and, with follow-up, whether episodes are longer or worse in patients who used chatbots heavily. One more question, in assessments that already cover sleep and alcohol, is a small price for a number regulators can use.

The question belongs wherever new psychosis and mania arrive, from the emergency department to the primary care office. It can be short: whether the patient uses chatbots, how much, about what, and whether the use grew before the symptoms began. Health systems and their medical staffs should give the answer a structured place in the record, so that it can be counted without the keyword searches both record studies needed. The quickest large count would come from the National Institute of Mental Health's network of early psychosis clinics, which gives newly enrolled patients a common battery of standardized measures. The institute should make sure that battery asks about chatbots, since first episodes are where the Vanderbilt cases clustered.

The definition is the specialty's job. The American Psychiatric Association's resource document on AI, approved in July, already lists reinforcing delusions among the risks of social AI for young people. The association should convene psychiatrists, psychologists and emergency physicians to agree what counts as a case: how much use, how close in time to the symptoms, and what part the chatbot played. The Vanderbilt raters judged the chatbot's part against four categories chosen in advance, in a rating system the authors say is not clinically validated. A definition for counting need not be a diagnosis: the association's diagnostic manual places internet gaming disorder in a section for conditions that need further research, with criteria proposed for study. Shared criteria would let one hospital's count be added to another's and give regulators a clinical check on what the companies report.

Vanderbilt's 28 cases may be close to the true number or far from it, and a keyword search of old notes cannot say which. A question asked of every patient with new psychosis or mania, and a definition every hospital shares, would get much closer.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.