This is a Physician in the Loop special topic.

Free advice

AI will soon give clinicians in much of the world a specialist's knowledge for free. Poor and rural patients will gain only if someone supplies the tests, drugs, specialists and money it calls for.

The most useful thing artificial intelligence will do for medicine in the next few years is to make the clinician who works alone a little less alone. Last month OpenEvidence, a tool that answers doctors' clinical questions from the medical literature, said it was rolling out a free version in about 100 low- and middle-income countries, from Uganda and Sudan to Haiti and Mongolia, with Anthropic providing its AI models, engineers and money. The company says that on an average day, most American doctors use it.

That is a real gain, and it will matter most where specialists are scarce: rural America has about a third as many doctors per person as its cities. But knowledge is the cheapest thing a sick person needs. What rural and poor places lack is what comes after the answer: the scan, the drug, the specialist who will take the referral, the road to reach them and the money to pay for it all. Many lack even electricity: close to a billion people in poorer countries depend on clinics with unreliable power or none. AI will widen access to care only where it is built into those things. Treating it as a substitute for them is the magical thinking that Silicon Valley keeps selling.

Some gains are already measurable. In a trial in Mayo Clinic's primary care practices, teams shown an AI reading of routine electrocardiograms diagnosed nearly a third more patients with a weak heart pump. Young people with diabetes offered an eye exam graded on the spot by AI all completed it; of those sent to an eye doctor, about a fifth did. In Zambia and North Carolina, novices using a cheap ultrasound device with built-in AI dated pregnancies as accurately as trained sonographers. The World Health Organization lets software stand in for human readers of chest X-rays when screening for tuberculosis. Duller technology helps too: when specialists answer primary care doctors' questions in writing, waits fall from months to days or weeks.

Money is following. The Gates Foundation and OpenAI have committed $50 million to bring AI to 1,000 primary care clinics by 2028, starting in Rwanda. OpenEvidence says it will tailor the tool to local guidelines, and Penn Medicine plans to design tools from the ground up with clinicians in Botswana. Patients are not waiting: OpenAI says Americans living more than half an hour's drive from a hospital send ChatGPT more than half a million health messages a week.

The first rigorous tests are sobering. A randomized trial in 16 Kenyan clinics found that an AI assistant built on OpenAI's GPT-4o was safe and helped clinicians reach appropriate diagnoses. It did not reduce the share of patients whose treatment failed. An earlier study at the same chain of clinics, paid for by OpenAI, had found 16% fewer diagnostic errors as judged by reviewing doctors, but no clear difference in whether patients felt better. In Britain, people given chatbots to work through medical scenarios did no better than people left to their own devices, though the chatbots alone usually named the right condition. Studies of OpenEvidence itself are few and small: a review of eleven found its answers generally relevant and backed by evidence, but also that it often reinforced decisions doctors had already made, and a disputed study rated its answers to doctors' real questions no better than Google's AI summaries.

The trouble is what comes after the answer. In the Mayo trial, about half the patients the AI flagged did not get an echocardiogram to confirm it. In the eye trial, more than a third of those with abnormal results had not seen an eye doctor six months later. When Google put its eye-screening AI into clinics in Thailand, a fifth of the images were rejected, slow internet delayed the uploads, and patients whose images failed were told to see a specialist elsewhere on another day. More than half of rural American counties have no hospital that delivers babies, and no chatbot can perform a cesarean section.

None of this dims the optimists. Vinod Khosla, a venture capitalist, predicted in 2012 that computers would replace 80% of what doctors do. Bill Gates said last year that within a decade great medical advice would be free and commonplace. Sam Altman, OpenAI's chief executive, has called ChatGPT a better diagnostician than most doctors in the world, though he said he did not want to entrust his own care to it without a human doctor. Mehmet Oz, who runs Medicare and Medicaid, has said that the best way to help some rural communities will be AI-based avatars; his agency later explained that he meant tools that extend clinicians' reach, not replace them. According to The New York Times, officials have discussed paying AI doctors run by technology companies as much as 60% to 80% of what human doctors earn.

The record should temper them. MD Anderson Cancer Center abandoned a Watson project after spending $62 million, and internal IBM documents cited unsafe and incorrect treatment recommendations from its cancer software. Babylon Health claimed in 2018 that its chatbot beat the pass mark on practice questions for Britain's exam for family doctors, and it ran Rwanda's national telemedicine service, with nearly four million consultations, until the company collapsed in 2023 and the service closed. Care that depends on a company's balance sheet ends when the balance sheet does. In April OpenEvidence itself stopped serving doctors in the European Union and Britain, citing regulatory uncertainty.

There are quieter risks too. Doctors defer to confident software: in a trial in Pakistan, physicians whose AI assistant was wrong in half the cases scored 14 points lower on diagnostic reasoning than those whose assistant was right, despite 20 hours of training in AI. Dermatology software tested on images of diverse skin tones did worse on dark skin. Chatbots can falsely reassure: given written scenarios describing emergencies, ChatGPT's health service under-triaged half, advising some to be seen within a day or two. OpenEvidence is supported by advertising, much of it from drug companies. And cheap software may become the only care that poor and rural patients are offered. "I'd be curious if Dr. Oz would want an avatar treating his own family," said Carrie Henning-Smith, who studies rural health at the University of Minnesota.

The strongest objection is that something beats nothing. Where there is no doctor for a hundred miles, an imperfect assistant is better than none, and insisting on trials first delays help for people who need it now. Where the alternative really is nothing, that is often right, and the Kenyan trial found its assistant safe. But the evidence so far shows that these tools help when they are tied to care, and expert care for places without experts is just what Watson and Babylon promised. Trials need not be slow: the Kenyan one enrolled nearly 10,000 patients in under three months. Patients in rural Uganda and rural Montana deserve the same proof as patients in Boston.

What would let AI widen access is mostly not AI. Governments should finish wiring and powering rural clinics. Medicare, Medicaid and health ministries should pay for, and keep paying for, the links that turn advice into care: specialist advice by message, video visits and the follow-up the software recommends. They should judge AI tools by completed referrals and patients' health rather than by use, and pay for software that acts on its own only after trials show it helps. Companies giving tools away should publish what happens to patients, build for local drugs, tests and weak connections, say how long free will last and keep advertising out of clinical tools. Big health systems should share the work of choosing and checking tools with small hospitals, as a new network in North Carolina means to. And clinicians should argue with the machine, making their own assessment and comparing it with the software's, an approach that helped doctors in trials led by Stanford researchers where simply handing them the tool did not.

A family doctor in Montana and a clinician in Uganda will soon carry a specialist's knowledge in their pockets. The advice is becoming free. The care it calls for still has to be paid for, staffed and built.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.