This is a Physician in the Loop special topic.

Practicing without a license

Software can do much of an office visit, but the law cannot yet hold it responsible. States should license its operators, with a doctor reviewing its work until trials prove it safe.

On September 28 McKinsey, a consultancy, estimated that AI can already do the work behind about a fifth of America's outpatient claims, two billion to three billion in 2024. Most were office visits at which little else was done; the rest were for reading scans and tests. The next day Robert F. Kennedy Jr., the health secretary, told a summit in Washington that AI offers "a second opinion that is much better informed than any doctor in the country." California's governor, meanwhile, had until September 30 to act on a bill that would stop hospitals and clinics from using AI to replace a licensed professional's clinical judgment.

McKinsey's figure measures what software can do, and says little about what the law lets it do, which is where the work will stall. Every one of those claims still needs a licensed person behind it, because American law has no settled way to hold a program responsible for a diagnosis. Licensing the software itself would not fix that, and banning it would throw away real gains. States should instead license the organizations that run clinical AI, make them answer for its mistakes, and keep a doctor reviewing its decisions until controlled trials show it can safely do without one.

The report is more careful than its headline. It counts only what it calls "discursive care," the reasoning and conversation in medicine that need no hands-on examination, and its authors write that "AI won't replace clinicians." Among the signals of progress they list is growth in "the number of states that allow AI systems to authorize clinical decision-making without physician intervention." By that yardstick progress is close to nil. Utah, which has gone furthest, lets two companies' AI renew prescriptions, but both began with a clinician checking every renewal before it reached the pharmacy, and the state has not reported that either has moved past that stage.

The software is plainly getting good. In a randomized, blinded study published in Nature in June, a Google system's plans for managing simulated patients were judged appropriate in 95 to 98 percent of cases, against 72 to 81 percent for primary care doctors' plans, though its authors said it was "not ready for real-world translation." How much care can shift to software without harming patients is, as McKinsey's authors concede, still to be studied.

The first obstacle is the license. Only licensed people may diagnose and treat, and the leaders of the Federation of State Medical Boards wrote in August that AI "is not ready to be independently licensed like a physician" and that boards "do not regulate machines or tools and are not set up to do so." A House bill that would let AI prescribe, where a state allowed it and the Food and Drug Administration had authorized it, has sat in committee since January 2025. Meanwhile a man who says ChatGPT's advice nearly killed him is suing OpenAI, claiming the chatbot practiced medicine without a license; his lawyer says it "acted like a medical authority while having none of the responsibility."

The second obstacle is the FDA, which treats diagnostic software as a medical device and clears it one narrow task at a time: more than 1,500 AI devices so far, each doing something like spotting a tumor or a clot. It has let software make a screening decision on its own, for diabetic eye disease, since 2018, and the first cleared device that talks with patients through a language model, by its maker's account, carries out a treatment plan a clinician wrote. The agency has no rules designed for chatbots or AI agents that could prescribe, according to the New York Times. And federal law rewards keeping a doctor in the loop: decision-support software escapes device rules only if a clinician can independently review the basis for its recommendations.

The third obstacle is money. Medicare has paid for an autonomous AI eye exam since 2022, but it has no way to pay for a visit that software conducts on its own. Officials are working on a payment category for AI software that supports care or diagnosis, the Times reported on September 14, and have discussed whether AI physicians run by technology companies should be paid as much as 60 to 80 percent of what human doctors earn for the same service; nothing of the kind has been formally proposed. The software category Medicare has proposed, which it calls "software as a medical service," would pay for algorithms that analyze scans and tests, not for programs that talk to patients. The money, like the license, runs through a clinician.

The fourth obstacle, on which the others turn, is liability. The American Medical Association calls the question "novel and complex" and says developers of autonomous systems "must accept this liability," backed by malpractice insurance; the state boards say physicians should remain accountable for harm from inappropriate reliance on AI. At the summit Mr. Kennedy said that Sam Altman, OpenAI's boss, had told him it would now be "malpractice" to diagnose or prescribe without consulting AI, and JD Vance, the vice president, said that "if you create a product that harms people, you should suffer consequences for it." Doctors may soon be liable both for ignoring the machine and for trusting it.

The result is an industry that borrows doctors' licenses. At one new $39-a-month service, a licensed physician makes "every prescription, note and clinical decision" while AI supports users between sessions. That fits the law as it stands, but it puts the risk in the wrong place. As the machine does more of the work, the physician's signature makes the claim payable and makes the physician the defendant when something goes wrong, for decisions the physician did not make. Paying for such care per visit, at a discount to a doctor's fee, would reward volume and make it worse.

The strongest objection is that licenses, supervising doctors and trials would mainly protect doctors' jobs and fees, at the expense of patients who cannot see a doctor at all. An estimated 92 million Americans live where primary care is scarce, and many who turn to AI with medical questions say they could not or would not pay a doctor, or could not reach one. For them, the argument runs, every new rule means a longer wait for help that cheap software could give today. But patients already have fast, cheap AI, from chatbots that answer to no medical board and disclaim responsibility; OpenAI's reply to the lawsuit was that "ChatGPT is not a doctor." What patients lack is recourse, and a licensed operator would give them someone to answer for mistakes. Requiring proof first protects them too: the people with nowhere else to turn are the ones most exposed when untested software fails.

States should create a license for the operators of clinical AI, much as they license pharmacies. The organization, not the software, would hold it; a named physician would direct it; it would carry malpractice insurance, report its results to the medical board and the public, and take on more work in stages. Utah's Doctronic pilot has the right shape but too low a bar: a physician checks every renewal in a group of drugs until 250 have been filled, after which, if the company meets agreed benchmarks and the state approves, the AI may send renewals straight to the pharmacist while the company checks one in ten. A few hundred checked renewals say little about the rare error that does real harm; measuring that takes thousands of cases, and comparing the software with doctors takes a controlled trial. And Utah's medical board said it learned of the pilot only once it was running, and asked for it to be suspended. Boards should set the stages, and the evidence each one requires, from the start.

Washington's part is to demand proof before software treats patients outside a trial, as it does before a drug reaches the pharmacy. It is heading the other way. Under a pilot called TEMPO, the FDA intends not to enforce requirements such as premarket authorization, or its rules for trials of unproven devices, for a handful of digital health products used in a Medicare program, among them an AI voice agent that delivers therapy for depression and anxiety. The agency weighed their risks before choosing them, they are meant to be used alongside care a clinician supervises, patients are told they are in a pilot, and their makers must report data as patients use them. But the FDA says it has not yet evaluated whether they work. That puts the test after the risk: any harm the agency did not foresee will be found in the patients who suffer it.

The heart-failure agents that a federal research agency is paying three companies to build are a better model: Kaiser Permanente is to test them in randomized trials. Medicare should pay for AI-run care only once it has proved itself, and then for results rather than per machine visit.

The health secretary's own family has a stake in how soon that happens. Last year one of Mr. Kennedy's sons, Finn, then at 8VC, a venture-capital firm, co-wrote essays arguing that AI doctors "won't work for free" and that Medicare should be allowed to pay "FDA-approved autonomous AI providers." He also set out a federal overhaul that included, the New York Times reported, a person at the FDA focused specifically on AI; on September 8 the health department created such a post. This year he started a venture fund. It was raising $100 million for young companies in health-care AI and consumer health, the Financial Times reported in April, and pitched itself to investors as poised to benefit from policy shifts tied to his father's movement. The department has said the secretary complies with all ethics and confidentiality requirements. Even so, his son's fund and the companies it backs stand to gain from any rule that lets software practice and bill sooner. That is one more reason to set the standard of proof in public, before the software reaches patients.

Doctors need not wait for Washington. They should refuse to sign for decisions they cannot review, press their medical staffs and boards to define supervision by what a physician can actually check, and seek the medical-director roles these licenses would create. The software may well be, as Mr. Kennedy says, better informed than any doctor in the country. A license answers a different question: who is to blame when it is wrong. Until the law names someone else, the answer will be whichever doctor signed.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.