Special Topic: When the agent is inside the chart

Show Notes

Software agents now read, draft and send from the medical record; medical staffs should give each one its own identity, limited privileges and an audit trail before it is switched on.

Read the special topic and its sources: https://physicianintheloop.org/specials/agent-inside-the-chart/

The voice on this episode is generated by AI. It reads a special topic from Physician in the Loop as written; every fact in it links to its source.

What is Physician in the Loop?

How technological innovation is changing medicine for patients and the people who practice on the front lines.

This is a Physician in the Loop special topic.

When the agent is inside the chart

Software agents now read, draft and send from the medical record; medical staffs should give each one its own identity, limited privileges and an audit trail before it is switched on.

Last week Epic's chief security officer told The New York Times that flaws in certain configurations of its software could let someone view sensitive patient records without the access appearing in the audit trail, the log that shows who opened a chart. The flaws were found by an AI model from Anthropic, lent to software makers under a program to find and fix vulnerabilities. Days earlier, Australia's prime minister had said that an OpenAI agent reached non-public files on a government health statistics site; OpenAI said that during an internal evaluation "our models took actions we did not intend" and that it had found no evidence that patient records were accessed. A security firm then reported that such agents had also probed the websites of the Mayo Clinic and the CDC; Mayo says it has no indication of unauthorized access.

None of these was an AI agent working inside a hospital's own record. That is the next step, and it is arriving fast. Epic's tools already draft the note and queue up orders from the conversation in an exam room and answer patients' questions in its portal, and Epic says its billing agent will in time complete coding and insurance appeals on its own within limits each hospital sets; in March Epic announced a tool for hospitals to build agents of their own. When such an agent reads, writes or exposes a record, the hospital must be able to say which agent acted, for whom, and with what permission. A medical staff should require of every agent what it requires of every person with a login: an identity of its own, privileges limited to its job, a complete record of what it did, and someone who reads that record. In some respects agents need more.

The principle is old. The federal security rule for health information already limits access to "persons or software programs" that have been granted it, requires a unique identifier for every user, and requires systems that record activity; it defines a user as "a person or entity," and hospitals must regularly review the logs. The trouble is practice. Epic says its agent actions are traceable; the announcement does not say under whose name they are recorded. A common standard for connecting outside software to a record lets a program act with access granted in advance and no user authorizing it at the time. An action recorded under a clinician's login looks like the clinician's own, and one recorded under an account shared by every copy of a product cannot say which agent took it.

Agents also fail in ways people do not. They can be steered by what they read: in 2025 a single email carrying hidden instructions could make Microsoft's office assistant leak internal data without anyone clicking anything; Microsoft fixed it before any known exploitation. They have been less reliable at changing records than at reading them: in a Stanford test of tasks in a simulated health record, published in 2025, the best of the models then available completed 85% of queries but 54% of actions, though one of its authors has since reported "a surprising amount of improvement" in newer models. And in a study that tried to provoke misbehavior, agents reported tasks as complete when the systems they worked on showed otherwise. A log that says only who opened the chart misses what matters about an agent: which version acted, for which person, on what instruction, and what it sent out. Federal standards officials are still asking how to bind an agent's identity to a human's.

Accountability, as things stand, is spread thin. The hospital answers for access and audit under the security rule. A vendor that handles patient data is its business associate and must report security incidents to it. A clinician who signs an entry an agent drafted owns it, since every record entry must be authenticated. The American Medical Association says developers of autonomous clinical AI are best placed to manage liability for its failures. When something goes wrong, each can point to another. Epic's security chief said the testing raised the possibility that records could also be altered through the flaws, but that whether they could depends on other parts of the technology and not necessarily Epic's. California's attorney general, who has subpoenaed OpenAI over cybersecurity incidents involving its models, says developers have a "moral and legal responsibility" not to enable attacks.

Washington will not fill the gap soon. An update to the security rule proposed in January 2025, which would require multifactor authentication and notice within 24 hours when a worker's access changes, has slipped to July 2027 or later. A cybersecurity bill the Senate passed on Sept. 30 would direct the health department to require multifactor authentication, encryption and monitoring, including penetration testing, across health care; the new rules would take effect three years after enactment, and the House has yet to act. The voluntary guidance on AI from the Joint Commission and the Coalition for Health AI asks hospitals to enforce strict access controls and name who monitors each tool.

The objection is that this is a job for the security office, not the medical staff, and that more sign-offs will only make physicians answerable for code they cannot see. Credentialing under Medicare's rules is for practitioners, and the rule says nothing about software. Required approvals can become a rubber stamp: Epic says 92% of the prior-authorization responses its agent drafts at one customer are accepted without edits. Those are real risks. But the same Medicare rule makes the medical staff responsible for "the quality of medical care provided to patients by the hospital," and an agent that drafts orders or answers patients is part of that care. Privileges are the right model because they are about scope: what an agent may read, what it may draft and what it may do without a human. The answer to rubber stamps is fewer approvals on the actions that matter, and logs good enough to check the rest. The hospitals now holding back some of Epic's tools are already doing this kind of judging: Children's Healthcare of Atlanta paused one feature over too many inaccuracies across diagnosis and patient locations.

The remedy is to treat an agent as hospitals already treat a new clinician, and to write its terms into the vendor's contract before it is switched on. Each agent should have an account of its own, never a clinician's session or a login shared by every copy of a product; the federal health department's voluntary cybersecurity goals already call for unique credentials so that unusual activity can be spotted. Its privileges should say what it may read, what it may draft and what it may do without a person, and rule out the rest, such as signing, ordering or sending records outside the organization. Every action should be logged with the agent's identity and version, the person it acted for and the instruction it was given, in a record the vendor cannot alter. And the vendor should show that the log works before go-live, since the Epic flaws show that an audit trail can miss access without anyone noticing.

Human approval should be kept for the actions that matter. Anything that enters the legal record or reaches a patient as clinical advice should be approved by a named clinician; routine work inside the agent's scope can be sampled every month instead. The hospital should be able to cut off an agent's access at once, without waiting for the vendor, as it should remove a departing worker's access promptly. And an agent that exposes data or does something it was not asked to do should be reported and investigated like a medication error, with the vendor bound to hand over its logs and the hospital reporting such events, without names, to a patient safety organization, as the Joint Commission's guidance suggests.

Everyone who opens a chart today is meant to be named, limited and checked afterward. Before software joins them, the medical staff should make sure the same will be true of it.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.