This is a Physician in the Loop special topic.

An alarm needs an answer

Epic's deterioration score has FDA clearance. It belongs only on wards where a staffed team answers its top alerts, the medical staff sets its thresholds and the hospital counts whom it helps and misses.

The Food and Drug Administration has cleared the deterioration score that Epic builds into its health record. Hospitals have used the index for years; on Sept. 18 the agency found its second version substantially equivalent to a competing score, eCART. The software reads 195 items in a patient's record and scores adults on hospital wards or under observation for the risk of needing a rapid response team, a code or a move to intensive care within a day, or of dying within three days. Each hospital sets its own alert thresholds, and the clearance summary leaves that choice to trained professionals at each organization, adding that the score is not to be the sole basis for decisions that may cause death or an irreversible decline in health.

That leaves the two decisions that matter most to each hospital: where to draw the line, and who answers when a patient crosses it. Where such scores have been tied to fewer deaths, the gains came with people assigned to answer the alerts, in studies that could not rule out other causes; in randomized trials, alerts sent to a nurse manager, or even to a nurse with no other duties, did not cut deaths. A deterioration score belongs on a ward only where a staffed team receives its most urgent alerts directly and the hospital measures what that team achieves, and the medical staff, not the information technology department, should set its thresholds and check whom it misses.

Two randomized trials in St. Louis, published in 2013 and 2014 with a different, older model, show the limits of an alert. In the first, a text page to the ward's nurse manager whenever a patient crossed the alert threshold made no difference to how many were moved to intensive care or died. In the second, the alerts went instead to the rapid response team's nurse, a post with no other duties, who was to respond to each one within 20 minutes. Transfers and deaths still did not fall, though patients left the hospital about a day sooner. That trial was small and was designed to detect fewer transfers to intensive care, not fewer deaths.

The programs tied to fewer deaths built more around the alert, though the studies of them are observational. Kaiser Permanente's own program scanned the records of its Northern California hospitals every hour and sent each alert to a centralized team of specially trained nurses. They reviewed the chart off the ward, sparing bedside staff unnecessary interruptions, and called the unit's rapid response team when a patient needed to be seen; that team made a structured assessment and worked with the patient's physician. In a study by Kaiser's own researchers, about 16% of hospital stays with an alert were followed by a death within a month, against 20% of comparison stays that would have set one off had the system been running; adjusted for differences between the groups, the risk of death was about a sixth lower.

RWJBarnabas Health in New Jersey used Epic's index itself. At 11 of its hospitals, alerts at the highest-risk tier went automatically to rapid response teams unless rules suppressed them, and after the change fewer than half of high-risk stays produced a notification. In a before-and-after comparison by the system's own doctors and researchers at Rutgers, deaths among high-risk patients fell from about 23% to under 19%. The researchers credited a combination of factors, including staff education, the alerts and the notifications. Stephen O'Mahony, the system's chief medical information officer and the study's senior author, said the study had combined Rutgers' rigor with the system's reach: "The mortality benefit was not produced by an algorithm but by the partnership around the algorithm."

Stanford University Hospital used Epic's index differently, in an earlier version than the one now cleared. When a patient's score crossed 65, the alert went to the nurse and the physician at once, starting a structured huddle with a checklist. In a retrospective comparison of patients just above and just below that line, who differ little except in whether an alert fired, those above it were about 10 percentage points less likely to need a rapid response team, a move to intensive care or a code, though deaths did not fall.

Where the line sits decides how often a team is called, and Epic's own test on past records, submitted to the FDA, shows the trade. At the score's top threshold, about one flagged score in three was followed by a deterioration. Moving the line down to the lower threshold tripled the number of scores flagged, and the added scores, those between the two lines, were followed by a deterioration only about one time in eight. A lower line catches more of the patients who are about to decline, and calls the team far more often for patients who are not.

The line also decides whom the score misses. Pooled across Epic's test sites, its sensitivity at the lower threshold was about half for white patients, a third for Hispanic patients and a quarter for Black patients. In a site-by-site analysis the gap between Black and white patients largely closed. The bigger difference lay between the sites, and the one that supplied most of the data on Black patients was where the score caught the smallest share of deteriorations, in patients of every race.

The hardest objection comes from hospitals with the least to give: a staffed response asks the most of them, for a benefit the randomized trials did not find. A small hospital has no rapid response team to spare, and an imperfect warning on a nurse's screen seems better than none. The studies that found fewer deaths are observational, and answering costs staff: at RWJBarnabas, rapid response calls among high-risk patients rose by about half. The second St. Louis trial is the best evidence for this view, and it narrows the case: a staffed response has not been shown to be enough. But it gives no reason to prefer a bare alert, which changed nothing in the first trial, and it was not designed to detect a difference in deaths. Kaiser's program also put more around the alert: nurses off the ward screened every one before anyone on the ward was called, and a structured assessment on the ward brought in the patient's physician. Whether that made the difference is unproven, which is why each hospital should measure what its own response achieves. Nor must the answer sit on every ward: Kaiser's 21 hospitals shared a central team of nurses, and at Stanford the alert went to the patient's own nurse and physician, arrangements a small hospital could share or copy.

The principle that matters most is that every top-tier alert has an owner. Hospitals and health systems should switch the score on only where its most urgent alerts go straight to someone whose job is to answer them: a rapid response team, or a central team of nurses that screens alerts for several hospitals and calls in the ward's rapid response team, which brings in the patient's doctor. The extra calls need a budget before the first one comes, and a hospital that cannot staff or share such a response is better off leaving the alerts off.

Just as important is who draws the line. That choice belongs to the medical staff and nursing, through their patient safety committee, which can weigh what each setting costs in calls and missed patients and publish its choice. The same committee should track what happens to the patients the score flags, review every patient who went to intensive care, needed a code or died on a ward without an alert, and check whom the score misses by race, ethnicity and sex, as Epic's own submission did, and by unit. If the misses or the calls begin to harm patients, the threshold has to change, or the alerts have to go off. A physician asked to work alongside the score is entitled to three answers before it goes live: who receives its top alerts, who chose its thresholds and whom it misses.

The FDA's clearance settles what Epic's score is and what it may be used for. Whether it helps the next patient who begins to slip on a ward at night depends on who answers when it sounds, who decided where to set the line, and whether anyone is keeping count.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.