Subpopulation-Specific Sepsis Identification Using Machine Learning

The focus of this study will be to conduct a prospective, randomized controlled trial (RCT) at Cape Regional Medical Center (CRMC), Oroville Hospital (OH), and UCSF Medical Center (UCSF) in which a subpopulation-optimized algorithm will be applied to EHR data for the detection of severe sepsis. For patients determined to have a high risk of severe sepsis, the algorithm will generate automated voice, telephone notification to nursing staff at CRMC, OH, and UCSF. The algorithm's performance will be measured by analysis of the primary endpoint, in-hospital SIRS-based mortality. The secondary endpoints will be in-hospital severe sepsis/shock-coded mortality, SIRS-based hospital length of stay, and severe sepsis/shock-coded hospital length of stay.

From July 2020 to February 2021, inclusive, we will perform a randomized controlled trial (RCT) across eight subpopulations at CRMC, OH, and UCSF. The subpopulations which will be included in the trial are: Cardiology, Gastroenterology (GI), Intensive Care Unit (ICU), Medicine, Oncology, Surgery, Transplant, and Emergency Department (ED). All aims of this study across both phases including the RCT have been approved by the Pearl Institutional Review Board with a waiver of informed consent (IRB00007772, FWA00026887). During the study period, all patients over the age of 18 presenting to the emergency department or admitted to an inpatient unit at the participating facilities will automatically be enrolled in the trial if they are a member of one of the eight subpopulations of interest in this study, until the target enrollment for the study is met. Enrollment will entail randomization to either the control or the experimental arms. Patients will be assigned to the experimental group or control group based on a random allocation sequence, generated by a computer program before the start of the trial, using simple randomization, with a 1:1 allocation ratio. This allocation sequence will be concealed to patients, healthcare providers and study investigators. However the trial will have an open-label design, as full blinding is not possible as some group assignments will become naturally revealed upon receipt of telephonic alerts. There will be two arms in the study. The control arm will involve patients monitored by the original version of InSight, and the experimental arm will involve patients monitored by the subpopulation-customized version of InSight. In both arms, if the applicable algorithm determines a patient to be at high risk for severe sepsis, a telephonic alert will be sent to the charge nurse on duty in the patient's current location. Response to alerts will follow the protocol from our previous sepsis clinical trial. The procedure consists of a nurse conducting a patient bedside evaluation to rule out suspected infection. This includes assessment of patient vital signs, EHR notes, and recent laboratory results. If the nurse suspects severe sepsis, a physician subsequently assesses the patient and, if appropriate, places an order for administration of the standard sepsis treatment bundle. In the administration of clinical trials, some open-label studies are cluster-randomized while others are randomized at an individual patient level. Cluster randomization is frequently used to minimize "contamination" between treatment and control groups, because exposure of providers to patients from both arms in an open-label study often invites unintentional behavioral biases. These biases may cause providers to adjust their interventions in the control group to mimic their actions in the experimental group, thereby masking the intervention's effect and skewing the study results towards the null. Although open-label, cluster-randomized trials are effective in minimizing contamination among groups, they have several significant disadvantages, including greater complexity in design and analysis as well as larger patient enrollment requirements to achieve the same statistical power. Because larger sample sizes often necessitate increases in cost, length, or complexity of a trial, current research has indicated that trialists should use individual randomization if possible due to the drawbacks of cluster allocation. Given these considerations, we concluded that individual randomization was the best strategy for our trial, as it affords a significant amount of increase in statistical power and allows each patient outcome to be assessed independently of every other patient. To minimize possible bias, we also decided to make the automated phone call text identical in both arms. The successful use of patient-level randomization in our previous sepsis clinical trial gives us confidence in this trial design. After the discharge of the last enrolled patient, we will evaluate whether the primary endpoint of in-hospital SIRS-based mortality and secondary endpoints of in-hospital severe sepsis/shock-coded mortality, SIRS-based hospital length of stay, and severe sepsis/shock-coded hospital length of stay are met. Additional outcome measures of interest for each SIRS-based and severe sepsis/shock-coded patient groups will include: time to completion of each element of the Surviving Sepsis Campaign (SSC) bundle; ventilator-free days; ICU days; and 30-day hospital readmission rate. The 1-hour SSC bundle consists of obtaining blood cultures, measuring lactate level, administering broad-spectrum antibiotics, administering 30 mL/kg of crystalloid fluid for hypertension or lactate >4 mmol/L, and applying vasopressors if patient is hypotensive during or after fluid resuscitation. Patients will be considered "SIRS-based" and included for primary endpoint analysis if they meet two or more SIRS criteria at any point during their stay. The reason for this inclusion criteria is that the algorithm may detect severe sepsis before it is apparent in the chart, and therefore sepsis may be resolved with early intervention prior to severe sepsis documentation in the medical record. For example, if a CDS alert results in treatment initiation before organ dysfunction indicative labs are drawn, the patient's state of "severe sepsis" may be censored out. Limiting inclusion criteria to the 2001 consensus (Sepsis-2) severe sepsis definition criteria or the Sepsis-3 criteria would exclude such patients from analysis; however, such censorship will be avoided with our use of SIRS-based inclusion criteria. We plan to draw from EHR-based clinical data for primary endpoint analysis, as opposed to claims-based data, due to its ability to provide more objective measurements on patient outcomes. However, to compare to other studies that use coding-based inclusion, claims data will be used in inclusion criteria for secondary endpoints. Patients will be considered to be documented "severe sepsis/septic shock-coded" and included for secondary endpoint analysis if they meet either Angus implementation criteria or any of the following diagnosis codes: R6520 and/or R6521 with septicemia codes A400, A401, A403, A408, A4101, A4102, A411, A412, A413, A414, A4150, A4151, A4152, A4153, A4159, A4181, A4189, A427, A021, A227, A267, A327, A5486, B377. The use of explicit ICD codes alone for tracking sepsis is known to have high specificity but low sensitivity. At the conclusion of the study, significant findings will be published as scientific papers.

Hospital length of stay attributed to patients meeting two or more SIRS criteria at some point during their stay

All adults above age 18 who are a member of one of the eight subpopulations studied in this trial (Cardiology, Gastroenterology (GI), Intensive Care Unit (ICU), Medicine, Oncology, Surgery, Transplant, and Emergency Department (ED)) are eligible to participate in the study.

Desautels T, Calvert J, Hoffman J, Mao Q, Jay M, Fletcher G, Barton C, Chettipally U, Kerem Y, Das R. Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting. Biomed Inform Insights. 2017 Jun 12;9:1178222617712994. doi: 10.1177/1178222617712994. eCollection 2017.

Calvert J, Mao Q, Rogers AJ, Barton C, Jay M, Desautels T, Mohamadlou H, Jan J, Das R. A computational approach to mortality prediction of alcohol use disorder inpatients. Comput Biol Med. 2016 Aug 1;75:74-9. doi: 10.1016/j.compbiomed.2016.05.015. Epub 2016 May 24.

Calvert JS, Price DA, Barton CW, Chettipally UK, Das R. Discharge recommendation based on a novel technique of homeostatic analysis. J Am Med Inform Assoc. 2017 Jan;24(1):24-29. doi: 10.1093/jamia/ocw014. Epub 2016 Mar 28.

Calvert J, Mao Q, Hoffman JL, Jay M, Desautels T, Mohamadlou H, Chettipally U, Das R. Using electronic health record collected clinical variables to predict medical intensive care unit mortality. Ann Med Surg (Lond). 2016 Sep 6;11:52-57. doi: 10.1016/j.amsu.2016.09.002. eCollection 2016 Nov.

Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017 Nov 9;4(1):e000234. doi: 10.1136/bmjresp-2017-000234. eCollection 2017.