Measuring and Improving the Safety of Test Result Follow-Up

Michael E. DeBakey VA Medical Center, Baylor College of Medicine, Birmingham Veterans Affairs Health Care System

Improving communication is foundational to improving patient safety. Electronic health records (EHRs) can improve communication, but also introduce unique vulnerabilities. Failure to follow-up abnormal test results (missed results) is a key preventable factor in diagnosis and treatment delays in the VHA and often involves EHR-based communication breakdowns. Effective methods are needed to detect diagnostic delays and intervene appropriately. Manual techniques to detect care delays, such as spontaneous reporting and random chart reviews, have limited effectiveness, due in part to bias and lack of provider awareness of delays. They are also inefficient and cost-prohibitive when applied to large numbers of patients. Diagnostic errors are considered harder to tackle, in part because they are difficult to measure. Rigorous measurement of diagnostic safety is essential and should be prioritized given the increasing amount of electronically available data. To create an effective measurement and learning program researchers must (1) ensure teams know how to take actionable steps on data and have assistance in doing so and (2) prioritize diagnostic safety at the organizational level by securing commitment from local VA leadership and clinical operations personnel. This will ensure that safety measurement will translate into action. The proposed study focuses on creating a novel program to develop and evaluate multifaceted socio-technical tools and strategies to help prevent, detect, mitigate, and ameliorate breakdowns in EHR-based communication that often lead to "missed" test results in the VHA.

The goal of this study is to develop and evaluate a new program for surveillance and improvement of test results-related diagnostic safety. The investigators will use a multifaceted measurement approach, the Virtual Breakthrough Series (VBTS) model, to account for processes of care and work systems issues as well as outcomes and implement change. This is for Aim 2 of the study which was approved under IRB Protocol Number: H-45450. This study will implement the Change Package which was developed in the research team's recent work (Aim1, prior IRB Protocol Number: H-43661 from October 2018 - September 2019) via Virtual Breakthrough Series using a stepped-wedge cluster-randomized control trial. The study design involves random and sequential crossover of clusters from control to intervention until all clusters are exposed. A Change Package is a catalogue of evidence-based practices, change concepts, and action steps/strategies that help guide improvement efforts. VBTS involves monthly learning sessions where participants are provided with education/review of a specific component of a Change Package along with reviewing de-identified data, and having a facilitated discussion about successes, challenges, and implementation progress related to needed changes. The intervention will consist of coached implementation of the SAFER Change Package using a VBTS Collaborative, plus automated near real-time surveillance data on potentially missed test results through monthly data extraction within the Corporate Data Warehouse (CDW) platform. The study's outcome measures will be the rate of missed test results, determined through random manual medical record review conducted nationally as part of the VHA performance-measurement system, known as the External Peer Review Program (EPRP) which has been collecting facility-level data on timeliness of communication of test results to patients within the time periods specified by VHA Directive 1088, as well as automated indicators (Triggers) of missed test results. Electronic surveillance will be made possible through the use of the research team's trigger algorithms which will determine the number of patients potentially lost to follow-up for the specified condition based on a previously validated timeframe. The triggers use the national EHR data warehouse (VINCI/CDW) to identify patients at risk for delays or patients who may have fallen through the cracks. For data surveillance, the investigators will apply five triggers to the medical record data contained within the CDW for the participating facilities during the Intervention Phase along with evaluating EPRP data. The research team hypothesizes there will be fewer missed test results in participating sites during the SAFER TRACKS Intervention as compared to during the pre-intervention period.

It's a cluster randomized control trial where 12 sites are randomized in groups of 4 in 3 clusters each.

VBTS participants, providers, and outcome assessors know about the intervention and in which conditions sites are in and when.

Each cluster starts receiving the intervention in sequence per cluster randomized control trial designs. Each cluster will participate in attending monthly coaching calls and compare their data on test results from pre-intervention to receiving the intervention.

When the cluster is not in active intervention, they are in the non-intervention period. The amount of time that each site contributes to the intervention depends on which cluster they belong to.

SAFER Change Package delivered using a Virtual Breakthrough Series [VBTS] Collaborative supplemented with automated surveillance data on test results.

Inclusion Criteria: Medical records identified via CDW which fall under the EPRP measures (FOBT/FIT, HCV, Mammogram, AFP, DEXA Scan, Pap/HPV, Chest X-ray, and Chest CT) Medical records containing clinical findings suspicious for breast cancer, lung cancer, bladder cancer, hepatocellular carcinoma, and colorectal cancer (CRC) Exclusion Criteria: Medical records that don't contain any tests, procedures, or appointments that need to be followed up on

Walter FM, Thompson MJ, Wellwood I, Abel GA, Hamilton W, Johnson M, Lyratzopoulos G, Messenger MP, Neal RD, Rubin G, Singh H, Spencer A, Sutton S, Vedsted P, Emery JD. Evaluating diagnostic strategies for early detection of cancer: the CanTest framework. BMC Cancer. 2019 Jun 14;19(1):586. doi: 10.1186/s12885-019-5746-6.

Murphy DR, Giardina TD, Satterly T, Sittig DF, Singh H. An Exploration of Barriers, Facilitators, and Suggestions for Improving Electronic Health Record Inbox-Related Usability: A Qualitative Analysis. JAMA Netw Open. 2019 Oct 2;2(10):e1912638. doi: 10.1001/jamanetworkopen.2019.12638.

Gandhi TK, Singh H. Reducing the Risk of Diagnostic Error in the COVID-19 Era. J Hosp Med. 2020 Jun;15(6):363-366. doi: 10.12788/jhm.3461. No abstract available.

Giardina TD, Royse KE, Khanna A, Haskell H, Hallisy J, Southwick F, Singh H. Health Care Provider Factors Associated with Patient-Reported Adverse Events and Harm. Jt Comm J Qual Patient Saf. 2020 May;46(5):282-290. doi: 10.1016/j.jcjq.2020.02.004. Epub 2020 Feb 21.

Cifra CL, Dukes KC, Ayres BS, Calomino KA, Herwaldt LA, Singh H, Reisinger HS. Referral communication for pediatric intensive care unit admission and the diagnosis of critically ill children: A pilot ethnography. J Crit Care. 2021 Jun;63:246-249. doi: 10.1016/j.jcrc.2020.09.011. Epub 2020 Sep 18.

Zimolzak AJ, Shahid U, Giardina TD, Memon SA, Mushtaq U, Zubkoff L, Murphy DR, Bradford A, Singh H. Why Test Results Are Still Getting "Lost" to Follow-up: a Qualitative Study of Implementation Gaps. J Gen Intern Med. 2022 Jan;37(1):137-144. doi: 10.1007/s11606-021-06772-y. Epub 2021 Apr 27.

Meyer AND, Giardina TD, Khawaja L, Singh H. Patient and clinician experiences of uncertainty in the diagnostic process: Current understanding and future directions. Patient Educ Couns. 2021 Nov;104(11):2606-2615. doi: 10.1016/j.pec.2021.07.028. Epub 2021 Jul 15.

Vaghani V, Wei L, Mushtaq U, Sittig DF, Bradford A, Singh H. Validation of an electronic trigger to measure missed diagnosis of stroke in emergency departments. J Am Med Inform Assoc. 2021 Sep 18;28(10):2202-2211. doi: 10.1093/jamia/ocab121.

Murphy DR, Savoy A, Satterly T, Sittig DF, Singh H. Dashboards for visual display of patient safety data: a systematic review. BMJ Health Care Inform. 2021 Oct;28(1):e100437. doi: 10.1136/bmjhci-2021-100437.

Sittig DF, Lakhani P, Singh H. Applying requisite imagination to safeguard electronic health record transitions. J Am Med Inform Assoc. 2022 Apr 13;29(5):1014-1018. doi: 10.1093/jamia/ocab291.

Sittig DF, Sengstack P, Singh H. Guidelines for US Hospitals and Clinicians on Assessment of Electronic Health Record Safety Using SAFER Guides. JAMA. 2022 Feb 22;327(8):719-720. doi: 10.1001/jama.2022.0085. No abstract available.

Singh H, Connor DM, Dhaliwal G. Five strategies for clinicians to advance diagnostic excellence. BMJ. 2022 Feb 16;376:e068044. doi: 10.1136/bmj-2021-068044. No abstract available.

Shafer GJ, Singh H, Thomas EJ, Thammasitboon S, Gautham KS. Frequency of diagnostic errors in the neonatal intensive care unit: a retrospective cohort study. J Perinatol. 2022 Oct;42(10):1312-1318. doi: 10.1038/s41372-022-01359-9. Epub 2022 Mar 4.

Giardina TD, Choi DT, Upadhyay DK, Korukonda S, Scott TM, Spitzmueller C, Schuerch C, Torretti D, Singh H. Inviting patients to identify diagnostic concerns through structured evaluation of their online visit notes. J Am Med Inform Assoc. 2022 May 11;29(6):1091-1100. doi: 10.1093/jamia/ocac036.

Shen L, Levie A, Singh H, Murray K, Desai S. Harnessing Event Report Data to Identify Diagnostic Error During the COVID-19 Pandemic. Jt Comm J Qual Patient Saf. 2022 Feb;48(2):71-80. doi: 10.1016/j.jcjq.2021.10.002. Epub 2021 Oct 29.

Read AJ, Waljee AK, Sussman JB, Singh H, Chen GY, Vijan S, Saini SD. Testing Practices, Interpretation, and Diagnostic Evaluation of Iron Deficiency Anemia by US Primary Care Physicians. JAMA Netw Open. 2021 Oct 1;4(10):e2127827. doi: 10.1001/jamanetworkopen.2021.27827.

Sittig DF, Singh H. Policies to Promote Shared Responsibility for Safer Electronic Health Records. JAMA. 2021 Oct 19;326(15):1477-1478. doi: 10.1001/jama.2021.13945. No abstract available.

Zhou Y, Walter FM, Mounce L, Abel GA, Singh H, Hamilton W, Stewart GD, Lyratzopoulos G. Identifying opportunities for timely diagnosis of bladder and renal cancer via abnormal blood tests: a longitudinal linked data study. Br J Gen Pract. 2021 Dec 31;72(714):e19-e25. doi: 10.3399/BJGP.2021.0282. Print 2022 Jan.

Bradford A, Shahid U, Schiff GD, Graber ML, Marinez A, DiStabile P, Timashenka A, Jalal H, Brady PJ, Singh H. Development and Usability Testing of the Agency for Healthcare Research and Quality Common Formats to Capture Diagnostic Safety Events. J Patient Saf. 2022 Sep 1;18(6):521-525. doi: 10.1097/PTS.0000000000001006. Epub 2022 Apr 22.

Ramesh S, Ayres B, Eyck PT, Dawson JD, Reisinger HS, Singh H, Herwaldt LA, Cifra CL. Impact of subspecialty consultations on diagnosis in the pediatric intensive care unit. Diagnosis (Berl). 2022 Apr 11;9(3):379-384. doi: 10.1515/dx-2021-0137. eCollection 2022 Aug 1.

Giardina TD, Hunte H, Hill MA, Heimlich SL, Singh H, Smith KM. Defining Diagnostic Error: A Scoping Review to Assess the Impact of the National Academies' Report Improving Diagnosis in Health Care. J Patient Saf. 2022 Dec 1;18(8):770-778. doi: 10.1097/PTS.0000000000000999. Epub 2022 Apr 27.

Meyer AND, Scott TMT, Singh H. Adherence to National Guidelines for Timeliness of Test Results Communication to Patients in the Veterans Affairs Health Care System. JAMA Netw Open. 2022 Apr 1;5(4):e228568. doi: 10.1001/jamanetworkopen.2022.8568.

Fischer H, Hahn EE, Li BH, Munoz-Plaza CE, Luong TQ, Harrison TN, Slezak JM, Sim JJ, Mittman BS, Lee EA, Singh H, Kanter MH, Reynolds K, Danforth KN. Potentially Harmful Medication Dispenses After a Fall or Hip Fracture: A Mixed Methods Study of a Commonly Used Quality Measure. Jt Comm J Qual Patient Saf. 2022 Apr;48(4):222-232. doi: 10.1016/j.jcjq.2022.01.003. Epub 2022 Jan 13.

Makris KI, Clark DL, Buffie AW, Steen EH, Ramsey DJ, Singh H. Missed Opportunities to Promptly Diagnose and Treat Adrenal Tumors. J Surg Res. 2022 Aug;276:174-181. doi: 10.1016/j.jss.2022.02.049. Epub 2022 Mar 30.

Cifra CL, Tigges CR, Miller SL, Curl N, Monson CD, Dukes KC, Reisinger HS, Pennathur PR, Sittig DF, Singh H. Reporting Outcomes of Pediatric Intensive Care Unit Patients to Referring Physicians via an Electronic Health Record-Based Feedback System. Appl Clin Inform. 2022 Mar;13(2):495-503. doi: 10.1055/s-0042-1748147. Epub 2022 May 11.