Effects of Performance Feedback on Imaging Use in the Emergency Department

DESIGN: An IRB-approved, HIPAA-compliant prospective randomized controlled trial conducted from January 1, 2012-December 31, 2013. SETTING: The ED of an urban Level-I adult trauma center with existing CDS for CTPE. PARTICIPANTS: All attending physicians were stratified into quartiles by 2012 CTPE use and randomized to receive feedback reporting or not. INTERVENTION: Quarterly performance feedback reports consisting of individual and anonymized group data on EBG adherence (using the Wells criteria), CTPE use (CTPEs per 1,000 patients), and yield (percentage of CTPEs positive for PE) beginning January 2013.

Setting and Subjects This IRB-approved and HIPAA-compliant prospective randomized controlled trial was conducted from January 1, 2012 to December 31, 2013 in the ED of an urban Level-I adult trauma center. Imaging CDS was deployed for all CTPE requests throughout the study period as previously reported. Ordering providers could ignore the evidence presented in CDS and proceed with CTPE requests deviating from evidence-based guidelines without interference. The investigators included all attending emergency physicians and, prior to randomization, stratified them into quartiles by 2012 CTPE use. Physicians were randomized by quartile into two groups using a random number generator: the intervention group received individualized feedback reports on CTPE adherence to EBG, use (defined as number of CTPEs per 1,000 patients), and yield (percentage of CTPEs positive for PE), and the control group did not. Data Collection Use of CTPEs for each physician was calculated using the number of completed CTPE examinations and the total number of patients seen during the quarter. Yield of CTPEs for acute PE was determined using a previously validated natural language processing tool and reported as a percentage of total CTPEs completed. Adherence to EBG was determined by applying the Wells Criteria and reviewing the serum D-dimer levels (if obtained). The discrete criteria making up the Wells Criteria were prospectively documented in the investigators computerized order entry (CPOE) system at the time of order entry, as previously reported. In order to determine whether any differences observed in guideline adherence were the result of "gaming" (erroneous data entry to either avoid potentially onerous CDS interactions or to enhancing the physician's apparent performance on feedback reports) the investigators performed manual chart reviews of 100 randomly chosen charts from each of the two groups. These chart reviews were performed by an attending physician to assess concordance between adherence to EBG calculated from the CDS data, and adherence to EBG calculated from data documented in the ED visit clinical note. The sample size was determined using a baseline concordance of 90% as previously reported, and chosen to detect a difference in concordance of 15% between groups with a power of 80% and an alpha error rate of 5%. Demographic data (including gender and both age and years since residency training as measured at the beginning of the study period) was also captured for all the attending physicians in the study.

The performance feedback reports displayed both individual physicians' statistics as well as their performance compared to anonymized results for the entire group of emergency physicians.