Improving Antibiotic Prescribing for Pediatric Respiratory Infections by Family Physicians With Peer Comparison

Improving Antibiotic Prescribing for Pediatric Respiratory Infections by Family Physicians With Peer Comparison

Improving Antibiotic Prescribing for Pediatric Acute Respiratory Tract Infections: Cluster Randomized Trial to Evaluate Individual Clinician Compared With Clinic-level Feedback

Findings from an ongoing improvement project to improve antibiotic prescribing for children and adolescents for three acute respiratory tract infections (ARTIs: upper respiratory tract infection, acute bacterial sinusitis, and acute otitis media) among pediatric and family medicine clinics revealed performance gaps between the two primary care specialties. An improvement project was then set up to address the lower performance by family medicine clinics. Literature review revealed that, in general, quality improvement feedback was more effective if provided to individual clinicians rather than to a group of clinicians, but very limited data existed for antibiotic prescribing practices actually comparing individual clinician feedback to group (clinic-level) feedback. The hypothesis is that individual clinician data feedback is superior to group (clinic-level) feedback in improving antibiotic prescribing for ARTIs in children and adolescents by family medicine clinicians. The aim is to determine if there are significant differences for antibiotic prescribing for ARTIs and for broad spectrum antibiotic prescribing percentage between an intervention group and a comparator group of family medicine clinics after the intervention starting November 2015 and ending December 2018. A cluster randomized trial was designed for 39 family medicine clinics. The intervention group received clinician-level and clinic-level data feedback monthly, and the comparator group received clinic-level only feedback monthly.

Introduction/Background Findings from an ongoing improvement project to improve antibiotic prescribing for children and adolescents for three acute respiratory tract infections (ARTIs: upper respiratory tract infection, acute bacterial sinusitis, and acute otitis media) among pediatric and family medicine clinics revealed performance gaps between the two primary care specialties. An improvement project was then set up to address the lower performance by family medicine clinics. Literature review revealed that, in general, quality improvement feedback was more effective if provided to individual clinicians rather than to a group of clinicians, but very limited data existed for antibiotic prescribing practices actually comparing individual clinician feedback to group (clinic-level) feedback. The hypothesis is that individual clinician data feedback is superior to group (clinic-level) feedback in improving antibiotic prescribing for ARTIs in children and adolescents by family medicine clinicians. The aim is to determine if there are significant differences for antibiotic prescribing for ARTIs and for broad spectrum antibiotic prescribing percentage between an intervention group and a comparator group of family medicine clinics after the intervention starting November 2015 and ending December 2018. Methods Design A cluster randomized trial was designed for all 98 family medicine clinics within Novant Health Medical Group. In August 2015, retrospective review of data from January 1, 2014 was conducted by the antimicrobial stewardship team, and the trial was developed. Clusters were a subset of clinics identified as under-performing, based on not avoiding antibiotics in at least 83% of patients with upper respiratory infection or common cold (URI) (2013 Healthcare Effectiveness Data Information Set mean in patients 3 months-18 years of age). Among these 47 clinics, 6 had recorded < 20 encounters in the 6 months, January-June 2015, for an illness diagnosis of URI and were excluded. The remaining 39 clinics sites in 26 practices (1 practice had 10 clinic sites and a second had 5 sites; all others were single site practices) agreed to participate and were stratified by size of clinic (number of clinics or clinicians as very small, small, large, very large). Clinics were then block randomized with selection of half of each stratum for two groups - an intervention group and a comparator group. The Institutional Review Board of Novant Health Presbyterian Medical Center granted a waiver of written, informed consent. An email was sent to the lead physicians for the 41 remaining clinics detailing the protocol, and all but two clinics agreed to participate. Intervention All 39 clinics received the same multifaceted intervention with one exception. This intervention included: A one-hour, in-person, educational session, in September-October 2015, with the lead clinician, clinic administrator, and stewardship team physicians describing the project and clinical guidelines for URI, ABS, and AOM. A tip sheet detailing how to improve scores (appropriate codes and documentation strategies). An after-visit summary for clinicians to give to patients and parents discussing antibiotic use and side effects. A presentation of summary results for all pediatric and family medicine clinics for the 3 ARTI metrics to for the six-month period, January-June 2015. Clinic comparison performance data were then provided monthly for all 39 clinics Discussion about a new clinical pathway for acute bacterial sinusitis with a request for adoption and implementation. A request of each practice to: Discuss the guidelines for the three metrics, the tip sheet, the AVS, and baseline performance at the outset. Adopt and implement the ABS clinical pathway. Review monthly the performance scores for the three metrics. The exception was the intervention group received monthly clinician-specific performance data for the three measures, while the comparator group received only clinic-level data. All family medicine clinics received composite clinic-level data for all family medicine and pediatric clinics so each clinician could view, at the clinic level, all other clinics' performances. Clinical decision support was not used. Data Collection Baseline performance data were collected retrospectively from January 1, 2014-October 31, 2015. The intervention period was November 1, 2015-December 31, 2017. A post-intervention period was from January 1-December 31, 2018, during which time only clinic-level data were provided monthly to all 39 clinics. Visit-level data included ICD-9 codes and, starting in October 2015, ICD-10 codes associated with an encounter (IE) and listed as a visit diagnosis. Antibiotic prescribing was determined by medication orders' search in the record associated with the encounter or within 30 days prior (if URI diagnosis) or within 60 days prior (if ABS or AOM diagnosis) and 3 days subsequent to the encounter. IEs were defined as evaluation and management visits for new patients (with codes 99201-99205) and for established patients (with codes 99212-99215). Measures Using guidelines from the American Academy of Pediatrics and the Infectious Diseases Society of America, customized clinical quality measures for the 3 ARTIs were developed and validated by selective, manual chart review of electronic health records. These measures represented the primary outcomes at cluster level and included as numerator, appropriate care, and denominator, IE, for each ARTI. A target of ≥ 90% for URI was set using the HEDIS® 2013 90th percentile, and at 80% for ABS and AOM, consistent with targets suggested by an outpatient antibiotic use target-setting workgroup. A secondary outcome was broad-spectrum antibiotic prescribing (BSAP) percentage, determined monthly by enumerating all antibiotics given, stratifying by narrow and broad spectrum and dividing by the total number of antibiotics prescribed for any condition, not limited to the 3 ARTIs. For this calculation, patients were excluded if their record showed an allergy to an antibiotic listed as narrow or broad spectrum and/or one of the listed antibiotics had been given 60 days prior. To determine if code shifting occurred or total antibiotic utilization changed after the intervention began, the investigators recorded the mean number of encounters per clinic for the 3 ARTIs and the total of all IEs and antibiotics prescribed for all patients seen for illness in the baseline and intervention periods in both groups. Statistical Analysis Power will be estimated based on the primary outcomes, baseline-to-intervention period change in the proportion of encounters with the appropriate treatment for URI, ABS, and AOM, respectively. The numbers of clusters are fixed at 22 for the intervention group and 17 for the comparator group. Based on retrospective data from the baseline period, the average cluster sizes, i.e., the average numbers of encounters per clinic over 22 months, for URI, ABS, and AOM, respectively, are 210.0, 72.6, and 129.2 for the intervention group and 211.0, 65.5, and 133.6 for the comparator group. This provided > 85% power to detect a study group difference in mean baseline-to-intervention period change of 0.4 standard deviations, corresponding to a medium effect size, with two-sided alpha = 0.05, for intra-cluster correlations (ICCs) ranging between 0.01 and 0.15. The power analysis was performed using PASS 15. The clinic is the unit of analysis for describing changes in clinical decision making. The primary outcomes, i.e., the proportion of relevant encounters with appropriate treatment (for URI, ABS, and AOM, respectively) will be recorded for each clinic during the baseline and intervention periods. To assess the influence of the intervention on clinical decision making, a generalized linear mixed model (GLMM) will be analyzed for each outcome using PROC GLIMMIX® in SAS. The response variable in the models is y/n, where y = the number of appropriate treatment occurrences and n = the total number of relevant encounters, and the response distribution is specified as binomial with a logit link. The independent variables are the study group (intervention, comparator), time period (baseline, intervention), a study-group-by-time interaction term, and clinic size (very small, small, large, and very large) (used as the stratification variable in the randomization). The models include a random intercept term for clinic to account for relatedness of clinical decisions made in the same clinic. The p-value for the interaction term will be used to assess significance between the intervention and comparator groups on the baseline-to-intervention period change. The overall alpha level is pre-specified at 0.05. To account for multiple testing, Holm's Step-Down procedure will be used to adjust p-values, where the family of inferences includes those for the three primary outcomes and the secondary outcome, BSAP%. In the case of a significant interaction effect, the pre-intervention and post-intervention outcome will be compared for the intervention and comparator groups, respectively, using pre-specified contrasts generated from the GLMM. Odds ratios, corresponding to these contrasts, will be calculated, along with 95% confidence intervals that are Bonferroni-corrected (at the α/2 = 0.025 level) to further adjust for multiple comparisons. For each outcome variable, the ICC will be calculated using the level-2 (i.e. random intercept) variance from the GLMM and a level-1 variance component assumed to be π‸2/3 = 3.29 for a logistic random intercept model. Analysis of the secondary outcome, BSAP percentage, will be conducted using the steps described for the primary outcomes. All analyses will be performed using SAS.

Peer Comparison, Feedback, Antibiotic Stewardship, Pediatric Acute Upper Respiratory Tract Infections

Group of 22 clinics to receive clinician-level and clinic-level monthly feedback on appropriate antibiotic use for three acute respiratory tract infections

Group of 17 clinics to receive clinic-level only monthly feedback on appropriate antibiotic use for three acute respiratory tract infections

Monthly emailed data on performance for guideline-concordant use of antibiotics for three acute upper respiratory tract infections either at the clinician and clinic levels or at the clinic level only

Proportion of encounters with guideline-appropriate use of antibiotics for upper respiratory tract infection (URI or common cold) in participants 3 months - 18 years of age

Measurement tool is electronic health record review of Illness encounters in the time frame for diagnosis codes for URI or common cold Antibiotics dispensed the day of the URI or common cold diagnosis encounter or 3 days subsequently Antibiotics dispensed in the previous 30 days

60 months, January 1, 2014 - October 31, 2015 (baseline period), November 1, 2015 - December 31, 2017 (intervention period), and January 1, 2017 - December 31, 2018 (post-intervention period)

Proportion of encounters with guideline-appropriate use of antibiotics for acute bacterial sinusitis in participants 1 - 18 years of age

Measurement tool is electronic health record review of Illness encounters in the time frame for diagnosis codes for acute bacterial sinusitis Antibiotics dispensed the day of the acute bacterial sinusitis diagnosis encounter or 3 days subsequently Antibiotics dispensed in the previous 60 days

60 months,January 1, 2014 - October 31, 2015 (baseline period), November 1 - December 31, 2017 (intervention period), and January 1, 2017 - December 31, 2018 (post-intervention period)

Proportion of encounters with guideline-appropriate use of antibiotics for acute otitis media in participants 6 months - 12 years years of age

Measurement tool is electronic health record review of Illness encounters in the time frame for diagnosis codes for acute otitis media Antibiotics dispensed the day of the acute otitis media diagnosis encounter or 3 days subsequently Antibiotics dispensed in the previous 60 days

60 months, January 1, 2014 - October 31, 2015 (baseline period), November 1, 2015 - December 31, 2017 (intervention period), and January 1, 2017 - December 31, 2018 (post-intervention period)

Measurement tool is electronic health record review of All antibiotics given for any diagnosis code Stratified by narrow- and broad-spectrum Then divided by total antibiotics given. Exclusions include Allergy to narrow- or broad-spectrum antibiotic One of the antibiotics listed given in the prior 60 days.

60 months, January 1, 2014 - October 31, 2015 (baseline period), November 1, 2015 - December 31, 2017 (intervention period), and January 1, 2017 - December 31, 2018 (post-intervention period)

Inclusion Criteria Family Medicine clinic in Novant Health Medical Group with performance on the upper respiratory infection measure < 85 per cent for the 6-month period, January- June 2015 Email agreement to participate for the clinic by the lead clinician for the clinic. Exclusion Criteria Family Medicine clinic in Novant Health Medical Group with performance on the upper respiratory infection measure ≥ 83 per cent for the 6-month period, January - June 2015, < 20 illness encounters for the upper respiratory infection measure recorded by the clinic for the 6-month period, January - June 2015, Email declination for the clinic by the lead clinician for the clinic

National Committee for Quality Assurance. Improving quality and patient experience. The State of Healthcare Quality 2013. October 2013, Washington, DC.

Hersh AL, Jackson MA, Hicks LA; American Academy of Pediatrics Committee on Infectious Diseases. Principles of judicious antibiotic prescribing for upper respiratory tract infections in pediatrics. Pediatrics. 2013 Dec;132(6):1146-54. doi: 10.1542/peds.2013-3260. Epub 2013 Nov 18.

Wald ER, Applegate KE, Bordley C, Darrow DH, Glode MP, Marcy SM, Nelson CE, Rosenfeld RM, Shaikh N, Smith MJ, Williams PV, Weinberg ST; American Academy of Pediatrics. Clinical practice guideline for the diagnosis and management of acute bacterial sinusitis in children aged 1 to 18 years. Pediatrics. 2013 Jul;132(1):e262-80. doi: 10.1542/peds.2013-1071.

Lieberthal AS, Carroll AE, Chonmaitree T, Ganiats TG, Hoberman A, Jackson MA, Joffe MD, Miller DT, Rosenfeld RM, Sevilla XD, Schwartz RH, Thomas PA, Tunkel DE. The diagnosis and management of acute otitis media. Pediatrics. 2013 Mar;131(3):e964-99. doi: 10.1542/peds.2012-3488. Epub 2013 Feb 25. Erratum In: Pediatrics. 2014 Feb;133(2):346. Dosage error in article text.

Chow AW, Benninger MS, Brook I, Brozek JL, Goldstein EJ, Hicks LA, Pankey GA, Seleznick M, Volturo G, Wald ER, File TM Jr; Infectious Diseases Society of America. IDSA clinical practice guideline for acute bacterial rhinosinusitis in children and adults. Clin Infect Dis. 2012 Apr;54(8):e72-e112. doi: 10.1093/cid/cir1043. Epub 2012 Mar 20.

The Pew Charitable Trust. Health experts establish national targets to improve outpatient antibiotic selection. https://www.pewtrusts.org/-/media/assets/2016/10/health_experts_establish_national_targets_to_improve_outpatient_antibiotic_selection.pdf.

Cohen, J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Routledge. ISBN 978-1-134-74270-7; 1988.

PASS 15 Power Analysis and Sample Size Software (2017). NCSS, LLC. Kaysville, Utah, USA, ncss.com/software/pass.

Wu S, Crespi CM, Wong WK. Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials. Contemp Clin Trials. 2012 Sep;33(5):869-80. doi: 10.1016/j.cct.2012.05.004. Epub 2012 May 22.

Clegg HW, Ezzo SJ, Flett KB, Anderson WE. Improving antibiotic prescribing for pediatric acute respiratory tract infections: A cluster randomized trial to evaluate individual versus clinic feedback. Antimicrob Steward Healthc Epidemiol. 2021 Nov 3;1(1):e43. doi: 10.1017/ash.2021.212. eCollection 2021.