Evaluating Improvement Strategies in Addiction Treatment (NIATx 200)

Addiction treatment is often characterized by long delays between first contact and treatment as well as high no-show and drop out rates leading to unused capacity in apparently full agencies. Patients do not get needed care and agency financial stability is threatened. The Network for Improvement of Addiction Treatment (NIATx) began as a high-intensity improvement collaborative of 39 addiction treatment agencies distributed across 25 states. NIATx substantially improved time to treatment and continuation in treatment by making improvements to organizational processes (such as first contact, intake and assessment, engagement, level of care transitions, paperwork, social support, outreach, and scheduling) in preliminary studies. While the results are very encouraging, they have, by intent, been obtained from a select group of agencies using a high-cost combination of services. A more practical diffusion model is needed to spread process improvements across the spectrum of treatment agencies. This study is a cluster-randomized trial to test the effectiveness and cost of less expensive combinations of the services that make up the NIATx collaborative (interest circles, coach calls, coach visits and learning sessions).

This cluster-RCT randomly assign 201 treatment agencies in 5 states to four experimental arms. The agencies were randomized to an intervention for 18 months with a 9 month sustainability period. The study aimed to: 1) Determine whether a state-based strategy can (with NIATx support) can lead mainstream treatment agencies to implement and sustain process changes that improve the study's primary outcomes: time to treatment, annual clinic admissions, and continuation in treatment; and 2) Evaluate the effectiveness and cost of the services making up NIATx. This study aims to create a practical model for improving efficiency and effectiveness of addiction treatment.

Interest Circles are monthly teleconferences where agency change leaders discuss change-related issues and progress. Circles address how to improve timeliness, continuation, admissions, dropouts and transitions. They also address specialty topics (e.g., programs for women, adolescents). Participants discuss successes, failures, and challenges, and get advice and assignments for their improvement plans. Meeting summaries appear on the Web site. Interest Circles are inexpensive, but are they are sufficient? Should Interest Circles prove effective, they would provide a low-cost, convenient diffusion approach.

Coaching assigns an expert in process improvement to work with an agency to make, sustain, and spread process improvement efforts. Consultations focus on executive directors, change leaders and improvement teams. Coaches help agencies address key issues, but also broker relationships with other agencies, offer process improvement training, and promote the innovations to make and how to make them. Coaching takes place during site visits, monthly phone conferences, and via email.

Learning Sessions occur bi-annually as change teams convene to learn and gather support from each other and outside experts who offer advice on how best to adopt the innovations and learn about new directions for the collaborative (e.g., the need to create business cases for improvements). Learning Sessions and Interest Circles (see below) have similar objectives-to help agencies learn and gather support from each other and from outside experts.

Learning Sessions occur bi-annually as change teams convene to learn and gather support from each other and outside experts who offer advice on how best to adopt the innovations and learn about new directions for the collaborative (e.g., the need to create business cases for improvements). Learning Sessions and Interest Circles (see below) have similar objectives-to help agencies learn and gather support from each other and from outside experts.

Interest Circles are monthly teleconferences where agency change leaders discuss change-related issues and progress. Circles address how to improve timeliness, continuation, admissions, dropouts and transitions. They also address specialty topics (e.g., programs for women, adolescents). Participants discuss successes, failures, and challenges, and get advice and assignments for their improvement plans. Meeting summaries appear on the Web site. Interest Circles are inexpensive, but are they are sufficient? Should Interest Circles prove effective, they would provide a low-cost, convenient diffusion approach

Coaching assigns an expert in process improvement to work with an agency to make, sustain, and spread process improvement efforts. Consultations focus on executive directors, change leaders and improvement teams. Coaches help agencies address key issues, but also broker relationships with other agencies, offer process improvement training, and promote the innovations to make and how to make them. Coaching takes place during site visits, monthly phone conferences, and via email.

The NIATx Web site features resources central to improvement. The site includes: 1) a catalog of change ideas and case studies; 2) a toolbox providing just-in-time training on topics such as conducting a walk-through and key innovations; 3) on-line tools to assess organizational (or project) readiness for and ability to sustain change; 4) electronic communication services to ask questions of experts, and participate in peer discussion groups; 5) links to relevant process improvement Web sites; and 6) a secure portion for treatment agencies to report and track progress. Hence, our control group will have access to the entire website.

The average length of time in days it takes from when a patient first calls for help to the time a patient was able to meet a clinician. In this quality improvement study, changes in this measure over time are reported. Estimates of improvement show the average days of improvement per month based on a best linear unbiased predictor estimate for each site. Note: this study has three primary outcomes. The number of participants analyzed varies for each outcome. The (higher) number of clinics shown in the flow diagram results because clinics may have been analyzed on a subset of the three primary outcomes (e.g., analyzed for waiting time and continuation, but not for annual number of new patients). To be considered "analyzed" in the flow diagram, a clinic must have been included in at least one primary outcomes analysis.

We aimed to increase clinics' treatment capacity in this quality improvement study. Capacity was measured by counting clinics' annual number of patient admissions. We monitored changes in admission counts, per clinic, in a pre-post analysis. Changes in the natural logarithm of annual admissions are presented, which approximates the average percentage change (year-to-year) in the number of new patient admissions per clinic. Note: this study has three primary outcomes. The number of participants analyzed varies for each outcome. The (higher) number of clinics shown in the flow diagram results because clinics may have been analyzed on a subset of the three primary outcomes (e.g., analyzed for waiting time and continuation, but not for annual number of new patients). To be considered "analyzed" in the flow diagram, a clinic must have been included in at least one primary outcomes analysis.

This outcome represents change in the rate at which a clinic's patients continue in treatment. Continuation rate is defined as the percentage of patients that make at least 4 visits to the clinic, on different days, before being discharged. Estimates of improvement show the average percentage points of improvement per month based on a best linear unbiased predictor estimate for each site. Note: this study has three primary outcomes. The number of participants analyzed varies for each outcome. The (higher) number of clinics shown in the flow diagram results because clinics may have been analyzed on a subset of the three primary outcomes (e.g., analyzed for waiting time and continuation, but not for annual number of new patients). To be considered "analyzed" in the flow diagram, a clinic must have been included in at least one primary outcomes analysis.

The goal of the economic analysis was to estimate costs of each group for governmental authorities who might organize improvement collaboratives. We collected the cost of personnel (state employees, NIATx employees, coaches and consultants), data management, buildings and facilities, lodging, travel, telephone calls and miscellaneous costs. Costs were categorized as group specific (such as hotel costs for the learning sessions group) or non-group-specific, which included state-incurred costs for outreach, data management and infrastructure, encouraging participation and administration. Cost data were collected three times during the study period and aggregated to create a total cost estimate. Figures reported below represent costs at the arm/group level (costs were not assessed at the organizational level). Measure type is "Number."

Inclusion Criteria: at least 60 admissions/year provide outpatient and intensive outpatient levels of care (as defined by ASAM) provide or use detox services provided by others have tax-exempt or government status or rely on public funding (e.g., block grants, Medicare, Medicaid, local government, private philanthropy) for at least 50% of their budget have adopted no more than two of the planned interventions Exclusion Criteria: are current NIATx members

Gustafson DH, Quanbeck AR, Robinson JM, Ford JH 2nd, Pulvermacher A, French MT, McConnell KJ, Batalden PB, Hoffman KA, McCarty D. Which elements of improvement collaboratives are most effective? A cluster-randomized trial. Addiction. 2013 Jun;108(6):1145-57. doi: 10.1111/add.12117. Epub 2013 Mar 1.

Choi D, Hoffman KA, Kim MO, McCarty D. A high-resolution analysis of process improvement: use of quantile regression for wait time. Health Serv Res. 2013 Feb;48(1):333-47. doi: 10.1111/j.1475-6773.2012.01436.x. Epub 2012 Jun 20.

Quanbeck A, Wheelock A, Ford JH 2nd, Pulvermacher A, Capoccia V, Gustafson D. Examining access to addiction treatment: scheduling processes and barriers. J Subst Abuse Treat. 2013 Mar;44(3):343-8. doi: 10.1016/j.jsat.2012.08.017. Epub 2012 Sep 27.

Hoffman KA, Quanbeck A, Ford JH 2nd, Wrede F, Wright D, Lambert-Wacey D, Chvojka P, Hanchett A, McCarty D. Improving substance abuse data systems to measure 'waiting time to treatment': lessons learned from a quality improvement initiative. Health Informatics J. 2011 Dec;17(4):256-65. doi: 10.1177/1460458211420090.

Quanbeck AR, Gustafson DH, Ford JH 2nd, Pulvermacher A, French MT, McConnell KJ, McCarty D. Disseminating quality improvement: study protocol for a large cluster-randomized trial. Implement Sci. 2011 Apr 27;6:44. doi: 10.1186/1748-5908-6-44.

Quanbeck AR, Madden L, Edmundson E, Ford JH 2nd, McConnell KJ, McCarty D, Gustafson DH. A business case for quality improvement in addiction treatment: evidence from the NIATx collaborative. J Behav Health Serv Res. 2012 Jan;39(1):91-100. doi: 10.1007/s11414-011-9259-6.

Roosa M, Scripa JS, Zastowny TR, Ford JH 2nd. Using a NIATx based local learning collaborative for performance improvement. Eval Program Plann. 2011 Nov;34(4):390-8. doi: 10.1016/j.evalprogplan.2011.02.006. Epub 2011 Mar 2.

Quanbeck A, Lang K, Enami K, Brown RL. A cost-benefit analysis of Wisconsin's screening, brief intervention, and referral to treatment program: adding the employer's perspective. WMJ. 2010 Feb;109(1):9-14.

McCarty D, Chandler RK. Understanding the importance of organizational and system variables on addiction treatment services within criminal justice settings. Drug Alcohol Depend. 2009 Aug 1;103 Suppl 1:S91-3. doi: 10.1016/j.drugalcdep.2009.03.001. Epub 2009 Apr 8.

McCarty D, Gustafson D, Capoccia VA, Cotter F. Improving care for the treatment of alcohol and drug disorders. J Behav Health Serv Res. 2009 Jan;36(1):52-60. doi: 10.1007/s11414-008-9108-4. Epub 2008 Feb 8.

McCarty D, Roman PM, Sorensen J, Weisner C. Health Services Research for Drug and Alcohol Treatment and Prevention. J Drug Issues. 2009 Jan;39(1):197-208. doi: 10.1177/002204260903900115.

McConnell KJ, Hoffman KA, Quanbeck A, McCarty D. Management practices in substance abuse treatment programs. J Subst Abuse Treat. 2009 Jul;37(1):79-89. doi: 10.1016/j.jsat.2008.11.002. Epub 2009 Feb 4.

Gustafson DH. Essential Ingredients for Successful Redesign of Addiction Treatment. Bridge (Kans City). 2012;2(2):v2i2_article01.

Ford JH 2nd, Gilson A. Influence of participation in a quality improvement collaborative on staff perceptions of organizational sustainability. BMC Health Serv Res. 2021 Jan 7;21(1):34. doi: 10.1186/s12913-020-06026-3.

Ford JH 2nd, Stumbo SP, Robinson JM. Assessing long-term sustainment of clinic participation in NIATx200: Results and a new methodological approach. J Subst Abuse Treat. 2018 Sep;92:51-63. doi: 10.1016/j.jsat.2018.06.012. Epub 2018 Jun 27.

Ford JH 2nd, Robinson JM, Wise ME. Adaptation of the Grasha Riechman Student Learning Style Survey and Teaching Style Inventory to assess individual teaching and learning styles in a quality improvement collaborative. BMC Med Educ. 2016 Sep 29;16(1):252. doi: 10.1186/s12909-016-0772-4.