Randomized Trial of a Data-driven Technical Assistance System for Drug Prevention Coalitions

This project is designed to test the Coalition Check-Up (CCU)-a theory-based and data-driven technical assistance (TA) system that supports community coalitions' implementation of evidence-based programs (EBPs) for drug prevention. The primary aims of the project are to: 1) Estimate the impact of the CCU on coalition capacity. Coalitions will be randomly assigned to the CCU or a 'TA as usual' condition to evaluate whether the CCU improves coalition capacity as measured by coalition member reports of team processes, network composition, and collaborative structure. 2) Estimate the impact of the CCU on implementation of evidence-based programs. The study will test the hypothesis that coalitions receiving the CCU will implement EBPs with greater: a) quantity, b) quality, and c) sustainability. The study will also test coalition capacity as a mediator of CCU impact on EBP implementation. 3) Estimate the impact of the CCU on youth substance use. The study will test the hypothesis that communities receiving the CCU will reduce youth substance use relative to communities in the comparison condition. The study will also test EBP implementation as a mediator of CCU impact on youth substance use.

The overall goal of this five-year R01 study is to test the Coalition Check-Up (CCU) technical assistance (TA) system for supporting community coalitions' implementation of evidence-based drug prevention programs (EBPs). Over 5,000 community anti-drug coalitions operating in the U.S serve as a cornerstone of federal drug prevention. These coalitions, however, have only demonstrated efficacy in preventing substance use when they use TA and implement EBPs, a key research-to-practice gap. The CCU supports coalitions by identifying and addressing gaps in EBP implementation capacity. The proposed study advances implementation science by applying Wandersman's Interactive Systems Framework to test the effects of CCU on coalition EBP implementation capacity and youth outcomes. Despite the popularity of community anti-drug coalitions as a mechanism for EBP dissemination, scant research addresses how to support coalitions for optimal EBP implementation. Lacking adequate support, coalitions and EBPs often fail. Intensive TA provided in evidence-based coalition models is effective but often too expensive to scale in real-world settings. The CCU provides a lower-cost TA system that is broadly applicable across coalition models. The study's main objective is to test the overall effectiveness of the CCU, including how it contributes to EBP implementation and prevention of youth substance use. Building on the Interactive System Framework, the central hypothesis is that the CCU can enhance the prevention support system, thereby increasing coalition capacity for EBP implementation and the probability that EBPs will reduce youth substance use. The study will test this central hypothesis by pursuing three specific aims. The first aim is to estimate the impact of the CCU on coalition capacity, including team processes, network composition, and collaborative structure. Coalitions will be randomly assigned to the CCU or a 'TA as usual' condition. The second aim is to estimate the impact of the CCU on implementation of EBPs, including EBP reach, implementation quality, and sustainability. The third aim is to estimate the impact of the CCU on youth substance use, including alcohol, tobacco, marijuana, and opioids. The CCU is innovative in its emphasis on proactive monitoring and data-driven TA, its use of motivational interviewing to enhance coalition-driven action planning, and its examination of network structure to enhance coalition capacity. The proposed study's contribution is highly significant because the field currently lacks clear evidence of the effectiveness of a TA model applicable to the heterogeneous mix of drug prevention coalitions in operation. The research will enhance community coalition ability to bridge the research to practice gap in drug prevention programming. Results are expected to have a positive impact on the field by establishing the evidence-base for a low-cost, data-driven, manualized TA model that identifies how to intervene with community coalitions to support sustained implementation of evidence-based drug prevention programs and policies known to promote community health.

At baseline, community coalitions will be randomly assigned to either the Coalition Check-Up experimental condition or the technical assistance as usual comparison condition.

The 4-step Coalition Check-Up technical assistance process provides proactive data-driven continuous quality improvement cycles. Step 1 assesses critical dimensions of the coalition's capacity and program implementation. A coalition profile based on assessment data is reviewed in step 2. Here the technical assistance provider works with the coalition to consider several dimensions of coalition capacity and program implementation, celebrating strengths and prioritizing weaknesses. Once priorities are set, the technical assistance provider uses structured action planning in step 3 to help coalition members establish consensus on how to improve prioritized weaknesses. In step 4, technical assistance providers review and support progress on action plan implementation with the coalition. Efforts are evaluated a year after the initial assessment in a continuous quality improvement cycle.

Coalitions in the comparison condition will receive a feedback report but no additional support from technical assistance providers beyond what is already available to them.

The Coalition Check-Up identifies and addresses coalition and implementation capacity deficiencies that frequently lead to failure, following recommendations from the audit and feedback literature.

the extent to which coalition members perceived feelings of unity, group spirit, trust, and belonging within their coalition. 3 likert scale items range from 1 = strongly disagree to 5 = strongly agree

the work ethic, efficiency, and task focus of the coalition members. 3 likert scale items range from 1 = strongly disagree to 5 = strongly agree

a combination of hierarchy, connectedness, average path length and clustering. Ranges from 0 (complete centralization) to 1 (complete decentralization).

the number of unique types of cooperation or interaction each partner reported for a relationship with another partner (e.g., sharing information, personnel, monetary resources, or other cooperation). Ranges from 0 (no multiplexity) to 4 (high multiplexity).

respondents name individuals in the coalition to whom they went to for advice about coalition matters. When an individual cited a person from a different sector as someone to whom they went for advice, that dyad is counted as an intersectoral tie. The level of intersectoral communication for each coalition is measured as the coalition's mean number of intersectoral ties per respondent, which ranges from 0 (no intersectoral ties) to 5 (all intersectoral ties).

a composite of 7 scales: a) Staff training (6 items); b) Staff motivation and competence (5 items); c) Fidelity monitoring (11 items); d) Evaluation (13 items); e) Dosage (2 items); f) Adherence (7 items); and g) Implementation barriers (11 items). Scores range from 0 (low implementation quality) to 6 (high implementation quality).

the sum number of years all EBPs are in operation during years 2-4 of the project, including both existing and new EBPs. An EBP will be designated as non-operational when: a) it has no reach; or b) an absence of ongoing training or TA in the past 12 months and no funding available to support the program

the mean of 12 items about the completion of sustainability planning activities. Ranges from 0 = no sustainability planning to 4 = extensive sustainability planning.

Past 30-day use of prescription pain relievers without a doctor's orders or heroin (yes = 1 / no = 0)

Inclusion Criteria: To be eligible coalitions must be fully operational, thus: 1) have been in existence for at least one year; 2) have a designated coordinator; 3) have at least quarterly meetings in which multiple sectors of the community attend; 4) currently support implementation of drug prevention activities or secured funding to do so; 5) be willing to complete coalition capacity and EBP implementation assessments annually; 7) be willing to participate in four in-person meetings annually with the CCU TA provider. Exclusion Criteria: Exist outside of Pennsylvania or Missouri

Upon request, we will share de-identified data from published studies with interested investigators once they have obtained IRB approval. Even though the final dataset will be stripped of identifiers prior to release for sharing, we believe that there remains the possibility of deductive disclosure of communities and schools with unusual characteristics. Thus, we will make the data and associated documentation available to users only under a data-sharing agreement that provides for: (1) a commitment to using the data only for research purposes and not to identify any participants; (2) a commitment to securing the data using appropriate computer technology; and (3) a commitment to destroying the data after analyses are completed.

We will make the data and associated documentation available to users only under a data-sharing agreement that provides for: (1) a commitment to using the data only for research purposes and not to identify any participants; (2) a commitment to securing the data using appropriate computer technology; and (3) a commitment to destroying the data after analyses are completed.

Brown LD, Chilenski SM, Wells R, Jones EC, Welsh JA, Gayles JG, Fernandez ME, Jones DE, Mallett KA, Feinberg ME. Protocol for a hybrid type 3 cluster randomized trial of a technical assistance system supporting coalitions and evidence-based drug prevention programs. Implement Sci. 2021 Jun 25;16(1):64. doi: 10.1186/s13012-021-01133-z.