Enhancing Prevention Capacity With Developmental Assets and Getting to Outcomes (AGTO)

National Institute on Drug Abuse (NIDA), Search Institute, Communities for Children and Youth, University of Southern Maine, Visions Training Associates

Alcohol and other drug use among youth is costly for communities. More research is needed about how to best support community based prevention programs and how community prevention expertise can inform the research process. The National Institute on Drug Abuse has funded a 5 year collaboration of the RAND Corporation, Search Institute and its training division, Vision Training Associates, Communities for Children and Youth, and the University of Southern Maine to implement and assess the impact on prevention coalitions, the combination of two complimentary, community-based interventions: Developmental Assets, which supports community mobilization and collaboration to promote positive youth development, and Getting To Outcomes (GTO), which enhances community capacity to complete critical prevention tasks (e.g., evaluation). The purpose of the project is to investigate: 1) How well is the Assets-GTO intervention delivered, how much is it used, and what coalitions think about it; 2) The extent to which the Assets-GTO approach enhances the prevention capacity (knowledge, attitudes, and skills) of individual coalition members and the quality of prevention performance; and 3) Whether enhanced prevention capacity improves alcohol and drug outcomes among youth. Twelve community-based prevention coalitions in Maine (part of Communities for Children and Youth) will participate. Six coalitions-determined at random-will receive manuals, training, and on-site technical assistance consisting of bi-Weekly meetings between A-GTO 4 ME! and key coalition staff. The other six coalitions will continue practice as usual, but will receive an abbreviated version of the Assets-GTO intervention near the end of the project. A Community Research Workgroup made of coalition representatives will review all aspects of the study and interim findings and facilitate dissemination on A-GTO 4 ME! The project will demonstrate and evaluate strategies to strengthen the prevention capacity of community organizations that can be used broadly across many types of programs.

Alcohol and drug (AOD) use is problematic in many communities. Despite the spread of evidence-based prevention, communities still face difficulty in achieving outcomes demonstrated by prevention science. This "gap" is because resources are limited, prevention is complex, and communities often lack the capacity to adapt and implement "off the shelf" programs. Also, many evidence-based programs aim to improve deficits-- despite evidence showing the need to also promote positive youth development through community-wide efforts. Common ways to bridge this gap, such as information dissemination, fail to change practice or outcomes at the local level in part because it does not sufficiently address capacity or use community input. Therefore, building a community's prevention capacity, through greater collaboration between scientists and practitioners, with a focus on positive youth development, is a method that could improve the quality of prevention and outcomes. This project will assess the combination of two models that are specifically designed to foster such an approach: Getting To Outcomes (GTO) and Developmental Assets. They are complimentary: GTO enhances local capacity for discrete prevention tasks (e.g., evaluation); Developmental Assets supports community mobilization and collaboration to promote positive youth development. Combining the content, tools, and resources of these two SAMHSA (Best Practice) prevention planning processes has the potential to improve the quality of prevention programming and accountability more than either would do alone. Quasi - experimental and case studies of both Assets and GTO have demonstrated feasibility in community settings and yielded evidence suggesting these models can help communities mobilize and improve prevention practices and outcomes. As a next step, we propose a randomized controlled efficacy trial with elements of an effectiveness study (i.e., implementation in community-based setting) comparing 6 AOD prevention coalitions using Assets-GTO with 6 similar coalitions who are not. Such blended designs that emphasize generalizability and external validity are now recommended for community-based research. We will use a participatory research approach in which a Workgroup of coalition representatives will be actively involved in all phases of the research. Assets-GTO's impact on prevention capacity will be assessed at the program level (5 per coalition) with staff interviews and at the individual level with a Coalition Survey (each has about 54 members). A survey of schools in which the coalitions operate will assess the impact on AOD use and positive developmental outcomes among the programs' target populations. Organizational change theories will guide Assets-GTO implementation; standardized measures will track Assets-GTO adoption. Results will have implications for how to ensure that prevention programming found to be effective through research trials is successfully delivered in real world settings, a priority for NIDA and NIAAA. Project Narrative The demonstration and evaluation of the Assets-Getting To Outcomes system for the prevention of alcohol and drug (AOD) use outlined in this proposal has direct relevance to public health. This is because AOD use among youth is a significant health problem facing US communities. The Assets-Getting To Outcomes system is designed to help communities engaged in AOD prevention work to better plan, implement, and self-evaluate their prevention strategies in order to help them achieve positive results, thereby positively impacting the mortality and morbidity of youth at the local level. PUBLIC HEALTH RELEVANCE: The demonstration and evaluation of the Assets-Getting to Outcomes system for the prevention of alcohol and drug (AOD) use outlined in this proposal has direct relevance to public health. This is because AOD use among youth is a significant health problem facing US communities. The Assets-Getting To Outcomes system is designed to help communities engaged in AOD prevention work to better plan, implement, and self-evaluate their prevention strategies in order to help them achieve positive results, thereby positively impacting the mortality and morbidity of youth at the local level.

Cohort 1: receives the Assets Getting To Outcomes intervention first. The AGTO intervention includes three types of assistance which are adapted to fit the needs and priorities of the individuals involved, as well as the inner and outer setting: (1) a manual of text and tools; (2) face-to-face training, and (3) onsite technical assistance (TA). These three types of assistance aim to improve the implementation process for each program. Two full-time, Maine-based staff, one with a master's and one with a bachelor's degree, provided AGTO tools, training, and TA to the intervention coalitions and programs during the two year intervention period. The tools are in the Search Institute-published manual, Getting To Outcomes with Developmental Assets: Ten steps to measuring success in youth programs and communities, which all intervention participants received.

Cohort 2: receives the Assets Getting To Outcomes intervention second, after Cohort 1 is done receiving the intervention.

Assessed in the Coalition Survey, prevention capacity was defined as efficacy and behaviors of practitioners. GTO efficacy scale is the sum of 10 items using a three-point scale (1="would need a great deal of help to carry out this task", 2="could carry out this task, but would need some help", 3="could carry out this task without any help") asking about activities associated with doing the AGTO 10 steps. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .02 change on the original 1-3 scale. A 50-percentage point change would be equivalent to a one-point change on the original 1-3 scale.

This scale is the sum of 11 items with seven-point scales (1="never" to 7="very often") assessing the frequency with which respondents engaged in AGTO activities during the previous 12 months. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .06 change on the original 1-7 scale. A 17-percentage point change would be equivalent to a one-point change on the original 1-7 scale.

This scale is the sum of 11 items with seven-point scales (1="never" to 7="very often") assessing the frequency with which respondents engaged in AGTO activities during the previous 12 months. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .06 change on the original 1-7 scale. A 17-percentage point change would be equivalent to a one-point change on the original 1-7 scale.

This scale is the sum of 11 items with seven-point scales (1="never" to 7="very often") assessing the frequency with which respondents engaged in assets activities during the previous 12 months. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .06 change on the original 1-7 scale. A 17-percentage point change would be equivalent to a one-point change on the original 1-7 scale.

Assessed in the Coalition Survey, prevention capacity was defined as efficacy and behaviors of practitioners. Assets efficacy scale is the sum of 10 items using a three-point scale (1="would need a great deal of help to carry out this task", 2="could carry out this task, but would need some help", 3="could carry out this task without any help") asking about activities associated with doing the Developmental Assets model. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .02 change on the original 1-3 scale. A 50-percentage point change would be equivalent to a one-point change on the original 1-3 scale.

A structured interview was used to assess the impact of AGTO on prevention practitioners' performance of tasks associated with high-quality prevention. Using the interview responses, a set of ratings were made assessing performance of activities in seven key domains: goals and objectives, best practices, planning, process evaluation, outcome evaluation, continuous quality improvement, and sustainability. The ratings are made on 10 items (or "components") that assess how well each of the above mentioned activities are performed over the last year. Each component has seven response choices, described with specific, observable behaviors, that range from "highly faithful=7" to "highly divergent=1" from ideal performance. The total score is an average of the 10 components, and has the same range as the individual components ("highly faithful=7" to "highly divergent=1" from ideal performance)

This scale is the sum of 11 items with seven-point scales (1="never" to 7="very often") assessing the frequency with which respondents engaged in GTO activities during the previous 12 months. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .06 change on the original 1-7 scale. A 17-percentage point change would be equivalent to a one-point change on the original 1-7 scale. Same analysis/measure as the intent to treat, but instead just comparing users of AGTO to non-users within the AGTO assigned group. "Use" was determined by six items added to the Mid and Post Coalition Survey, called the AGTO Participation Index. If individuals received any hours of technical assistance, they received an additional point on the Index. Then, a dichotomous measure was created if a user participated (AGTO Participation Index >=1) at either Mid or Post.

The GTO efficacy scale is the sum of 10 items using a three-point scale (1="would need a great deal of help to carry out this task", 2="could carry out this task, but would need some help", 3="could carry out this task without any help") asking about activities associated with doing the AGTO 10 steps. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .02 change on the original 1-3 scale. A 50-percentage point change would be equivalent to a one-point change on the original 1-3 scale. Same analysis/measure as the intent to treat, but instead just comparing users of AGTO to non-users within the AGTO assigned group. "Use" was determined by six items added to the Mid and Post Coalition Survey, called the AGTO Participation Index. If individuals received any hours of technical assistance, they received an additional point on the Index.

This scale is the sum of 11 items with seven-point scales (1="never" to 7="very often") assessing the frequency with which respondents engaged in AGTO activities during the previous 12 months. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .06 change on the original 1-7 scale. A 17-percentage point change would be equivalent to a one-point change on the original 1-7 scale. Same analysis/measure as the intent to treat, but instead just comparing users of AGTO to non-users within the AGTO assigned group. "Use" was determined by six items added to the Mid and Post Coalition Survey, called the AGTO Participation Index. If individuals received any hours of technical assistance, they received an additional point on the Index. Then, a dichotomous measure was created if a user participated (AGTO Participation Index >=1) at either Mid or Post.

This scale is the sum of 11 items with seven-point scales (1="never" to 7="very often") assessing the frequency with which respondents engaged in assets activities during the previous 12 months. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .06 change on the original 1-7 scale. A 17-percentage point change would be equivalent to a one-point change on the original 1-7 scale. Same analysis/measure as the intent to treat, but instead just comparing users of AGTO to non-users within the AGTO assigned group. "Use" was determined by six items added to the Mid and Post Coalition Survey, called the AGTO Participation Index. If individuals received any hours of technical assistance, they received an additional point on the Index. Then, a dichotomous measure was created if a user participated (AGTO Participation Index >=1) at either Mid or Post.

The Assets efficacy scale is the sum of 10 items using a three-point scale (1="would need a great deal of help to carry out this task", 2="could carry out this task, but would need some help", 3="could carry out this task without any help") asking about activities associated with doing assets activities. The sum was then transformed to be on a 1-100% scale. A percentage point change is equivalent to a .02 change on the original 1-3 scale. A 50-percentage point change would be equivalent to a one-point change on the original 1-3 scale. Same analysis/measure as the intent to treat, but instead just comparing users of AGTO to non-users within the AGTO assigned group. "Use" was determined by six items added to the Mid and Post Coalition Survey, called the AGTO Participation Index. If individuals received any hours of technical assistance, they received an additional point on the Index.

A structured interview was used to assess the impact of AGTO on prevention practitioners' performance of tasks associated with high-quality prevention. Using the interview responses, a set of ratings were made assessing performance of activities in seven key domains: goals and objectives, best practices, planning, process evaluation, outcome evaluation, continuous quality improvement, and sustainability. The ratings are made on 10 items (or "components") that assess how well each of the above mentioned activities are performed over the last year. Each component has seven response choices, described with specific, observable behaviors, that range from "highly faithful=7" to "highly divergent=1" from ideal performance. The total score is an average of the 10 components, and has the same range as the individual components ("highly faithful=7" to "highly divergent=1" from ideal performance)

Inclusion Criteria: Related to the programs of the 12 participating coalitions, the inclusion criterion is being an AOD-related universal, selective, or indicated prevention program or initiative. Exclusion Criteria: The exclusion criterion is being a case identification, treatment, or after-care program.

Chinman M, Acosta J, Ebener P, Q Burkhart, Clifford M, Corsello M, Duffey T, Hunter S, Jones M, Lahti M, Malone PS, Paddock S, Phillips A, Savell S, Scales PC, Tellett-Royce N. Establishing and evaluating the key functions of an interactive systems framework using an assets-getting to outcomes intervention. Am J Community Psychol. 2012 Dec;50(3-4):295-310. doi: 10.1007/s10464-012-9504-z.

Acosta J, Chinman M, Ebener P, Malone PS, Paddock S, Phillips A, Scales P, Slaughter ME. An intervention to improve program implementation: findings from a two-year cluster randomized trial of Assets-Getting To Outcomes. Implement Sci. 2013 Aug 7;8:87. doi: 10.1186/1748-5908-8-87.

Chinman M, Acosta J, Ebener P, Burkhart Q, Malone PS, Paddock SM, Clifford M, Corsello M, Duffey T, Hunter S, Jones M, Lahti M, Phillips A, Savell S, Scales PC, Tellett-Royce N. Intervening with practitioners to improve the quality of prevention: one-year findings from a randomized trial of assets-getting to outcomes. J Prim Prev. 2013 Jun;34(3):173-91. doi: 10.1007/s10935-013-0302-7.