Evaluation of Project TEAM (Teens Making Environmental and Activity Modifications) (ProjectTEAM)

Evaluation of Project TEAM (Teens Making Environmental and Activity Modifications) - Effectiveness, Social Validity and Feasibility

The purpose of this study is to determine the extent to which Project TEAM (Teens making Environment and Activity Modifications) is an effective, socially valid, and feasible intervention that prepares youth with developmental disabilities ages 14-21 to respond to environmental barriers and increases participation in school, work, and the community. Project TEAM is a manualized intervention co- facilitated by a disability advocate and a licensed professional. The intervention includes eight group sessions and two experiential learning field trips. In addition, young adults with disabilities serve as peer mentors on field trips and contact youth weekly to support attainment of goals. Project TEAM outcomes are to: increase youths' knowledge of environmental factors and modification strategies; reduce the impact of environmental barriers on participation; increase self-efficacy and self-determination; and increase participation in a personal activity goal in the area of education, employment, or community life. This project builds on a participatory action research partnership with disability community stakeholders to address the following research questions: (1) To what extent do youth with disabilities participating in Project TEAM achieve intervention outcomes? (2) What are the characteristics of youth with disabilities who most benefit from Project TEAM? (3) To what extent are goals, procedures, and outcomes of Project TEAM important and acceptable (socially valid) to youth with disabilities?.

Disparities in school, work, and community participation impact the 15% of youth in the United States estimated to have a developmental disability. A growing body of literature suggests disparities in participation are due to barriers in the physical and social environment. Practitioners and advocates have developed skill and advocacy interventions in an attempt to increase the participation of youth with disabilities. Yet none of these interventions focus specifically on acquiring the problem-solving skills needed to identify environmental barriers and generate modification strategies to resolve barriers to participation. Research indicates that there is a vital need to develop manualized, theory-based interventions that empower youth with developmental disabilities to identify and advocate for environments that support their participation in school, work, and the community. The purpose of this study is to determine the extent to which Project TEAM is an effective, socially valid, and feasible intervention that prepares youth with developmental disabilities ages 14-21 to respond to environmental barriers and increases participation in school, work and the community. Project TEAM is a manualized intervention co- facilitated by a disability advocate and a licensed professional. The intervention includes eight group sessions and two experiential learning field trips. In addition, young adults with disabilities serve as peer mentors on field trips and contact youth weekly to support attainment of goals. Project TEAM outcomes are to: increase youths' knowledge of environmental factors and modification strategies; reduce the impact of environmental barriers on participation; increase self-efficacy and self-determination; and increase participation in a personal activity goal in the area of education, employment, or community life. Pilot research showed that Project TEAM participants (N=20) had a significant increase in knowledge of environmental barriers and modification strategies (t(19) = -6.37, p=.00), and 57% increased their participation in one activity. This project builds on a Participatory Action Research (PAR) partnership with disability community stakeholders to address the following research questions: 1) To what extent do youth with disabilities participating in Project TEAM achieve intervention outcomes? 2) What are the characteristics of youth with disabilities who most benefit from Project TEAM? 3) To what extent are goals, procedures, and outcomes of Project TEAM important and acceptable (socially valid) to youth with disabilities? This project uses a multi-site, quasi-experimental repeated measures design with matched controls to evaluate Project TEAM. Sixty-four youth ages 14-21 with developmental disabilities will participate in Project TEAM and complete outcome measures at three time points: 2 weeks pre-intervention, 2 weeks post- intervention, and follow-up 6 weeks post-intervention. A control group of 64 youth, matched to intervention participant characteristics using a three tiered approach, will complete outcome measures at time points that correspond with the intervention group. Outcome measures assess goal attainment (Goal Attainment Scaling), knowledge and application of skills acquired during intervention (Project TEAM Knowledge Test), changes in participation and impact of barriers on participation (Participation and Environment Measure- Child &Youth), self-efficacy (Generalized Self Efficacy Scale), and self-determination (AIR Self-Determination Scale). Characteristics that may influence the extent to which youth benefit from Project TEAM will be assessed using a battery of descriptive measures. Outcomes will be analyzed within and across groups to evaluate the effectiveness of Project TEAM. Feasibility and adherence to the proposed design will be evaluated using a process evaluation. To evaluate social validity, a Youth Research Panel (YRP) of 6 youth with disabilities ages 14-21 and a Consumer Research Specialist will administer a satisfaction survey and focus group interview to Project TEAM participants. Parents (n= 64) will also participate in on-line focus groups to evaluate the feasibility and usefulness of Project TEAM. The YRP and other members of the research team will use an action/reflection process to interpret data and revise Project TEAM to maximize outcomes for future implementation. The YRP will disseminate information about Project TEAM to local and national capacity- building organizations targeted to youth with disabilities. The PI will also disseminate findings to professionals and the disability community via a website, presentations, and peer- reviewed journals.

repeated measures, matched comparison, manualized intervention, self-determination, participation, advocacy, problem-solving, commuinity participation, occupational therapy, environment

Project TEAM is a manualized intervention co- facilitated by a disability advocate and a licensed professional. The intervention includes eight group sessions and two experiential learning field trips. In addition, young adults with disabilities serve as peer mentors on field trips and contact youth weekly to support attainment of goals. Project TEAM outcomes are to: increase youths' knowledge of environmental factors and modification strategies; reduce the impact of environmental barriers on participation; increase self-efficacy and self-determination; and increase participation in a personal activity goal in the area of education, employment, or community life.

Youth with disabilities who are matched controls will receive their typical educational or therapeutic services. Youth will receive a stipend to participate in a preferred activity in the community; youth will document what they did and with whom they participated. Attempts to control for the impact of resources on participation and goal achievement.

Project TEAM is a manualized, group-based intervention designed to be co-facilitated by an experienced leader with a disability (disability advocate) and a licensed service provider (such as an occupational therapist, social worker, or educator). Project TEAM includes eight group sessions and two experiential learning field trips for each participant. Weekly phone calls with peer mentors with disabilities support achievement of each participant's personal activity goal.

All youth had four goals in the following areas: 1) a participation goal, 2) their ability to identify environmental barriers to their goal, 3) their ability to generate solutions to barriers, and 4) their ability to advocate for needed changes to achieve their goal. Each goal used a five-point goal attainment scale with baseline at -1. Goals levels were created at intake (initial assessment). For the knowledge application goals (goal 2-4), we created standardized goal levels to ensure content validity and reliability within and across youth. Goal attainment for all four goals was rated 12 weeks following intake (outcome) and transformed into a t-score. A t-score of 50 indicates all goals were achieved at the expected level; t-scores greater than 50 indicate individuals exceeded the expected level of goal attainment. Scores range from 0-100 (100 indicates greater than expected goal attainment).

Part I: Knowledge of parts of the environment, modification strategies, and the "Game Plan." Higher scores indicate more correct responses. Part I responses were independently coded as correct/incorrect by the study facilitator and a trained graduate student; discrepancies were resolved by a third scorer (the PI). To establish unidimensionality, we applied a dichotomous Rasch model and removed 24% of the items with Outfit Mean Square >2; values higher than 2 can indicate guessing. The resulting interval sum scores, in logits, were used for analysis; higher logit scores indicate more knowledge (Minimum: -4.05 to maximum 6.69). Higher scores indicate greater problem solving.

The AIR measured the capacity and opportunity to act in a self-determined manner at home and school. Parallel youth and parent forms used a 5-point frequency scale (never-always), with higher scores reflecting more self-determination. Reported here are parent self-reported sum scores at outcome. Sum scores range from minimum 18- to maximum 90 (90/higher scores = more self determination)

We revised a disability self-efficacy scale for this study and created additional questions to assess self-efficacy for addressing environmental barriers. We used a modified three point response scale (Not like me, Sort of like me, Really like me) that incorporated visuals to support comprehension. Higher scores indicated higher self-efficacy. Sum scores range from minimum 11 to maximum 33.

Frequency of participation in home, school, and the community. Parent report. We examined change in scores between baseline and outcome only for the context in which the individuals' goal occurred (e.g., for GAS goals regarding going to a concert, the parent only completed "community" at outcome. Higher scores indicate higher frequency of participation. Below, we only report outcomes for the youth with community data at outcome, as it was the most frequently occurring goal context . 0 is do not ever participate, and 7 is participate daily. HIgher scores indicate more frequent participation in the context

readiness to engage in advocacy based on transtheoretical model of change. This is a single question with a 5 possible responses (1= minimum, 5= maximum), where higher responses (5) indicate higher readiness for advocacy. Data is reported as increase (improvement in readiness), no change, or decrease (decline in readiness). Improvement is a better outcome. Below, reported for number of participants in each group with improvement between intake and 12 weeks following intake (outcome).

Inclusion Criteria: 1) A developmental disability as defined by the Developmental Disabilities Assistance and Bill of Rights Act of 2000 (Public Law No.106-402) 9(example diagnoses include autism, intellectual disability, and cerebral palsy), 2) Age 14 to 21 years at time of enrollment, 3) Communicate in English verbally or using other means as needed, 4) Able to attend to a task for 10 minutes and follow a two-step direction with support, 5) Able to categorize objects and concepts, and 6) Self-identify as a youth with a disability as reported on a modified functional strengths and challenges questionnaire Exclusion Criteria: Learning disabilities without any other co-occuring diagnosis. living outside of the university recruitment regions

Levin MR, Kramer JM. Key elements supporting goal attainment for transition-age young adults: A case illustration from Project TEAM. Inclusion 3: 145-161, 2015

Kramer J, Hwang I, Helfrich C, Samuel P, Carralles A, YELL Youth Research Team. Evaluating the social validity of Project TEAM: A problem-solving intervention to teach transition age youth with developmental disabilities to resolve environmental barriers. International Journal of Disability, Development, & Education 65(1): 57-75, 2018

Schwartz AE, Kramer JM. "I just had to be flexible and show good patience": management of interactional approaches to enact mentoring roles by peer mentors with developmental disabilities. Disabil Rehabil. 2018 Oct;40(20):2364-2371. doi: 10.1080/09638288.2017.1334835. Epub 2017 Jun 8.

Kramer JM, Ryan CT, Moore R, Schwartz A. Feasibility of electronic peer mentoring for transition-age youth and young adults with intellectual and developmental disabilities: Project Teens making Environment and Activity Modifications. J Appl Res Intellect Disabil. 2018 Jan;31(1):e118-e129. doi: 10.1111/jar.12346. Epub 2017 Mar 1.

Ryan CT, Kramer JM, Cohn ES. Exploring the Self-Disclosure Process in Peer Mentoring Relationships for Transition-Age Youth With Developmental Disabilities. Intellect Dev Disabil. 2016 Aug;54(4):245-59. doi: 10.1352/1934-9556-54.4.245. Erratum In: Intellect Dev Disabil. 2016 Oct;54(5):fmi.