Youth FORWARD Phase 2 YRI and EPP Study

German Society for International Cooperation, Caritas Freetown, University of Georgia, Innovations for Poverty Action

Objectives: The objectives of this research proposal are to study the delivery of an evidence-based mental health intervention in the alternate setting of youth employment programs tied to regional economic development and to examine the use of a Collaborative Team Approach (CTA) as an implementation scale-up strategy that addresses the human resource shortage and related access to care and capacity challenges in low- and middle-income countries (LMICs). Specifically, this study aims to examine the incorporation of the evidence-based Youth Readiness Intervention (YRI) into a program that promotes employment among youth (EPP/Entrepreneurship Training program) through a pilot study and scale-up intervention study in Sierra Leone. Study population: The study population includes youth, ages 18-30, with elevated t-scores on assessments of functional impairment and emotional dysregulation, who live in the Kono, Koinadugu and Kailhun districts of Sierra Leone. Scale-up study design: For the scale-up study, a Hybrid Type 2 Effectiveness-Implementation Cluster Randomized Three-arm trial will be employed. We estimate the entire sample size for the scale-up study to be 3,630 participants, including 1200 youth, 10 agency heads, 20 intervention facilitators, and 2400 third-party informants. Upon enrollment into the study, youth will be assigned to community level sites based on geographical location. Each of these community level sites will make up one cluster. These clusters will be randomized into the three study conditions: the control condition - where youth do not receive the YRI or the EPP but are able to utilize as available resources in the community - the EPP-only condition, and the YRI+EPP condition. Data will be collected at baseline, post-YRI, post-EPP, and 12-months follow-up. Scale-up study outcomes: Implementation outcomes of the Hybrid Type II study are focused on process and implementation aspects including a costing analysis, measures of fidelity and the sustainment and quality of delivering YRI within a Collaborative Team Approach to support intervention delivery, training and supervision. Effectiveness outcomes of the Hybrid Type II study are development of emotion regulation, mental health assessed as anxiety and depression, and interpersonal functioning, including self report and by third-party reporters for assessment of the YRI's ability to improve youth's interpersonal skills and functioning in the community and the entrepreneurship training program. In a Hybrid Type II study implementation and effectiveness aims are dual and equally important aspects of the study. (Curran et al., 2015) Outcomes associated with both aims are considered primary outcomes in this study.

Scale-up Study Objectives We propose to implement and test an evidence-based intervention called the Youth Readiness Intervention (YRI) alongside GIZ's youth entrepreneurship training (EPP). This scale-up study expects to enroll 1200 youth across three treatment arms (control, EPP, YRI+EPP) across GIZ's three target districts. Typically, GIZ targets youth considered to be underemployed or unemployed and youth with low literacy levels. While that criteria will remain, given the socioemotional aims of the YRI, youth must also demonstrate impaired functioning and emotional dysregulation as assessed by a screening tool (see section 4b for more information regarding inclusion criteria). Together, this model is intended to improve the mental health and functioning of vulnerable Sierra Leonean youth while demonstrating the capacity for a technical behavioral change intervention to be delivered via an alternative delivery platform like employment programs. Past implementation of the YRI in Sierra Leone has led to the creation of an evidence-informed intervention that is both relevant and culturally responsive. Most importantly, however, the previous implementation of the YRI RCT within education relied on trained counselors. By demonstrating that the YRI can be delivered by service professionals who may not have clinical training while integrated into employment programming the ability for Youth FORWARD to be adopted for scale-up across Sierra Leone and other LMICs is further strengthened. Specifically, the objects and aims of this scale up study are: AIM 1 (Implementation Impact Evaluation): To utilize an innovative Collaborative Team Approach (CTA) to scaling and sustaining the YRI in terms of feasibility of this approach, facilitator preparedness and sense of satisfaction and impacts on fidelity and sustainment of delivery of YRI. AIM 2 (Implementation Process Evaluation): To identify internal and external factors influencing the integration of the YRI into the youth EPP programs via a process evaluation documenting barriers and facilitators to effective implementation and integration. AIM 3 (Clinical Effectiveness): To compare clinical effectiveness of YRI when delivered via the EPP platform to results of our previous randomized control trial (RCT) of YRI as measured by improved mental health and reduced functional impairments among high-risk youth. Emotion regulation will be examined as a major mechanism by which the YRI improves behavior of treated youth and their functioning in the EPP. Subjects The total sample size across sites is 3630. The breakdown of study participants by size and target population is as follows: 1200 youth participants, 2400 third-party reporters, 20 YRI facilitators, and 10 agency stakeholders. Study Design and Methods The scale-up study reflects a Hybrid Type 2 Effectiveness-Implementation Cluster Randomized three-arm trial which simultaneously evaluates the use of a CTA for service delivery, supervision, intervention fidelity monitoring, and sustainment of evidence-based practices. The study will be conducted in three rural districts in Sierra Leone and involve a total sample size across sites of 3630: 1200 youth participants, 2400 third-party reporters, 20 facilitators, and 10 agency leaders. The 1200 youth (50% females, ages 18-30 years) will be randomized into three study arms: 1) control (n=400 youth); 2) EPP-only (n=400 youth); 3) YRI+EPP (n=400 youth). Data will be collected at baseline, post-YRI (sub-sample of youth randomized to receive the YRI+EPP), post-EPP, and 12-months follow-up. Study location The scale-up study will be conducted in the districts of Kono, Koinadugu and Kailhun in Sierra Leone, as GIZ's country programming operates exclusively in these locations. The most commonly used language in these districts is Krio and all study related materials for the field will be translated to Krio. Program delivery The YRI is organized into 12, 90-minute group meetings or sessions to be delivered over the course of 12 weeks. The YRI is designed to be delivered by a range of lay workers in LMICs. GIZ will use a competitive bidding process to hire service provision agencies that will design and implement the youth EPP training modules. A selection of individuals from the agencies will be trained as YRI facilitators. Training and supervision Facilitators will receive an intensive training regarding all YRI components. This training will be delivered by local experts who have received previous training from Youth FORWARD leadership and who have been integral into the creation of the YRI Data collection IPA will oversee the youth participant quantitative data collection while the Caritas Freetown research team will oversee qualitative data collection as well as the quantitative data collection regarding the third-party reporters. IPA's local research assistants will conduct all screening, baseline, and post-intervention quantitative assessments. Youth will be asked to complete the quantitative assessment battery at baseline, post-YRI (sub-sample of youth), post-EPP 12-months follow-up (from baseline). To strengthen youth self-report and provide more insight into youths' psychosocial functioning, we will utilize a third-party reporter to gain further insight into programmatic effects. Two third-party reporters will be selected per each youth across all study arms (control, EPP and YRI+EPP) from a list provided by the youth participant, and quantitatively surveyed at four time points (baseline, post-YRI (sub-sample of youth), post-EPP, and 12 months). The surveys administered to the third-party reporter include questions related to youth participant emotional regulation, workplace performance and interpersonal functioning. Study Procedures Step 1. Youth who apply and meet the eligibility criteria for the EPP will be contacted by our research team. Information about our study will be provided and appointments for in-person meetings with research team members will be scheduled. Step 2. Consent. Consent forms will be available in Krio. The consent form will be read out loud to participants to ensure illiteracy does not interfere with the consenting process. Step 3. Once consent is obtained, information for the third-party reporters will be collected and youth will be screened to determine if they have elevated t-scores on functional impairment based on thresholds from our previous research using questions adapted from the World Health Organization Disability Assessment Schedule (WHODAS) (scoring at least a 1 on this scale) and have elevated t-scores (62.5 or above) on the Disturbances in Emotion Regulation Scale (DERS) as determined through previously validated screening procedures. Step 4. Youth who meet our study criteria and are interested in participating will be collated into one list. The locations of the residences of these youth will inform our clustering approach. Clusters of youth will be assigned to one of the three study arms. Step 5. Youth will be contacted and informed of their assigned study arm. The entire intervention is expected to last six months with the YRI delivered during the first three months and the EPP delivered during the second half of the intervention period. Step 6. Youth participants across all treatment arms will be asked to complete quantitative assessment batteries at baseline, post-YRI (sub-sample of youth), post-EPP and 12, follow-up. The assessment batteries will be used for research purposes. The intervention is delivered both for the treatment of the subjects' condition and research purposes.

Under the supervision of Innovations for Poverty Action (IPA) Sierra Leone, local research assistants (blind to site and youth condition assignments) will conduct all screening, baseline, and follow up quantitative assessments. Ongoing data collection at all sites will include youth attendance and satisfaction with YRI content, counselor satisfaction with content and delivery via scales and qualitative exit interviews, as well as monitored audio-recording of all sessions for fidelity by an independent expert rater hired by CARITAS. The expert fidelity monitor will be a skilled YRI trainer and supervisor from our prior RCT and will not be a part of the CTA Seed Team, thus allowing them to provide impartial and blinded fidelity ratings of all audio-transcripts.

Youth randomized into the control arm of this study will be quantitatively assessed at the same time points as youth randomized into the two treatment arms (baseline, 3-months, 12-months). Due to access to funding and feasibility, youth in the control arm and EPP-only arm will not have the opportunity to receive the YRI once the study period concludes. However, GIZ is conducting a phased roll out of their EPP programming, meaning they will offer their EPP throughout 2018 past their participation in Youth FORWARD. Youth will also be compensated for their participation in our quantitative assessments.

Youth randomized into the Entrepreneurship Training-only arm will receive GIZ's employment programming approximately three months after the completion of the baseline assessments. The GIZ-supported Entrepreneurship Training program includes six training modules, delivered over three weeks, to include skills development, financial literacy, and other skills necessary to secure employment or engage in livelihood activities.

Youth randomized into the YRI+Entrepreneurship Training arm will begin the YRI module within two weeks after the completion of baseline assessments. The YRI curriculum (12 modules), will be delivered in 12 weekly 90-minute sessions, with additional time taken as needed. These weekly sessions assume a peer-to-peer group learning format deemed culturally appropriate for a setting like Sierra Leone where limited resources for mental health services exist. Once youth complete the YRI they will complete a post-intervention (3-months) quantitative assessment. Following the completion of the quantitative assessments, youth will receive the Entrepreneurship Training.

The Youth Readiness Intervention integrates six CBT-based, empirically-supported practice elements shown to have transdiagnostic efficacy across disorders ranging from major depressive disorder to PTSD and conduct disorders. The YRI's six evidence-based components are delivered in three phases common in trauma-informed interventions: stabilization, integration, and connection.

The youth capacity development component of GIZ's Entrepreneurship Training includes the following core components: financial literacy, entrepreneurship, and vocational training delivered through six modules (agro-processing, solar photovoltaic installation and maintenance, governance and conflict resolution, psychosocial competencies, entrepreneurship, employability). Youth will be trained at training sites and have access to a youth center in the area. These centers will have agricultural equipment and machinery available for youth to use.

A costing analysis will be conducted to determine the financial feasibility of the YRI. We will conduct a return on investment (ROI) analysis. The main categories of cost to be quantified are: Salaries/wages of staff involved in the scale-up study (e.g. salaries of agency staff)-including supplementary benefits, allowances, and employer contributions to local/national taxes or health insurance plans Facility operating costs associated with running the establishments involved in the scale-up study (e.g. EPP training centers) Overhead and capital costs relating to the intervention Cost estimates will be obtained from various sources, such as invoices, quotes, employee and contracts. For costs that are difficult to determine with accuracy, we will use estimates by consulting sources to provide effectiveness estimates of the YRI and cost per participant and cost-per-outcome.

At the facilitator level we are most interested in assessing one's experience delivering the intervention, including information regarding fidelity, feasibility, and acceptability. These measures include: Seed Team Assessment Battery, sum scores, higher scores denote higher levels of collaboration and cohesion Evidence-based Practice Attitudes Scale (adapted) Levels of Collaboration Scale (adapted) Team Assessment Questionnaire (adapted) Perceived Cohesion Scale (adapted) Facilitator Qualitative Interview Guide

YRI Fidelity Checklist Dissemination and Implementation Facilitator Survey, higher scores denote higher elements of dissemination and implementation Dissemination and Implementation Agency Survey, see above for scoring Dissemination and Implementation Participant Survey, see above for scoring

Changes in youth emotion regulation will be assessed via: • Difficulties in Emotion Regulation (DERS), clinical cut point of 62.5, higher scores denote higher emotion dysregulation

Changes in youth psychosocial adjustment will be assessed via the Oxford Measure of Psychosocial Adjustment which assesses mental health, prosocial attitudes, and functioning. Higher scores on this measure indicate worse psychosocial adjustment.

Changes in youth mental health will be assessed via the Hopkins Symptom Checklist which assesses anxiety and depression. Higher scores on this measure indicate worse mental health.

Changes in youth functioning will be assessed via: the WHO Disability Assessment Schedule, there is no cut point for this measure but higher scores indicted greater functional impairment.

Youth economic self sufficiency will be assessed using a template that captures employment across different employment sectors, income generated, and hours worked. This template looks at change over time.

Inclusion Criteria: Youth aged 18-30 will be included in the Youth FORWARD scale-up study based upon eligibility to the employment program, focusing specifically on socioeconomic status and willingness and interest in participation in an employment program, as well as mental health screening, which will include questions on daily functioning and emotional regulation. Specifically, inclusion criteria for the youth sample is: be Sierra Leonean, be female or male aged 18-26, be considered a vulnerable youth who is disengaged from both education and employment programs, have elevated t-scores on functional impairment based on thresholds from our previous research using questions adapted from the World Health Organization Disability Assessment Schedule (WHODAS)54 and, have elevated t-scores (62.5 or above) on the Disturbances in Emotion Regulation Scale (DERS).55 Inclusion criteria for agencies in this study is: be a "service provision agency" in Sierra Leone, have a minimum of two experts employed, one with subject-specific knowledge and; experience, and another with pedagogical experience in adult education and working with youth with low literacy, and sign a Memorandum Of Understanding (MOU) with Boston College (BC) regarding intellectual property of the YRI. Inclusion criteria for facilitators in this study is: be employed by one of the agencies selected in GIZ's competitive bidding process, and agree to participation in the ICTA which utilizes an enhanced supervision structure throughout the intervention. Inclusion criteria for third-party reporters in this study is: be over age 18, be nominated by the youth participant, provide oral consent to participate in the quantitative assessments, and have daily to weekly contact with the youth participant throughout the duration of this scale-up study, which will be assess via verbal confirmation when obtaining oral consent. Inclusion criteria for agency leaders is: must be a part of one of the agencies selected in GIZ's competitive bidding process and, must hold a leadership position in the agency. Exclusion Criteria: Exclusion criteria for the youth: Youth will be excluded from the study if they do not fulfill the inclusion criteria or: present with severe mental illness (e.g., psychosis), or developmental disability that precludes comprehension of the intervention content and assessments. Specifically, mental health exclusion criteria are: severe, active suicidality or psychosis as assessed via the MINI-KID diagnostic assessment administered by a study social worker and, serious cognitive impairments that might preclude one's ability to comprehend informed consent and complete study assessments as determined by a study social worker. Youth who report active suicidality or symptoms of psychosis requiring more intensive mental health care will be referred for immediate individual mental health services Exclusion criteria for the agencies: Agencies will be excluded from participation if: they are not considered a "service provision agency", and if they do not have a minimum of two experts employed: one with subject-specific knowledge and experience, and another with pedagogical experience in adult education and working with youth with low literacy. Those holding leadership positions within the agencies selected to deliver the intervention will be excluded from participation in the qualitative interviews if: they are not employed at the agencies selected, or are not considered to be in a leadership position within the agency. Exclusion criteria for the facilitators: Facilitators will be excluded from participation in delivering the intervention if: they are unable to deliver core components of the YRI and EPP intervention, and if they are unable to meet the logistical demands of the ICTA Exclusion criteria for the third-party reporters: The third-party reporter will be excluded from the study if: they are under age 18, present with severe mental illness (e.g., psychosis), or developmental disability that precludes comprehension of the intervention content and assessments (as determined by a study social worker), and if the third-party reporter does not have at least monthly contact with the youth participant

Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012 Mar;50(3):217-26. doi: 10.1097/MLR.0b013e3182408812.

Bond L, Farrar J, Borg RC, Keegan K, Journeay K, Hansen N, Mac-Boima E, Rassin A, Betancourt TS. Alternate delivery platforms and implementation models for bringing evidence-based behavioral interventions to scale for youth facing adversity: a case study in West Africa. Implement Sci Commun. 2022 Feb 16;3(1):16. doi: 10.1186/s43058-022-00259-5.

Betancourt TS, Hansen N, Farrar J, Borg RC, Callands T, Desrosiers A, Antonaccio CM, Williams MJ, Bangura J, Brennan RT. Youth Functioning and Organizational Success for West African Regional Development (Youth FORWARD): Study Protocol. Psychiatr Serv. 2021 May 1;72(5):563-570. doi: 10.1176/appi.ps.202000009. Epub 2020 Dec 9.