School Health Implementation Network: Eastern Mediterranean (SHINE)

Evaluating the Effectiveness of Technology Assisted School Mental Health Program to Improve Socio-emotional Wellbeing of School Going Children in Public Schools of Rural Pakistan: A Cluster Randomized Controlled Trial (cRCT)

University of Liverpool, University of Washington, Pakistan Ministry of Health, World Health Organization, Institute of Psychiatry, WHO Collaborating Center for Mental Health, Rawalpindi, National Institute of Mental Health (NIMH), Liverpool School of Tropical Medicine, Harvard University

Background An estimated 10-20% of children globally are affected by a mental health problem. Child mental health has been identified as a priority issue by the World Health Organization's Eastern Mediterranean Regional Office (WHO EMRO). Following consultations with international and regional experts and stakeholders, WHO EMRO developed an evidence-based School Mental Health Program (SMHP), endorsed by WHO EMRO member countries, including Pakistan. The federal and provincial health departments in Pakistan made recommendations for a phased implementation of the SMHP in a pilot district. In the formative phase of this program, a number of implementation challenges were identified by the stakeholders. Broadly, these included the need to operationalize and adapt the existing components of the intervention to the local context and to develop sustainable mechanisms for delivery of quality training and supervision. Informed by the results of a formative phase investigations, the SHINE scale-up research team adapted the SMHP (henceforth called Conventional SMHP or cSMHP) to address these implementation challenges. The enhanced version of the intervention is called Enhanced School Mental Health Program (eSMHP). Enhancements to cSMHP have occurred at two levels: A) Content enhancements, such as a collaborative care model for engaging parents/primary caregivers, strategies for teacher's wellbeing, and adaptation and operationalization of particular clinical intervention strategies and B) Technological enhancements which include adaptation of the training manual for delivery using an online training platform, and a 'Chat-bot' to aid the implementation of intervention strategies in classroom settings. Objectives The primary objective of the study is to evaluate the effectiveness of eSMHP in reducing socio-emotional difficulties in school-going children, aged 8-13, compared to cSMHP in Gujar Khan, a rural sub-district of Rawalpindi, Pakistan. The secondary objectives are to compare the cost-effectiveness, acceptability, adoption, appropriateness (including cultural appropriateness), feasibility, penetration and sustainability of scaled-up implementation of eSMHP and cSMHP. It is hypothesized that eSMHP will prove to be both more effective and more scalable than cSMHP. Study population The research is embedded within the phased district level implementation of the cSMHP in Rawalpindi, Pakistan. The study population will consist of children of both genders, aged 8-13 (n=960) with socio-emotional difficulties, studying in rural public schools of sub-district Gujar Khan in Rawalpindi. Design The proposed study design is a cluster randomized controlled trial (cRCT), embedded within the conventional implementation of the SMHP. Following relevant ethics committees and regulatory approvals, 80 eligible schools, stratified by gender, will be randomized into intervention and control arms with a 1:1 allocation ratio. Following informed consent from the parent/ primary caregiver, children will be screened for socio-emotional difficulties using Strengths and Difficulties Questionnaire (SDQ). 960 children scoring > 12 on the teacher-rated SDQ total difficulty scores and > 14 on the parent-rated SDQ total difficulty scores will be recruited and equally randomized into intervention and control arms (480 in each arm). Teachers in the intervention arm will receive training in eSMHP, whereas teachers in the active control will be trained in cSMHP. Trained teachers will deliver the program to children in their respective arms. Outcome measures Primary Outcome: The primary outcome is reduction in socio-emotional total difficulties scores, measured with the parent-rated SDQ, 9 months after commencing intervention delivery. Secondary Outcomes: Implementation data on acceptability, adoption, appropriateness (including cultural appropriateness), feasibility, penetration and sustainability outcomes will be collected from children, parents/primary caregivers, head teachers and teachers. In addition, data will be collected on self-reported Psychological Outcome Profiles (PSYCHLOPS)-KIDS to measure progress on psycho-social problems and wellbeing; annual academic performance; classroom absenteeism, stigmatizing experiences and parent-teacher interaction. Data on teachers' sense of efficacy and subjective well-being, and on the schools' psychosocial environment profile will be collected. All secondary outcome data will be collected at baseline and 9 months after commencing intervention delivery. Outcomes will be analyzed on an intention to treat basis. The role of various factors as potential mediators and moderators eSMHP effectiveness will be explored. Cost-effectiveness evaluation of SMHP shall be evaluated in terms of costs associated with implementation of eSMHP compared with cSMHP.

Behavioral Symptoms, Behavioral Problem, Emotional Problem, Emotional Stress, ADHD, Emotional Disorder, Depression, Anxiety, Conduct Disorder, Emotional Trauma

Study investigators and outcome assessors will be blind to the allocations status of participants. To ensure blinding, participants will be instructed to not disclose their allocation during assessment. Fidelity of masking will be ensured by having assessors guess the allocation status of participants at the end of assessments.

The teachers of schools randomized to control arm will receive training in World Health Organization (WHO) School Mental Health Program (SMHP) by mental health experts at WHO collaborating center for mental health research and training, Institute of Psychiatry. The training of teachers will consist of a mix of both didactic and interactive training methodologies in the form of a workshop. The workshop will consist of lectures/presentations, incorporating group discussions/activities. Through didactic methods, teachers will be taught basic theoretical knowledge related to mental health in schools. Training will be followed by monthly supervision meeting of teachers for 9-months.

The teachers of schools randomized to intervention arm will receive online training in adapted version of School Mental Health Program. The online training in adapted School Mental Health Program consists of 4-5 hour, self-paced online training course for teachers. The teachers will register themselves in the online course in the form of a group of 4-5 teachers from each school. The teachers will complete the online training course in a group, with interactive group activities and role plays. Progress to the next module in the online training is conditional upon completion of post-module mental health literacy quiz. A certificate of training completion in adapted SMHP shall be awarded to those teachers who complete the post-test. Teachers will be supported online and in-person by the trainers who are trained in SMHP in monthly supervision meeting of teachers for 9-months.

World Health Organization (WHO) School Mental Health Program (SMHP) is a manual based multi-component, multi-tiered and evidence-informed intervention for common mental health problems in school going children. SMHP is designed to be introduced into the normal classroom and school setting by trained teachers. The intervention has a universal component which takes a whole school approach that aims to promote mental health among all school children. It includes basic counseling skills for teachers, core values of mental health promoting schools and other health promoting efforts that impact upon mental health and can be administered to all students in school and classroom settings. The manual also contains targeted intervention strategies on anxiety, separation anxiety/ school refusal, post-trauma, depression, suicide, ADHD, autism, psychosis, conduct problems and substance use problems that can be implemented by teachers in classroom settings.

Informed by the results of pilot implementation, a number of content and delivery adaptations have been made to the School Mental Health Program (SMHP) to address the implementation challenges to scale-up of program in Pakistan. The adapted version of the intervention is called Enhanced School Mental Health Program (eSMHP). Enhancements to conventional SMHP have occurred at two levels: A) Content enhancements, such as a collaborative care model for engaging parents/primary caregivers, strategies for teacher's well-being, and adaptation and operationalization of particular clinical intervention strategies and B) Technological enhancements which include adaptation of the training manual for delivery using an online training platform, and a 'Chat-bot' to aid the implementation of intervention strategies in classroom settings.

The change in total difficulties scores of children will be measured at 9 months after commencing intervention delivery using the parent-rated Strengths and Difficulties Questionnaire (SDQ) . SDQ has 25 items and consists of sub-scales to measure emotional symptoms, conduct problems, hyperactivity/inattention, peer problems and prosocial behavior. Items are rated on a three-point Likert scale (0= not true, 1= somewhat true, 2=certainly true). Total difficulty score is calculated by summing the responses of each item in all domains except pro-social behavior items. SDQ has been culturally adapted and validated in Pakistan

Externalizing problems in children will be measured at 9 months after commencing intervention delivery, by combining conduct and hyperactivity sub-scales scores on parent-rated SDQ. Internalizing problems in children will be measured at 6 months after commencing intervention delivery, by combining emotional and peer problems subscales on parent-rated SDQ.

Child's insight into his/her problems and wellbeing would be measured using the self-administered PSYCHLOPS Kids. The outcome measure assesses three domains, including problems, functioning and well-being. PSYCHLOPS KIDS has three questionnaire forms i.e. pre-therapy, during therapy and post therapy version. Some of the items are qualitative and provide additional information about child's problem, functioning and wellbeing, but are not provided a score. Other items are rated on 0-4 scale. The maximum score for each question is 4 (scored 0-4). PSYCHLOPS pre, during and post intervention versions would be administered by assessment team at baseline, and at 9 months after commencing intervention delivery, respectively.

Child's health related quality of life during past month will be measured by parent-rated Pediatric Quality of Life (Peds-QL). The Peds-QL is 23 item impact module scale that encompasses 4 sub-scales namely physical functioning, emotional functioning, social functioning and school functioning . Items are rated on a 4-point Likert scale (1 = no problem to 4 = almost always a problem). Items are then reverse-scored and linearly transformed to a 0-100, so that higher scores indicate better quality of life. This tool yields a total score (all 23 items), physical health Summary score (8 items), Psychosocial Health Summary score (10 items) and School Functioning score (5 items).

WHODAS Child 12 is used to measure individuals' difficulties due to mental health problems across six domains including cognition, mobility, self-care, getting along, life activities, and participation, during the last 30 days. It has 12-items which are rated on a scale of 0 to 4, with summed total scores ranging from 0 to 48. The parent-rated WHO-DAS 12 - Child Version will be used to measure child's functioning at baseline and 9 months' after commencing intervention delivery.

The record of attendance and academic grades will be obtained from the school records at 9 months after commencing intervention delivery

The 12 item teachers' sense of efficacy scale will be used to assess teacher's beliefs about his or her capabilities in enhancing students' learning and ability to get through to students who are difficult or unmotivated. The scale measures teacher's sense of efficacy on three subscales namely, instructional strategies, student engagement and classroom management. The items are rated on 9 points Likert scale ranging from (1) 'None at all' to (9) 'A great deal'.

The Self-Reporting Questionnaire (SRQ) is a 20-item self-report measure to detect non-specific psychological distress, developed by the World Health Organization. Psychological distress is represented by subscales of physical symptoms and emotional symptoms . The SRQ items are scored 0 or 1. A score of 1 indicates the presence of symptoms of psychological distress during past month and a score of 0 indicates absence of symptoms. The maximum score indicates presence of higher psychological distress.

The Parent Teacher Involvement Questionnaire (PTIQ) is used to measure the frequency and quality of parental involvement in children's educational progress in school and at home. The parent version of PTIQ has 26 items while the teacher version has 21 items. The four subscales of parent-rated PTIQ include frequency of contact between parents/primary caregivers and teachers, quality of parent teacher relationship, parent's/primary caregivers' volunteering and involvement of parents/primary caregivers, parent's/primary caregivers' endorsement of their child's school, while the three subscales of teacher-rated PTIQ include quality and frequency of contact between parents/primary caregivers and teacher, parent's/primary caregivers volunteering and involvement of parents/primary caregivers, parent's/primary caregivers' endorsement of their child's school . It's a 5-point Likert scale with more than one type of response options. Higher scores indicate more parental involvement.

Psycho-Social Environment (PSE) Profile will be administered with head teachers and teachers to identify school's capacity to create healthy psycho-social environment for its staff, teachers and students. The Psychosocial Environment Profile is a 98 item scale developed by the World Health Organization to evaluate the extent to which a school's environment contributes to the social and emotional well-being of its students and staff. Each question is scored on a scale from 1 to 4, with 1 representing the lowest and 4 the highest rating of social and emotional support.

DIBQ will be used to measure the change in teachers' behavior to implement the SMHP in school and classroom settings. It is a tool based on 'The Theoretical Domains Framework (TDF)' to measure behavior change of providers. DIBQ will be used to explore the determinants that inhibit or promote the implementation of evidence-based intervention strategies by teachers. An adapted version of DIBQ, based on 18 domains of TDF and consisting of 93 items will be used to assess the change in teachers' behavior regarding implementation of SMHP. Responses on DIBQ are scored from 1 'strongly disagree' to 7 'strongly agree'. Total sum for each domain is calculated and divided by maximum score for the given domain.

The self-stigma subscale of the PaedS will be used to measure stigma in children and adolescents .It has 5 items that measure sense of shame, embarrassment, and worry about others' responses towards mental health problems. The items are rated on 4 point likert scale, where higher scores indicate greater stigmatization.

Service use and out-of-pocket expenditure of the research participants (costs for: seeing a doctor or other health care providers; admission to hospital, medicines, tests and extra help at home needed) will be collected using a validated version of the Client Services Received Inventory (CSRI). It has previously been adapted for childhood developmental disorders and autism in study settings.

Eligibility Criteria for Children Inclusion criteria for children: Children meeting the following criteria will be included in the study; i. Aged 8-13 years living with parents/primary caregivers ii. Written parent/primary caregiver informed consent or witnessed consent (in case the parent is unable to read and write, the informed consent will be obtained from both parent and witness) and child assent for participation in the study. iii. Screen positive on teacher-rated SDQ (total difficulties score > 12) and parent-rated SDQ (total difficulties score ≥ 14). Exclusion criteria for children: i. Children at high risk of abuse or harm to self or others as reported by the students themselves, teachers or parents/primary caregivers, or identified by the trained assessment team during screening. ii. Children who require immediate or on-going in-patient medical or psychiatric care, as reported by student themselves or teachers or parents/primary caregivers or identified by the trained assessment team during screening. iii. Children with deafness, blindness and speech difficulties or with developmental disorders as defined by the WHO mhGAP intervention guide identified by the trained assessment team during screening. Exclusion from the evaluation will not impact the care received by the school children as all children studying in public schools will receive intervention as part of routine implementation of WHO SMHP.

In compliance with the data sharing agreement, the unidentifiable data set will be periodically submitted to the online NIMH data repository- National Database for Clinical Trials Related to Mental Illness (NDCT).

Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997 Jul;38(5):581-6. doi: 10.1111/j.1469-7610.1997.tb01545.x.

Godfrey E, Aubrey M, Crockford S, Haythorne D, Kordowicz M, Ashworth M. The development and testing of PSYCHLOPS Kids: a new child-centred outcome measure. Child Adolesc Ment Health. 2019 Feb;24(1):54-65. doi: 10.1111/camh.12271. Epub 2018 Mar 24.

Janca A, Kastrup M, Katschnig H, Lopez-Ibor JJ Jr, Mezzich JE, Sartorius N. The World Health Organization Short Disability Assessment Schedule (WHO DAS-S): a tool for the assessment of difficulties in selected areas of functioning of patients with mental disorders. Soc Psychiatry Psychiatr Epidemiol. 1996 Nov;31(6):349-54. doi: 10.1007/BF00783424.

Varni JW, Seid M, Rode CA. The PedsQL: measurement model for the pediatric quality of life inventory. Med Care. 1999 Feb;37(2):126-39. doi: 10.1097/00005650-199902000-00003.

Tschannen-Moran, M. and A.W. Hoy, Teacher efficacy: Capturing an elusive construct. Teaching and teacher education, 2001. 17(7): p. 783-805.

Beusenberg, M., J.H. Orley, and W.H. Organization, A User's guide to the self reporting questionnaire (SRQ. 1994, Geneva: World Health Organization

Organization, W.H., Creating an environment for emotional and social well-being: an important responsibility of a health promoting and child-friendly school. 2003

Huijg JM, Gebhardt WA, Dusseldorp E, Verheijden MW, van der Zouwe N, Middelkoop BJ, Crone MR. Measuring determinants of implementation behavior: psychometric properties of a questionnaire based on the theoretical domains framework. Implement Sci. 2014 Mar 19;9:33. doi: 10.1186/1748-5908-9-33.

Moses T. Stigma and self-concept among adolescents receiving mental health treatment. Am J Orthopsychiatry. 2009 Apr;79(2):261-74. doi: 10.1037/a0015696.

Hamdani SU; Zill-e-Huma; Warraitch A, Suleman N, Muzzafar N, Minhas FA; F.R.C.Psych; Nizami AT; F.C.P.S.; Sikander S; F.C.P.S.; Pringle B, Hamoda HM, Wang D, Rahman A, Wissow LS. Technology-Assisted Teachers' Training to Promote Socioemotional Well-Being of Children in Public Schools in Rural Pakistan. Psychiatr Serv. 2021 Jan 1;72(1):69-76. doi: 10.1176/appi.ps.202000005. Epub 2020 Aug 25.