Evaluate the Value of Telehomecare for Mental Health

Lakeridge Health Corporation, Ontario Shores Centre for Mental Health Sciences, Ontario Telemedicine Network (OTN)

This is a mixed methods study that evaluates the effectiveness of the Big White Wall (BWW) online community and its implementation in 3 hospital/health system sites across Ontario. The 6 month study includes 1000 participants that are randomized to an immediate treatment group or a delayed treatment group. The primary outcome includes a change in total and subscale scores on the Recovery Assessment Scale. Qualitative interviews with various stakeholders will explore issues relating to successful implementation.

The study consists of a pragmatic randomized controlled trial (RCT) with a nested comparative effectiveness arm to assess the value of intervention extension, with concurrent Realist Evaluation. The pragmatic randomized trial will assess whether the intervention (the Big White Wall) works when introduced into a public health or clinical setting, i.e., in real life conditions. The qualitative realist evaluation will assess the contextual influences and strategies by which the Big White Wall is adopted or rejected. Pragmatic Randomized Control Trial (RCT) with nested comparative effectiveness arm: The RCT will include a delayed treatment crossover control. The immediate treatment group (ITG) will get access to the BWW at time of enrollment and have continuous access for a period of 3 months. The delayed treatment group (DTG) will have no access for the first 3 months, then receive access to the BWW for 3 consecutive months. A 2:1 allocation ratio will be used to randomized participants to ITG or DTG. After 3 months of intervention, ITG participants will have the opportunity to opt-in to an intervention extension arm. Those who opt-in will be randomized at a 1:1 ratio to receive an additional 3 months of BWW or no additional intervention. Objective: To determine the incremental benefit and cost-effectiveness of the BWW for patients with mental health problems who seek care at hospital-based mental health programs. Hypothesis: The investigators hypothesize that users of the BWW will experience an increase in their mental health recovery orientation as evidenced using a standardized recovery assessment tool. The investigators also hypothesize that improvements will be observed in symptoms of depression and anxiety, along with quality of life and community integration. Recruitment/Study Setting: A total of 1000 study participants will be recruited from 3 large mental health settings in Ontario: Ontario Shores Mental Health Sciences, Lakeridge Health, and Women's College Hospital (WCH). Lakeridge Health and Ontario Shores both have inpatient and outpatient mental health services that support individuals with mental health issues across the lifespan while Women's College Hospital has a large outpatient mental health program. Intervention: All study participants will be given access to the BWW, free of charge, for 3 months or 6 months for those randomized to the ITG extension arm. Services on the BWW include peer support, self-guided and facilitated self-help resources, psycho-educational material and interactive creative activities. All participants will maintain anonymity on the site through a unique non-identifiable user identifier (ID). All participants will receive a unique access code for the BWW. Use of the BWW is participant dependent, with the ability to log in and utilize the services at any frequency. Data Collection: Baseline socio-demographic data will be collected following consent. Self-report data on all outcomes will be collected via questionnaires at baseline, 3 months and 6 months. Baseline outcome questionnaires will be completed in 1 of 3 ways: 1) web survey (default); 2) by phone; 3) in-person hard copy. All other questionnaires will be administered with web-based surveys. Data Analysis: Data analysis will be done at study completion, after all data has been collected. Analysis will be blinded to treatment allocation. Initially, descriptive analysis will be conducted for all variables. The primary outcome, The Recovery Assessment Scale-Revised (RAS-r) at 3 months, will be analyzed with an intent-to-treat analysis using an ANCOVA controlling for baseline RAS-r score as well as treatment group, unadjusted and adjusted for baseline Patient Health Questionnaire (PHQ-9), baseline Generalized Anxiety Disorder (GAD-7), age, sex, education, relationship status, household income, duration of episode, and recruitment setting. In sensitivity analysis, the investigators will repeat this using a marginal structural model to account for attrition. The same analysis will be repeated for all secondary outcomes at 3 months controlling for baseline score and treatment group. In the subset of ITG participants who opt in to the nested extension study, the investigators will examine outcomes at 6 months between treatment groups. Analysis of primary and secondary outcomes will be repeated as described above, controlling for scores at both baseline and 3 months. The first exploratory analysis will examine a subset of the ITG group who had a PHQ-9 or GAD-7 score of at least 10 at baseline. Participants will be categorized as 'responders' or 'non-responders' based on whether or not they achieved at least a 50% reduction in the PHQ-9 or GAD-7 at 3 months relative to baseline. Second, the investigators will examine engagement with the BWW among ITG participants. The number of logins and total time on the site will be separately predicted with age, gender, education, relationship status, living situation, household income, baseline belief in treatment credibility and outcome expectancy, baseline PHQ-9 and GAD-7 scores, duration of current episode, recruitment setting and outpatient mental health visits. In terms of economic evaluation, the combination of program costs, out-of-pocket costs and costs in the health care system will be used to determine the total costs associated with participants in the program. Incremental cost-effectiveness will be assessed using an incremental cost-effectiveness ratio calculated as the difference in costs between DTG and ITG divided by the difference in outcomes between these groups. Qualitative Realist Evaluation Protocol The Realist Evaluation will occur alongside the pragmatic randomized trials described above, and will include two key methods. The first is qualitative interviews with key stakeholders involved in the implementation and use of BWW, and the second is observations of the introduction and use of the virtual care interventions among health care provider participants. Objective: The objective of the qualitative realist evaluation component of this study is to (a) understand perspectives of mental health care providers regarding the appropriateness of BWW for their patients, (b) explore participants' perspectives of the usability and acceptability of the BWW online community, and (c) examine the key issues associated with scaling BWW up across Ontario. Recruitment: At the time of recruitment into the broader pragmatic randomized trial of which this qualitative study is a part, patient participants will be asked if they consent to be contacted with further information regarding the qualitative element of the study. The qualitative research team will purposefully sample from the pool of interested ITG participants who consented to be contacted for an interview at study enrollment. The investigators aim to interview 4-6 participants at each site, spanning different age categories, for a total of 12-15 patient participants. Qualitative interviews will be completed with 5-7 health care providers at each site; 2-4 organizational leaders at each site (for example, clinical managers); and 5-7 health system decision makers in Ontario. These participants will be identified by clinical site leads using a snowball sampling process to identify key informants. Data Collection: Patient interviews will be conducted via telephone or personal video conferencing. Patients will be invited to participate in two interviews 1-2 weeks after gaining access to the BWW and again at 8-12 weeks after gaining access. Observation of the educational session in which health care providers and other staff are educated regarding the BWW will take place. Participants who opt in for the intervention extension arm of the study will again be asked whether they are interested in participating in a qualitative interview. Participants who identify as willing to be contacted may or may not have participated in a qualitative interview in the first phase of the study, and will be purposefully sampled and interviewed 2-4 weeks after the re-randomization process in the same manner as the initial interviews. The investigators will again recruit 2-3 participants at each of the 3 sites for a total of 12-15 patient participants from the extension arm. Healthcare providers will also be invited to participate in an interview 1-2 weeks post-education session on the BWW. Health care provider interviews will take place at their location of work or via telephone. Organizational leaders will be invited to participate in an interview in person or via telephone 3 to 4 months post BWW implementation. Health system decision makers will be invited to participate in an interview in person or via telephone 3 to 6 months post BWW implementation. Patient interviews will include general questions about how the patient manages his or her mental health in the community, their perspectives of the BWW, and how they use the BWW. If the patient has not engaged with the BWW, the qualitative interview will include questions about why not, and what alternative strategies the patient uses to manage mental health in the community. Health care provider interviews will include questions about health care providers' perspectives on the BWW community, mental health services in Ontario, and the value of virtual care interventions such as the BWW for promoting self-management of mental health overall. Interviews with organizational leaders and health system decision makers will include questions about the effectiveness of virtual care interventions to promote mental health self-management in the community, the context of mental health services in Ontario, and the procurement and implementation of virtual care interventions such as the BWW. Data Analysis: Observation and interview data will be analyzed using thematic analysis strategies, identifying key themes that demonstrate important contextual influences and practices related to the implementation and evaluation of the virtual care technologies in actual contexts of health care delivery. The findings of the qualitative data will be used to develop statements of the relationships between (a) key contextual factors, (b) the mechanisms by which they effect the implementation of the virtual care interventions, and (c) the impact on the outcomes of the intervention themselves. These statements will be used to develop understanding of (a) the specific mechanisms by which BWW is effective for users, and (b) strategies to inform the future implementation of BWW and/or similar interventions on a larger scale.

The Immediate Treatment Group will receive access to the study intervention - the Big White Wall, immediately after consenting for a total duration of 3 months.

The Delayed Treatment Group will have no access to the study intervention - the Big White Wall for the first 3 months, then receive access to the BWW for 3 consecutive months.

Immediate Treatment Group participants who opt in to the nested study and are randomized to the The Immediate Treatment Group - Extension, will receive access to the Big White Wall for an additional 3 months (months 4-6), immediately after receiving the initial 3 months of access.

Immediate Treatment Group participants who opt in to the nested study and are randomized to The Immediate Treatment Group - No extension, will not receive extended access to the Big White Wall.

The BWW is an internet-based intervention that incorporates several elements of mental health care that have been shown to be effective, in a virtual environment that provides anonymity (see www.bigwhitewall.com). The on-line portal gives users access to self-assessment tools to track symptoms for a wide range of conditions over time, educational materials, a social network that allows communication between people who may have similar problems, and access to on-line courses. The service is monitored at all times by "Wall Guides" to ensure that users are responded to and that the content posted is appropriate and safe.

Community integration questionnaires assesses home, social, and work integration. Outcome is total and subscale scores.

RAS-r Subscales, total score on 5 subscales: 1) personal confidence and hope, 2) willingness to ask for help, 3) goal and success orientation, 4) reliance on others, and 5) not dominated by symptoms.

Use of hospital, community and home services, self-help resources, medication, and out of pocket costs of care.

Rate of utilization of BWW including: logins (number; time of day; average length and/or total time on site), activity (number of comments in talkabouts, number of bricks and brick comments, number of useful stuff pages viewed), guided support (number and type of courses taken, test scores - where tests are compulsory), and support network (results with dates).

Community integration questionnaires assesses home, social, and work integration. Outcome is total and subscale scores.

RAS-r Subscales, total score on 5 subscales: 1) personal confidence and hope, 2) willingness to ask for help, 3) goal and success orientation, 4) reliance on others, and 5) not dominated by symptoms.

Inclusion Criteria: 16 or older Referred or assessed for a mental health problem (any diagnosis may be considered, but main target is individuals with mood or anxiety symptoms and related difficulties) Able to provide informed consent Willing and able to access an on-line mental health intervention Has or is willing to obtain and use an email address (the study can support participants to get access to an email address if needed) Able to read English Able to navigate an on-line tool independently or with minimal assistance If relevant: Being discharged from the emergency department (ED) or prompt care clinic Clinician assessing patient feels the BWW may be useful (applies to ED and prompt care) Exclusion Criteria: 1. Symptoms or behaviours that the assessing clinician feels may be disruptive or negatively impactful for the individual or others on the BWW

There are agreements in place to share individual participant data between all the study sites and Women's College Hospital.

Hensel JM, Shaw J, Ivers NM, Desveaux L, Vigod SN, Cohen A, Onabajo N, Agarwal P, Mukerji G, Yang R, Nguyen M, Bouck Z, Wong I, Jeffs L, Jamieson T, Bhatia RS. A Web-Based Mental Health Platform for Individuals Seeking Specialized Mental Health Care Services: Multicenter Pragmatic Randomized Controlled Trial. J Med Internet Res. 2019 Jun 4;21(6):e10838. doi: 10.2196/10838.

Hensel JM, Shaw J, Ivers NM, Desveaux L, Vigod SN, Bouck Z, Onabajo N, Agarwal P, Mukerji G, Yang R, Nguyen M, Jeffs L, Jamieson T, Bhatia RS. Extending access to a web-based mental health intervention: who wants more, what happens to use over time, and is it helpful? Results of a concealed, randomized controlled extension study. BMC Psychiatry. 2019 Jan 24;19(1):39. doi: 10.1186/s12888-019-2030-x.

Hensel JM, Shaw J, Jeffs L, Ivers NM, Desveaux L, Cohen A, Agarwal P, Wodchis WP, Tepper J, Larsen D, McGahan A, Cram P, Mukerji G, Mamdani M, Yang R, Wong I, Onabajo N, Jamieson T, Bhatia RS. A pragmatic randomized control trial and realist evaluation on the implementation and effectiveness of an internet application to support self-management among individuals seeking specialized mental health care: a study protocol. BMC Psychiatry. 2016 Oct 18;16(1):350. doi: 10.1186/s12888-016-1057-5.