Examining the Feasibility of a Mobile Mental Health Application in Psychotherapy

Examining the Feasibility of a Mobile Mental Health Application (Rose) for Augmentation of In-person Psychotherapy

There is an urgent need to better understand and manage the mental health burden among working adults. Up to 40% of persons with serious mental illnesses do not receive care or stop intervention programs due to negative stigma. Additionally, nearly 50 percent of respondents diagnosed with an anxiety disorder said it interfered with their relationships with coworkers. Close to $6000 USD is lost per working person each year due to depression and its effects. Rose (Recognition of Speech and Emotion) is a mental health-focused application that utilizes artificial intelligence to identify symptoms and provides targeted, tailored in-person therapy solutions for its users. The RoSE application provides solutions to supplement ongoing in-person psychotherapy. This includes journaling and daily assessments that provide curated content and feedback. The current feasibility study aims to recruit 45 established outpatients with at least mild depressive and/or anxiety symptoms who will be consented and enrolled in a five to ten-week study. There are two study arms: (1) the intervention arm and (2) a waitlist control arm. During the course of the study, the participants in the intervention arm will use the RoSE application daily. They will receive either weekly in-person psychotherapy with their established psychotherapist for a total of four sessions over four weeks or biweekly in-person psychotherapy with their established psychotherapist for a total of four sessions over eight weeks. The participants in the waitlist arm will serve as controls unless there is attrition from the intervention group at which time waitlist participants will be offered a spot in the intervention arm. The primary objectives of the study are (1) To assess the usability of the RoSE application and (2) To evaluate the short-term impact on mood and anxiety of using the RoSE application to augment in-person psychotherapy. The secondary objectives are (1) To examine the usage and utility of an in-application journaling function and (2) To examine the usage and utility of in-application curated insights.

Problem description Mental illness among working adults is of huge detriment to society on a large scale. Not only does mental illness lead to a lower quality of life and negative outcomes for the individual, but it places a burden on the economy, resulting in lost productivity, lost workdays and reduced economic growth. According to the National Institute of Mental Health an estimated 17.3 million adults in the United States had at least one major depressive episode in 2017. This number represented 7.1% of all U.S. adults. Furthermore, according to ADAA anxiety is one of the most common mental illnesses in America with around 40 million adults in the US having anxiety disorders every year. In addition to the millions lost on medical costs from mental illness, studies show an added societal burden. On average depressive symptoms can lead to close to 27 lost workdays per year and 18 days of lost productivity. While mental illness negatively affects adults in significant ways, primary care doctors identify less than one-third of all depression cases and up to 40% of persons with serious mental illnesses do not receive care or stop intervention programs due to negative stigma. With only one in four adults disclosing their anxiety disorder in time, a much smaller proportion are ever diagnosed and receive treatment for major depression. Lastly, but perhaps most importantly, according to the Center for Workplace Mental Health, 80 percent of adults treated for mental health problems report improvements in their overall satisfaction and productivity. Study model This study uses is a mental health-focused application that utilizes artificial intelligence to identify symptoms and provides targeted, tailored in-person therapy solutions for its users. The application is meant to augment in-person psychotherapy utilizing online journaling and daily patient assessments to provide curated content and feedback. In brief, the application utilizes machine learning and artificial intelligence to select content (e.g., self-help articles) and provide feedback (e.g., weekly summary score of mood and anxiety) based on user input (e.g., journal entries, daily mood rating) Study Design The proposed study is a 5-week feasibility study to assess the usability and potential short-term benefits of the mental health platform in a population of adults with established mental health care. Potential participants will be informed of the study using study brochures provided to their established psychotherapists. All written materials are submitted for approval. Interested patients will be directed to a secure online survey, which will screen them for eligibility. Patients that meet all inclusion criteria and no exclusion criteria will be offered entrance into the study. Of note, no medical records of the patients will be accessed as part of the study. After the initial screening survey, 45 eligible patients will complete the consent process. The consented participants will be randomized to one of the two study arms: (1) the intervention arm or (2) the waitlist control arm in a 2:1 ratio. The intervention arm is described in detail below. The participants in the waitlist control arm will serve as controls for the study. They will complete the pre-pilot assessment and the post-pilot assessment. It will be made clear that the participants can continue to seek a standard of care services. During their time on the waitlist, participants may reach out to study personnel if they need assistance with their psychiatric care. We aim to have 22 participants complete the intervention arm and in the setting of participant attrition from the intervention arm, participants in the waitlist control arm will be offered entrance to the intervention arm. If recruited off the waitlist, the participant will complete consent, this time selecting "Intervention" as the study arm and will be entered into the five-week intervention study. Participants in the intervention arm will be sent an individualized redemption code to download the mobile app application on to their personal mobile phones. The participants enter individual login details. During the first week of the study, each participant will be prompted by the application to complete key surveys and assessments at predetermined time intervals. The mental health platform generates a summary health metric on-demand that is a combination of responses entered from the in-app assessments. A summary health metric will be pushed in-app after the first week of use. Details for scheduling the first in-person therapy session will be provided at the same time as the summary health metric. Participants will be informed that the general health metrics will be shared with the therapist to help the therapist better plan the sessions. The therapy session will be scheduled in a private room on-campus. The therapists will offer standard-of-care psychotherapy (e.g., supportive psychotherapy, cognitive-behavioral therapy). The participants use the app for a total of 5 weeks and receive weekly in-person therapy for a total of 4 weeks (one-week application only lead-in, four weeks application plus in-person psychotherapy). Two functions of the application that will be independently examined for usage and utility are its journaling function and curated insights. Participants will be asked to use the journaling function daily (more details below). The curated insights are selected by the application based on participant responses to the daily mood and anxiety surveys, as well as built-in sentiment analysis of the journaling. At the end of 5 weeks, all participants in both study arms will complete the post-pilot assessment using the same assessments as pre-pilot. Participants who agree to be contacted post-pilot study will be given the option at 3, 6, and 12 months to take the same online assessments as at pre- and post-pilot. This evaluation will be used to test the potential long-term impact of the app. In addition, post-pilot focus groups may be completed. There would be one focus group with therapists and separate focus groups with participants. These optional assessments will be pending study results, funding, and resources and would be submitted in the future as a change in research to the current Institutional Review Board (IRB). Details of Study Assessments Scales / Questionnaires Mood / Anxiety Meter: The application will send timed notifications three times a day (morning, afternoon and evening) for the user to pick current mood and anxiety on a Likert scale General Anxiety Disorder-7 (GAD-7) anxiety scale. The app will prompt the user to fill in this assessment every 2 weeks. The GAD-7 parallels the 7 diagnostic symptom criteria that define DSM-IV GAD. The format and temporal framework of the items also correspond to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria and will facilitate the follow-up review of symptoms and diagnostic processes. At only 7 items, the GAD-7 is substantially shorter than most anxiety screening measures. This instrument has also demonstrated acceptability among non-psychiatric patients and among busy primary care providers and is a valid and reliable screening tool for anxiety. Patient Health Questionnaire-9 (PHQ-9) depression scale. The app will prompt the user to fill in this assessment every 2 weeks. The PHQ-9 parallels the 9 diagnostic symptom criteria that define DSM-IV. The format and temporal framework of the items also correspond to the DSM-IV criteria and will facilitate the follow-up review of symptoms and diagnostic processes. At only 9 items, the PHQ-9 is substantially shorter than most depression screening measures. This instrument has also demonstrated acceptability among non-psychiatric patients and among busy primary care providers and is a valid and reliable screening tool for major depression. Mobile Application Rating Scale (MARS): The MARS is a well-established framework for classifying and assessing the objective and subjective quality of apps, as well as the app's perceived impact. It is designed to score apps on the criteria of engagement, functionality, aesthetics, and information quality, as well as app subjective quality. Each MARS item used a 5-point scale (1-Inadequate, 2-Poor, 3-Acceptable, 4-Good, 5-Excellent). The MARS is calculated as mean scores of the engagement, functionality, aesthetics, and information quality objective sub-scales, and an overall mean app quality total score. Other Online Journaling: The online journaling feature allows the user to enter his/her feelings and thoughts, like in the case of a diary. The user is asked daily "How has your day been? (please provide at least four sentences to describe how you are doing and feeling today)". Every two weeks the user is asked "How have you been the last two weeks? (please provide at least four sentences describing your emotional well-being over the last two weeks)". This second prompt is meant to allow comparison with the PHQ-9 and GAD-7, both of which ask about a two-week time frame's neural network is built and calibrated to provide an emotional context through pre-labeled datasets that tag words, phrases, and sentences to a specific emotion (e.g., fear, sadness, anxiety). The neural network model then analyzes the input from the journaling features and assigns it an emotion weighting. User input is then analyzed and the output indicates what condition participants are likely currently experiencing based on participant's entry.

The current study aims to recruit 45 patients with at least mild depressive and/or anxiety symptoms who will be consented and enrolled in a five-week study.The consented participants will be randomized to one of the two study arms: (1) the intervention arm or (2) the waitlist control arm in a 2:1 ratio. During the course of the study, the participants in the intervention arm will use the mental health application daily. Participants will receive weekly in-person psychotherapy for a total of four sessions over four weeks. Licensed therapists will provide the in-person psychotherapy. The participants in the waitlist arm will serve as controls unless there is attrition from the intervention group at which time waitlist participants will be offered a spot in the intervention arm.

Eligible participants had been scheduled for either weekly or biweekly sessions with their therapists. Participants in the intervention arm were asked to download the Rose application on to their personal mobile phones and were given their therapist's unique ID for verification in the app. During the first week of the study, participants were prompted to complete key surveys and assessments at predetermined time intervals. Participants seen weekly will use the app for a total of 5 weeks and receive weekly in-person therapy for a total of 4 weeks (one-week application only lead-in, four weeks application plus in-person psychotherapy). Participants seen biweekly will use the app for a total of 10 weeks and receive biweekly in-person therapy for a total of 8w weeks (two-week application only lead-in, eight weeks application plus in-person psychotherapy).

The participants in the waitlist control arm served as controls for the study. They completed the pre-pilot and post-pilot assessments only. The waitlist participants continued their standard care and were offered entrance to the intervention arm at the end of the study period or earlier if patients in the intervention arm dropped out mid-study. During their time on the waitlist, participants could reach out to study personnel if they needed assistance with their psychiatric care.

Rose is a patient-centric, mobile mental health platform designed for in-between clinical mental health monitoring in form of mood tracking, clinical surveys and daily journaling. Rose is used as a participant-facing mobile app and clinician-facing dashboard. Each day, they complete mood and anxiety scales and update their journals. Biweekly, they are also asked to complete the PHQ-9 and GAD-7 assessments as well. The Rose mobile app sends timely self-help articles and notifications for users. The curated insights are selected based on participant responses to the daily mood and anxiety surveys, as well as a built-in sentiment analysis of the journaling. The clinical dashboard is a secure, web-based portal that has a unique login for each provider. Providers have access to only their patients through the system and are able to see all the health metrics collected with the Rose mobile app and the summarized metrics produced by our models.

Over the study period, the mobile app will collect time (date/time) the user fills in the Mood Likert Scale. We will calculate the average number of times the user fills in the scale each week.

Over the study period, the mobile app will collect time (date/time) the user fills in the Anxiety Likert Scale. We will calculate the average number of times the user fills in the scale each week.

Over the study period, the mobile app will collect time (date/time) the user fills the online journal. We will calculate the average number of times the user fills in the scale each week.

Over the study period, the mobile app will prompt the user to fill in the PHQ-9 survey. The app will collect time (date/time) the user fills this in. We will record the level of adherence: Full ( Baseline filled) vs. Partial (Baseline not filled)

Over the study period, the mobile app will prompt the user to fill in the PHQ-9 survey. The app will collect time (date/time) the user fills this in. We will record the level of adherence: Full ( Week 2 filled) vs. Partial (Week 2 not filled)

Over the study period, the mobile app will prompt the user to fill in the PHQ-9 survey. The app will collect time (date/time) the user fills this in. We will record the level of adherence: Full ( Week 4 filled) vs. Partial (Week 4 not filled)

Over the study period, the mobile app will prompt the user to fill in the GAD-7 survey. The app will collect time (date/time) the user fills this in. We will record the level of adherence: Full ( Baseline filled) vs. Partial (Baseline not filled)

Over the study period, the mobile app will prompt the user to fill in the GAD-7 survey. The app will collect time (date/time) the user fills this in. We will record the level of adherence: Full ( Week 2 filled) vs. Partial (Week 2 not filled)

Over the study period, the mobile app will prompt the user to fill in the GAD-7 survey. The app will collect time (date/time) the user fills this in. We will record the level of adherence: Full ( Week 4 filled) vs. Partial (Week 4 not filled)

At the end of the study (week 5), participants will be asked to complete the Mobile Application Rating Scale (MARS) to score the intervention app on the criteria of engagement, functionality, aesthetics, and information quality, as well as app subjective quality. Each MARS item used a 5-point scale (1-Inadequate, 2-Poor, 3-Acceptable, 4-Good, 5-Excellent). The MARS is calculated as mean scores of the engagement, functionality, aesthetics, and information quality objective sub-scales, and an overall mean app quality total score.

The GAD-7 is a self-administered 7 item instrument that uses some of the DSM-V criteria for anxiety to identify probable cases of GAD along with measuring anxiety symptom severity. Responders are asked to rate the frequency of anxiety symptoms in the last 2 weeks on a Likert scale ranging from 0-3. Severity can be determined by examining the total score. A meaningful change is 5 or more points.

The PHQ-9 is a self-administered 9 item instrument that uses some of the DSM-V diagnostic symptom criteria that define DSM-IV Major Depressive Disorder. Responders are asked to rate the frequency of depressive symptoms in the last 2 weeks on a Likert scale ranging from 0-3. Severity can be determined by examining the total score. A meaningful change is 5 or more points.

Inclusion Criteria: English speaking; ->=18 years of age; Score >5 on PHQ-9 and/or >5 on GAD-7; Have an accessible smartphone with a data plan; Ability to dedicate one hour a week for therapy sessions; Ability to give informed consent and understand the tasks involved. Exclusion Criteria: Current psychosis (e.g., hallucinations, delusions, thought disorder); Current suicidal ideation or homicidal ideation

Economides M, Ranta K, Nazander A, Hilgert O, Goldin PR, Raevuori A, Forman-Hoffman V. Long-Term Outcomes of a Therapist-Supported, Smartphone-Based Intervention for Elevated Symptoms of Depression and Anxiety: Quasiexperimental, Pre-Postintervention Study. JMIR Mhealth Uhealth. 2019 Aug 26;7(8):e14284. doi: 10.2196/14284.

Goldin PR, Lindholm R, Ranta K, Hilgert O, Helteenvuori T, Raevuori A. Feasibility of a Therapist-Supported, Mobile Phone-Delivered Online Intervention for Depression: Longitudinal Observational Study. JMIR Form Res. 2019 Jan 22;3(1):e11509. doi: 10.2196/11509.

Giosan C, Cobeanu O, Mogoase C, Szentagotai A, Muresan V, Boian R. Reducing depressive symptomatology with a smartphone app: study protocol for a randomized, placebo-controlled trial. Trials. 2017 May 12;18(1):215. doi: 10.1186/s13063-017-1960-1.

Adam A, Jain A, Pletnikova A, Bagga R, Vita A, N Richey L, Gould N, Munshaw S, Misrilall K, Peters ME. Use of a Mobile App to Augment Psychotherapy in a Community Psychiatric Clinic: Feasibility and Fidelity Trial. JMIR Form Res. 2020 Jul 3;4(7):e17722. doi: 10.2196/17722.