California Collaborative Network to Promote Data Driven Care and Improve Outcomes in Early Psychosis (EPI-CAL)

University of California, San Francisco, National Institute of Mental Health (NIMH), Washington University School of Medicine

The proposed project seeks to create a California early psychosis network using a core assessment battery of valid, low burden measures and mHealth technology platform to collect client-level data, visualize data via clinician dashboard for treatment planning, and integrate across clinics to provide de-identified data to the national coordinating hub. Research capacity for the network will be tested via development and validation of a measure of the Duration of Untreated Psychosis (DUP) that is feasible for use in community settings. The proposed California network will contribute systematically collected outcomes data on over 100 FEP clients per year, from 12 community and university EP clinics, to enhance the development of a national EP network, supported by the NIMH EPINET program.

A prolonged first episode of psychosis (FEP) without adequate treatment is the most consistent predictor of poor clinical and functional outcomes, poor health outcomes and significant economic burden. Team-based "coordinated specialty care" (CSC) for early psychosis (EP) has established effectiveness in promoting clinical and functional recovery. EP treatment programs have expanded rapidly with increased funding across the US without formal coordination of training or implementation. While EP programs share many features, the lack of state and national coordination and data infrastructure limits the capacity for large-scale evaluation or accelerated dissemination of best practices. Based on prior collaborations with 30 California (CA) EP programs and experiences using mobile health (MOBI mHealth) technology to measure individual outcomes in EP care, the UC Davis (UCD) team is uniquely poised to create EPI-CAL, a CA network that will contribute systematically collected outcomes data on 1329 FEP clients per year, from 6 community and 6 university EP clinics, to a national EP network supported by the NIMH EPINET program. Building on the team's prior work evaluating CA EP programs, EPI-CAL programs will participate in a formative evaluation in Year 1 to define core EP clinical features, intervention targets, and outcomes needed to harmonize network input. A "core battery" based on current measures collected at the sites, the PhenX toolkit and expanded to cover all critical domains, will be installed across the network in Year 2. Core client outcomes and metrics of data use for treatment decisions will be collected using the custom MOBI mHealth data network at the client, program, and state level to allow easy data analysis, interpretation and dissemination. Training and ongoing monitoring will be provided at all EPI-CAL sites to ensure appropriate implementation. EPI-CAL will contribute de-identified data to the national coordinating hub. Using the RE-AIM implementation science framework, the investigators will systematically evaluate the impact of MOBI on EP programs across 5 dimensions: reach, efficacy, adoption, implementation, and maintenance. To demonstrate the network's research capacity, in the R34 component of this application, the investigators propose to develop and validate a measure of the Duration of Untreated Psychosis (DUP) that is feasible for use in community settings and psychometrically sound. Although DUP is a significant predictor of both short-term CSC treatment response and long-term outcomes for FEP, no measure currently exists that has been rigorously validated and is feasible for use by community providers. The investigators will utilize stakeholder feedback (clients, family members, academic experts and CSC staff) to develop a tool with standardized DUP definitions that includes anchored assessment of psychosis onset and start of treatment. Developing such a tool will allow standardized assessment of this critical moderator of CSC outcomes across the entire EPINET. FEP (and CHR) individuals receiving early psychosis treatment services at one of the participating sites will be invited to participate in all aspects of the study. Family members/ support persons will be asked to participate in tablet data collection and provide feedback via surveys, interviews and focus groups. EP providers will complete questionnaires and provide feedback via surveys, interviews and focus groups. Stakeholders (e.g. EP program and county administrators, support staff, and local community groups) will participate in focus groups and feedback interviews. Aim 1: To create a sustainable CA EP network using a core battery of evidence-based measures.To address this aim, the investigators will test the following hypotheses: H1.1: 70% of eligible FEP participants, representative of the target population, and 50% of available family members across the network will enroll and complete baseline (Reach). Client-, provider- and program-level barriers to engagement will be identified through analyses of qualitative data (Reach). H1.2: Clinician use of MOBI over 12 months of care, as measured by MOBI, will be associated with reduced psychotic symptom severity for FEP at 12 months (Efficacy). H1.3: Clinician use of MOBI will be positively associated with reduction in psychotic symptom severity at 24 months and higher patient satisfaction with care (Maintenance). Aim 2. To develop an integrated data network that provides real-time feedback to improve clinical care and program quality, and contribute de-identified data to the national coordinating hub. The investigators hypothesize that: H2.1: Compared to pre-MOBI, providers will report increased use of data to determine treatment choices after training and using MOBI for 6 months (Adoption); H2.2: Over 12 months, EP providers will use MOBI in direct care to FEP clients for at least 50% of completed assessments (Implementation). Client-, provider- and program-level barriers to implementation will be identified through analyses of qualitative data; H2.3: Exploratory analysis will examine level of clinician expertise and training needed to effectively implement clinician review of FEP participant outcome data using MOBI at 80% of available time points (Adoption). Aim 3. To develop and validate a novel DUP measure for use in community settings (R34). The investigators hypothesize this new measure will show: H3.1: inter-rater reliability (IRR) between CSC providers and a MA-level assessor with an intra-class coefficient (ICC) of at least .80 for days from initial assessment to DUP start point, days from assessment to DUP end point, and days from start point to end point DUP (total DUP); H3.2: convergent validity, with ICCs of at least .80 between CSC providers and centralized study team assessors using the Symptom Onset in Schizophrenia Inventory (SOS) (reference standard); H3.3: Predictive validity, defined by significant relationships between shorter DUP and greater improvements in functioning and quality of life at 6 and 12 months; H3.4: Feasibility and acceptability to EP providers and clients, with a mean administration time of less than 40 minutes.

Psychosis, first episode, clinical high risk, outcomes, mobile health, data driven care, learning healthcare network

This arm of the study involves the use of the mobile health technology ("app") to measure outcomes within an early psychosis (EP) program.

A subset of individuals will participate in interviews to validate a tool to determine the duration of untreated psychosis in community settings

This mobile, app-based platform was designed to: 1) enable outcomes data collection from clients and family members/support person who are receiving care at an early psychosis program, 2) summarize the data visually for clients and providers on a secure web-based dashboard, and 3) allow download of de-identified data for program or research analysis.

A tool will be developed to enable measurement of the duration of untreated psychosis (DUP) for FEP individuals based on 1) data other assessments that are typically completed during the intake process (e.g. SIPS, SCID) or 2) specific questions, prompts, a rating scale, and anchor points to enable rating of the DUP. Participants would have their DUP rated on the new tool and also complete a second assessment of DUP by research evaluators using the Symptom Onset in Schizophrenia Inventory (SOS) to determine reliability and validity of the new tool.

Show adequate reach of enrollment using descriptive statistics showing 70% of eligible FEP participants, who are representative of the target population based on current program demographics, and 50% of their available family members, across the network were enrolled and complete baseline

Efficacy of measurement-based care, comparing adjusted mean differences in baseline to 12-month change in psychotic symptom severity on the Colorado Symptom Index (CSI), between groups defined by clinician metrics from mHealth app assessed during this 12-month period. The CSI is a 14-item, self-report scale designed to assess frequency of positive mood and cognitive symptoms. Each item is scored on a 0-4 Likert-style scale and added together to give a score between 0 and 56, with higher scores indicating greater emotional distress. Reduction in score over time is considered clinical improvement. Maintenance of measurement-based care, based on adjusted mean differences in baseline to 24-month change in psychotic symptom severity between groups defined by clinician metrics from mHealth app aggregated over the 6-, 12- and 18-month assessment period, with the primary analysis based on a composite indicator for any endorsement of "impact on treatment plan" across these three periods.

Compared to pre-implementation period, providers will report a change in the use of data to determine treatment choices after training and using the app for 6 months (Adoption). Adoption of data in care is measure by pre- and post-surveys of randomly sampled client sessions)

Over 12 months, EP providers will use mHealth app in direct care to FEP clients for at least 50% of completed assessments (Implementation) as measured by metrics gathered in mHealth app.

New DUP tool will show inter-rater reliability (IRR) between CSC providers and a MA-level assessor with an intra-class coefficient (ICC) of at least .80 for days from initial assessment to DUP start point, days from assessment to DUP end point, and days from start point to end point DUP (total DUP).

New DUP tool will show convergent validity, with ICCs of at least .80 between CSC providers and centralized study team assessors using the Symptom Onset in Schizophrenia Inventory (SOS) (reference standard).

New DUP tool will show predictive validity, defined by significant relationships between shorter DUP and greater improvements in functioning (Global Social and Role Functioning scales) at 6 and 12 months - shown by regression coefficients between DUP and change from baseline to 6 and 12 months in functioning.

New DUP tool will show predictive validity, defined by significant relationships between shorter DUP and greater improvements in quality of life (Lehman Quality of Life scale) at 6 and 12 months - shown by regression coefficients between DUP and change from baseline to 6 and 12 months in quality of life.

Feasibility and acceptability to EP providers and clients, with a mean administration time of less than 40 minutes for the brief and full versions of the new DUP Tool

Patient rated satisfaction with EP care at end of study (Maintenance) as measured on the Client Satisfaction Questionnaire (CSQ-8). The CSQ-8 is an 8-item measurement that is designed to measure client satisfaction with services. Each item is scored on a 1-4 Likert-style scale and added together to give a score between 8 and 32, with higher scores indicating greater satisfaction with care.

Exploratory analysis will examine level of clinician expertise and training needed to effectively implement clinician review of FEP participant outcome data using MOBI at 80% of available time points (Adoption).

Inclusion Criteria: FEP individuals, ages 12-30, who have experienced the onset of an affective (bipolar or major depression with psychotic features) or non-affective psychotic disorder (schizophrenia, schizoaffective, schizophreniform, brief psychotic, other specified or unspecified schizophrenia spectrum disorders) within the past 5 years and are receiving early psychosis services at one of the study sites. Family members/support persons, over age 18, of the participating FEP (or CHR) individuals are receiving early psychosis services at one of the study sites Early psychosis (EP) care providers (e.g. clinicians, physicians, nurses, support staff) who are providing care at one of the study sites. Additional stakeholders from the communities served by the study sites, including EP program and county administrators, state representatives, and local community groups as well as researchers and other experts in relevant domains. Clinical high risk (CHR) individuals, ages 12-30, who have no history of psychosis and will demonstrate attenuated psychotic symptoms consistent with the Structured Interview for Prodromal Syndromes (SIPS), or genetic risk (first-degree relative with psychosis) in conjunction with a substantial drop in functioning over the past year. Exclusion Criteria: neurological illness or injury leading to psychotic symptoms reported diagnosis of intellectual disability or estimated IQ below 70 according to the Pennsylvania Computerized Neuropsychological Test Battery.