An Implementation Study of Suicide Risk Management Among Discharged Psychiatric Patients

An Implementation Study of Suicide Risk Management Among Discharged Psychiatric Patients Based on Brief Contact Interventions and Sequential Multiple Assignment Randomized Trial

The post-discharge suicide risk among psychiatric patients is significantly higher than it among patients with other diseases and general population. The brief contact interventions (BCIs) are recommended to decrease the risk in areas with limited mental health service resource like China, however the best frequency to implement BCIs is unknown. This implementation study aims to 1) to develop an intervention strategy against post-discharge suicide based on BCIs for Chinese psychiatric patients; 2) to determine the best frequency of BCIs based on Sequential Multiple Assignment Randomized Trial; 3) to evaluate the effectiveness of the intervention strategy and explore its implementability based on the Implementation Outcome Framework (IOF). Based on the community-based participatory research (CBPR) approach, this study will invite psychiatric patients and family members, psychiatrist and nurses, community mental health workers and social workers as the community team to develop a post-discharge suicide intervention strategy. The study will recruit patients with psychotic symptoms and with major depressive disorder discharged from Shenzhen Kangning Hospital (SKH) in a Sequential Multiple Assignment Randomized Trial (SMART) to determine the best frequency for implementing BCIs and to evaluate the effectiveness. Participants will be randomized into two intervention groups to receive BCIs at different frequencies. Follow-ups to evaluate participants' suicide risk are scheduled at 1, 3, 6 and 12 months after discharge. The re-randomization will be applied at 3 months after discharge. With the Intent-to-treat (ITT) approach, generalized estimating equation (GEE) and survival analysis (SA) will be applied to compare the effectiveness among groups and to explore factors associated with suicide risk. Meanwhile, this study will collect qualitative and quantitative information on implementation and service outcomes from the community team.

This is a mixed-methods study with two stages. The first stage is to develop the intervention strategy by individual in-depth and focus groups interviews; and the second stage is to implement the strategy and evaluate the implementation quantitatively by a randomized trial and qualitatively by focus group interviews. The community-based participatory research The investigators aim to recruit discharged psychiatric patients and their lay health care supporters (LHSs) who are usually their family members, psychiatrists and nurses, psycho-crisis intervention team members, community mental health workers and mental health social workers as the community team that will provide a Chinese context under the community-based participatory research (CBPR) framework. In specific, the framework would help this study: explore the feasibility of implementing BCIs against suicide risk after discharge, understand the needs for suicide risk management after discharge from related health care service providers and acceptors, integrate suicide risk management experiences from the community, discuss, develop, and revise the intervention strategy with the community. The investigators categorize the community team into three sub-groups, the patients-LHSs group, the clinic mental health service provider group (psychiatrists and nurses, and psycho-crisis intervention team members), and the community mental health service provider group (community mental health workers and mental health social workers). 1.1 Intervention development The investigators will conduct three focus group interviews in each sub-group and ten to fifteen cases of individual in-depth interview with the community to avoid bias in focus groups and to protect privacy related to personal experience in suicide and suicide intervention. The themes include: 1) key points in suicide risk management after discharge, 2) how to develop BCIs content and delivery BCIs appropriately and feasibly to increase social connectedness and social support, 3) how to improve compliance to treatment and increase subsequent visits after discharge. There will be scheduled meetings with the community to discuss and revise the intervention strategy before implementation. 1.2 Implementation evaluation Based on IOF, the investigators will conduct three focus group interviews in each sub-group to explore 1) patients' and LHSs' attitudes, acceptability, and understanding of the strategy, 2) the clinic and community mental health service providers' willingness, feasibility and sustainability to implement the strategy, 3) the effectiveness, efficiency, equity, safety and timeliness of the strategy and whether it is patient-centered. 1.3 The qualitative study sample Purposive sampling will be applied to recruit participants for the community team. For each type of sub-group, there will be five to eight members. The sequential multiple assignment randomized trial The investigators will conduct the sequential multiple assignment randomized trial (SMART) to determine the best frequency to implement BCIs and investigate the patient outcomes in IOF. The SMART design reflects the idea of adaptive treatment strategies and dynamic treatment regimens that provide a sequence of decisions about the points at which to offer different interventions and a set of intervention options for each decision point. There will be two stages of treatment. Stage 1: After recruitment and baseline survey, participants will be randomized into Group 1 and Group 2 where BCIs will be implemented monthly and weekly, respectively. Because suicide risk is the highest in the first three months among discharged psychiatric patients, the investigators set the check point at three months after discharge to assess participants' suicide risk in both groups. Stage 2: At the check point, for participants in Group 1, if the suicide risk increased, they will be re-randomized into Group 1a and Group 1b where BCIs will be implemented weekly and bi-weekly, respectively; if the suicide risk decreased or did not change, they will remain receiving BCIs monthly as Group 1c. For participants in Group 2, if the suicide risk increased or did not change, they will remain receiving BCIs weekly as Group 2a; if the suicide risk decreased, they will be re-randomized into Group 2b and Group 2c where BCIs will be implemented monthly and bi-weekly, respectively. After the re-randomization, participants will continue to receive BCIs until 12 months after discharge, and the suicide risk will be evaluated at 1, 3, 6 and 12 months after discharge. 2.1 The quantitative study sample The investigators plan to implement the strategy in patients with psychotic symptoms and patients with major depressive disorder (MDD), as in representative of severe and non-severe mental disorders, separately. 2.2 Sample size The sample size was calculated to estimate the primary effect in SMART trial. The investigators set the rate of type I error α at 0.05, the rate of type II error β at 0.20, the power (1-β) at 0.80, the moderate effect size d at 0.3566, and the sample size is 130 for Group 1 and Group 2, 260 in total; considering dropout, the sample size will be increased by 20%, and the final sample size is 312 participants. The investigators will conduct two SMART trials in patients with psychotic symptoms and MDD separately, and the sample size for each trial is 312 (624 patients in total). 2.3 Randomization and mask After recruitment and the baseline survey, participants will be assigned into Group 1 and Group 2 by simple randomization in R program. At the check point in the SMART trial, participants will be re-assigned into Group 1a, Group 1b, Group 1c, Group 2a, Group 2b, and Group 2c based on their suicide risk by simple randomization in R program. The allocation ratio in randomization will be 1:1. Participants, LHSs, nurses who perform recruitment and baseline survey, and investigators who perform follow-ups will be blinded to the assignment. 2.4 Brief contact intervention The BCI in this study is a series of structured messages. Messages will be delivered to participants by pushing feeds through WeChat and an iOS/Android application developed for this study. If participants did not use smartphones, messages will be delivered by mobile text messages or by phone calls. Though the final details is yet to be determined by the CBPR study, the investigators expect to structure messages into six components including introduction, greetings for previous complains, mental health promotion, encouragement, and coping strategies, remind of treatment and subsequent visit, and crisis intervention resource. Noted, the same messages will also be sent to patients' LHSs to remind patients through their families for subsequent visits and upcoming follow-up surveys, and to remind LHSs that patients are at risk of post-discharge suicide and need attention and care, and the necessities of seeking crisis intervention in a timely manner. 2.5 Data collection To evaluate post-discharge suicide risk more cautiously and to provide crisis intervention in time, we will conduct face-to-face interview to collect information. After research assistants introduce the study and obtain written informed consent, trained nurses in SKH will recruit participants and perform baseline survey before discharge. As mentioned, we encourage subsequent visits to SKH out-patient clinics in BCIs, and research assistants will contact participants to schedule out-patient visits and complete follow-up surveys during the visits at 1, 3, 6 and 12 months after discharge. If participants refused follow-ups in out-patient settings, we would schedule home visits to complete the survey by research assistants and community mental health workers. If patients did not respond, research assistants will contact their LHSs to obtain participants' recent updates and help them schedule out-patient visits for patients if necessary. Dropout is defined as 1) participants or their LHSs request to quit the study and stop receiving any brief contact messages; 2) participants or their LHSs refuse follow-up surveys either at out-patient clinics or at home; 3) participants pass away by accidents or other health problems except suicide. Particularly, at each time point of follow-ups, we will contact patients and LHSs up to three times. If neither of them responded, they would be treated as dropout. Study outcomes and measurements The implementation outcomes of the overall project are based on the Implementation Outcomes Framework, in which primary and secondary outcomes of the SMART trial will be collected at 1, 3, 6 and 12 months after discharge.

Psychiatric patients, Post-discharge suicide, Brief contact interventions, Sequential multiple assignment randomized trial, Implementation science

After recruitment and baseline survey, participants will be randomized into two groups Group 1 and Group 2 where BCIs will be implemented monthly and weekly. Participants' suicide risk will be evaluated at three months after discharge. At the check point, for participants in Group 1, if the suicide risk increased, they will be re-randomized into Group 1a and Group 1b where BCIs will be implemented weekly and bi-weekly; otherwise, they will remain receiving BCIs monthly as Group 1c. For participants in Group 2, if the suicide risk increased or did not change, they will remain receiving BCIs weekly as Group 2a; otherwise, they will be re-randomized into Group 2b and Group 2c where BCIs will be implemented monthly and bi-weekly.

After recruitment and the baseline survey, we will assign participants into Group 1 and Group 2 by block randomization in R program. At the check point in the SMART trial, we will re-assign participants into Group 1a, Group 1b, Group 1c, Group 2a, Group 2b, and Group 2c based on their suicide risk by simple randomization in R program. The allocation ratio in randomization will be 1:1. The randomization will be performed by a statistician in the research team. Patients, LHSs, nurses who perform recruitment and baseline survey, the statistician who performs randomization, and investigators who perform follow-ups will be blinded to the assignment.

After recruitment and baseline survey, participants will be randomized into Group 1 and Group 2. In Group 1, participants will receive brief contact intervention (BCI) monthly. If participants' suicide risk increased at 3 months after discharge, they will be re-randomized into Group 1a and Group 1b to receive BCIs weekly (Group 1a) and bi-weekly (Group 1b).

After recruitment and baseline survey, participants will be randomized into Group 1 and Group 2. In Group 1, participants will receive brief contact intervention (BCI) monthly. If participants' suicide risk increased at 3 months after discharge, they will be re-randomized to receive BCIs weekly (Group 1a) and bi-weekly (Group 1b).

After recruitment and baseline survey, participants will be randomized into Group 1 and Group 2. In Group 1, participants will receive brief contact intervention (BCI) monthly. If participants' suicide risk decreased or did not change, they will remain receiving BCIs monthly as Group 1c.

After recruitment and baseline survey, participants will be randomized into Group 1 and Group 2. In Group 2, participants will receive brief contact intervention (BCI) weekly. If the suicide risk increased or did not change, they will remain receiving BCIs weekly as Group 2a.

After recruitment and baseline survey, participants will be randomized into Group 1 and Group 2. In Group 2, participants will receive brief contact intervention (BCI) weekly. If the suicide risk decreased, they will be re-randomized to receive BCIs monthly (Group 2b) and bi-weekly (Group 2c).

After recruitment and baseline survey, participants will be randomized into Group 1 and Group 2. In Group 2, participants will receive brief contact intervention (BCI) weekly. If the suicide risk decreased, they will be re-randomized to receive BCIs monthly (Group 2b) and bi-weekly (Group 2c).

The BCI in this study is a series of structured messages, and messages will be delivered to participants by pushing feeds through WeChat and an iOS/Android application developed for this study. If participants did not use smartphones, messages will be delivered by mobile text messages or by phone calls. Though the final details are yet to be determined by the CBPR study, the investigators expect to structure messages into six components including introduction, greetings for previous complains, mental health promotion, encouragement, and coping strategies, remind of treatment and subsequent visit, and crisis intervention resource. Noted, the same messages will also be sent to patients' LHSs.

This study will use the Beck Suicide Ideation Scale-Chinese Version (BSI-CV) to assess participants' suicide ideation. The BSI-CV includes 19 items, and each item scores from 0 to 2 with a total score ranging from 0 to 38, and a higher score indicates a higher level of suicide ideation. The total score's trajectory from baseline to three months after discharge will be recorded and compared.

This study will use the Beck Suicide Ideation Scale-Chinese Version (BSI-CV) to assess participants' suicide ideation. The BSI-CV includes 19 items, and each item scores from 0 to 2 with a total score ranging from 0 to 38, and a higher score indicates a higher level of suicide ideation. The total score's trajectory from baseline to 12 months after discharge will be recorded and compared.

Social connectedness will be measured by the Social Connectedness Scale (SCS) to evaluate participants' social connected ness after discharge. The SCS is a 20-item scale, and each item is on a 6-Likert continuum (from "Strongly disagree" to "Strongly agree") scoring from 1 to 6. A higher total score indicates a higher level of social connectedness. The total score's trajectory from baseline to three months after discharge will be recorded and compared.

Social connectedness will be measured by the Social Connectedness Scale (SCS) to evaluate participants' social connected ness after discharge. The SCS is a 20-item scale, and each item is on a 6-Likert continuum (from "Strongly disagree" to "Strongly agree") scoring from 1 to 6. A higher total score indicates a higher level of social connectedness. The total score's trajectory from baseline to 12 months after discharge will be recorded and compared.

Social support will be measured by the 23-item Duke Social Support Index (DSSI) to evaluate participants' social support after discharge. The DSSI investigates social support by social interaction, perceived social support and instrumental social support. Every answer has been assigned a score, and the total reflects the sum of the items ranging from 11 to 45. A higher total score indicates a higher level of social support. The total score's trajectory from baseline to three months after discharge will be recorded and compared.

Social support will be measured by the 23-item Duke Social Support Index (DSSI) to evaluate participants' social support after discharge. The DSSI investigates social support by social interaction, perceived social support and instrumental social support. Every answer has been assigned a score, and the total reflects the sum of the items ranging from 11 to 45. A higher total score indicates a higher level of social support. The total score's trajectory from baseline to 12 months after discharge will be recorded and compared.

This study will use the Beck Suicide Ideation Scale-Chinese Version (BSI-CV) to assess participants' suicide ideation. The BSI-CV includes 19 items, and each item scores from 0 to 2 with a total score ranging from 0 to 38, and a higher score indicates a higher level of suicide ideation. The total score's trajectory from three months after discharge to 12 months after discharge will be recorded and compared.

Responses to the question "How many times have you been hospitalized for mental disorders" will be recorded and compared.

Responses to the question "How many times have you called the research team or the Crisis Intervention Hotline for help after discharged from hospital?" will be recorded and compared.

This is a mixed methods study, and the investigators will conduct community-based participatory research (CBPR) to evaluate the implementation of the intervention. In the second implementation evaluation stage of the study, focus group interviews will be conducted with the CBPR team understand the community's attitudes towards the acceptability of the intervention. Of note, participants (the patients, LHSs, clinical and community mental health providers) are recruited for the qualitative interviews only, who are different from the SMART trial sample.

It will be evaluated by three focus group interviews during the implementation of the SMART trial, and each interview will be about 60 to 120 minutes.

This is a mixed methods study, and the investigators will conduct community-based participatory research (CBPR) to evaluate the implementation of the intervention. In the second implementation evaluation stage of the study, focus group interviews will be conducted with the CBPR team understand the community's attitudes towards the adoption of the intervention. Of note, participants (the patients, LHSs, clinical and community mental health providers) are recruited for the qualitative interviews only, who are different from the SMART trial sample.

It will be evaluated by three focus group interviews during the implementation of the SMART trial, and each interview will be about 60 to 120 minutes.

The rate will be measured by the number of participants who subscribe to follow the study's WeChat Platform and the application divided by the number of participants who remain as followers and users at the end of the study. Of note, this outcome evaluates the implementation process of the intervention.

This is a mixed methods study, and the investigators will conduct community-based participatory research (CBPR) to evaluate the implementation of the intervention. In the second implementation evaluation stage of the study, focus group interviews will be conducted with the CBPR team understand the community's attitudes towards the equity of the intervention. Of note, participants (the patients, LHSs, clinical and community mental health providers) are recruited for the qualitative interviews only, who are different from the SMART trial sample.

It will be evaluated by three focus group interviews during the implementation of the SMART trial, and each interview will be about 60 to 120 minutes.

This is a mixed methods study, and the investigators will conduct community-based participatory research (CBPR) to evaluate the implementation of the intervention. In the second implementation evaluation stage of the study, focus group interviews will be conducted with the CBPR team understand the community's attitudes towards the feasibility of the intervention. Of note, participants (the patients, LHSs, clinical and community mental health providers) are recruited for the qualitative interviews only, who are different from the SMART trial sample.

It will be evaluated by three focus group interviews during the implementation of the SMART trial, and each interview will be about 60 to 120 minutes.

This is a mixed methods study, and the investigators will conduct community-based participatory research (CBPR) to evaluate the implementation of the intervention. In the second implementation evaluation stage of the study, focus group interviews will be conducted with the CBPR team understand the community's attitudes towards the patient-centeredness of the intervention. Of note, participants (the patients, LHSs, clinical and community mental health providers) are recruited for the qualitative interviews only, who are different from the SMART trial sample.

It will be evaluated by three focus group interviews during the implementation of the SMART trial, and each interview will be about 60 to 120 minutes.

The total cost of implementing the SMART trial will be recorded to assess the economic benefits of the intervention. Of note, this outcome evaluates the implementation process of the intervention.

The investigators will develop a checklist to evaluate the fidelity of the overall study. Of note, this outcome evaluates the implementation process of the intervention.

During the SMART trial, the investigators will record the number of daily brief contacts delivered to participants through WeChat and the application, text messages and phone calls. Of note, this outcome evaluates the implementation process of the intervention.

This is a mixed methods study, and the investigators will conduct community-based participatory research (CBPR) to evaluate the implementation of the intervention. In the second implementation evaluation stage of the study, focus group interviews will be conducted with the CBPR team understand the community's attitudes towards the safety of the intervention. Of note, participants (the patients, LHSs, clinical and community mental health providers) are recruited for the qualitative interviews only, who are different from the SMART trial sample.

It will be evaluated by three focus group interviews during the implementation of the SMART trial, and each interview will be about 60 to 120 minutes.

During the SMART trial, the investigators will record the time for the research team cost to response to participants' requests for crisis intervention and feedbacks.

Perceived stigma will be evaluated the Chinese version of Link Perceived Devaluation-Discrimination Scale. The scale contains 12 items assessing the extent to which a person believes that other people will devalue or discriminate against someone with a mental illness. Each item is on a 4-Likert continuum (from "Strongly disagree" to "Strongly agree") scoring from 1 to 4. A higher total score indicates a higher level of perceived stigma. The total score's trajectory from baseline to three months after discharge will be recorded and compared.

Perceived stigma will be evaluated the Chinese version of Link Perceived Devaluation-Discrimination Scale. The scale contains 12 items assessing the extent to which a person believes that other people will devalue or discriminate against someone with a mental illness. Each item is on a 4-Likert continuum (from "Strongly disagree" to "Strongly agree") scoring from 1 to 4. A higher total score indicates a higher level of perceived stigma. The total score's trajectory from baseline to 12 months after discharge will be recorded and compared.

Self-efficacy will be evaluated by the Chinese version of the General Self-Efficacy Scale. The scale contains 10 items, and each item is on a 4-Likert continuum (from "Not at all true" to "Exactly true") scoring from 1 to 4. A higher total score indicates a higher level of self-efficacy. The total score's trajectory from baseline to three months after discharge will be recorded and compared.

Self-efficacy will be evaluated by the Chinese version of the General Self-Efficacy Scale. The scale contains 10 items, and each item is on a 4-Likert continuum (from "Not at all true" to "Exactly true") scoring from 1 to 4. A higher total score indicates a higher level of self-efficacy. The total score's trajectory from baseline to 12 months after discharge will be recorded and compared.

Compliance to treatment will be evaluated by a 4-item self-administered questionnaire. The questionnaire inquires whether the patients take medications under the instruction on prescriptions (frequency, dosage, time, and self-discontinued medication). Each item is on a 4-Likert continuum (from "Not following the instruction" to "Exactly following the instruction") scoring from 1 to 4. A higher total score indicates a higher level of compliance to treatment. The total score's trajectory from baseline to three months after discharge will be recorded and compared.

Compliance to treatment will be evaluated by a 4-item self-administered questionnaire. The questionnaire inquires whether the patients take medications under the instruction on prescriptions. Each item is on a 4-Likert continuum (from "Not following the instruction" to "Exactly following the instruction") scoring from 1 to 4. A higher total score indicates a higher level of compliance to treatment. The total score's trajectory from baseline to 12 months after discharge will be recorded and compared.

This is a mixed methods study, and the investigators will conduct community-based participatory research (CBPR) to evaluate the implementation of the intervention. In the second implementation evaluation stage of the study, focus group interviews will be conducted with the CBPR team understand the community's attitudes towards the sustainability of the intervention. Of note, participants (the patients, LHSs, clinical and community mental health providers) are recruited for the qualitative interviews only, who are different from the SMART trial sample.

It will be evaluated by three focus group interviews during the implementation of the SMART trial, and each interview will be about 60 to 120 minutes.

Inclusion criteria: For patients in the SMART trial and qualitative interviews Being 18 years and above; Being diagnosed with psychotic symptoms or MDD; Having received in-patient care for three days or more; Living in Shenzhen and having no plan to leave Shenzhen in the following 12 months after discharge; Being able to read text messages, answer phone calls on mobile phones, use WeChat or any application on smart phones. For lay health care supporters (LHSs) in qualitative interviews Being 18 years and above; Being without any diagnosis of mental disorder; Being the lay health care supporter in the family; Living in Shenzhen and having no plan to leave Shenzhen in the following 12 months after discharge; Being able to read text messages, answer phone calls on mobile phones, use WeChat or any application on smart phones. For the clinic and community mental health service providers in qualitative interviews Being 18 years and above; Having practiced in mental health service at least for 12 months. Exclusion criteria for patients in the SMART trial and qualitative interviews Unable to provide written consent due to any cognitive problems. Being discharged by the patient's or LHSs' demand against medical advice. With no ID, no stable residence nor any source of income.

Quantitative data will be publicly available through FigShare 12 months after the main studies are published in peer-reviewed journals. The data will contain de-identified demographic information, primary and secondary outcomes, and other covariate outcomes.

Twelve months after the main studies are published in peer-reviewed journals. The study protocol will be shared once published in peer-reviewed journal.;

Please contact the PI Fengsu Hou to request for the use of the data, and the requests should include detail contact information of applicants, the purpose of study, and the analysis plan.

Goldacre M, Seagroatt V, Hawton K. Suicide after discharge from psychiatric inpatient care. Lancet. 1993 Jul 31;342(8866):283-6. doi: 10.1016/0140-6736(93)91822-4.

Geddes JR, Juszczak E. Period trends in rate of suicide in first 28 days after discharge from psychiatric hospital in Scotland, 1968-92. BMJ. 1995 Aug 5;311(7001):357-60. doi: 10.1136/bmj.311.7001.357.

Meehan J, Kapur N, Hunt IM, Turnbull P, Robinson J, Bickley H, Parsons R, Flynn S, Burns J, Amos T, Shaw J, Appleby L. Suicide in mental health in-patients and within 3 months of discharge. National clinical survey. Br J Psychiatry. 2006 Feb;188:129-34. doi: 10.1192/bjp.188.2.129.

Haglund A, Lysell H, Larsson H, Lichtenstein P, Runeson B. Suicide Immediately After Discharge From Psychiatric Inpatient Care: A Cohort Study of Nearly 2.9 Million Discharges. J Clin Psychiatry. 2019 Feb 12;80(2):18m12172. doi: 10.4088/JCP.18m12172.

Chung D, Hadzi-Pavlovic D, Wang M, Swaraj S, Olfson M, Large M. Meta-analysis of suicide rates in the first week and the first month after psychiatric hospitalisation. BMJ Open. 2019 Mar 23;9(3):e023883. doi: 10.1136/bmjopen-2018-023883.

Fleischmann A, Bertolote JM, Wasserman D, De Leo D, Bolhari J, Botega NJ, De Silva D, Phillips M, Vijayakumar L, Varnik A, Schlebusch L, Thanh HT. Effectiveness of brief intervention and contact for suicide attempters: a randomized controlled trial in five countries. Bull World Health Organ. 2008 Sep;86(9):703-9. doi: 10.2471/blt.07.046995.

Zalsman G, Hawton K, Wasserman D, van Heeringen K, Arensman E, Sarchiapone M, Carli V, Hoschl C, Barzilay R, Balazs J, Purebl G, Kahn JP, Saiz PA, Lipsicas CB, Bobes J, Cozman D, Hegerl U, Zohar J. Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry. 2016 Jul;3(7):646-59. doi: 10.1016/S2215-0366(16)30030-X. Epub 2016 Jun 8.

Motto JA, Bostrom AG. A randomized controlled trial of postcrisis suicide prevention. Psychiatr Serv. 2001 Jun;52(6):828-33. doi: 10.1176/appi.ps.52.6.828.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011 Mar;38(2):65-76. doi: 10.1007/s10488-010-0319-7.

Lei H, Nahum-Shani I, Lynch K, Oslin D, Murphy SA. A "SMART" design for building individualized treatment sequences. Annu Rev Clin Psychol. 2012;8:21-48. doi: 10.1146/annurev-clinpsy-032511-143152. Epub 2011 Dec 12.

Liu H, Chen G, Li J, Hao C, Zhang B, Bai Y, Song L, Chen C, Xie H, Liu T, Caine ED, Hou F. Sequential multiple assignment randomised trial of a brief contact intervention for suicide risk management among discharged psychiatric patients: an implementation study protocol. BMJ Open. 2021 Nov 26;11(11):e054131. doi: 10.1136/bmjopen-2021-054131.