Increasing Access to Evidence-based Treatments for Depression

Increasing Access to Evidence-based Treatments for Depression: The Development and Evaluation of a Digital Training Platform for Interpersonal Psychotherapy.

The goal of this single-blind, randomized controlled pilot study is to evaluate the acceptability and feasibility of an Interpersonal Psychotherapy (IPT) asynchronous self-directed digital training platform for psychiatry residents, as compared to synchronous large group online workshop teaching. This study has the potential to improve access to competency-based training and dissemination of IPT, impacting healthcare delivery with increasing access to this evidence-based psychological depression treatment.

Background/Rational The COVID-19 pandemic is expected to have an unprecedented impact on the mental health and well-being of patients, healthcare workers and our society at large. Among these illnesses, depression is expected to be the most prevalent to emerge and is already the leading cause of disability worldwide; it affects up to 1.8 million Canadians. While physical distancing is required to curb the spread of COVID-19, it is well known that social isolation is a substantive risk factor of mental illness and suicide. Effective treatments exist but as few as 20% of Canadians receive minimally-adequate treatment for depression. One effective treatment for depression includes Interpersonal Psychotherapy (IPT) which may be particularly relevant for the isolation and life changes brought on by the COVID-19 pandemic. IPT is recommended in key consensus guidelines from the World Health Organization and the Canadian Network for Mood and Anxiety Treatments. IPT is a brief, structured psychological treatment that has been shown to effectively reduce depressive symptoms and enhance remission rates for patients across the lifespan. IPT proposes that the key to reducing depression is to increase connection with social supports and help patients work through triggering interpersonal experiences of loss/grief, life changes/social role transitions, interpersonal sensitivity/loneliness, or relational conflicts/role disputes. We selected IPT for three reasons: i) its strong evidence base for effectiveness as an antidepressant treatment, comparable in efficacy to CBT, with fewer patients dropping out of treatments; ii) its effectiveness in treating depression among culturally diverse patients across the lifespan; and iii) its parsimonious approach that is easy to understand and implement. This is critical when training novice clinicians. The poor dissemination of effective psychological therapies such as IPT is partly due to the limited number of available mental health professionals who can train novice clinicians with high fidelity; and one key obstacle is how these individuals are trained. Prior to the COVID-19 pandemic, this typically involved a trainee attending an expert-led workshop and then practicing the treatment under the supervision of someone experienced in delivering that treatment. The recent development of digital training platforms in various psychological treatments has attempted to address this training gap. In addition, web-conferencing applications have opened the door for distance psychotherapy training and supervision. Online training methods need to be developed and examined to address this significant bottleneck, particularly in light of the current COVID-19 pandemic where in-person trainings have stopped but the need for evidence-based psychotherapy grows. The current study aims to examine two online training methods with consenting psychiatry resident/trainee therapists: a) asynchronous self-directed training through a novel digital training course which they will complete on their own schedule; and b) synchronous large group workshop where residents will participate in a live training workshop at a specific time. Asynchronous training is a more cost-effective and scalable method of training which permits residents (and other trainees) to learn psychotherapy according to their preferred schedules. To date, several asynchronous IPT training courses have been developed, however none have been compared to synchronous large group online workshop. The current study will provide preliminary evidence to examine the acceptability and feasibility of both models among psychiatry residents at University of Toronto affiliated hospitals. Purpose The main purpose of the current study is to examine the acceptability and feasibility of a digital training platform for psychiatry residents in IPT for depression treatment. Aim 1. What is the feasibility of an asynchronous self-directed online digital training platform? To assess feasibility, we will examine psychiatric resident recruitment, and participation (completion of the online modules and homework assignments) for the intervention condition vs control (synchronous large group online workshop). Aim 2. What is the acceptability of the asynchronous self-directed online digital training process and platform? We will assess acceptability by conducting a process evaluation; this will take the form of semi-structured interviews with all residents in the intervention to explore experiences of learning through the asynchronous self-directed online curriculum and platform along with facilitators and barriers to attaining the learning objectives of achieving competency in delivering IPT. Aim 3. Does the asynchronous self-directed online training platform demonstrate preliminary efficacy when training PGY 2-5 psychiatry residents in IPT for depressed patients, as compared to a synchronous large group online workshop? This will be estimated by assessing changes in patient outcomes of depressive symptoms over the course of treatment. A secondary patient outcome will be changes in patient anxiety symptoms over the course of IPT treatment. In addition, therapist-related outcomes will be assessed including therapeutic alliance, therapy quality as assessed by IPT- specific skills and non-specific therapeutic communication skills, IPT-knowledge, and counseling self-efficacy. Target Population The target population are psychiatry residents in good standing with the University of Toronto (UofT), Faculty of Medicine, post-graduate psychiatry residency program, all of whom require training in IPT as part of the residency requirements, and their IPT patients. Specifically, this will include psychiatry residents from four UofT-affiliated hospitals, Mount Sinai Hospital (MSH), Women's College Hospital (WCH), Sunnybrook Health Sciences Centre (SHSC) and the Centre for Addition and Mental Health (CAMH). Study Design This is a mixed-methods feasibility and acceptability study, with a single blind randomized controlled design comparing asynchronous self-directed digital IPT training to a synchronous large group online IPT workshop. Patients will be blind to therapist study arm allocation. Procedure Residents doing clinical rotations at MSH, SHSC,WCH and CAMH as part of their Psychiatry Residency training at the University of Toronto will be approached for consent by the Research Coordinator (RC) at the beginning of each academic semester. Consenting resident participants will complete a measure of their counseling self-efficacy and IPT knowledge before randomization to the intervention or the control condition, stratified by study site (MSH, WCH, SHSC and CAMH). Patient recruitment will be through referrals via treatment providers at all study sites (MSH, SHSC, WCH, and CAMH). Patient participants will be stratified by site and then randomly assigned to residents in the intervention or control condition. Resident participants will be randomized to one of two training conditions: (1) synchronous self-directed digital training (intervention condition), where residents will complete an interactive modular curriculum covering evidence-based psychotherapy principles and IPT-specific principles and strategies at their own pace over a two week time period; or (2) a synchronous online training workshop (control condition), which will involve the same content contained in the asynchronous self-directed digital training platform, except delivered over a one day online workshop. Immediately after training has been completed, each resident participant will participate in structured role plays designed to assess the resident's competence on IPT-specific and general psychotherapy skills, as rated by two IPT experts. Resident participants will then begin their 12-week course of therapy with their assigned patient participant. During treatment delivery, residents will have once weekly one-on-one supervisory sessions with an IPT expert supervisor/faculty member of the University of Toronto, Department of Psychiatry, who will be blind to residents' foundational didactic training modality. Patient participants will complete measures of their depressive and anxiety symptoms just before each weekly session, as well as a measure of therapeutic alliance at the 3rd and final therapy sessions. At every session resident participants will complete a measure of their impression of their patient's improvement (or lack thereof) as well as a measure of therapy quality. They will complete the measures of counseling self-efficacy and IPT knowledge at the conclusion of the course of therapy with their patient. Consenting resident-patient pairs will audio record each therapy session; these recordings will be transcribed and coded for therapy quality. Resident participants randomized to the intervention condition (asynchronous digital training) will complete a semi-structured interview exploring the barriers and facilitators to engaging with the digital training platform in order to assess the acceptability of the intervention.

This is a mixed-methods feasibility and acceptability study, with a single blind randomized controlled design comparing asynchronous self-directed digital IPT training to a synchronous large group online IPT workshop.

Patient participants will be blind to which training modality their resident therapist has been randomized to.

A digital training program platform that delivers an interactive case-based modular curriculum covering evidence-based psychotherapy principles and IPT-specific principles and strategies with homework will take roughly 13 hours to complete, with 4.5 hours of online learning, and an additional 7.5 hours of reading and homework assignments. The reading and homework includes viewing captioned videotaped role plays and completing self-directed lesson plans. The digital curriculum leverages audio, video, and visual content, and is self-directed - for completion within a 2-week period. Residents can access this content through a smartphone, tablet or computer at their own pace, revisit modules and digital content as needed, and access a curated list of additional resources to supplement their learning.

This condition will reflect training as usual. Training will involve the same content contained in the asynchronous self-directed digital training platform, except delivered over a one day (4.5 hours of online instruction) workshop; and residents will be required to do the same reading and homework of lesson plans and viewing of the on-line videotaped role plays (an additional ~7.5 hours in total).

Resident participants will complete the digital training platform in real time over the course of a one day, live workshop (4.5 hours of online instruction).

Percentage of patient and resident participants retained and assessed with valid primary outcome data for the intervention vs control.

Percentage of resident participants who complete the online modules and homework assignments for the intervention vs control.

We will assess acceptability by conducting a process evaluation; this will take the form of semi-structured interviews with all residents in the intervention to explore experiences of learning through the asynchronous self-directed online curriculum and platform along with facilitators and barriers to attaining the learning objectives of achieving competency in delivering IPT.

Depressive symptoms as measured by the Patient Health Questionnaire-9 (PHQ-9), a self-rating instrument that assesses symptoms of depression experienced during the past 2 weeks and is based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria for MDD. Each of the nine items includes four possible responses related to symptom duration (e.g. not at all to nearly every day). Scores ≥10 (scale range 0-27) are often used to identify major depression.

Anxiety symptoms as measured by the General Anxiety Disorder-7 scale (GAD-7). The GAD-7 is a 7-item, self-rated scale developed as a screening tool and severity indicator for Generalized Anxiety Disorder. Items are rated on a 4-point Likert-type scale (0 = not at all to 3 = nearly every day). Scores range from 0 to 21 with higher scores indicating more severe GAD symptoms.

Resident participants will fill in a measure of their global impression of their patient's improvement via the Clinical Global Impressions of Improvement (CGI-I) scale. This is a clinician rated scale that assesses the extent of clinical change in the patient at the point of assessment compared with baseline, and has a 7-point range, from 'Very much improved' 1 to 'Very much worse' 7. Higher scores signify greater severity and/or worse outcomes. The CGI-I has been found to be highly sensitive to change.

The Working Alliance Inventory-Short Revised (WAI-SR) is used by psychotherapists to assess and measure the therapeutic alliance for supervision and research purposes. The WAI-SR is a 12- item scale and consists of three subscales that measure three aspects of the therapeutic alliance: goals (agreement on the goals of therapy), tasks (agreement on the agenda of the therapy) and affective bond (development of relational bond between the client and the therapist). Each item is rated on a 7- point scale ranging from 1 (never) to 7 (always).

The Brief IPT Checklist (BIC) will assess therapy quality. Developed by the study PI and used in other IPT-training initiatives in Ethiopia, China and Canada, this 15-item checklist includes essential IPT therapist behaviours across all sessions and within specific phases (beginning, middle or end). Response options for each item are recorded on a Likert scale of 0 (not done) to 4 (excellent). For consenting resident and patient pairs, each IPT therapy session will be audio recorded, and 1 randomly chosen from each of the early (sessions 1-2), middle (sessions 3-10) and late (sessions 11-12) phases of the therapy will be rated by independent experts in IPT (not study supervisors) and the resident therapist (after each therapy session) using the BIC.

IPT knowledge will be measured by a 25-item quiz derived from principles of IPT. The quiz will include case-based multiple-choice and short-answer questions that examine the trainees understanding of phase- and focus-specific therapeutic guidelines of IPT.

Counselling self-efficacy will be measured by using the counselling self-estimate inventory (COSE). It is a 42-item scale and includes both positive and negative statements about counselling self-efficacy. Respondents are asked to rate on a 6-point Likert scale from 1 (strongly disagree) to 6 (strongly agree) how they would perform in a counselling situation at the present time. The higher the score the stronger perception of counselling self-efficacy.

Resident participants will be assessed for competency through structured role plays using the investigator-developed Brief IPT Checklist (BIC) Using standardized patients, there will be two vignettes per resident on 2 differing IPT focal areas of social role transitions and disputes which will be rated by two experts in IPT (not study supervisors) on IPT-specific and general psychotherapy skills, on a scale of 0 (not done) to 2 (done well). The average score between the two role plays will be selected as a measure of the trainee competence. After the role play is finished, the standardized patients will be asked four questions to assess their impression of the resident participant's performance, and whether they would want to return for psychotherapy treatment with the resident.

Inclusion Criteria - Residents Mount Sinai Hospital (MSH), Sunnybrook Health Sciences Centre (SHSC), Women's College Hospital (WCH) and the Centre for Addition and Mental Health (CAMH) PGY2-5 psychiatry residents treating patients Able to participate in the asynchronous self-directed on-line/synchronous large group online workshop Ability to add 1-2 patients to their caseload and provide a minimum of 12 IPT sessions over 6 months. Exclusion Criteria - Residents Academic concerns as flagged by the University of Toronto General Psychiatry program as evidenced by a referral to the University of Toronto Psychiatry Resident Evaluation Subcommittee OR significant concerns regarding promotion or progression as flagged by the University of Toronto Psychiatry Post-graduate Competence Sub-committee (responsible for reviewing all resident files twice a year). Inclusion Criteria - Patients Adults (≥18 years) Experiencing clinically significant depressive symptoms (defined as a score of ≥10 on the Patient Health Questionnaire-9, PHQ-9, during eligibility screening). Able to commit to attending a minimum of 12 IPT sessions over 6 months Sufficient English and literacy levels to fill in self-report measures Exclusion Criteria - Patients Diagnosis of borderline personality disorder, or psychotic or manic symptoms Active alcohol or substance use disorder within the past 6 months, assessed by a score >1 on items "2" through "5" the Global Appraisal of Individual Needs-Short Screener (GAIN-SS). Self-harm and/or at high risk of suicide or suicidal behaviors, as assessed by a score of >1 on question 9 of the PHQ-9, followed by an assessment of active suicidal ideation on the Mini International Neuropsychiatric Interview (MINI), as confirmed by the site PI Severe symptoms of depression, defined as a score of ≥20 on the PHQ-9 during eligibility screening <6 months discharged from psychiatric in-patient unit

Ferrari AJ, Charlson FJ, Norman RE, Patten SB, Freedman G, Murray CJ, Vos T, Whiteford HA. Burden of depressive disorders by country, sex, age, and year: findings from the global burden of disease study 2010. PLoS Med. 2013 Nov;10(11):e1001547. doi: 10.1371/journal.pmed.1001547. Epub 2013 Nov 5.

Heinrich LM, Gullone E. The clinical significance of loneliness: a literature review. Clin Psychol Rev. 2006 Oct;26(6):695-718. doi: 10.1016/j.cpr.2006.04.002. Epub 2006 Jun 19.

Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, Al-Hamzawi A, Alonso J, Andrade L, Borges G, Bruffaerts R, Bunting B, de Almeida JM, Florescu S, de Girolamo G, Gureje O, Haro JM, He Y, Hinkov H, Karam E, Kawakami N, Lee S, Navarro-Mateu F, Piazza M, Posada-Villa J, de Galvis YT, Kessler RC. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry. 2017 Feb;210(2):119-124. doi: 10.1192/bjp.bp.116.188078. Epub 2016 Dec 1.

Cuijpers P, Donker T, Weissman MM, Ravitz P, Cristea IA. Interpersonal Psychotherapy for Mental Health Problems: A Comprehensive Meta-Analysis. Am J Psychiatry. 2016 Jul 1;173(7):680-7. doi: 10.1176/appi.ajp.2015.15091141. Epub 2016 Apr 1.

Ravitz P, Watson P, Lawson A, Constantino MJ, Bernecker S, Park J, Swartz HA. Interpersonal Psychotherapy: A Scoping Review and Historical Perspective (1974-2017). Harv Rev Psychiatry. 2019 May/Jun;27(3):165-180. doi: 10.1097/HRP.0000000000000219.

Elkin I, Shea MT, Watkins JT, Imber SD, Sotsky SM, Collins JF, Glass DR, Pilkonis PA, Leber WR, Docherty JP, et al. National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments. Arch Gen Psychiatry. 1989 Nov;46(11):971-82; discussion 983. doi: 10.1001/archpsyc.1989.01810110013002.

Bass J, Neugebauer R, Clougherty KF, Verdeli H, Wickramaratne P, Ndogoni L, Speelman L, Weissman M, Bolton P. Group interpersonal psychotherapy for depression in rural Uganda: 6-month outcomes: randomised controlled trial. Br J Psychiatry. 2006 Jun;188:567-73. doi: 10.1192/bjp.188.6.567.

Bolton P, Bass J, Betancourt T, Speelman L, Onyango G, Clougherty KF, Neugebauer R, Murray L, Verdeli H. Interventions for depression symptoms among adolescent survivors of war and displacement in northern Uganda: a randomized controlled trial. JAMA. 2007 Aug 1;298(5):519-27. doi: 10.1001/jama.298.5.519.

Mufson L, Dorta KP, Wickramaratne P, Nomura Y, Olfson M, Weissman MM. A randomized effectiveness trial of interpersonal psychotherapy for depressed adolescents. Arch Gen Psychiatry. 2004 Jun;61(6):577-84. doi: 10.1001/archpsyc.61.6.577.

Cooper Z, Bailey-Straebler S, Morgan KE, O'Connor ME, Caddy C, Hamadi L, Fairburn CG. Using the Internet to Train Therapists: Randomized Comparison of Two Scalable Methods. J Med Internet Res. 2017 Oct 18;19(10):e355. doi: 10.2196/jmir.8336.

Ravitz P, Lawson A, Fefergrad M, Rawkins S, Lancee W, Maunder R, Leszcz M, Kivlighan DM Jr. Psychotherapy Competency Milestones: an Exploratory Pilot of CBT and Psychodynamic Psychotherapy Skills Acquisition in Junior Psychiatry Residents. Acad Psychiatry. 2019 Feb;43(1):61-66. doi: 10.1007/s40596-018-0940-4. Epub 2018 Jun 1.

Ravitz P, Cooke RG, Mitchell S, Reeves S, Teshima J, Lokuge B, Lawson A, McNaughton N, Skinner W, Cooper C, Fefergrad M, Zaretsky A. Continuing education to go: capacity building in psychotherapies for front-line mental health workers in underserviced communities. Can J Psychiatry. 2013 Jun;58(6):335-43. doi: 10.1177/070674371305800605.

Weingardt KR, Cucciare MA, Bellotti C, Lai WP. A randomized trial comparing two models of web-based training in cognitive-behavioral therapy for substance abuse counselors. J Subst Abuse Treat. 2009 Oct;37(3):219-27. doi: 10.1016/j.jsat.2009.01.002. Epub 2009 Mar 31.

Kobak KA, Lipsitz JD, Markowitz JC, Bleiberg KL. Web-Based Therapist Training in Interpersonal Psychotherapy for Depression: Pilot Study. J Med Internet Res. 2017 Jul 17;19(7):e257. doi: 10.2196/jmir.7966.

Fluckiger C, Del Re AC, Wampold BE, Horvath AO. The alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy (Chic). 2018 Dec;55(4):316-340. doi: 10.1037/pst0000172. Epub 2018 May 24.

Fairburn CG, Cooper Z. Therapist competence, therapy quality, and therapist training. Behav Res Ther. 2011 Jun;49(6-7):373-8. doi: 10.1016/j.brat.2011.03.005. Epub 2011 Mar 21.

Kohrt BA, Jordans MJ, Rai S, Shrestha P, Luitel NP, Ramaiya MK, Singla DR, Patel V. Therapist competence in global mental health: Development of the ENhancing Assessment of Common Therapeutic factors (ENACT) rating scale. Behav Res Ther. 2015 Jun;69:11-21. doi: 10.1016/j.brat.2015.03.009. Epub 2015 Mar 24.

Singla DR, Weobong B, Nadkarni A, Chowdhary N, Shinde S, Anand A, Fairburn CG, Dimijdan S, Velleman R, Weiss H, Patel V. Improving the scalability of psychological treatments in developing countries: an evaluation of peer-led therapy quality assessment in Goa, India. Behav Res Ther. 2014 Sep;60:53-9. doi: 10.1016/j.brat.2014.06.006. Epub 2014 Jun 30.

Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606-13. doi: 10.1046/j.1525-1497.2001.016009606.x.

Titus JC, Dennis ML, Lennox R, Scott CK. Development and validation of short versions of the internal mental distress and behavior complexity scales in the Global Appraisal of Individual Needs (GAIN). J Behav Health Serv Res. 2008 Apr;35(2):195-214. doi: 10.1007/s11414-008-9107-5.

Shinn M, Gottlieb J, Wett JL, Bahl A, Cohen A, Baron Ellis D. Predictors of homelessness among older adults in New York city: disability, economic, human and social capital and stressful events. J Health Psychol. 2007 Sep;12(5):696-708. doi: 10.1177/1359105307080581.

Stucky BD, Edelen MO, Ramchand R. A psychometric assessment of the GAIN individual severity scale (GAIN-GISS) and short screeners (GAIN-SS) among adolescents in outpatient treatment programs. J Subst Abuse Treat. 2014 Feb;46(2):165-73. doi: 10.1016/j.jsat.2013.07.007. Epub 2013 Aug 29.

Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 Suppl 20:22-33;quiz 34-57.

Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006 May 22;166(10):1092-7. doi: 10.1001/archinte.166.10.1092.

Kroenke K, Bair M, Damush T, Hoke S, Nicholas G, Kempf C, Huffman M, Wu J, Sutherland J. Stepped Care for Affective Disorders and Musculoskeletal Pain (SCAMP) study: design and practical implications of an intervention for comorbid pain and depression. Gen Hosp Psychiatry. 2007 Nov-Dec;29(6):506-17. doi: 10.1016/j.genhosppsych.2007.08.005.

Lowe B, Decker O, Muller S, Brahler E, Schellberg D, Herzog W, Herzberg PY. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care. 2008 Mar;46(3):266-74. doi: 10.1097/MLR.0b013e318160d093.

Beard C, Bjorgvinsson T. Beyond generalized anxiety disorder: psychometric properties of the GAD-7 in a heterogeneous psychiatric sample. J Anxiety Disord. 2014 Aug;28(6):547-52. doi: 10.1016/j.janxdis.2014.06.002. Epub 2014 Jun 14.

Kertz S, Bigda-Peyton J, Bjorgvinsson T. Validity of the Generalized Anxiety Disorder-7 scale in an acute psychiatric sample. Clin Psychol Psychother. 2013 Sep-Oct;20(5):456-64. doi: 10.1002/cpp.1802. Epub 2012 May 17.

Munder T, Wilmers F, Leonhart R, Linster HW, Barth J. Working Alliance Inventory-Short Revised (WAI-SR): psychometric properties in outpatients and inpatients. Clin Psychol Psychother. 2010 May-Jun;17(3):231-9. doi: 10.1002/cpp.658.

Ravitz P, Wondimagegn D, Pain C, Araya M, Alem A, Baheretibeb Y, Hanlon C, Fekadu A, Park J, Fefergrad M, Leszcz M. Psychotherapy Knowledge Translation and Interpersonal Psychotherapy: Using Best-Education Practices to Transform Mental Health Care in Canada and Ethiopia. Am J Psychother. 2014;68(4):463-88. doi: 10.1176/appi.psychotherapy.2014.68.4.463.

Bowlby J. The making and breaking of affectional bonds. I. Aetiology and psychopathology in the light of attachment theory. An expanded version of the Fiftieth Maudsley Lecture, delivered before the Royal College of Psychiatrists, 19 November 1976. Br J Psychiatry. 1977 Mar;130:201-10. doi: 10.1192/bjp.130.3.201.

Bowlby J. Attachment and loss: retrospect and prospect. Am J Orthopsychiatry. 1982 Oct;52(4):664-678. doi: 10.1111/j.1939-0025.1982.tb01456.x. No abstract available.

Levy KN, Kivity Y, Johnson BN, Gooch CV. Adult attachment as a predictor and moderator of psychotherapy outcome: A meta-analysis. J Clin Psychol. 2018 Nov;74(11):1996-2013. doi: 10.1002/jclp.22685. Epub 2018 Sep 21.

Hunter JJ, Maunder RG. Using attachment theory to understand illness behavior. Gen Hosp Psychiatry. 2001 Jul-Aug;23(4):177-82. doi: 10.1016/s0163-8343(01)00141-4.

Krupnick JL, Sotsky SM, Simmens S, Moyer J, Elkin I, Watkins J, Pilkonis PA. The role of the therapeutic alliance in psychotherapy and pharmacotherapy outcome: findings in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. J Consult Clin Psychol. 1996 Jun;64(3):532-9. doi: 10.1037//0022-006x.64.3.532.

Young JF, Mufson L, Gallop R. Preventing depression: a randomized trial of interpersonal psychotherapy-adolescent skills training. Depress Anxiety. 2010 May;27(5):426-33. doi: 10.1002/da.20664.

Frank E, Kupfer DJ, Thase ME, Mallinger AG, Swartz HA, Fagiolini AM, Grochocinski V, Houck P, Scott J, Thompson W, Monk T. Two-year outcomes for interpersonal and social rhythm therapy in individuals with bipolar I disorder. Arch Gen Psychiatry. 2005 Sep;62(9):996-1004. doi: 10.1001/archpsyc.62.9.996.

Wilfley DE, Welch RR, Stein RI, Spurrell EB, Cohen LR, Saelens BE, Dounchis JZ, Frank MA, Wiseman CV, Matt GE. A randomized comparison of group cognitive-behavioral therapy and group interpersonal psychotherapy for the treatment of overweight individuals with binge-eating disorder. Arch Gen Psychiatry. 2002 Aug;59(8):713-21. doi: 10.1001/archpsyc.59.8.713.

Markowitz JC, Petkova E, Neria Y, Van Meter PE, Zhao Y, Hembree E, Lovell K, Biyanova T, Marshall RD. Is Exposure Necessary? A Randomized Clinical Trial of Interpersonal Psychotherapy for PTSD. Am J Psychiatry. 2015 May;172(5):430-40. doi: 10.1176/appi.ajp.2014.14070908. Epub 2015 Feb 13.

Markowitz JC, Lipsitz J, Milrod BL. Critical review of outcome research on interpersonal psychotherapy for anxiety disorders. Depress Anxiety. 2014 Apr;31(4):316-25. doi: 10.1002/da.22238. Epub 2014 Feb 3.

Cyranowski JM, Frank E, Shear MK, Swartz H, Fagiolini A, Scott J, Kupfer DJ. Interpersonal psychotherapy for depression with panic spectrum symptoms: a pilot study. Depress Anxiety. 2005;21(3):140-2. doi: 10.1002/da.20069.

Kirmayer LJ, Groleau D, Guzder J, Blake C, Jarvis E. Cultural consultation: a model of mental health service for multicultural societies. Can J Psychiatry. 2003 Apr;48(3):145-53. doi: 10.1177/070674370304800302.

Watson P, Raju P, Soklaridis S. Teaching Not-Knowing: Strategies for Cultural Competence in Psychotherapy Supervision. Acad Psychiatry. 2017 Feb;41(1):55-61. doi: 10.1007/s40596-016-0552-9. Epub 2016 May 16. No abstract available.

Grote NK, Spieker SJ, Lohr MJ, Geibel SL, Swartz HA, Frank E, Houck PR, Katon W. Impact of childhood trauma on the outcomes of a perinatal depression trial. Depress Anxiety. 2012 Jul;29(7):563-73. doi: 10.1002/da.21929. Epub 2012 Mar 23.

Miniati M, Rucci P, Benvenuti A, Frank E, Buttenfield J, Giorgi G, Cassano GB. Clinical characteristics and treatment outcome of depression in patients with and without a history of emotional and physical abuse. J Psychiatr Res. 2010 Apr;44(5):302-9. doi: 10.1016/j.jpsychires.2009.09.008. Epub 2009 Oct 2.

Ravitz P, McBride C, Maunder R. Failures in interpersonal psychotherapy (IPT): factors related to treatment resistances. J Clin Psychol. 2011 Nov;67(11):1129-39. doi: 10.1002/jclp.20850. Epub 2011 Oct 3.

Frank E, Kupfer DJ, Perel JM, Cornes C, Jarrett DB, Mallinger AG, Thase ME, McEachran AB, Grochocinski VJ. Three-year outcomes for maintenance therapies in recurrent depression. Arch Gen Psychiatry. 1990 Dec;47(12):1093-9. doi: 10.1001/archpsyc.1990.01810240013002.