Phase II Motivational Interviewing: An Experiential Online Training Tool

National Institutes of Health (NIH), National Institute of Mental Health (NIMH), American Institutes for Research

Clinical social workers, doctors, outreach workers, and all other staff providing direct care to vulnerable populations face multiple challenges in obtaining and maintaining training and implementing evidence based practices in diverse community settings. Motivational Interviewing is a well-established evidence based practice that is challenging to implement across programs and agencies because it requires that skills gained from training be reinforced as a service provider employs the practice. This study builds on findings from a Phase I study. With funding from a Phase I Small Business Innovation Research grant from the National Institute of Health, the Center for Social Innovation developed the Motivational Interviewing Simulator: An Experiential Online Training Tool. The interactive, case-based, multiplayer web-based game allows service providers to deepen their skills in Motivational Interviewing (MI), a widely recognized evidence-based practice that supports people to make positive behavior changes related to health, wellness, mental illness, and addiction. The Phase II randomized controlled trial's primary aim is to assess the relative effectiveness of three interventions (MI Training Only; MI Training + eBook; MI Training + SIM) in increasing provider MI knowledge and skill retention over time. During Phase II of this study, we will recruit 180 providers from 18 community agencies serving individuals who live in supportive housing, many of whom have histories of mental illness, addiction, homelessness, and medical problems. After receiving a standardized two-day onsite MI training, participants will be randomized into one of three conditions: 1) MI Training Only; 2) MI Training + eBook (an online comparison with comparable information to the Simulator without the interactive elements); or 3) MI Training + Simulator. The longitudinal, mixed methods study will assess providers' acquisition and retention of MI knowledge and skills through surveys and coding of standardized client interviews; barriers and facilitators of MI implementation via focus groups; organizational-level data via key informant interviews and site visits; and client outcomes for 3,600 clients through surveys, administrative, data and focus groups.

Between 2011 and 2012, the Center for Social Innovation, LLC (C4), developed and evaluated a prototype of the Motivational Interviewing Simulator: An Experiential Online Training Tool. The interactive, case-based, multiplayer web-based game allows service providers to deepen their skills in Motivational Interviewing (MI), a widely recognized evidence-based practice that supports people to make positive behavior changes related to health, wellness, mental illness, and addiction. Phase I study findings support the product's feasibility and the need for a large-scale randomized trial to determine the tool's effectiveness. During Phase I, participants in the Simulator group showed significant improvements in MI skills as measured by Motivational Interviewing Treatment Integrity (MITI) scores (34% improvement over baseline) and Helpful Responses Questionnaire (HRQ) scores (13% improvement over baseline)-both substantially higher than the rates of improvement for participants in the control groups. This project represents Phase II of the work to further develop the MI Simulator and to study its impact on provider behavior and client outcomes. Phase II will create multiple "play arenas"-Single Player, Two Player, and Advanced Free-play spaces-and multiple case-based scenarios related to mental health, homelessness, addiction, and primary care. The significantly expanded product will have wide appeal across a large workforce of providers serving vulnerable populations, and it will support National Institute of Mental Health's strategy of "closing the gap between the development of new, research-tested interventions and their widespread use by those most in need." The Phase II randomized controlled trial is guided by four specific aims. The first aim is to diversify the prototype Motivational Interviewing Simulator to meet the needs of a wider audience. Based on Phase I findings, we will build on the prototype by adding a Single Player Arena, expanding the existing Two-Player Arena, and creating an Advanced Free-play Arena where learners can go beyond basic MI skills. Additionally, the expanded product will include multiple scenarios, story-lines, and characters for each arena, and multiple levels of play for each scenario. The second aim is to evaluate the Simulator through a randomized controlled trial. The third aim is to disseminate findings. And the fourth aim is to prepare for commercialization. Phase II seeks to answer a number of Research Questions that concern service providers, clients, and organizations. Research questions pertaining to service providers are as follows: What are the relative impacts of these interventions on providers' confidence, knowledge and skills acquisition and retention? What supports are needed for providers to adhere to the MI model with fidelity over time? And Does the impact of the treatment condition vary by the amount of time the service provider spends engaged with the Sim or eBook tools? One key research questions pertaining to clients is as follows: Do clients served by providers receiving the Sim or eBook interventions show improved outcomes such as housing stability and receptiveness to mental health treatment, medication adherence, greater satisfaction with the provider, and greater confidence in individual treatment or recovery plans, relative to the Training Only group? Two key research questions focusing on organizations implementing MI are as follows: How does agency context influence provider adherence to the MI model? And, What are the organizational-level barriers and facilitators to successful MI implementation and fidelity? Phase II will test four hypotheses (H1, H2, H3, H4). H1 predicts that providers in the Simulation condition will show the strongest knowledge and skill retention during the study period when compared to MI Training Only and MI Training + eBook conditions. H2 predicts that providers in the eBook condition will have stronger knowledge skill and retention over the study period when compared to the MI Training Only Condition. H3 predicts that clients served by providers in the Simulation condition will show greater improvements in housing stability and receptiveness to mental health treatment, medication adherence, greater satisfaction with the provider, and greater sense of confidence in individual treatment or recovery plans when compared to clients of MI Training + eBook and MI Training Only conditions. H4 predicts that clients served by providers in the Simulation condition will show improved communication with their provider and greater sense of confidence in individual treatment or recovery plans compared to those clients with providers in the MI training + eBook and MI Training Only. The Motivational Interviewing Simulator: An Experiential Online Training Tool (the Sim) has the potential to make a significant contribution to the fields of behavioral health, primary care, adult learning, dissemination science, and beyond. Phase I of the project demonstrated the feasibility of the product development effort and research approach. Phase II support will allow the Center for Social Innovation (C4) to expand the depth and sophistication of the product, evaluate its efficacy, and lay the groundwork for Phase III commercial roll-out. Specifically, Phase II will add multiple arenas and levels to the prototype simulator, allowing users to advance more deeply into Motivational Interviewing (MI) practice. The Phase II product expansion will also create multiple story-lines, a variety of characters, and expanded opportunities for feedback, features that will expand the product's marketability in Phase III. As described below, the Sim has strong commercial potential and fits well into C4's existing commercial training platform, the t3 (think. teach. transform.) learning initiative. The project is valuable in various ways. Firstly, the SIM has the potential for improving the lives of vulnerable people. The ultimate goal of this product is to improve the lives of people who have experienced mental illness, addiction, homelessness, violence, and other traumatic events. MI has been proven to have positive impact on clinical outcomes for these marginalized groups. We believe that effective training of the workforce to use MI will result in better care for people who would otherwise slip through the cracks of our systems. Secondly, the SIM has the potential to add value to the workforce. As C4 has expanded its reach to thousands of direct service providers across the United States and internationally, we have documented a lack of high-quality training opportunities for the behavioral health workforce and a hunger for training on the part of agencies and individual providers-particularly for MI. Much existing training is limited to onsite workshops, conferences, and other face-to-face training. The online products that do exist do not provide opportunities for peer-to-peer learning and have specific focuses (diabetes and substance use) that do not include mental health. This product will provide an engaging, highly interactive opportunity to learn a set of skills that many providers want to acquire. Thirdly, the SIM has the potential to have an impact on dissemination science. If the Sim proves to be a successful mode of learning, primarily as a "booster shot" following face-to-face training, the implications for dissemination of evidence-based practices are significant. We envision a family of learning simulation products built upon the architecture developed for this project, covering a wide range of practices and skills-helping to close the research- practice gap and move effective practices into the field more quickly. The expected outcomes of this study include findings that document the product's effectiveness. Expected research outcomes include improvements in provider knowledge and skill, documentation of organizational strategies that support the uptake of best practices, and improved outcomes for clients, including increased housing stability and greater engagement in mental health services. The Sim brings together a well-documented best practice, the need for ongoing provider training, and cutting-edge simulation technology. Through widespread dissemination following Phase II, the Sim has the potential to have tremendous impact, improving the skills of service providers and the lives of the people they serve.

Motivational Interviewing, Evidence Based Practice, Online Simulation, Workforce development, Training

Service Providers in all group will receive a free two-day Motivational Interviewing training. Service Providers in this group will also receive access and a brief introduction to the Motivational Interviewing Simulator (SIM). Sim is designed to help Service Providers practice, learn, maintain, and implement MI skills. Clients of Service Providers from this group will receive no study related intervention. All Clients will receive identical surveys to collect client centered observational data of the interventions effect.

Service Providers in all group will receive a free two-day Motivational Interviewing training. Service Providers in this group will also receive a Motivational Interviewing eBook designed to help Service Providers learn, maintain, and implement MI skills. Clients of Service Providers from this group will receive no study related intervention. All Clients will receive identical surveys to collect client centered observational data of the interventions effect.

Service Providers in this group will only receive a free two-day Motivational Interviewing training. Clients of Service Providers from this group will receive no study related intervention. All Clients will receive identical surveys to collect client centered observational data of the interventions effect.

The interactive, case-based, multiplayer web-based game allows service providers to deepen their skills in Motivational Interviewing (MI), a widely recognized evidence-based practice that supports people to make positive behavior changes related to health, wellness, mental illness, and addiction.The SIM will have three arenas for play, so that users can practice more advanced MI skills and include: Single Player, Two Player, and Advanced Free play. The SIM is equipped with multiple story-lines and characters to keep learners engaged over time and to provide practice on diverse case scenarios, making the Sim relevant to a wide audience. Lastly the SIM will provide an opportunity for feedback from peers and MI experts.

The Phase I research team developed a 69-page eBook as a "credible online comparator" that presents information comparable to that included in the Sim, but without the interactive, role-playing, peer learning functionalities embodied in the Sim. Developed in Hyperlink Markup Language, the eBook presented MI skills, followed by transcripts of effective and not-effective samples of MI from the Sim itself. The e-Book includes the story-lines along with sample dialogue and expert feedback on MI skills.

The training intervention consists of a free two-day standardized Motivational Interviewing (MI) training provided by experienced MI trainers.

Change in Provider Motivational Interviewing Ability: Motivational Interviewing Treatment Integrity (MITI) Assessments

Change in Provider Motivational Interviewing Knowledge and Skill: Helpfulness Response Questionnaire (HRQ), Motivational Interviewing Knowledge and Attitudes Test (MIKAT)

Focus groups will examine perceptions of implementation, confidence with content, barriers and facilitators to implementation, agency support, client characteristics, and experience with Evidence Based Practices, including Motivational Interviewing.

Available Online and in paper form, this questionnaire will examine client perceptions of providers application of empathy, promotion of self-efficacy, ability to adapt, and develop therapeutic discrepancy. This questionnaire will employ unstandardized questions adapted from the HRQ, MITI, and MIKAT.

Each interview will examine client perceptions of providers application of empathy, promotion of self-efficacy, ability to adapt and develop therapeutic discrepancy. Each interview will employ unstandardized questions adapted from the HRQ, MITI, and MIKAT.

Interviews will examine organizational variables such as, size, number of full-time employees, annual budget, budget for training, experience with training, experience implementing Evidence Based Practices (including MI), organizational preferences for trainings, and perceived value of training boosters.

Focus Groups will examine organizational variables such as, size, number of full-time employees, annual budget, budget for training, experience with training, experience implementing Evidence Based Practices (including MI), organizational preferences for trainings, and perceived value of training boosters.

Client outcomes will be measured by examining data from mandated reporting measures, such as the Government Performance and Results Act (GPRA), to identify how long a client has been in treatment, how often he or she meets with a provider, frequency of missed appointments, and progress toward treatment goals. De-identified data will be collected from Homeless Management Information Systems (HMIS) and the Transformation Accountability System (TRAC)

Inclusion Criteria: Providers who have not had formal MI training in the past 3 months. Providers who are willing to engage in MI training and evaluation, including obtaining Internal Review Board (IRB) certification for the collection of client data, distribution of client surveys for four three-day periods during the study, and agreement to participate in a focus group if their agency is selected for a site visit. Providers who carry caseloads of clients in human services or housing programs Exclusion Criteria: Providers who have had formal MI training in the past 3 months. Providers who are not willing to engage in MI training and evaluation, including obtaining IRB certification for the collection of client data, distribution of client surveys for four three-day periods during the study, and agreement to participate in a focus group if their agency is selected for a site visit. Providers who do not carry caseloads of clients in human services or housing programs.