Trial of Simulation-based Mastery Learning to Communicate Diagnostic Uncertainty

Simulation-based Mastery Learning to Improve Communication in Times of Diagnostic Uncertainty: Randomized Control Trial

The goal of this study is to teach emergency medicine resident physicians to communicate more effectively with patients at the time of emergency department discharge in the setting of diagnostic uncertainty (i.e. no definitive cause identified for the patient's symptoms). All residents will complete baseline and follow up assessments by completing in-person simulated patient discharged. After the baseline assessment, they will complete an online educational curriculum that has been developed by the study team, and will participate in video-based simulation deliberate practice (DP) and feedback sessions using a simulation-based mastery learning (SBML) approach. They will be assessed with the Uncertainty Communication Checklist (UCC), a tool already developed by the study team, that has a minimum passing standard (MPS) that was established through engagement of both patients and physicians. The investigators will perform a 2-arm wait-list randomized control trial with resident physicians to test the efficacy of the SBML curriculum in training residents to have a discharge discussion with patients discharged from the emergency department with diagnostic uncertainty.

This study will employ a 2-arm wait-list control trial. Participants will be evenly randomly assigned to Group A or Group B. Investigators will use a control arm (Group B) to compare the impact of exposure to the educational curriculum to the impact of the baseline testing (T1) alone on subsequent test scores. However, to maintain the goal of mastery learning curriculum of allowing all learners to achieve mastery, following the second test (T2), the Group B participants will also be exposed to the intervention. All participants will return for a third test (T3). For participants in Group A, T3 will provide information about the decay of learned skills and if performance on the task is maintained by a single intervention or would require further follow-up education.

Standardized patients will score the residents upon completion of each simulation encounter, and they will be masked to the assignment of the participant.

Participants will have at least three study visits. The first visit (T1) will consist of a baseline test, during which participants will complete a simulated encounter with a standardized patient (SP) to establish a baseline score. Participants in Group A will be given access to the online interactive module and will have the opportunity to have video-based DP sessions with the SPs. The second visit (T2) will be scheduled 4-8 weeks after T1 and will consist of another simulated encounter with the SP. Group A will return for a third visit (T3) to provide information about the decay of learned skills and if performance on the task is maintained by a single intervention.

Participants will have at least three study visits. The first visit (T1) will consist of a baseline test, during which participants will complete a simulated encounter with a standardized patient (SP) to establish a baseline score. The second visit (T2) will be scheduled 4-8 weeks after T1 and will consist of another simulated encounter with the SP; following this session, participants in Group B will be introduced to the intervention. Group B will return for a third visit (T3) to provide information on how the curriculum impacts their score.

The intervention includes: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allow trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions will be conducted with standardized patients via a video platform, during a scheduled appointment.

Investigators will measure the percentage of residents who meet or exceed the Minimum Passing Standard (MPS) on the Uncertainty Communication Checklist (UCC) in the intervention group versus the control group at the second assessment (T2). At T2, resident physicians completed a simulation encounter with a standardized patient (SP) and the SP assessed the physician's performance using a checklist using the UCC. Our team developed the UCC in prior work with extensive patient input. The UCC includes 21 checklist items; items were assessed as either performed correctly (1 point) or incorrectly (0 points) and all items were added into a total score, which can range from 0-21 points. Since 1 = achieved and 0 = did not achieve, 1 is always better for each individual item and the higher the total score the better. Physicians achieved mastery when they met or exceeded the MPS, i.e. when they correctly performed at least 19 of the 21 items.

Although we originally planned for T2 to occur 4-8 weeks after the baseline assessment, T2 ended up taking place 16-19 weeks after baseline assessment due to the residency schedule.

Investigators will evaluate the association between number of Deliberate Practice (DP) sessions completed and achieving the Minimum Passing Standard (MPS)

This will assess the change within groups on the percentage pass rate on the Uncertainty Communication Checklist (UCC) from the second assessment (T2) to the third assessment (T3), which will assess retention of mastery in the intervention group and will be a supplemental assessment of the interventions' efficacy in the delayed intervention (control) group.

Thus this outcome measure was assessed at T2 (16-19 weeks after baseline assessment) and T3; although we originally planned for T3 to occur 4-8 weeks after T2, T3 ended up taking place 11-15 weeks after T2 due to the residency schedule.

Inclusion Criteria: -Eligible participants include all emergency medicine residents at Thomas Jefferson University and Northwestern University. Exclusion Criteria: -None

McCarthy DM, Powell RE, Cameron KA, Salzman DH, Papanagnou D, Doty AM, Leiby BE, Piserchia K, Klein MR, Zhang XC, McGaghie WC, Rising KL. Simulation-based mastery learning compared to standard education for discussing diagnostic uncertainty with patients in the emergency department: a randomized controlled trial. BMC Med Educ. 2020 Feb 19;20(1):49. doi: 10.1186/s12909-020-1926-y.