Impact of Augmented Reality in Polypectomy Skills Acquisition in Simulation-based Endoscopy Training

Impact of Augmented Reality on Procedural Skills Acquisition in a Simulation-based Training Curriculum for Polypectomy: A Randomized Controlled Trial

Simulation-based training (SBT) is a safe and effective strategy for improving skills development in gastrointestinal endoscopy. The use of curricula based on progressive learning, as well as comprehensive structured curricula, have been demonstrated to be effective in enhancing simulation-based training. With current advancements in technologies, another possible enhancement to SBT is the use of augmented reality (AR). To date, no other studies have examined the benefits of AR technology in endoscopy training. This study aims to evaluate the effectiveness of a simulation-based AR curriculum in developing technical skills, self-assessment accuracy, and clinical performance; as compared to a conventional simulation curriculum. Thirty six novice endoscopists will be recruited from the gastroenterology and general surgery programs at the University of Toronto. Participants will be randomized and assigned to two groups. The Conventional Simulation Training Curriculum group will receive 6 hours of simulated training, with expert feedback, and four 1-hour didactic teaching sessions. The AR Training Curriculum group will receive the same number of training hours and didactic sessions as the control group. The main difference is the use of superimposed videos to guide the intervention group through simulated polypectomy cases. During the didactic teaching sessions, the intervention group will also receive a brief introduction to principles of AR and its uses for endoscopy simulation. Participants will be trained to perform colonoscopies on two validated simulator models: (1) a bench-top colonoscopy simulator; and (2) the EndoVR® virtual reality simulator. Performance will be assessed before training (pre-test), immediately after training (acquisition post-test) and 4-6 weeks after training (retention test). On the same day as the retention test, the participants will perform two live colonoscopies and use a mechanical polypectomy simulation test (transfer tests), assessed by two blinded expert endoscopists. The main hypothesis of this study is that novices trained under the AR-enhanced curriculum will have better technical skill performance during simulated polypectomies and live colonoscopies.

Simulation-based training (SBT) provides a safe and effective means to enhance skills development. Simulation-based curricula have been developed for a number of minimally invasive procedures, and more recently in gastrointestinal endoscopy. Previous studies have demonstrated the effectiveness of comprehensive structured curricula and curricula based on progressive learning in endoscopic simulation. However, as digital technologies evolve, other educational strategies using the latest innovations may aid to further enhance procedural skills training during simulation. One such strategy may lie in augmented reality (AR). The use of AR involves the superimposition of a computer-generated image on a user's view of the real world, which enhances the user's perception of reality. There has been a recent, growing interest for the integration of AR in medical education, such as in the visualization of anatomical structures, training for various medical procedures and telemedicine guidance for remote procedural training. The real time interactive nature of AR provides immersion, immediate learner support, and has the ability to lower the complexity of learning new tasks by showing the content of the tasks using different perspectives, which is beneficial for both engagement and the learning process. To date, there are only a few studies which have investigated the application of AR for medical education and procedural learning specifically. Some of the benefits demonstrated by these studies include decreased amount of practice needed, reduced failure rate, improved performance accuracy, accelerated learning, shortened learning curve and better understanding of spatial relationships. However, the majority of studies on the application of AR have been on training for laparoscopy, neurosurgical procedures, and echocardiography. There are currently no studies showing the benefits of AR in endoscopy training. Furthermore, several studies have shown the importance of self-assessment for suitable lifelong learning of new techniques in minimally invasive surgery and gastrointestinal endoscopy. A study by our group showed that experienced endoscopists had better self-assessment accuracy compared to novice endoscopists, therefore, a targeted intervention to help novice endoscopists is needed so that they may seek appropriate preceptorship during their training. One such targeted intervention to improve self-assessment accuracy may be AR, as it provides immediate and repeated visual support of the perfect approach to perform a specific task, therefore, allowing the trainee to reflect on their performance. In order to bridge the gap related to AR in endoscopy training, the investigators aim to determine the impact of a simulation-based AR curriculum on areas such as knowledge acquisition, technical performance during simulated polypectomies, non-technical skills during integrated scenarios, and self-assessment accuracy. The investigators hypothesize that AR is an instructional strategy with the potential to offer a highly realistic and meaningful learning experience for novice endoscopists with transferable skills to clinical practice.

Expert endoscopists will assess the performance of the participants during simulated colonoscopy, simulated polypectomies, simulated scenarios, and two live colonoscopy procedures. Assessors will have no knowledge of the group assignments and assessments will be done by viewing video recordings of the participants. The videos will be edited to remove the faces and other identifiable characteristics of the participants and patients.

Four, one-hour small-group sessions on the theory of colonoscopy including pathology, anatomy, and therapeutic technique. Following each session, a multiple choice test on topics covered will be administered. In addition, this group will be given a total of six hours of expert-assisted instruction on both the low-fidelity simulator (1 hour) and the high-fidelity VR simulator (5 hours). During the high-fidelity simulation, endoscopic procedures will be performed with instructor support. The difficulty of the therapeutic intervention (polypectomy) will rise after each successfully completed module. The last two hours of training on the high-fidelity simulator will consist of two integrated scenarios.

This group will receive the same 4 hours of small group teaching and 6-hours of hands-on simulator training. The intervention is the augmented reality-based curriculum: (1) a brief explanation of the principles of AR and how it will be used during the VR simulations and (2) performance of the therapeutic procedure (polypectomy) as demonstrated by the real-time AR platform. Specific videos corresponding to the therapeutic intervention and pathology (e.g pedunculated vs non-pedunculated polyp) will be available every time a polypectomy is required. Each new module will come with increased technical challenges and will require adjustment to previously used technique by the learner. The last two hours of training on the high-fidelity simulator will consist of two integrated scenarios.

Participants in the Augmented Reality Group will be taught principles in augmented reality and training sessions will be enhanced by an augmented reality platform. This involves training on virtual reality simulators with superimposed videos and guides demonstrating correct procedure. In this study, the videos and guides will be on identifying and removing polyps.

Participants in this group will will train from low- to high-fidelity colonoscopy simulation. During training, the participants will receive guidance and feedback from expert endoscopists. In addition, there will be didactic sessions covering theory of colonoscopy.

The primary outcome measure is change in polypectomy technical performance between during the Integrated Scenario Tests and in Polypectomy Simulator Tests from pre-tests as assessed using the Direct Observation of Polypectomy Skills assessment tool (DOPyS; a validated task-specific polypectomy scoring tool) by two experienced endoscopists who will be blinded to group assignment. Range is scored from 0 to 100%; higher values represent a better outcome.

The change in DOPyS score will be assessed between pre-test and post-test done 6 weeks after the intervention.

Performance during simulated (pre, post, delayed) and two live clinical colonoscopies will be assessed using the Joint Advisory Group Direct Observation of Procedural Skills tool, a validated task-specific tool for performance in colonoscopy. Range is scored from 0 to 100%; higher values represent a better outcome.

The change in technical performance will be measured between a pre-test and post-test 6 weeks after the intervention.

Participants will be asked to assess their own colonoscopy performance using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). This will be compared with assessor evaluations. The purpose of this outcome measure is to investigate whether augmented reality helps improve self-assessment accuracy in novice endoscopists. The GiECAT is represented as a percentage from 0-100, with higher percentage score indicating greater self-assessment of performance, a better outcome.

The self-assessment will be at the time of the Integrated Scenario Tests 6 weeks after the intervention.

The non-technical skill performance will be measured using the Modified Objective Structured Assessment of Non-Technical Skills (MOSANTS) tool for colonoscopy. This is a score from 5-25, with higher scores indicating greater non-technical skill performance.

Participant non-technical skill performance will be measured during the integrated scenario six weeks after the intervention.

Assessed by the Nurse-Assessed Patient Comfort Score (NAPCOMS), a validated score for patient comfort in colonoscopy, ranging from 0-9, with higher scores indicating less comfort, a worse outcome.

Participants will be asked to complete a questionnaire using the adapted General Self-Efficacy Scale. This validated score of self-efficacy has a scale from 4-40, with higher scores indicating higher self-efficacy (a better outcome).

Participants will be asked to complete a questionnaire that includes the Cognitive Load Inventory for Colonoscopy (CLIC) Scale, a validated colonoscopy-specific cognitive load scale. This scale ranges from 0 to 150, with higher scores indicating greater cognitive load (a worse outcome).

Measured by the change in the colonoscopy-specific knowledge multiple choice test (a validated cognitive assessment tool for colonoscopy) between pre-test and post-test. This tool has been used in many prior studies in simulation in colonoscopy, is scored from 0-20 with higher scores indicating greater knowledge (a better outcome).

This will be measured as the difference in score between pre-test and post-test, administered 6 weeks after the intervention.

Inclusion Criteria: Postgraduate trainees in the gastroenterology, general surgery, or internal medicine residency programs Performed 25 or less live or simulated endoscopic procedures Exclusion Criteria: Performed more than 25 live or simulated endoscopic procedures