Use of an Observer Tool to Improve Learning Outcomes for Cardiac Arrest Management

Does the Use of an Observer Tool During Simulation Training Improves Learning Outcomes for Cardiac Arrest Management: a Randomized Controlled Trial

Only approximately 10.4% of patients with out-of-hospital cardiac arrest (OHCA) survive their initial hospitalization. The very early management of cardiac arrest is indeed considered an extremely important criterion to improve patients' outcome. Despite recent advances and improved results, outcomes remain poor, especially because bystanders are not well trained to manage such a crisis situation. According to the International Liaison Committee on Resuscitation, teaching and learning the knowledge, skills and attitudes of resuscitation with the ultimate aim of improving patient survival after cardiac arrest is thus an essential goal. Teaching and learning strategies are numerous and have been well described and reviewed recently. Considering initial training methods already requires attention as study results often fail to show complete and effective learning after the initial training. Educational efficiency may be improved by several means (for example use of digital media) but it is now clear that more effective training strategies, i.e. those focusing on the cognitive process which leads to inclusion in memory), are needed. One recently introduced strategy, mainly in the field of simulation, is the use of observer tools. It has been shown that although effectiveness was still poorly demonstrated, this method had the potential to improve learning outcomes. An observer tool is a document which is used by observers during a scenario played by other learners and immediately compares the player's technical or non-technical skills with a referential guide. As it has been shown that observers learn less than those on the "hot seat" and as time for training is limited and the number of trainees is huge, focusing on observers is important. By visually assessing the practice of others repeatedly during the course and comparing each performance to the reference (i.e. the observer tool), the learner may increase his engagement in the training process with an expected final result of improved training efficacy. In a previous study performed in our simulation center, it has been have shown that using an observer tool increases immediate learning scores during operating room crisis management simulation. Good data on the use of such an observer tool in other training fields is needed to confirm these preliminary positive results.

Outcomes of cardiac arrest remain poor (1). The International Liaison Committee on Resuscitation (ILCOR) emphasizes that one essential component for good resuscitation outcomes is effective education of the lay public and resuscitation providers (2). Even when modern learning methods (i.e. for example digital aids) are used, the final result at the end of initial training can be described as providing incomplete knowledge and performance. During training sessions, it is often seen that the time during which the learners really act is short and they are positioned in a role of observer for a variable duration (but often a long proportion of the course duration). As it has been shown that observers learn less than those on the "hot seat" (3) and as time for training is limited and the number of trainees is huge, focusing on observers is important to increase the overall level of knowledge. To increase the positive effects of the simulation when the learner is in the role of observer, some authors have proposed to strengthen the educational effect through the use of an observer tool that observers must complete by analyzing the progress of the task performed by their colleagues (4). This is a list describing the set of key points to be achieved. However, data concerning the educational value of these observer tools are limited. Studies on the use of these tools during crisis management training in the operating room assessed by high-fidelity simulation have been already conducted (5) but good data on the use of such an observer tool in other training fields is needed to confirm these preliminary positive results. The objective of this study will be to assess the value of using an observer tool (OT) (including mainly the key points of technical performance) to improve learning outcomes of medical students during cardiac arrest training involving simulation. For medical students, learning how to manage a cardiac arrest is taught early in their curriculum, at a stage where their medical knowledge is low, and their overall competence is close to a layperson. This means that improving medical students' learning outcomes may also be a reasonable mean to improve training methods of non-medical personnel. This prospective and randomized study will be performed in the Paris-Saclay University (LabForSIMS) simulation center. The study will be carried out with the use of the CONSORT tool adapted for simulation studies and the GREET Tool for educational studies (6). Students will be in the first part of the 2nd year of their curriculum, explaining why only basic life support is taught at this stage. The training session lasts the whole day and 20-30 students are trained each day. Sessions are mainly focused on cardiac arrest management but a third of the day is dedicated to learning other resuscitation techniques (management of limb hemorrhage, Heimlich maneuver, recovery position for an unconscious patient who maintains spontaneous breathing…). The cardiac arrest part of the workshop is divided into five portions (diagnosis of unconsciousness, call for help/emergency services, diagnosis of apnea, external cardiac massage and use of an automated defibrillator). At each stage, two students are encouraged to make a spontaneous preliminary performance followed by other students' and teacher's comments and explanations. Students are then invited to perform by themselves the scenario, with progressive increments at each of the five additional portions. In practical terms, students working in dyads, perform initially the procedure which leads to diagnosis of unconsciousness but at the end of the session, they perform the whole five steps described above. When students are not players, they observe the dyad and are encouraged to analyze what has been well done and what could have better performed. Cardiac massage training is facilitated by the use of a feed-back device (Q-CPR Laerdal ® mannequin equipped with the wireless Skill reporter®software) which displays on a screen the main parameters (i.e. frequency, depth of massage, hand position, rib relaxation), allowing trainees to assess and improve immediately their performance. In the present study, after informed consent, students will be randomized in to groups: Group with an observation tool (OT +) filled out at each time the students are not on the hot seat. The observer tool is paper-based and is divided in columns (each column filled out for a dyad observed) with lines representing the above-described steps of learning and presented with visual incremental actions to be performed. The observer tool will only be used to reinforce learning and will not be used for summative evaluation. Instructors have already been involved in previous simulation studies performed in our center, and especially those in which an observer tool has been assessed (one study published, two manuscripts in preparation). Group without observation tool (OT-): these students will be trained as usual, i.e. will be encouraged to express their thoughts after each dyad has finished but will not be given the observer tool. The randomization will be performed by day as it is not possible to randomize students themselves on a given day for practical reasons. At the end of the training day, all learners will be again invited to perform a complete cardiac arrest management (basic life support part), using the previously described mannequin with feed-back information. During this ultimate performance, students will not be allowed to see the screen and will not know if they perform adequately. The overall score automatically provided by the Q-CPR/Skill Reporter® system will only be available to the assessor and the result provided to the student at the end of the test with a short feed-back from the instructor. A 10 question MCQ test will also be applied at the end of the session. This MCQ will be prepared by the instructors who had prepared the checklist to highlight important theoretical knowledge on cardiac arrest management.

In this group (OT +), each student will complete an observer tool each time another dyad is performing

In this group (OT-), the student will not use the observer tool and will observe other students without any physical support.

The observer tool is paper-based and is divided in columns (each column filled out for a dyad observed) with lines representing the above-described steps of learning and presented with visual incremental actions to be performed. The observer tool will only be used to reinforce learning and will not be used for summative evaluation. Instructors have already been involved in previous simulation studies performed in our center, and especially those in which an observer tool has been assessed (one study published, two manuscripts in preparation).

The primary outcome will be a global score describing acquisition of the skills immediately after training by comparing the groups using or not the OT (level 1-2 of the Kirkpatrick model). This score will be the sum of adequately performed items (yes/no) of the checklist which forms the observer tool itself. Although this checklist has not been validated previously, it has been constructed by experts in education, intensive care and simulation who are well aware of the constraints which are associated with such an assessment method. In addition, checklists are a usual mode of evaluation on procedural simulation. The total score is a 25 point-checklist which describes the overall performance of the dyad. Each student will also be assessed separately and points related to his (her) specific action will be counted.

The Q-CPR Laerdal ® mannequin equipped with the wireless Skill reporter® software) displays on a screen the main parameters (i.e. frequency, depth of massage, hand position and rib relaxation) and calculates an overall score which includes also the number of compressions per minute. Criteria of the compression score are taken up by the ERC 2015 recommendations in the definition of "High quality CPR". This global score has been used in previous published studies to evaluate the overall performance during cardiac massage. Individual components of the score are also provided by the software.

Inclusion Criteria: 2nd-year medical students Exclusion Criteria: unwilling to be assessed absence to the training

Panchal AR, Bartos JA, Cabanas JG, Donnino MW, Drennan IR, Hirsch KG, Kudenchuk PJ, Kurz MC, Lavonas EJ, Morley PT, O'Neil BJ, Peberdy MA, Rittenberger JC, Rodriguez AJ, Sawyer KN, Berg KM; Adult Basic and Advanced Life Support Writing Group. Part 3: Adult Basic and Advanced Life Support: 2020 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2020 Oct 20;142(16_suppl_2):S366-S468. doi: 10.1161/CIR.0000000000000916. Epub 2020 Oct 21. No abstract available.

Greif R, Lockey A, Breckwoldt J, Carmona F, Conaghan P, Kuzovlev A, Pflanzl-Knizacek L, Sari F, Shammet S, Scapigliati A, Turner N, Yeung J, Monsieurs KG. European Resuscitation Council Guidelines 2021: Education for resuscitation. Resuscitation. 2021 Apr;161:388-407. doi: 10.1016/j.resuscitation.2021.02.016. Epub 2021 Mar 24.

Suet G, Blanie A, De Montblanc J, Benhamou D. Use of an Observer Tool to Enhance Observers' Learning of Anesthesia Residents During High-Fidelity Simulation: A Randomized Controlled Trial. Simul Healthc. 2022 Feb 1;17(1):e75-e82. doi: 10.1097/SIH.0000000000000584.

Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R. Anaesthetists' Non-Technical Skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth. 2003 May;90(5):580-8. doi: 10.1093/bja/aeg112.