Video-based Collaborative Learning to Improve Ventral Hernia Repair

Agency for Healthcare Research and Quality (AHRQ), University of Wisconsin, Madison, Abdominal Core Health Quality Collaborative

Recent studies demonstrate the critical role of individual surgeon performance, including both the approach they take to an operation as well as their technical skill, in determining patient outcomes. Utilizing the rich data collected by the Abdominal Core Health Quality Collaborative (ACHQC), formerly Americas Hernia Society Quality Collaborative (AHSQC), for its 200 members performing ventral hernia repair in the United States, the investigators will examine the effectiveness of video-based collaborative learning to provide feedback and improve surgical performance and patient outcomes. A prospective randomized trial comparing two interventions is proposed, comparing live video-based surgical coaching and video-based feedback using time-stamped annotations that can be reviewed at a later time to a wait-list control group drawn from the same cohort of surgeons. The results have the immediate potential to improve the quality of care for the 350,000 patients requiring ventral hernia repair each year, while also providing critical evidence to support a novel approach to surgical performance improvement more broadly.

Hernias of the abdominal wall are among the most common surgical disorders, with over 350,000 patients requiring repair annually for a total cost of $3.2 billion. Despite their prevalence, outcomes remain poor and variable, with recurrence rates that range between 32% and 63% at 10 years and a 5-fold difference in recurrence rates between surgeons. With the introduction of new repair techniques and continuous advances in biomedical devices and technology, it is difficult for practicing surgeons to stay abreast of new developments and incorporate these advances into their practice. Collaborative learning through video review can support continuous professional development and aid practicing surgeons in adopting new approaches and continually improving their own technical skills. Surgeons have limited time to invest in surgical quality improvement despite recognizing its importance. A critical gap in our current knowledge is the optimal approach to video-based collaborative learning. The effectiveness of two approaches to video review and performance feedback are herein proposed to be compared: synchronous surgical coaching versus asynchronous feedback. This study will be performed in partnership with the Abdominal Core Health Quality Collaborative (ACHQC), formerly Americas Hernia Society Quality Collaborative (AHSQC). This name change took place in July 2020. Surgeons will be paired with a coach and will participate in collaborative review of ventral hernia repairs. Participants will be randomized to undergo in-person coaching or asynchronous feedback utilizing a web-based video platform or will be assigned to a wait-list control group. All participants will submit 2 procedural videos pre- and post-intervention for self-assessment and blinded expert review utilizing a previously validated assessment instrument to measure technical skill. The primary outcome is change in 30-day procedural outcomes, which are captured and risk adjusted in the ACHQC. Participants will be followed for 2 years to assess long-term recurrence rates. Other secondary outcomes include intervention adherence, time spent in review, and the perceived value of the different approaches. These interventions will have immediate impact. Working with the ACHQC, the potential exists to improve quality and decrease costs for the 350,000 patients that require a ventral hernia repair annually. In addition, this will provide a new paradigm for surgical quality improvement and continuing professional development.

Coaches will facilitate an initial, individual, introductory phone call with participants prior to the first formal coaching session. The objective of this call is to develop rapport, explore each other's background, experience, and motivation for participation in the program, set overall goals for the program, set specific goals for the first coaching session, develop an action plan including identification of the key characteristics of the first case for review, and develop a timeline and plan for meetings. Peer coaching sessions will be scheduled at three national meetings that are commonly attended by ACHQC surgeons. In advance of each meeting, participants will record and upload a self-selected video to a secure server maintained by the study team, and coaches will have the opportunity to review the video if they wish to prepare. A live coaching session will be organized at the meeting where the coaches and participants will have parallel one-hour coaching sessions.

There will be no real-time interpersonal contact between coaches and participants in this arm. Participants will upload their self-selected procedural video to the video review platform, together with a short description of the case and any specific questions. The coach will review the video within one week of its posting and provide time-stamped feedback on the video platform. Participants will then review the coach's feedback within one week with the ability to respond to the comments. The coach and participant will continue communication via the internet-based review platform until no further comments are made by either party. Coach-participant dyads are expected to review three videos during the 6 month intervention period.

One-third of participants will be randomized to an intervention, but wait-listed to provide a control group. These surgeons will submit two videos for technical skill evaluation during each of the baseline and follow-up periods, and ACHQC data will be tracked for short-term outcomes prior to their crossover to the intervention for long-term follow-up. Selecting the control group using the identical sampling frame of ACHQC surgeons participating in the interventions affords the opportunity for a comparable group with outcome metrics recorded systematically.

Surgical coaching is based on a peer coaching model that has been proven to be highly effective in other disciplines and for surgeons in training. Surgical coaching can improve technical performance in one of two ways: 1) identifying new or alternative approaches to the procedure; or 2) improving surgeon technical skill, which can lead to improvements in patient outcomes. In this proposal, our primary objective is to evaluate the effectiveness of video-based collaborative learning for surgeons.

Constructive feedback is critical for performance improvement. It is currently unknown, however, whether asynchronous constructive feedback alone, without interactive discussions with a coach, would be as acceptable and valued as formal in-person coaching. Furthermore, the effectiveness of either approach to collaborative learning in improving surgical performance and outcomes is not known.

A repeated measures ANOVA will be used to assess the Objective Structured Assessment of Technical Skills (OSATS) score between the two baseline and two follow-up measures for surgeons randomly assigned to the Coaching and Constructive Feedback intervention arms, as compared with the wait-list control group. The modified OSATS instrument is a validated rubric that assesses a surgeon's overall technical skill. Skill is measured as both a summary score on a scale from 1 (deficient) to 5 (expert), or as a total score across five subdomains (gentleness, tissue exposure, instrument handling, time and motion, flow of operation). Each domain is rated on the same 1 to 5 point Likert scale, leading to overall scores ranging from 5-25. Scoring will continue until a minimum of 15 scores are obtained for each video.

The approach to the assessment of change in Surgical Site Occurrence (SSO) rates from baseline to follow-up will be assessed similarly to technical skill; however, in this case longitudinal mixed model Poisson regression will be used, with group assignment as the primary explanatory variable. The data will be examined for over dispersion. SSO is a well-defined measure for ventral hernia repair and a composite measure of several outcomes collected through the ACHQC. This includes surgical site infection, seroma, wound dehiscence, and enterocutaneous fistula.

Intervention Adherence: Number of Surgeon Participants who Complete All Requisite Training and Session

Intervention adherence will be assessed as the proportion of surgeon participants in each of the arms that complete all of the requisite training and sessions. Based on previous experience, an estimate of 75% of surgeons will complete all 3 sessions in the real-time coaching arm, while 50% will complete all 3 cases in the Asynchronous Feedback arm. This is the hypothesis based on challenges in prior studies in coordinating stand-alone activities with surgeons. Prior work suggests all surgeons who volunteer will participate in a minimum of one intervention component.

Another secondary analysis will assess differences in recurrence rates at one and two-years post-intervention, by comparing the Coaching and Constructive Feedback interventions to ACHQC surgeons who are neither surgeons nor coaches in this study. The analysis of long-term follow up is exploratory as a primary analysis would require that we maintain the control group without intervention for 2.5 years to ensure sufficient power, which is not feasible and could adversely impact loss-to- follow-up in the control group.

The perceived value of coaching will be measured at the completion of the intervention by asking coaches and participants to rate on a 1 (low) - 5 (high) scale the perceived value of the intervention in terms of personal performance improvement. Since previous participants have rated the live coaching as highly valuable, we aim to test whether Asynchronous Feedback is perceived to be of significantly lower value than live coaching (non-inferiority, margin of 0.3).

Coaches and surgeons will log any time spent in activities related to the program in a logbook provided by the study team. Reported time will include time in training, preparing for sessions, reviewing videos, and providing feedback (whether as written asynchronous feedback via Asynchronous Feedback or Live Coaching). The total amount of time spent will be calculated in minutes by coaches and participants for each intervention. This will test a hypothesis that Asynchronous Feedback is less time efficient than live coaching (non-inferiority, margin of 0.5).

Inclusion Criteria: ACHQC membership in good standing submission of a minimum of 10 eligible cases within the 6 months preceding the time of enrollment in the trial Exclusion Criteria: not an ACHQC member not submitting cases to ACHQC within the 6 months preceding the time of enrollment in the trial