A Novel Augmented Reality System (ARssist) for the Assistant Surgeon in Robotic Assisted Surgery

A Novel Augmented Reality System (ARssist) for the Assistant Surgeon in Robotic Assisted Surgery: A Pilot Study

Robotic prostatectomy is a surgery for treating localized prostate cancer. The ARssist system is a novel augmented reality system designed for the assistant surgeon, allowing delivery of augmented reality information via Microsoft HoloLens 2 (a head mount display developed by Microsoft) to better delineate the 3-D operative environment and enable better visualization. To date, there is no prospective study on the clinical performance and utilization of the ARssist system. This study is to evaluate the clinical feasibility and safety of the ARssist system during robotic surgery with the da Vinci Xi system.

The da Vinci robotic surgery system offers advantages such as immersive three-dimensional visualization, intuitive control, and high degree of movement freedom to the chief surgeon. However, major surgery remains a team-based effort. Apart from the chief surgeon who remotely controls the da Vinci system at the console, the success of robotic assisted surgical procedures also relies on the assistant surgeon positioned at the patient side, who provides assistance laparoscopically. Throughout a robotic assisted operation, the patient side surgeon is responsible for tasks such as the exchange of instruments, retraction of tissue to enhance operative fields, manipulation of instruments, etc. Literature has demonstrated that the performance of the assistant surgeon has an effect on the outcomes of robotic surgery such as operative time. Traditionally the patient side surgeon relies on the monitor mounted on the vision cart to guide his/her movement. The monitor provides real-time relay of the image captured from the endoscope, but does not provide the full three-dimensional stereo endoscopy view unless set up with specialized stereo-vision equipment. The position of the monitor is also often awkward, creating problems such as obstructed views and non-ergonomic positioning of the patient side surgeon to overcome the view obstruction. The complex three-dimensional set-up of endoscope, robotic instruments, and hand-held instruments inside the patient body could prove difficult to imagine from the patient side surgeon's perspective, and guesswork could be involved during transfer of instruments/objects towards the operative field as the hand-held instruments are often out of the visualized field of the endoscope. Augmented reality (AR) technology delivered via optical see-through head-mounted display (OST-HMD) could potentially be the solution to the aforementioned issues. OST-HMD, such as Microsoft HoloLens, can superimpose computer graphics on top of real-world view through optical combiners. The clinical application of such a technology has been gaining interest in the surgical community, with preliminary study demonstrating feasibility of AR technology in ureteroscopic procedures. The ARssist system is a novel AR system designed for the patient side surgeon in robotic assisted surgeries. It integrates the da Vinci surgical system and Microsoft HoloLens, and provides valuable AR information to the patient side surgeon including (i) three-dimensional real-time rendering of the endoscope, robotic instruments, and hand-held instruments within the patient body, and (ii) real-time stereo endoscopy that is configurable for the assistant surgeon's preferred hand-eye coordination. The ARssist system would in theory grant the patient side surgeon improved orientation and navigation of hand-held laparoscopic instruments, thus improving their laparoscopic performance and the performance of the surgery as a whole. Based on the Innovation, Development, Exploration, Assessment, Long-term Study (IDEAL) Collaboration group methodology of promoting surgical innovation and research through planned prospective studies within an established staged process, we propose a stage 1 (Innovation) study to evaluate the feasibility and safety of the ARssist system.

This is a prospective case series to be conducted as a pilot study to evaluate the clinical feasibility and safety of the ARssist system during robotic surgery with the da Vinci Xi system.

The surgery will follow the same steps of a standard robotic assisted radical prostatectomy procedure, with the addition of the ARssist system used by the patient side surgeon.

Performance of the patient side surgeon will be assessed using Global Operative Assessment of Laparoscopic Skills (GOALS) by different evaluators, including the patient side surgeon him/herself (self-evaluation), the console surgeon, and a third-party evaluator

The surgeries will be recorded audio-visually, with the captured video clips later reviewed by an independent assessor to determine objective performance outcomes of included tasks via motion analysis parameters

Specific feedback on the use of ARssist system will be collected from the patient side surgeon at the pre-operative and post-operative time points using a customized questionnaire of 10 items on a 5-point Likert scale

Inclusion Criteria: Age ≥ 18 years old with informed consent Suitable for minimally invasive surgery Clinically diagnosed with urological conditions that are indicated for robotic assisted radical prostatectomy with or without lymph node dissection Exclusion Criteria: Body mass index ≥ 35 kg/m2 Contraindication to general anaesthesia Severe concomitant illness that drastically shortens life expectancy or increases risk of therapeutic intervention Untreated active infection Uncorrectable coagulopathy Presence of another malignancy or distant metastasis Emergency surgery

Nayyar R, Yadav S, Singh P, Dogra PN. Impact of assistant surgeon on outcomes in robotic surgery. Indian J Urol. 2016 Jul-Sep;32(3):204-9. doi: 10.4103/0970-1591.185095.

Sgarbura O, Vasilescu C. The decisive role of the patient-side surgeon in robotic surgery. Surg Endosc. 2010 Dec;24(12):3149-55. doi: 10.1007/s00464-010-1108-9. Epub 2010 May 22.

Al Janabi HF, Aydin A, Palaneer S, Macchione N, Al-Jabir A, Khan MS, Dasgupta P, Ahmed K. Effectiveness of the HoloLens mixed-reality headset in minimally invasive surgery: a simulation-based feasibility study. Surg Endosc. 2020 Mar;34(3):1143-1149. doi: 10.1007/s00464-019-06862-3. Epub 2019 Jun 18.

Qian L, Deguet A, Kazanzides P. ARssist: augmented reality on a head-mounted display for the first assistant in robotic surgery. Healthc Technol Lett. 2018 Sep 17;5(5):194-200. doi: 10.1049/htl.2018.5065. eCollection 2018 Oct.

McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, Nicholl J; Balliol Collaboration; Aronson JK, Barkun JS, Blazeby JM, Boutron IC, Campbell WB, Clavien PA, Cook JA, Ergina PL, Feldman LS, Flum DR, Maddern GJ, Nicholl J, Reeves BC, Seiler CM, Strasberg SM, Meakins JL, Ashby D, Black N, Bunker J, Burton M, Campbell M, Chalkidou K, Chalmers I, de Leval M, Deeks J, Ergina PL, Grant A, Gray M, Greenhalgh R, Jenicek M, Kehoe S, Lilford R, Littlejohns P, Loke Y, Madhock R, McPherson K, Meakins J, Rothwell P, Summerskill B, Taggart D, Tekkis P, Thompson M, Treasure T, Trohler U, Vandenbroucke J. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009 Sep 26;374(9695):1105-12. doi: 10.1016/S0140-6736(09)61116-8.

Barkun JS, Aronson JK, Feldman LS, Maddern GJ, Strasberg SM; Balliol Collaboration; Altman DG, Barkun JS, Blazeby JM, Boutron IC, Campbell WB, Clavien PA, Cook JA, Ergina PL, Flum DR, Glasziou P, Marshall JC, McCulloch P, Nicholl J, Reeves BC, Seiler CM, Meakins JL, Ashby D, Black N, Bunker J, Burton M, Campbell M, Chalkidou K, Chalmers I, de Leval M, Deeks J, Grant A, Gray M, Greenhalgh R, Jenicek M, Kehoe S, Lilford R, Littlejohns P, Loke Y, Madhock R, McPherson K, Rothwell P, Summerskill B, Taggart D, Tekkis P, Thompson M, Treasure T, Trohler U, Vandenbroucke J. Evaluation and stages of surgical innovations. Lancet. 2009 Sep 26;374(9695):1089-96. doi: 10.1016/S0140-6736(09)61083-7.

Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB. Observational tools for assessment of procedural skills: a systematic review. Am J Surg. 2011 Oct;202(4):469-480.e6. doi: 10.1016/j.amjsurg.2010.10.020. Epub 2011 Jul 28.

Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005 Jul;190(1):107-13. doi: 10.1016/j.amjsurg.2005.04.004.

Wolf R, Medici M, Fiard G, Long JA, Moreau-Gaudry A, Cinquin P, Voros S. Comparison of the goals and MISTELS scores for the evaluation of surgeons on training benches. Int J Comput Assist Radiol Surg. 2018 Jan;13(1):95-103. doi: 10.1007/s11548-017-1645-y. Epub 2017 Aug 20.

Faulkner H, Regehr G, Martin J, Reznick R. Validation of an objective structured assessment of technical skill for surgical residents. Acad Med. 1996 Dec;71(12):1363-5. doi: 10.1097/00001888-199612000-00023.

Mason JD, Ansell J, Warren N, Torkington J. Is motion analysis a valid tool for assessing laparoscopic skill? Surg Endosc. 2013 May;27(5):1468-77. doi: 10.1007/s00464-012-2631-7. Epub 2012 Dec 12.

Dindo D, Demartines N, Clavien PA. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg. 2004 Aug;240(2):205-13. doi: 10.1097/01.sla.0000133083.54934.ae.