Holographic Screens as a Replacement of Monitors During GI Endoscopies

Mixed-reality Holographic Screens as the Replacement of Monitors in the Endoscopy Suite: a Multicenter Study

Nowadays, the application and development of spatial technologies have shown an increased interest in different fields of medicine, especially in procedural specialties. Many studies have shown the utility of augmented and virtual reality; however, studies evaluating mixed reality are scarce. In gastroenterology, some proposed advantages of MR are the 3D space guidance, its increased situational awareness, remote assistance, and the reduction of surgical monitors in the units. Based on this, the researchers proposed a multicenter trial to assess the added value of MR through a holographic device during gastroenterology endoscopic procedures.

In the medical field, many technologies have emerged to aid healthcare providers during training, communication, and management of patients. The area of spatial computing is becoming part of the immersive developing technology. The main components of spatial computing are augmented, virtual, and mixed reality (AR, VR, and MR, respectively). VR is a human-computer interface that simulates a realistic environment, while AR superimpose elements of VR in a real-world environment. AR is an enhanced version of the real-world achieved through sensory information delivered via technology. Moreover, the combination of both is known as MR. Mixed reality is an interactive, real-time processed, three-dimension registered technology. In medicine, published data has broadly demonstrated the utility and advantages of AR and VR in many fields like neurosurgery, cardiology, maxillofacial, hepatobiliary, and orthopedics. However, information about MR for health purposes is scarce. Based on the above, it is necessary to develop studies to assess the utility of MR in other procedural medical fields like gastroenterology. Some of the proposed advantages of MR in gastroenterology, mainly during endoscopic procedures, are the 3D space guidance and gesture operation (translation and rotation) without touching the object, the increase in situational awareness by reduction of real-world and content refocusing, the remote assistance during procedures, and the replacement/reduction of the use of monitors during the procedures. To evaluate the advantages and added value of MR in gastroenterology, the researchers proposed a multicenter trial in the endoscopic units by using a built developed application (HXtendTM, mdconsgroup, Guayaquil, Ecuador) for the HoloLens2TM (Microsoft, Redmond, WA, USA) holographic device.

This group is comprised by expert gastrointestinal endoscopists designated to perform diagnostic procedures. The procedure will be performed without physical surgical monitors, and through the guidance of Mixed Reality by using the HXtend™ application holographic monitors.

Included expert gastrointestinal endoscopists performing diagnostic procedures with mixed reality guidance. All the procedures will be performed by using the HoloLens2TM (Microsoft, Redmond, WA, USA), an ergonomic self-contained holographic device. HoloLens2TM (Microsoft, Redmond, WA, USA) uses HXtend™ developed software (mdconsgroup, Guayaquil, Ecuador) installed and running on Windows holographic OS. The monitors signals are replicated into holograms by an app developed with the unity Software package (Version 2022.1.22, Unity Technologies, San Francisco, CA, USA). After each procedure, the researchers will evaluate the device performance along with the acceptance and experience of the endoscopists using this technology.

After performing the endoscopic procedure with mixed reality, the expert endoscopist´s experience will be evaluated. The investigators will use a Likert's scale to assess user´s experience (comfort, functionality, and learnability).

The quality of image during the endoscopic procedure (hologram and surgical field) will be assess through perceptual judgment of the user by using a Likert scale.

Latency, as a measurement of the time it takes to the data to travel from one node to the other, will be measured by the device´s developed software using a networking library. The software will display the network metrics of the transmission. The latency will be measured in milliseconds.

Interoperability, as the ability of computerized products to readily connect and exchange information with one another, will be measured between multiple video processors (Boston Scientific, Micro-tech and Pentax Medical) and the HoloLens2. Its assessment will be made by measuring the latency (ms) and the frames per second (FPS).

Interconnectivity, as the connection between to HoloLens through the HXtend™, and the use of gestures from one HoloLens displayed in the second HoloLens, will be assessed by measuring the latency in milliseconds.

Inclusion Criteria: Expert gastrointestinal endoscopists performing diagnostic colonoscopy, cholangioscopy, ERCP or EUS evaluation/intervention Written informed consent provided Exclusion Criteria: Refuse to participate in the study or to sign corresponding informed consent Internet connection less than 100 MBs per second.

Zhang J, Zhu L, Yao L, Ding X, Chen D, Wu H, Lu Z, Zhou W, Zhang L, An P, Xu B, Tan W, Hu S, Cheng F, Yu H. Deep learning-based pancreas segmentation and station recognition system in EUS: development and validation of a useful training tool (with video). Gastrointest Endosc. 2020 Oct;92(4):874-885.e3. doi: 10.1016/j.gie.2020.04.071. Epub 2020 May 6. Erratum In: Gastrointest Endosc. 2021 Mar;93(3):781.

Yeung AWK, Tosevska A, Klager E, Eibensteiner F, Laxar D, Stoyanov J, Glisic M, Zeiner S, Kulnik ST, Crutzen R, Kimberger O, Kletecka-Pulker M, Atanasov AG, Willschke H. Virtual and Augmented Reality Applications in Medicine: Analysis of the Scientific Literature. J Med Internet Res. 2021 Feb 10;23(2):e25499. doi: 10.2196/25499.

Reis G, Yilmaz M, Rambach J, Pagani A, Suarez-Ibarrola R, Miernik A, Lesur P, Minaskan N. Mixed reality applications in urology: Requirements and future potential. Ann Med Surg (Lond). 2021 May 13;66:102394. doi: 10.1016/j.amsu.2021.102394. eCollection 2021 Jun.

Verhey JT, Haglin JM, Verhey EM, Hartigan DE. Virtual, augmented, and mixed reality applications in orthopedic surgery. Int J Med Robot. 2020 Apr;16(2):e2067. doi: 10.1002/rcs.2067. Epub 2020 Feb 6.

Lee GK, Moshrefi S, Fuertes V, Veeravagu L, Nazerali R, Lin SJ. What Is Your Reality? Virtual, Augmented, and Mixed Reality in Plastic Surgery Training, Education, and Practice. Plast Reconstr Surg. 2021 Feb 1;147(2):505-511. doi: 10.1097/PRS.0000000000007595.