(VR/AR/MR) Reality in Healthcare

By Gajen Sunthara, Director of Innovation R&D, Innovation Program, Boston Children’s Hospital

Gajen Sunthara, Director of Innovation R&D, Innovation Program, Boston Children’s Hospital

During the Google Glass (GG) evolution, all types of “reality” became mainstream including Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). During the debut of the GG era, I wrote my thesis at Harvard University on “GlassSurgeon: Enhanced workflow for surgeons using Google Glass”. Fast forwarding to now, the industry has made a stunning progress to bringing users an “immersive” experience in all types of realities. Let’s understand the performance of these technologies and think about a few different applications within healthcare. All types of “realities” provide “immersive” experience for the users while differentiating in the following.

"VR has the potential to lead the space of simulation and training programs, where it can immerse the clinicians into a virtual reality experience"

• Virtual Reality (VR): Computer-simulated reality
• Augmented Reality (AR): Real life (live) viewed with enhanced or supplemented input
• Mixed Reality (MR): Hybrid of both VR and AR where there are real and virtual simulations.

Healthcare offers a number of unique cases that can be leveraged by both VR and AR, while AR provides more low-hanging-fruit use cases and does not completely take people out of the real world.

Simulator, Training and Education Programs

The flight simulator program that trains the pilot before they fly the actual plane has been around since the 1920s. Healthcare consists of the very same scenario where it deals with life or death situations and very high-risk industries. In academic and teaching health systems, it is very common to have a simulator and training programs that train healthcare providers like surgeons, an anesthesiologist on a complicated procedure before on actual patients.

There could be phased approaches to training the next generation clinicians with VR and AR. First, new trainees will be able to completely visualize and walk through with just the virtual reality immersive experience with VR tech. Second, with a clinical scenario and mannequins with AR tech, they can aid in showing the real world, life view with an augmented display that walks them through with step by step instructions for the complicated procedure.

Medical training also can be enhanced by revolution of these “reality” technologies where trainers are able to observe and grade medical students.

Surgical

In a real world scenario; there is no room for errors as the VRs only show virtual world and it is not going to work best as the user want to see the real world surgical situations with enhanced information. Imagine the following clinical scenario: a patient on a full sedation in the operating room where AR technology can show the real world surgical scenario “blueprint” of the whole procedure for the surgeon. For example, the surgeon just received the medical scan for the area of procedure, now AR tech can align the medical image with the patient’s tumor that needs to be removed.

Surgical environments are usually complex and the “reality” technology needs to be sterile, wire-free, with LED lights, black glass that can be replaced with clear glass or protective glass, with supported magnifying surgical loupes, point of view video recording of a surgical procedure with enhanced information like highlighting veins, showing the tumors and information with computer visions is the next level AR applications.

In conclusion, there are potential uses of VR, AR, and MR technologies in the Healthcare systems. VR has the potential to lead the space of simulation and training programs, where it can immerse the clinicians into a virtual reality experience. AR/MR is still enduring an early phase, especially the “computer vision”. Healthcare is complex in terms of knowing medicine that is a high-risk complex industry. A major disadvantage in these technologies would be the instance that they are still tethered, not wireless and it is not medical grade equipment. However it can be hacked to show proof of the concept.