A view from the Center

Deloitte's Life Sciences & Health Care Blog

From center ice to the operating room: AR/VR technologies could transform consumer, clinician, and patient experiences

I live outside of Washington, D.C., and I was thrilled to see the Washington Capitals advance to the Stanley Cup finals for the first time in 20 years. I always enjoy watching sports on TV, but the experience has changed little since sports became televised. Now imagine donning an immersive experience headset and watching a championship game from center ice…with players gliding all around you. Or maybe you want to watch from the perspective of the goalie, or through the eyes of your favorite player. With tiny cameras embedded in each player’s helmet, the fans’ viewing experience could be individualized.

Virtual reality (VR) and augmented reality (AR) extends that immersive and individualizing capability in significant ways. Total spending on VR and AR products and services is projected to grow from $9.1 billion in 2017 to nearly $160 billion in 2021, representing a compound annual growth rate of 113.2 percent, according to a report from Deloitte Insights.1

While the entertainment and gaming industries are demonstrating leadership in the adoption of VR technology,2 other industries are also interested in changing the consumer experience. Automobile manufacturers, for example, might let you take a car for a virtual spin around a racetrack, or through the European countryside.3 Some Australian wineries are looking to create a virtual tourist experience by offering tours via headset.4

But what about health care? I see tremendous potential for AR/VR in health care. I see it altering workflows and changing the way clinicians work. Novel treatment options could emerge for acute and chronic conditions. But I also see barriers that we need to overcome.

VR/AR could facilitate the choreography of care
The operating room is a busy place. Pre-operatively, the surgeon needs to have accurate, timely, and relevant information about the patient (e.g., x-rays, MRIs, lab results, opinions from primary care physicians and specialists). Today, this information often must be collected manually, even if some of it is available in the patient’s electronic health record. Before the procedure begins, the appropriate surgical tools need to be prepared, ready to go, and confirmed with a checklist.

In the operating room, there are often multiple zones of activity, with the anesthesiologist, nurses, assistants, and other professionals playing important roles. Depending on the complexity of the procedure, there might be teams of clinicians entering and exiting. A pathologist might come in to collect tissue samples. A circulating nurse might be troubleshooting issues raised by the surgeon or other clinicians, while trainees observe and ask questions. The bursts of activity in a typical operating room can create a certain level of chaos that can be distracting, and could lead to an adverse event if unchecked.

How can AR/VR help? The potential exists for AR or VR—in conjunction with other technology enablement—to improve the choreography of care in the operating room. Here’s how that might look:

  • While performing a procedure, a surgeon equipped with an AR headset reviews relevant imaging data (historical or in-room C-arm), and can note real-time physiologic parameters from the anesthesiologist without needing to look away from the patient field.
  • A running count of instruments floats just inside of the surgeon’s peripheral view, placed wherever she or he deems fit. All of this information can disappear and reappear with prompts from the surgeon (e.g., a wave of the hand).
  • RFID tags embedded in medical sponges ensure that all sponges have been removed before the patient’s incisions are closed, with a real-time count available in the AR field of vision.
  • A camera built into the headset allows the surgeon to take pictures of the procedure and include them in the procedural documentation, or share with medical students and other learners.
  • Sensors attached to the surgeon can measure both case-specific parameters (e.g., cumulative case time) and surgeon-specific parameters (e.g., operator vital signs).

What are the potential benefits of a virtual or augmented world?
Along with assisting surgical care, VR also could be useful in the broader treatment of the patient, or in education of clinicians, caregivers, patients and their families: Consider these applications:

Treating patients: VR can simulate an environment that might assist behavior change in patients in a way that is safer, more convenient, and more accessible. Therapy for a patient trying to overcome opioid addiction, for example, might include some time immersed in a virtual world full of tailored distractions. A patient could spend an hour in the mountains or on the beach, or even painting in a virtual art studio. This sort of brain stimulation might help reduce the need for chemical stimulation and make it possible to alter the dose-response relationship with drugs for pain management. Researchers at the University of Houston are using VR-based addiction treatment to study innovative methods to combat substance-use disorder.5 Their research initially focused on smoking cessation, and then migrated to alcohol abuse and, more recently, IV-drug abuse. VR applications are not envisioned as replacements for comprehensive treatment, but they might have supplemental treatment roles.

Educating clinicians: Physicians could learn about procedures, and virtually practice a surgical technique, while wearing an AR or VR headset. Some academic medical centers are already incorporating AR/VR into their curricula. Last fall, Ohio’s Case Western Reserve University, piloted AR for 32 medical students.5 Rather than relying entirely on an actual human cadaver, students can use a smartphone or tablet to access a virtual cadaver for more extensive analysis and practice. The virtual cadaver remains in pristine condition no matter how many students use it. I certainly appreciate the value of undisturbed anatomy when I recall trying to learn structures from a cadaver that had already been prodded and manipulated by classmates in medical school.

What are the potential barriers to moving to a virtual world?
We are only in initial stages and are just beginning to see some demonstration projects. VR and AR likely won’t get much traction in health care if clinicians don’t see the value themselves and for their patients. There often is a level of provider apathy or disinterest in technology until it becomes mainstream or supports a relevant clinical workflow. The value of this technology is usually dependent upon content. An AR or VR headset without content can have little value. The algorithms that can turn MRI scans into 3-D images that are easy to manipulate creates value for the surgeon and for the surgical team. Specialty-specific tailoring will likely be essential for gaining acceptance.

We have limited but growing knowledge about the impact AR/VR might have on health outcomes. But we do know there could be some clinical risks. Similar to motion sickness, visually-induced motion sickness7 is a demonstrated phenomenon, as is the risk of real-world falls or injuries while wearing a VR headset.

While AR/VR technology could be transformative in health care, we are early in the first period (to get back to my hockey analogy). Health care will likely stay a step or two behind gaming and entertainment industries in developing AR/VR applications and adopting them. Next year, rather than watching the Stanley Cup finals from my couch, I might be watching through an AR headset, standing next to Braden Holtby as he swats pucks away from the goal. Once this type of experience becomes common, AR/VR as a routine tool in the OR might not be far off.

Go Caps!

(And PS, check out our newly launched series which highlights how eight tech trends could transform the life sciences and health care industry. You’ll find videos, podcasts, and short-form content rolling out over the coming months. Bookmark the page and check back for updates!)


1International Data Corp., Worldwide Semiannual Augmented and Virtual Reality Spending Guide, October 28, 2017.
2https://www.forbes.com/sites/kurtbadenhausen/2018/05/08/the-nba-and-intel-launch-a-joint-initiative-to-invest-in-startups-in-sports/#4e7d7b9e5ab1
3https://www.cnbc.com/2018/03/30/auto-companies-feature-virtual-reality-and-other-tech-heavy-exhibits-at-new-york-auto-show.html
4http://www.abc.net.au/news/rural/2018-05-08/virtual-reality-boosts-australian-wine-industry/9728440
5Michael Rass, The Pioneers of Virtual Reality in Addiction Treatment, Lakeview Health, December 18, 2017
6https://www.medscape.com/viewarticle/886718
7Mitigating visually induced motion sickness in virtual reality, Stanford University, 2016:
https://stanford.edu/class/ee267/Spring2016/report_becker_ngo.pdf

Author bio

Dr. Rangappa is an Internal Medicine trained physician with over 20 years of health care practice and consulting experience. He is a senior physician at Deloitte Consulting LLP and provides strategic guidance and operational assistance for many clients, across both the commercial and federal/government sectors. He has broad experience working in conjunction with teams of clinicians and administrators. Dr. Rangappa has been heavily involved with development of clinical content, clinical decision support, and the implementation of electronic medical record systems across large organizations. He has focused on physician adoption, engagement, and retention strategies needed for success in the face of technology- enabled transformation and digital innovations in the clinical workflow. Dr. Rangappa holds a Master’s degree in health care administration and is well-versed in ongoing and emerging health care policy issues and innovation trends.