WashU-developed holograms help physicians during cardiac procedure

Jennifer Silva, MD, uses the holographic display during a cardiac ablation procedure. Courtesy photo.

Bringing a little bit of science fiction into an operating room, a team of engineers and physicians at Washington University in St. Louis has shown for the first time that using a holographic display improves physician accuracy when performing a procedure to treat irregular heartbeat.

Jennifer N. Avari Silva, MD, associate professor of pediatrics at the School of Medicine, and Jonathan Silva, associate professor of biomedical engineering in the McKelvey School of Engineering, co-led a team that tested a Microsoft HoloLens headset with custom software during cardiac ablation procedures on patients at St. Louis Children's Hospital. Results of the trial were published in the Journal of the American College of Cardiology – Clinical Electrophysiology July 2020, International Conference on Human Computer Interaction July 10, 2020 and IEEE Journal of Translational Engineering in Health and Medicine, July 3, 2020.

In an ablation procedure using existing technology, known as electroanatomic mapping system (EAMS), a technician controls the catheters while the physician views the images on monitors presented in two different planes. The physician then has to mentally create the image of the heart. Jennifer Silva, also director of pediatric electrophysiology at St. Louis Children's Hospital, said that standard-of-care technology is antiquated, and the team believed it could do better.

Jon Silva and his team of engineers created software for the Microsoft HoloLens headset that converts the data from the catheters fed into the patient's heart into a geometrical holographic image that hovers over the patient. The headset, which weighs roughly a pound, allows the physician to take control of the procedure by using his or her gaze to guide the controls and to keep hands free and sterile. Their system, Enhanced Electrophysiology Visualization and Interaction System (ĒLVIS), provides a 3D digital image of the patient's electroanatomic maps that provide a picture of the inside of the heart, which they can measure and manipulate during the procedure.

Jon Silva said that the technology has caught up with the application. The headset allows the user to view his or her whole environment, including the patient, unlike virtual reality, which take a user completely out of his or her environment.

"The old headsets were slow to update and made the users sick," he said. "The development of mobile phones and mobile technology and computer has enabled these kinds of displays and headsets."

To test the device, two physicians at St. Louis Children's Hospital received a short training session on the device before using it on a total of 16 pediatric patients. During a post-procedure waiting phase, the physicians were given 60 seconds to navigate to each of five target markers within the geometry of the heart, using both the 3D ĒLVIS and the 2D EAMS technology. The physicians were significantly more accurate with the ĒLVIS technology.

"Without the use of the ĒLVIS 3D display, a significant fraction of ablation lesions, 34%, would be made outside of the target area, as opposed to 6% with ĒLVIS 3D display," Jon Silva said. "We expect that this will improve patient outcomes and potentially reduce the need for repeat procedures."

Jennifer Silva said the team learned a lot from taking something from the lab to nearly market-ready.

"What ended up being equally important, if not more important, was that this was the springboard for everything that is to come, not only that we can visualize it better, but that we can control it," Jennifer Silva said. "There are people working in this extended reality space who have come to conclusions that the control is the strongest value-add, particularly in medical applications."

The Silvas licensed their technology to SentiAR, which is further developing the augmented reality software. SentiAR has funding from the National Institutes of Health as well as St. Louis investment firms Cultivation Capital and BioGenerator. They are working with the university's Office of Technology Management to bring the technology to market and have submitted it to the U.S. Food & Drug Administration for approval.


The McKelvey School of Engineering at Washington University in St. Louis promotes independent inquiry and education with an emphasis on scientific excellence, innovation and collaboration without boundaries. McKelvey Engineering has top-ranked research and graduate programs across departments, particularly in biomedical engineering, environmental engineering and computing, and has one of the most selective undergraduate programs in the country. With 140 full-time faculty, 1,387 undergraduate students, 1,448 graduate students and 21,000 living alumni, we are working to solve some of society’s greatest challenges; to prepare students to become leaders and innovate throughout their careers; and to be a catalyst of economic development for the St. Louis region and beyond.

Washington University School of Medicine's 1,500 faculty physicians also are the medical staff of Barnes-Jewish and St. Louis Children's hospitals. The School of Medicine is a leader in medical research, teaching and patient care, ranking among the top 10 medical schools in the nation by U.S. News & World Report. Through its affiliations with Barnes-Jewish and St. Louis Children's hospitals, the School of Medicine is linked to BJC HealthCare.

Silva J N A, Southworth M, Blume W, Andrews C, Van Hare G, Dalal A, Miller N, Sodhi S, Silva J R. First-In-Human Use of a Mixed Reality Display for Guidance during Transcatheter Cardiac Mapping and Ablation. Journal of the American College of Cardiology –Clinical Electrophysiology. July 2020, https://doi.org/10.1016/j.jacep.2020.04.036.

Silva J N A, Privitera M B, Southworth M, Silva J R. Development and Human Factors Considerations for Extended Reality Applications in Medicine: The Enhanced Electrophysiology Visualization and Interaction System (ĒLVIS). Human-Computer Interaction, July 10, 2020, https://doi.org/10.1007/978-3-030-49698-2_23.

Southworth M, Silva J N A, Blume W, Van Hare G, Dalal A, Silva J. Performance Evaluation of Mixed Reality Display for Guidance during Transcatheter Cardiac Mapping and Ablation. IEEE Journal of Translational Engineering in Health and Medicine, July 3, 2020, 10.1109/JTEHM.2020.3007031