Disclosures: Athwal reports being a consultant and design surgeon for and receiving royalties from Stryker for the implants and technology discussed. Vigdorchik reports being a consultant for Medacta. Witham reports being on the medical advisory board for and being an investor in Augmedics. Allen and Khazzam report no relevant financial disclosures.

Augmented reality used in knee-replacement surgery in the U.S. first. Available at: https://news.hss.edu/dr-jonathan-vigdorchik-performs-1st-knee-replacement-with-ar-technology-in-us/ . Accessed Sept. 16, 2021.

Save

Augmented reality puts holograms, data at fingertips of surgeons

Issue: October 2021

By Casey Tingle

Source/Disclosures

Published by:

View Issue
Source:

Augmented reality used in knee-replacement surgery in the U.S. first. Available at: https://news.hss.edu/dr-jonathan-vigdorchik-performs-1st-knee-replacement-with-ar-technology-in-us/. Accessed Sept. 16, 2021.

Disclosures:
Athwal reports being a consultant and design surgeon for and receiving royalties from Stryker for the implants and technology discussed. Vigdorchik reports being a consultant for Medacta. Witham reports being on the medical advisory board for and being an investor in Augmedics. Allen and Khazzam report no relevant financial disclosures.

ADD TOPIC TO EMAIL ALERTS

Receive an email when new articles are posted on

.

Please provide your email address to receive an email when new articles are posted on

ADDED TO EMAIL ALERTS

You’ve successfully added

to your alerts. You will receive an email when new content is published.

Click Here to Manage Email Alerts You’ve successfully added

to your alerts. You will receive an email when new content is published.

Click Here to Manage Email Alerts

We were unable to process your request. Please try again later. If you continue to have this issue please contact [email protected].

Across many orthopedic subspecialties, surgeons are experiencing a taste of the future in the OR with the use of augmented reality.

Although augmented reality (AR) can be found in the published literature starting in the late 1990s and early 2000s, George S. Athwal, MD, of Western University in London, Ontario, Canada, noted the use of AR in orthopedic surgery began as a “one off.”

George S. Athwal, MD, said augmented reality systems that are more user friendly are now available. These allow intraoperative, real-time, 3D holographic visualization of the preoperative surgical plan and imaging to assist the surgeon with implant insertion.

Source: Jonathan Tufts

“[Augmented reality] in the 1990s did not take off because it was so new and so clunky,” Athwal, who presently uses Blueprint Mixed Reality Technology (Stryker), told Orthopedics Today, adding that many more AR systems have become commercially available within the last 2 years.

“All of a sudden, the technology is becoming more adaptable because it is user friendly and it is mass produced,” Athwal said.

Expansion potential

Currently used in total hip, knee and shoulder replacement, as well as certain spine surgery procedures, sources who spoke with Orthopedics Today said the use of AR could be expanded to include other surgical procedures.

“You could make an argument to use it on everything, because any time you are adding information, you are adding value to the surgeon,” Jonathan M. Vigdorchik, MD, assistant professor of orthopedic surgery at Hospital for Special Surgery, said.

With the use of computer navigation and robotics in orthopedic surgery in the last few decades, Vigdorchik said AR is the “next step toward a small profile, easy to use, efficient, low-cost computer-aided technology.” Although orthopedic surgeons have traditionally used techniques that involved “eyeballing” implant placement, AR provides surgeons with information during each step of the procedure, according to Vigdorchik.

Jonathan M. Vigdorchik

“That is the key function of all these different technologies. When you are going to cut something, instead of closing one eye, eyeballing it, lining it up with your hand or some metal rod, you are doing it with the latest technological advances and it is giving you information at the time that you need it,” Vigdorchik said.

Intraoperative interaction

Prior to the advent of AR, surgeons were limited to the computer screen. However, now they can interact with patient CT scans and X-rays in three dimensions in real time, according to Michael Khazzam, MD.

R. Todd Allen

“Now we can take AR and interact with the plan and make the appropriate adjustments in real time as we follow the preoperative plan. But it also lets us, in real time, move the CT scan around, move the shoulder in three dimensions in positions that you typically cannot see in your operative field,” Khazzam, chief of shoulder surgery, director of clinical orthopedic research and associate professor of orthopedic surgery at the University of Texas Southwestern Medical Center, said.

R. Todd Allen, MD, PhD, chief of spine surgery and director of the spine fellowship in the department of orthopedic surgery at UC San Diego Health System, said AR provides spine surgeons with multiple views of the spine and a topographical map of the spine compared with navigation-based systems.

The ability to see more of the patient’s spine during surgery may provide surgeons with added confidence that the instrumentation is being placed accurately, according to Timothy F. Witham, MD, FAANS, FACS, professor of neurosurgery and orthopedic surgery at Johns Hopkins University School of Medicine.

Timothy F. Witham

“There are always cases where somebody puts in a screw and they did not know that they put it in the wrong place,” Witham said. “If the accuracy is improved, which I think it is with this technology, then you are more confident that the screws are being placed properly, and that benefits the patient because then they might not need a revision surgery for a screw that has been misplaced.”

Preoperative use

Although mainly used in the intraoperative setting, some AR systems may be used in the preoperative setting to plan the surgical procedure, which is a critical step that can improve surgical accuracy, Khazzam said.

“It is important to plan every case. It is important to understand in three dimensions what exactly you are trying to accomplish and what anatomy you are trying to restore with the implants you are using and AR gives you a better understanding of all those things,” Khazzam, who uses HoloLens AR glasses (Microsoft), said.

Michael Khazzam

Athwal said AR may be used preoperatively to educate patients on how the surgery will be performed. It involves two headsets. One is worn by the patient and the surgeon wears the other headset.

“You can walk through the surgery with them,” Athwal said. “It is amazing for them to see the holograms of what their shoulder looks like, what their bones look like and how the implants will be positioned.”

Similarly, AR can be used as a tool to educate fellows and residents, as well as allow surgeons to help one another in the OR from anywhere in the country, Vigdorchik noted.

“Instead of surgeons having to fly across the country or to different countries to learn how to do something, operating rooms are now being equipped with technology that the surgeon can feel like they are there with me,” Vigdorchik told Orthopedics Today.

Potential postoperative use

Athwal said AR will be beneficial postoperatively by integrating the information collected intraoperatively into patient outcomes, a feature which is not currently available.

“When I am doing the surgery, the system records what implants I put in and can visually record the position they were in,” Athwal said. “Afterward, the system will be able to record the patient’s range of motion, calculate their functional scores, input their X-rays and then, using artificial intelligence, it can coalesce all of the preoperative, intraoperative and postoperative data with pattern recognition algorithms to predict how patients will do and what potential complications they are at risk for.”

Athwal said any information recorded intraoperatively may be able to be used during postoperative patient visits.

“If I move the arm around [intraoperatively] and test the limits of motion, [the AR] will record that information and then afterward, when I am with the patient, I can put my [HoloLens] on and ask them to actively move their arm in the same positions and will be able to determine where the differences are,” Athwal said.

But, as with every new technology, AR has its drawbacks, sources who spoke with Orthopedics Today said.

“This is all in its infancy, so it is a work in progress,” Khazzam told Orthopedics Today. “I think this first generation is incredible in terms of what it is providing for us, but we are relying on CT-scan quality. We are relying that the software is generating accurate mapping of the three-dimensional structure that we are trying to recreate.”

Khazzam said the preoperative planning software available with some AR systems varies from company to company, which has been a cause for concern among orthopedic surgeons regarding overall accuracy.

“[The preoperative planning software] is relying on math, and so there have been some comparisons that the math is not always the same in how they arrive at the reconstructions. Obviously, the concern is which one is correct,” Khazzam said.

During primary reverse total shoulder arthroplasty performed using AR, the humerus hologram is shown being overlayed after implantation of the humeral component.

Source: Michael Khazzam, MD

Improvements needed

Equipment improvements, such as to the headsets, to make these lighter and more ergonomic are needed, according to Athwal.

“The other issue we had initially was OR lights. The lighting inside the OR is so bright and focused on the incision that sometimes it would wash out the hologram,” Athwal said. “It is like watching a drive-in movie in the middle of the day. It’s difficult to see because the sun is out. However, advancements in the hologram projections have substantially improved visualization.”

Some AR systems also require intraoperative CT scans to be able to use it, Witham said.

Data collected from the intraoperative scans takes time to register, which can slow the surgical process, making AR not ideal for smaller, easier cases, according to Witham. Although AR systems can save time during the surgical procedure and improve OR workflow, he said some surgeons may be hesitant to adopt AR in their practices due to the disruption that will occur with the learning curve during the initial adoption.

“A lot of surgeons are … set in their ways in terms of they have their way of doing it that works for them, and they get into a habit of doing it the same way every time,” Witham, who uses the xvision Spine System (Augmedics), told Orthopedics Today. “A lot of people do not like to change their habits or change the way they do things and one of the reasons for that is that if it disrupts your workflow and disrupts your efficiency, then you are not going to want to try it.”

Learning curve with AR

However, sources who spoke with Orthopedics Today said the learning curve for using AR is not steep, especially for any surgeons who already use computer navigation.

“If you are already a computer navigation surgeon, the learning curve [for AR] is one or two surgeries, just to figure out the nuances of that system vs. the other one,” Vigdorchik said. “If you are going from nothing to augmented reality, it is similar to computer navigation, less than 10 surgeries.”

Although any surgeon of any age with any number of years practicing can easily learn how to integrate AR into their practice, Witham said the up-and-coming generation of surgeons may be quicker adopters.

“I can take a resident who may not be able to put some of the spinal instrumentation in easily using some of the other techniques and when they get this headset on … they are good at adopting the technology and applying it,” Witham said.

However, Vigdorchik said AR needs to be proven to be “super easy, super efficient, low cost and show an outcomes difference” for surgeons who do not perform a high volume of complex orthopedic surgeries to want to adopt it.

“[Augmented reality needs to be] more readily adoptable so that the surgeon doing two or three [total joint replacements] a month can easily use it,” Vigdorchik said. “If there is a learning curve and the learning curve is 10 cases, doing two or three a month, it is going to take you 6 months to learn how to do it efficiently and you may not be willing to invest in that learning curve.”

Consider costs

The cost of AR is another consideration, sources told Orthopedics Today. However, while AR costs more to use than manual techniques, it is considerably less expensive than robotics, which may be an advantage, Witham said.

“The price point for augmented reality is competitive to a robotic system,” Witham said. “That might be an advantage, but still there is a capital cost that maybe a small hospital cannot afford or a low volume hospital may not want to invest in because it is just not worth spending that extra money for it.”

The use of AR in orthopedic surgery still needs to be vetted through research, according to Allen.

“We need to vet if [AR is] going to tease out to be a truly incremental benefit to our patients, to society, to health care,” Allen told Orthopedics Today. “That is a tall task, so to speak, and it is something we need to look into further as we advance technology to make sure that it is vetted appropriately.”

With many AR systems gaining FDA approval and commercialized in the last 12 to 24 months, Athwal said head-to-head research into performing orthopedic surgery with and without AR technology does not exist.

More research needed

Allen said research is needed to identify the incremental cost-effectiveness ratios of using AR, while Witham said there is a lack of data on accuracy rates and potential complications.

“Certainly, we will need research on efficiency and does it save in operative time and cost,” he said. “Other components will require research, as well, if we are talking about using this to remove tumors. How are the outcomes when we use it? There is still a lot of research that needs to be done.”

Athwal contends the benefits of AR are clear even without those data, which may take 2 to 4 years to be published.

“Intuitively, if you look at X-rays before you operate, and you look at patients’ CT scans before you operate, and now with AR we are going to give you all that data at the time of surgery in the form of a hologram at your fingertips, it seems logical that surgeons would find value in that information,” Athwal said.

Surgeon trial

Although AR may not be for every surgeon, Witham said it would be worthwhile for surgeons to try it through a cadaver lab or surgeon visit to see the various ways it may benefit their practice.

“I would recommend checking it out,” Witham said. “If for no other reason, it is good to know that it is out there and that it is a different way of doing things because, as the technology improves, more people will want to adopt it when they see what it has to offer, and it can make their lives easier.”

While familiarity with computer navigation would be helpful for surgeons interested in adopting AR, surgeons do not have to have any previous computer navigation experience to use AR, Allen said, but he noted it is important for surgeons to guard against being over-confident when using AR intraoperatively.

“We need to guard ourselves from being overconfident in technology as 100% accurate. There is no system currently that is 100% accurate and, in part, that may be due to multiple factors involved with surgery, with cases, with any mobility of something you placed on the patient as a reference marker,” Allen said. “We still have to use the mechanisms that we have been trained to use, which is why I do not think we are to the robotics stage yet, but this is a nice transition to reinforce what we normally do in open surgery and gives us that same tactile and haptic feedback that we would normally want to feel.”

As with all new technology, Athwal said there will be innovators, early adopters, late adopters and laggards. However, as more orthopedic surgeons put AR to use into their practices, Athwal said the technology will gain traction and will be here to stay.

“Whenever a new technology is introduced, once it gets through a critical mass … where people start to identify it, talk about it, they realize the benefit of it,” Athwal said. “I cannot see us ever going away from [AR] now because it provides so much valuable information at our fingertips.”

• References:
• Johns Hopkins performs its first augmented reality surgeries in patients. Available at: www.hopkinsmedicine.org/news/articles/johns-hopkins-performs-its-first-augmented-reality-surgeries-in-patients. Accessed Sept. 16, 2021.
• UC San Diego Health is region’s first to use augmented reality in spine surgery. Available at: https://health.ucsd.edu/news/releases/Pages/2021-04-29-uc-san-diego-health-first-in-region-to-use-augmented-reality-in-spine-surgery.aspx. Accessed Sept. 16, 2021.
• UT Southwestern orthopedic surgeon first in Texas to use AR shoulder replacement surgery. Available at: www.utsouthwestern.edu/newsroom/articles/year-2021/ar-shoulder-replacement-surgery.html. Accessed Sept. 16, 2021.

• For more information:
• R. Todd Allen, MD, PhD, can be reached at 9400 Campus Point Dr., La Jolla, CA 92037; email: [email protected].
• George S. Athwal, MD, can be reached at 268 Grosvenor St., London, Ontario, Canada N6A 3N3; email: [email protected].
• Michael Khazzam, MD, can be reached at 1801 Inwood Road, Dallas, TX 75390; email: [email protected].
• Jonathan M. Vigdorchik, MD, can be reached at 541 East 7th St., 6th Fl., New York, NY 10021; email: [email protected].
• Timothy F. Witham, MD, FAANS, FACS, can be reached at 600 N. Wolfe St., Meyer Building 7-109, Baltimore, MD 21287; email: [email protected].

Click here to read the Point/Counter to this Cover Story.