3 | The AI-Enabled Imaging Revolution in Surgery

How machine learning is changing the way surgeons interact with imaging tools for diagnosis and management in surgical care

How AI Changes the way Surgeons Interact with Images Imaging is important in all aspects of surgical care from diagnosis to management to monitoring. AI can reshape the way that orthopedic surgeons interact with imaging by allowing an unbiased view of images, detect subtleties that are invisible to the human eye, and recognize patterns over thousands of images that would be difficult to contextualize without years of experience.

AI will change orthopedics in 3 main ways: decrease the need for advanced imaging like CT scans and MRIs, help with the prediction of diagnoses and prognosis, and finally, provide real-time intra-operative guidance.

1. Decreased Need for Advanced Imaging: 2-Dimensional Imaging (X-Rays, Ultrasound) is often ordered before 3-Dimensional imaging (CT scans, MRI scans). Machine learning has provided an ability to recreate 3-D images from 2-D images to answer the questions that surgeons are investigating when ordering 3D imaging, such as predicting canal compromise in spinal trauma or the evaluation of the ankle joint in a spiral distal tibia fracture. If 3D images can be replaced with models applied to 2D imaging, there will be a significant reduction in time and cost, as well as a decrease in the radiation given to patients. Finally, it would allow for a better understanding of a patient’s injury in places where 3D imaging capabilities are not present (such as rural areas of America or the third world). Read more about how this is being done in ultrasound here, and how it is being done with bony anatomy at turning X-Rays to CT scans here.

2. Improve Diagnosis and Prognosis Predictions: This is likely the area that has been most publicized and utilized with machine learning in orthopedics. Ever since an algorithm was shown to be superior to a dermatologist in diagnosing skin cancer, thousands of papers have been published on the use of machine learning for improving diagnosis prediction from imaging whether it is predicting COVID from a CT scan, a MI from an ECG, or a cobb angle from a spine X-Ray. It is well-publicized, so I will not discuss it much here, but I think the emerging innovation in this space is aiding in diagnosis in pathology found in surgery. Traditionally, during cancer surgery, a surgeon collects a small amount of the tissue from a mass and sends it to a pathologist for diagnosis – a process which can take almost an hour while the patient lays open on the table. Several studies have shown that ML algorithms can help automate this process in a matter of seconds. Read more here.

3. Intra-Operative Guidance: this may be the sexiest application of AI in orthopedics, but it likely has the longest road to go before becoming useful. Machine learning can detect anatomical structures from open wounds and through arthroscopic cameras helping young surgeons avoid important structures, recognize the different steps of a surgical procedure, and ultimately has been shown to aid in the thought process of a surgeon. In orthopedic fractures, a bone is often broken into several pieces, and mapping it together to its anatomical normal position is often challenging and time-consuming. A paper last year showed that AI techniques can take intra-operative images of a bone broken in several pieces, localize and segment each bone fragment by analyzing its surface anatomy, and put it together in a way that recreates normal anatomy to help the surgeon piece together a 3D puzzle. This has the potential to help young surgeons save hours of operating room time by helping them visualize normal anatomy. Read more about the way machine learning is being used in the operating room today here, as well as a new lab at Mass General re-thinking the operating room through AI here.

I am excited about each of these ways that AI will help how surgeons interact with imaging, which plays a key part in our current treatment model. There is a lot of hype in this space currently with only a few companies having FDA approval: the next year will show us what products are the most promising.

Interesting Links:

• Starting January 1st, hospitals must be transparent with the prices they charge patients for all their procedures. This has a lot of interesting implications in health policy. Learn more here, and you can compare prices here.

• Smart knee implants may allow the detection of aseptic loosening or infection before aggressive bone loss occurs and symptoms appear. Read more here.

• Using 3D Holograms (re: J.A.R.V.I.S. from Iron Man) to Analyze Anatomy During Surgery – a new term “Holomedicine” emerges, learn more here.

Give Back

I have been fundraising for a clinic in Guatemala that provides free care for their community. Learn more about my history with the clinic and opportunities to help here.