Artificial intelligence-assisted three-dimensional reconstruction in thoracic surgery: a narrative review

Introduction

Background

Before the advent of precision medicine, surgical decisions were predominantly based on evidence tailored to the average patient (1). Today, the integration of comprehensive patient-specific data and powerful computational tools has centered surgical planning around the individual (2). This shift is particularly evident in thoracic surgery, where targeted anatomical lung resection with maximal lung preservation has gained prominence. Previously, lobectomy was the standard approach for early-stage non-small cell lung cancer (NSCLC). However, recent trials, such as JCOG0802/WJOG4607L and CALGB 140503 (3,4), have shown that segmentectomy and sublobar resection can achieve comparable or superior outcomes in overall and disease-free survival, particularly for small peripheral lesions. These findings support tailoring surgical plans to lesion location rather than adhering to a one-size-fits-all approach. Despite these advances, segmentectomy remains more technically demanding, requiring precise localization of lesions and detailed identification of sublobar pulmonary structures. These challenges contribute to longer operating times (5), higher conversion rates to open surgery, and an increased risk of major complications (6). Thus, a significant challenge in thoracic surgery is the accurate localization of lesions and the identification of sublobar pulmonary structures to facilitate these more precise, lung-preserving procedures.

Traditionally, several techniques have been employed for lesion localization in thoracic surgery, including percutaneous puncture-assisted methods using metallic hooks, dyes, and contrast agents; bronchoscopic preoperative pulmonary nodule localization; and intraoperative ultrasound-guided localization. After identifying the lesion, the surgeon relies on two-dimensional computed tomography (CT) scans, mentally reconstructing them into a three-dimensional (3D) architecture to guide surgery, which is then confirmed through intraoperative findings. However, each localization method has its limitations, ranging from patient discomfort and the risk of marker displacement to technical challenges (7). Additionally, mastering chest CT interpretation at the segmental or subsegmental level can be difficult, even for experienced surgeons. Misdetection and misclassification of remote pulmonary arteries (PAs) and veins (PVs) may still occur, which can lead to serious complications such as bleeding, incorrect vessel ligation, or other potentially catastrophic consequences.

Over the past two decades, 3D CT reconstruction imaging has gained popularity in thoracic surgery for anatomical evaluation and preoperative planning. A variety of sophisticated 3D CT imaging software tools have been developed, including platforms such as Materialise Mimics Innovation Suite 23 (Materialise NV, Leuven, Belgium), Ceevra (Ceevra, Inc., San Francisco, USA), Iris (Intuitive Surgical, Sunnyvale, USA), Visible Patient (Visible Patient, Strasbourg, France), Synapse Vincent (Fujifilm Medical Co., Tokyo, Japan), and REVORAS (Ziosoft, Inc., Tokyo, Japan). These 3D reconstructions provide a more intuitive understanding of anatomy compared to traditional two-dimensional images, allowing surgeons to interact with and manipulate the images to visualize complex anatomical structures from multiple angles. Hagiwara et al. demonstrated that 3D reconstruction imaging successfully identified all anomalous PA branching patterns (8). Similarly, Nakao et al. reported that in the vast majority (90%) of segmentectomy cases, 3D reconstruction findings were consistent with intraoperative observations (9). Overall, 3D CT imaging has proven to be a valuable adjunct in thoracic surgery, with the potential to reduce the risk of complications and improve patient outcomes.

Despite the numerous advancements in 3D CT imaging, several limitations hinder its broader application (10). One major challenge is that the manual reconstruction process is time-consuming and requires specialized expertise, as skilled professionals must annotate each layer individually. Another significant limitation is that 3D CT imaging generates static images, which cannot account for deformable changes of lungs that occur during surgery, such as deflation or surgical manipulation. This inability to reflect real-time deformations reduces its utility in dynamic surgical environments. Additionally, these systems rely heavily on contrast-enhanced CT scans, which may not be suitable for all patients, particularly those with contrast allergies or impaired renal function. Addressing these limitations is crucial to enhancing the efficiency and reliability of 3D CT reconstruction imaging in thoracic surgery.

Rationale and knowledge gap

In recent years, artificial intelligence (AI) technology has advanced rapidly. AI-driven algorithms, particularly deep learning techniques like convolutional neural networks (CNNs), have demonstrated remarkable capabilities in analyzing medical images (11). CNNs can automatically identify patterns through multiple layers of filters, excelling in tasks such as detecting and classifying patterns within images and reconstructing images with minimal human input—key functions in thoracic surgery. By integrating AI algorithms into 3D reconstruction imaging, AI-assisted 3D reconstruction (AI-3DR) technology has the potential to accurately detect anatomical structures like blood vessels, bronchi, and tumors, reducing the workload for surgeons and radiologists and minimizing the risk of misidentifying critical structures. AI-3DR exemplifies the principles of precision medicine by enabling the extraction of comprehensive imaging features (e.g., intensity, shape, volume, texture of the region of interest) beyond the limited key aspects typically recognized by human clinicians (2). Additionally, this technology has the potential to augment healthcare providers by informing and refining surgical decisions, leading to more individualized treatment plans.

However, with this growing field comes the need for a thorough understanding of both the applications and limitations of AI-3DR. Questions remain about its clinical integration, particularly in terms of its reconstruction accuracy and its handling of anatomical variations during surgery. The purpose of this paper is to review the advancements in AI-3DR systems and assess how this novel technology can contribute to improved surgical outcomes in thoracic surgery.

Objective

This review will critically evaluate the status of AI-assisted 3D CT reconstruction and its utilization in thoracic surgery, with a particular emphasis on its ability to streamline preoperative planning and facilitate more precise surgical interventions. We present this article in accordance with the Narrative Review reporting checklist (available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-40/rc).

Methods

A comprehensive literature search of PubMed, Embase, and Cochrane Library databases was conducted (with the study selection flow-chart depicted in Figure 1). Keywords related to the use of (artificial intelligence) AND (3D reconstruction) AND (thoracic surgery) were searched on September 30, 2024 (Table 1). Additional articles identified through cross-referencing were also assessed. Published original articles that met the following criteria were included: (I) the study utilized an AI algorithm specifically for 3D reconstruction, and (II) the AI-3DR technology was applied in a manner relevant to thoracic surgical procedures, such as segmentectomy. Studies were excluded if the 3D reconstructions were primarily performed by specialized radiological technicians who manually contoured all anatomical structures rather than being generated by an AI algorithm.

Figure 1 Outline of literature search process. Literature search with keywords ‘Artificial intelligence’, ‘3D reconstruction’, and ‘thoracic surgery’ was performed on September 30, 2024. The selection criteria included studies focusing on the use of AI-3DR in thoracic surgery. Additional articles identified through cross-referencing were also reviewed. 3D, three-dimensional; AI, artificial intelligence; AI-3DR, artificial intelligence-assisted three-dimensional reconstruction.

Results

Development of AI-3DR

The AI-3DR technology was built using a CNN, a type of deep learning model designed to process and analyze complex medical images (12). These models were trained on chest CT scans from a large cohort of patients, annotated by experienced thoracic surgeons (Figure S1). The surgeons meticulously labeled key anatomical structures in the scans, such as lobes, bronchi, and blood vessels, making their annotations the reference standard for training the AI. The system uses multiple layers of the neural network to analyze and classify imaging features, enabling it to identify and segment structures accurately. These segmented structures are then used to create detailed 3D models. Its reconstruction accuracy was evaluated using a metric called the Dice similarity coefficient (DSC), which measures how closely the AI’s results match human annotations. The DSC improved as more cases were added, reaching a reliable level at 400 cases, after which additional data only provided marginal improvements to performance (13). For comparison, a well-known study by Liu used 80,000 skin lesion images to train an AI system for diagnosing skin conditions, achieving high accuracy (14). In Liu’s study, the reference standard was established by aggregating independent opinions from multiple dermatologists. Similarly, in the development of AI-3DR, the involvement of senior thoracic surgeons ensured the system’s annotations were clinically robust and aligned with expert standards.

Reconstruction procedure

The manual 3D reconstruction process requires expertise in software use and significant time to delineate areas of interest. These challenges are addressed by the AI-3DR technology, which operates automatically and only requires the surgeon to upload CT images for image generation. Wang et al. reported that AI-3DR reduced reconstruction time from 30 minutes (manual) to 5 minutes, while Li et al. found similar results (6.8 vs. 23 minutes) (7,13). These findings highlight the system’s superior efficiency in labor and time. Notably, in Sadeghi’s study, where real-time visualization during surgery was enabled, the average rendering time for generating a new image was just 15 seconds (15). This highlights that prompt, dynamic reconstruction is achievable with AI, a feature that would be impossible with manual methods.

Sublobar localization of the lesion

Accurate sublobar localization of pulmonary lesions is crucial in determining the extent of surgery. Zheng et al. found that AI-3DR achieved 100% accuracy in predicting the affected lung segment (mean tumor size: 1.3 cm), outperforming radiologists (94.4%), especially for tumors at segment junctions, which are challenging to locate with 2D CT (16) (Table 2). Bakhuis et al. conducted a survey with surgeons and radiologists regarding congenital lung abnormalities, asking them to identify the lesion’s location (18). Using 2D CT, the surgeon and radiologist agreed on the segment in only 20% of cases, indicating uncertainty. However, after AI-3DR generated images were introduced, the consensus increased to 80%, demonstrating improved agreement. Nevertheless, the clinical utility of this study is limited by the small sample size and the absence of pathological confirmation.

Detection accuracy of pulmonary structures

Detection, the outlining of pulmonary structures, is the first crucial step in reconstruction. The detection accuracy of these structures is calculated by dividing the number of AI-detected structures by the number observed during surgery. AI-3DR demonstrated excellent detection accuracy for pulmonary structures. Zheng et al. reported 92.1% accuracy for lobar and segmental vessels and bronchi, slightly higher than 89.5% achieved with manual systems like MIMICS. Accuracy was highest for bronchi (100%), followed by arteries (94.7%) and veins (92.1%). In cases where contrast was not used for the original CT (16), the overall detection accuracy decreased but remained higher with AI-3DR (85%) than with manual reconstruction systems (80%). The most common reason for detection errors occurred with blood vessels smaller than 2 mm in diameter (16), as the small size makes them challenging to detect. Li et al. suggested that increasing the number of training samples could improve the identification of vascular trunks and arteriovenous boundaries, though the issue of misidentifying small vessels remains (13).

Classification accuracy of pulmonary structures

Classification differs from detection in that it involves categorizing detected structures into three groups: artery, vein, or bronchi. Classification accuracy is defined as the number of correctly classified structures divided by the number of detected structures. The literature generally shows high classification accuracy for the AI-3DR technology, though there are conflicting results regarding its superiority to manual reconstruction systems. In a prospective study of 139 patients, Li et al. reported that AI-3DR achieved classification accuracies of 97%, 99%, and 98% for segmental bronchi, PAs, and veins, respectively—higher than the 96%, 91%, and 92% achieved using manual reconstruction systems (13). Similarly, Suzuki et al. demonstrated that deep learning algorithms significantly outperformed traditional methods, with classification accuracies of 97% for segmental arteries and 94.4% for veins, compared to 67% and 77% using filtered back projection, and 77% and 80% using hybrid iterative reconstruction (17).

Conversely, earlier studies by Zheng et al. found that classification accuracy of AI-3DR was lower for segmental arteries (89.5%) and veins (86.8%) compared to manual reconstruction systems (94.7% and 92.1%) (16). Chen’s study further confirmed this, showing an overall classification accuracy of 80% for AI-3DR, compared to 85% for manual reconstruction systems (19). However, the authors concluded that misclassification was less concerning than misdetection, as the former could be corrected intraoperatively, whereas the latter could lead to serious complications such as bleeding. Overall, the AI-3DR technology demonstrated at least comparable accuracy in classifying pulmonary structures.

The most commonly misclassified structures in the AI-3DR technology were the segmental veins in the right upper lobe (V1, V2, V3), which were difficult to distinguish from the proximal superior vena cava due to their similar CT values (17). In Chen et al.’s study, the most frequently misclassified structures were V6 in the left lower lobe, A3 in the right upper lobe, and A8 in the right lower lobe, likely due to anatomical variations in the test patients that were not present in the training cohort (19). A more recent study by Li et al. demonstrated near-perfect classification accuracy for all segmental structures, likely due to the larger training dataset and advancements in technology (13).

Safety of the AI-3DR

Despite its accuracy, the safety of the use of AI-3DR must be evaluated to ensure reliable surgical outcomes. In Zheng’s study of 90 patients scheduled for minimally invasive segmentectomy with preoperative AI-3DR generated images (16), one case (1.1%) was converted to open surgery due to severe adhesions, and four cases (4.4%) were converted to lobectomy due to the need for adequate oncologic margins. None of these conversions were attributed to inaccuracies in the reconstructed anatomy. Importantly, no patient required reoperation or readmission. In Chen’s retrospective analysis of 20 segmentectomy cases, three (15%) errors in the AI-reconstructed images were deemed potential surgical risks, as they could lead to incorrect vessel ligation and possible bleeding during segmentectomy (19). On the other hand, Li et al. reported that the AI-3DR technology reduced operating time by an average of 24.5 minutes for lobectomy and 20 minutes for segmentectomy, without significantly increasing intraoperative blood loss (13).

Efficacy of the AI-3DR in surgical planning and execution

In addition to its safety profile, the AI-3DR technology has been shown to enhance surgical planning and improve precision during surgery. In Bakhuis’ study, four cases initially planned for lobectomy based on 2D CT images were reassessed using AI-3DR generated images, and three cases (75%) were downgraded to segmentectomy (18). This resulted in an estimated lung volume preservation of 257 mL due to the change in surgical approach. Furthermore, the surgeon indicated a greater likelihood of consulting the AI-3DR generated images over 2D CT images during surgery. Kudo’s study demonstrated that the solid-part volume of lung cancer <2 cm, as calculated by the AI-3DR technology, was a better predictor of recurrence risk than the commonly used consolidation-to-tumor ratio on 2D CT images (20). This suggests that AI-3DR can aid in determining whether to perform lobectomy or segmentectomy.

Zheng et al. compared the preoperative surgical plans based on AI-3DR generated images with the actual procedures carried out in surgery, finding a consistency rate of 68.4% (16). The discrepancies were mainly due to factors such as fissure malformations, widespread adhesions, and difficult-to-dissect lymph nodes, which prevented the surgery from proceeding as planned. These finding highlights that while AI-3DR generated images are valuable for preoperative planning, surgeons must remain flexible and adjust their approach based on intraoperative findings. Additionally, Sadeghi et al. achieved the first AI-assisted augmented reality robotic lung surgery by superimposing reconstructed images onto the surgeon’s view in real-time, visible through the console (15). The reconstruction model successfully guided the anterior hilar, fissure, and posterior hilar views and adapted dynamically to lung manipulation. Notably, the AI-3DR technology enabled real-time rendering of a deformable lung model with just 15 seconds of rendering time, allowing for interactive adjustments that mirrored anatomical changes caused by surgical manipulation.

Enhancing the generalizability and robustness of the AI-3DR

To enhance the generalizability and robustness of the AI-3DR, the training dataset should be developed and processed using diverse patient data (21). This includes representing a broad demographic spectrum with variations in age, gender, and race. Additionally, variability in 2D CT imaging settings, such as contrast usage, slice thickness, and imaging protocols, should be incorporated to ensure comprehensive applicability. Furthermore, the AI-3DR technology could benefit from integrating supplementary data, such as patient demographics and clinical history, into the algorithm. For instance, Liu et al. demonstrated the value of including patient histories of autoimmune diseases for accurate AI-assisted diagnosis of skin conditions (14). A similar approach could be applied to lung AI-3DR, incorporating histories of asthma or chronic obstructive pulmonary disease (COPD) to refine decision-making (22). This strategy would further individualize the system, enhancing its precision and applicability across diverse clinical scenarios.

Limitations

Our review has several limitations. Firstly, the studies did not comment on image quality, which can sometimes be distorted or corrupted, potentially affecting the system’s actual performance depending on the quality of images provided. Additionally, the studies did not address the explainability of the classification of the AI-3DR, which is crucial information for thoracic surgeons to gain confidence and become more receptive to the technology. Lastly, AI-3DR technology is still in the experimental phase, with many barriers to clinical use, including high initial costs, especially in resource-limited settings. However, widespread adoption could potentially make the technology more cost-effective than manual reconstruction, saving time and resources such as the expertise of experienced surgeons or radiologists. Additionally, AI-3DR technology could help reduce health disparities by eliminating the need for highly specialized expertise. In resource-constrained environments, where access to radiologists or surgeons skilled in manual reconstruction may be limited, AI-3DR could provide a more accessible solution. Since many surgeons are already familiar with interpreting images derived from manual reconstruction, the transition to AI-generated outputs would be seamless.

Conclusions

The AI-3DR technology has seen significant advancements in recent years. In addition to being fully automated and time-efficient, it has demonstrated comparable performance in detecting thoracic lesions and classifying pulmonary structures. Studies have shown its potential to aid in surgical planning and intraoperative decision-making. While improvements are needed in the system’s generalizability and robustness, it is clear that surgeons can benefit from familiarizing themselves with this revolutionary technology, which may enhance the precision and outcomes of future surgeries.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the Narrative Review reporting checklist. Available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-40/rc

Peer Review File: Available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-40/prf

Funding: This research was fully supported by the Department of Surgery of the University of Alabama at Birmingham.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-40/coif). B.W. serves as an unpaid editorial board member of Current Challenges in Thoracic Surgery from September 2024 to August 2026. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Johnson KB, Wei WQ, Weeraratne D, et al. Precision Medicine, AI, and the Future of Personalized Health Care. Clin Transl Sci 2021;14:86-93. [Crossref] [PubMed]
2. Ibrahim A, Primakov S, Beuque M, et al. Radiomics for precision medicine: Current challenges, future prospects, and the proposal of a new framework. Methods 2021;188:20-9. [Crossref] [PubMed]
3. Saji H, Okada M, Tsuboi M, et al. Segmentectomy versus lobectomy in small-sized peripheral non-small-cell lung cancer (JCOG0802/WJOG4607L): a multicentre, open-label, phase 3, randomised, controlled, non-inferiority trial. Lancet 2022;399:1607-17. [Crossref] [PubMed]
4. Altorki N, Wang X, Kozono D, et al. Lobar or Sublobar Resection for Peripheral Stage IA Non-Small-Cell Lung Cancer. N Engl J Med 2023;388:489-98. [Crossref] [PubMed]
5. Tane S, Kitamura Y, Kimura K, et al. Segmentectomy versus lobectomy for inner-located small-sized early non-small-cell lung cancer. Interact Cardiovasc Thorac Surg 2022;35:ivac218. [Crossref] [PubMed]
6. Decaluwe H, Petersen RH, Hansen H, et al. Major intraoperative complications during video-assisted thoracoscopic anatomical lung resections: an intention-to-treat analysis. Eur J Cardiothorac Surg 2015;48:588-98; discussion 599. [Crossref] [PubMed]
7. Wang Y, Chen E. Advances in the localization of pulmonary nodules: a comprehensive review. J Cardiothorac Surg 2024;19:396. [Crossref] [PubMed]
8. Hagiwara M, Shimada Y, Kato Y, et al. High-quality 3-dimensional image simulation for pulmonary lobectomy and segmentectomy: results of preoperative assessment of pulmonary vessels and short-term surgical outcomes in consecutive patients undergoing video-assisted thoracic surgery†. Eur J Cardiothorac Surg 2014;46:e120-6. [Crossref] [PubMed]
9. Nakao M, Omura K, Hashimoto K, et al. Novel three-dimensional image simulation for lung segmentectomy developed with surgeons’ perspective. Gen Thorac Cardiovasc Surg 2021;69:1360-5. [Crossref] [PubMed]
10. Chen-Yoshikawa TF. Evolution of Three-Dimensional Computed Tomography Imaging in Thoracic Surgery. Cancers (Basel) 2024;16:2161. [Crossref] [PubMed]
11. Chan HP, Samala RK, Hadjiiski LM, et al. Deep Learning in Medical Image Analysis. Adv Exp Med Biol 2020;1213:3-21. [Crossref] [PubMed]
12. Rajkomar A, Dean J, Kohane I. Machine Learning in Medicine. N Engl J Med 2019;380:1347-58. [Crossref] [PubMed]
13. Li X, Zhang S, Luo X, et al. Accuracy and efficiency of an artificial intelligence-based pulmonary broncho-vascular three-dimensional reconstruction system supporting thoracic surgery: retrospective and prospective validation study. EBioMedicine 2023;87:104422. [Crossref] [PubMed]
14. Liu Y, Jain A, Eng C, et al. A deep learning system for differential diagnosis of skin diseases. Nat Med 2020;26:900-8. [Crossref] [PubMed]
15. Sadeghi AH, Mank Q, Tuzcu AS, et al. Artificial intelligence-assisted augmented reality robotic lung surgery: Navigating the future of thoracic surgery. JTCVS Tech 2024;26:121-5. [Crossref] [PubMed]
16. Zheng Z, Ren M, Li B, et al. Application Value of Artificial Intelligence-assisted Three-dimensional Reconstruction in Planning Thoracoscopic Segmentectomy. Zhongguo Fei Ai Za Zhi 2023;26:515-22. [PubMed]
17. Suzuki C, Nakano J, Matsubara K. Effect of Automatic Extraction Accuracy by Different Image Reconstruction Methods Using a Three-dimensional Image Analysis System for Pulmonary Segmentectomy Preoperative CT Angiography. Nihon Hoshasen Gijutsu Gakkai Zasshi 2021;77:1309-16. [Crossref] [PubMed]
18. Bakhuis W, Kersten CM, Sadeghi AH, et al. Preoperative visualization of congenital lung abnormalities: hybridizing artificial intelligence and virtual reality. Eur J Cardiothorac Surg 2022;63:ezad014. [Crossref] [PubMed]
19. Chen X, Wang Z, Qi Q, et al. A fully automated noncontrast CT 3-D reconstruction algorithm enabled accurate anatomical demonstration for lung segmentectomy. Thorac Cancer 2022;13:795-803. [Crossref] [PubMed]
20. Kudo Y, Shimada Y, Matsubayashi J, et al. Artificial intelligence analysis of three-dimensional imaging data derives factors associated with postoperative recurrence in patients with radiologically solid-predominant small-sized lung cancers. Eur J Cardiothorac Surg 2022;61:751-60. [Crossref] [PubMed]
21. Bajwa J, Munir U, Nori A, et al. Artificial intelligence in healthcare: transforming the practice of medicine. Future Healthc J 2021;8:e188-94. [Crossref] [PubMed]
22. Tanabe N, Nakagawa H, Sakao S, et al. Lung imaging in COPD and asthma. Respir Investig 2024;62:995-1005. [Crossref] [PubMed]