Institutional learning curve for operative chest injury management as defined by outcomes

Introduction

Patients with severe chest wall injuries have been shown to have significant mortality and morbidity, often resulting from pulmonary complications (1-3). In particular, as many as 40% of patients with flail chest injuries develop pneumonia (PNA) as a consequence of their injury (4). While historically management has primarily consisted of non-operative supportive treatments such as analgesia, mechanical ventilation, and pulmonary toilet (2), recent evidence has supported the use of operative rib stabilization to improve a variety of patient outcomes (3-5).

Given the increasing adoption of surgical techniques to manage chest wall injuries, including surgical stabilization of rib fractures (SSRF) and surgical stabilization of sternal fractures (SSSF), the ability to stably grow and mature operative programs has become a topic of interest (6,7). Three recent analyses have focused on SSRF education by measuring operating time as a proxy for individual surgeon learning (8-10). Such approximations can be useful in evaluating trainee progress (11), and attempts to use composite operative times have been used as surrogate measures for institutional performance (9). However, while operative time can be useful as an individually controlled benchmark, numerous elements ranging from equipment to work flow can confound surgical time. Additionally, operative time does not adequately reflect the most important element of clinical care: patient-centered outcomes. As such, programmatic optimization or institutional learning may be better evaluated by incorporating patient outcome metrics. Currently, there is a paucity of literature evaluating institutional learning curves for operative chest wall stabilization using relevant clinical outcomes.

To address this gap, we sought to determine whether clinical outcomes can accurately reflect trends in institutional learning during the adoption of an operative chest wall injury program. By analyzing patient outcomes over time, we aimed to evaluate the trajectory of institutional learning in comparison to current outcome-based evidence in rib plating (12). We hypothesized that clinical outcomes, such as postoperative pulmonary complications, readmissions, and ventilator-free days (VFDs), can be used to assess institutional progress and establish institutional learning curves. We present this article in accordance with the STROBE reporting checklist (available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-39/rc).

Methods

Data collection and study population

A formal comprehensive chest wall injury program was developed to incorporate operative management of severe chest wall injuries in our health care system in 2017. A prospectively kept registry of all operative and non-operative patients was established as a part of ongoing performance improvement efforts. This registry has been previously reviewed for early outcomes and education analysis (9,13), and expanded to include additional relevant outcomes. Data was collected over 7 years from the initiation of the operative chest injury program.

Operative patients included those undergoing SSRF and SSSF, respectively. Operative indications included both flail chest and flail segments, severe displacement/chest deformity (noted either radiographically or clinically), reduction in disability, and failure of anesthesia. Contraindications to surgery included active infections, hemodynamic instability, cervical spinal cord injury with paralysis, or other critical illness precluding operative intervention. All operations were performed by trauma and acute care surgeons, whose experience and individual analyses have been studied in prior investigations (9,10). All potentially operative patients were evaluated and agreed upon as operative candidates prospectively between the trauma surgeon, the chest wall operative team, and intensive care team to minimize confounding and selection bias. Additionally, all patients received perioperative analgesia treatment as per institutional protocol which includes both pharmacological (such as opioids, non-steroidal analgesics, muscle relaxers, gabapentin, and topical therapies) as well as non-pharmacologic (positional changes, physical therapy, thermotherapy, etc.) interventions. During the period of study, the institutional practice evolved from routine open thoracotomy to video-assisted thoracoscopic surgery (VATS) for the clearance of hemothorax. Patient characteristics collected included age, sex, body mass index (BMI), respiratory comorbidities [including smoking status, chronic obstructive pulmonary disease (COPD), and asthma], injury characteristics, and operative characteristics. Patient outcomes collected included hospital course, post-operative pulmonary complications, readmissions, and 28-day VFDs. Pulmonary complications were defined as PNA according to Centers for Disease Control criteria, acute respiratory distress syndrome (ARDS), pulmonary embolism (PE), unplanned intubation, or persistent respiratory failure. Persistent respiratory failure was defined as failure to liberate from mechanical respiratory support for non-neurological reasons following SSRF resulting in tracheostomy. Readmission was defined as an unexpected hospital encounter within 30 days of discharge for pulmonary reasons or complications related to chest wall stabilization (14), with or without readmission. Twenty-eight-day VFDs were calculated according to prior literature, and if a patient had a tracheostomy, VFD was set to zero (12,15).

Cumulative summation (CUSUM) analysis, learning curves, and statistical analysis

CUSUM modeling is used to construct learning curves for pulmonary complications, hospital encounters, and VFDs. The CUSUM score (Ct) is a serially calculated value where Ct = Ct−1 + X1 − X0. In this context, Ct−1 is the score from the previous iteration. X1 is whether the current iteration is a success (0 points) or misstep (1 point), and X0 is the expected misstep incidence rate. A misstep was defined as any pulmonary complication, readmission, or VFDs less than 28 days (12).

Learning curves were calculated to anticipated incidence rates of 5%, 10%, 20%, and 25% to aid in calibration. For example, a misstep incidence rate of 10% (indicating an anticipated 90% success rate) would result in each success decreasing the CUSUM score by 0.1, while each misstep would increase the score by 0.9. Graphically, a series of successes would be reflected in a negative slope, while a series of missteps would be reflected in a positive slope. An expected learning curve would have an initial positive slope followed by an inflection point, demonstrating learning. Thus, both descriptive statistics and slope analysis were performed. Regression analyses of the outcomes were also performed with significance set at 0.05 and were performed using JMP Pro v16 (SAS Institute, Cary, NC, USA).

Ethical consideration

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the University of Cincinnati Institutional Review Board, Study #2017-2673 and #2023-0385, and individual consent for this retrospective analysis was waived.

Results

Over the course of 7 years, 168 patients underwent surgical chest wall stabilization. Of these, 122 (72.6%) were male. In patients undergoing chest stabilization, median Injury Severity Score (ISS) was 24 [interquartile range (IQR), 16–34] with a median Blunt Pulmonary Contusion 18 (BPC-18) score of 4 (IQR, 2–6). These injury characteristics did not change substantially over our period of study. Operations were performed at a median of 3 days following injury, with on mean of 3.5 rib levels repaired and a 0.5 plate-to-fracture ratio. Five patients (3.0%) underwent sternal stabilization along with SSRF. Overall, 33.9% of patients were current smokers at the time of their injury and 23.2% had COPD or another respiratory disorder (Table 1).

The pulmonary complication cohort included 32 patients (19%) experiencing one or more complications and are detailed in Table 1. Of these patients, 9.4% were current smokers at the time of injury and 25% had a history of COPD or another respiratory disorder. The most common pulmonary complications encountered included PNA (8.3%), unplanned intubation (8.9%), and persistent respiratory failure resulting in tracheostomy (10.1%). The readmission cohort included 14 patients (8.3%) who are also detailed in Table 1. Of these patients, 50% were current smokers, and 28.6% had a history of COPD or another respiratory disorder. The two cohorts were not exclusive and included overlap (i.e., a patient with a pulmonary complication may also have had a readmission for that complication) with three patients with both pulmonary complications and readmission.

Pulmonary complications and readmissions

Pulmonary complication CUSUM curves were plotted and slopes were calculated in increments of five cases (Figure 1). Negative slopes initially occurred around operation 16–20 for all four incidence rate curves, indicating consistent successes. By the end of the series, the 5% and 10% curves had an overall positive slope. By contrast, the 20% and 25% curves had negative slopes generally over the course of the series. None of the curves appear to correlate with the expected shape of a learning curve, with the 20% incidence rate generally following the X-axis.

Figure 1 Pulmonary complications CUSUM curves: CUSUM scores for pulmonary complications plotted across consecutive chest-injury stabilization operations. CUSUM curves are calibrated at anticipated pulmonary complication incidence rates of 5%, 10%, 20%, and 25%. CUSUM, cumulative summation.

Readmission CUSUM curves were similarly plotted, and interval slopes were calculated (Figure 2). The readmission CUSUM curves all demonstrated an initial negative slope indicating a string of early successes until case 24, after which they differentiated along incidence rates. Readmission curves plotted at 10%, 20%, and 25% incidence all developed an overall downward slope beyond case 64. The CUSUM curve at the 10% incidence rate similarly demonstrated an overall flat appearance along the X-axis, without clear demonstration of learning. Multivariable regression analysis was performed to analyze the impact of study year with previously identified covariates (12), and no significant time-oriented pulmonary complications change was identified (Table 2).

Figure 2 Thirty-day readmissions CUSUM curves: CUSUM scores for readmissions plotted across consecutive chest-injury stabilization operations. CUSUM curves are calibrated at anticipated readmission incidence rates of 5%, 10%, 20%, and 25%. CUSUM, cumulative summation.

28-day VFDs

Over the series, 60% of patients had zero VFDs. CUSUM analysis for VFDs showed consistent morphological changes on all incidence curves: an initial series of missteps with several improvements around cases 21–25 and then more distinctly after operation 86 (Figure 3). Morphologically the more sustained flattening of slopes occurred after operation 86 and resulted in negative slopes for the 20% and 25% incidence curves, thus indicating sustained improvement beyond those incidences. The 5% and 10% incidence curves briefly reached negative slopes by cases 91 to 95, respectively, before continuing overall positive deflections until the end of the series, indicating that these may not be the ideal incidence curves to demonstrate learning. The 10% incidence curves for pulmonary complications, readmissions, and VFDs are overlaid for comparison in Figure 4.

Figure 3 VFDs CUSUM curves: CUSUM scores for VFDs plotted across consecutive chest-injury stabilization operations. CUSUM curves are calibrated at anticipated VFD incidence rates of 5%, 10%, 20%, and 25%. CUSUM, cumulative summation; VFD, ventilator-free day.

Figure 4 CUSUM curves for 10% incidences of pulmonary complications and readmissions and VFDs: institutional learning curves for serial incidences of complications following consecutive chest-injury stabilization operations. Pulmonary complications, readmissions, and VFDs are all presented at a 10% incidence rate. CUSUM, cumulative summation; VFD, ventilator-free day.

Discussion

The use of SSRF and SSSF continues to become more widely available as a tool for the comprehensive management of the injured patient. In our study, we present an ongoing measurement of the optimization of an operative chest injury program. To the best of our knowledge, this is the first investigation to use patient-centered metrics to derive institutional learning measurements. By using patient-centered outcomes as novel milestones for “success” and “failure”, we offer insight into expectations of a new and developing chest wall injury program. Over 7 years, we show calibration curves that yield consistent rates of post-operative pulmonary complications, unplanned readmissions, and intensive care unit (ICU) pulmonary metrics. While pulmonary complications and readmissions do not appear to show iteration-related improvement, VFDs appear to have a definitive improvement between the 80th and 90th operation, thus indicating that VFDs may be a good variable to measure learning in this setting.

Previous evidence about learning curves and optimizations in SSRF and SSSF focused on operative time and assess early experience of novice surgeons (9,10). While there are significant limitations to operative time as a surrogate for institutional improvement, within that methodology institutional improvement was shown to occur by the 20th operation (9). A closer analysis of that same data shows a second inflection point and further institutional optimization in the 50–80 procedure range. This mirrors the observed learning curves in VFD, where the slope of all curves (but particularly the 20% incidence rate) appears to flatten around the 80th operation. A regression analysis of our VFD data appears to re-demonstrate the same reduction in odds ratios as previously identified (12). While there are no single identifiable factors to explain the change, this improvement over time may reflect advancement in respiratory care as well as the maturation of our process to identify and select patients needing surgical chest wall stabilization early on in their hospitalization course.

By contrast, the absence of a clear change in slope to the CUSUM curve, particularly a slope that changes from positive to negative, for either pulmonary complications or readmissions may suggest that both complications are at steady states. Therefore, the finding of a flat CUSUM curve, or a curve with an overall slope of zero, suggests the tested incidence rate is the local incidence rate and that change or optimization is not happening. For example, readmissions have a zero slope at an incidence of 10% and pulmonary complications at 20%. Past literature identifying a 10.3% rate of readmission within 90 days following rib injury (without SSRF) appears to align with our findings (14). The observed steady-state readmission and pulmonary complication rates may suggest such metrics are not “learnable”. A variety of factors may influence both readmission and pulmonary complications, and this analysis only suggests that over the study period, further optimizations to improve these rates were not identified.

The importance of identifying a learning curve for any medical treatment or procedure lies in how that information may be used. For a novice surgeon, knowing an expected series of metrics may help track growth and learning. For an educator, knowing the expected course of a student may help tailor mentorship. An operative program director or administrator may want to understand expectations for budgeting, growth, and performance evaluation. While the results presented here may have certain limitations as detailed below, they represent the further movement towards understanding outcomes and expectations in an expanding operative discipline such as SSRF.

Our study has several limitations. First, as a single-institutional retrospective review, local factors such as patient population and institutional variation may affect reproducibility. However, we feel that being able to evaluate a program from inception through a maturing process provides valuable qualitative if not quantitative insight. Secondly, as a time-series analysis, there may be institutional changes that have not been accounted for that may be impacting results, though the overall consistent surgeon complement, consistency in training (10), and minimal practice pattern variation may balance this out. Thirdly, given that most patients experienced polytrauma beyond only rib fractures, our outcome of readmission may have been impacted by other factors apart from only rib injuries. As isolated thoracic injury is rare, we feel that confounding from these other injuries is part of regular practice, unavoidable and equally related to patient selection. Lastly, the choice of patient outcome metric selection was not chosen for prospective analysis. As a result, we specifically incorporated a controversial metric like VFDs (16) to align our novel analysis with those observed in a mature operative rib program (12). The strict 28-day interpretation of VFDs, defining success as avoiding mechanical respiratory support, may not be an ideal goal for chest injury management.

Conclusions

The use of CUSUM analysis to assess learning has been well documented in a variety of surgical domains (17,18). Effective learning curve analyses require measurable benchmarks that can accurately reflect competency. Results from our study suggest that 28-day VFDs may be a reasonable metric to assess institutional learning for surgical stabilization of chest wall injuries. Future multi-institutional studies, possibly utilizing large chest-injury specific prospective databases such as the Chest Injury International Database, may illuminate other benchmarks for programmatic improvement.

Acknowledgments

The authors would like to acknowledge the assistance of Drs. Mary Stuever, D Millar, and Aaron Seitz in addition to Mr. Christopher Dawes for their respective roles in guiding this study. Research was presented at the Academic Surgical Congress, Washington D.C., February 7, 2024.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-39/rc

Data Sharing Statement: Available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-39/dss

Peer Review File: Available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-39/prf

Funding: None.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://ccts.amegroups.com/article/view/10.21037/ccts-24-39/coif). C.F.J. serves as an unpaid editorial board member of Current Challenges in Thoracic Surgery from September 2024 to August 2026. M.D.G. received personal consulting fees from Grifols and personal payment from Johnson & Johnson. He is also a board member of the MATRCS study DSMB. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the University of Cincinnati Institutional Review Board, Study #2017-2673 and #2023-0385, and individual consent for this retrospective analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Dehghan N, de Mestral C, McKee MD, et al. Flail chest injuries: a review of outcomes and treatment practices from the National Trauma Data Bank. J Trauma Acute Care Surg 2014;76:462-8. [Crossref] [PubMed]
2. Kasotakis G, Hasenboehler EA, Streib EW, et al. Operative fixation of rib fractures after blunt trauma: A practice management guideline from the Eastern Association for the Surgery of Trauma. J Trauma Acute Care Surg 2017;82:618-26. [Crossref] [PubMed]
3. Sawyer E, Wullschleger M, Muller N, et al. Surgical Rib Fixation of Multiple Rib Fractures and Flail Chest: A Systematic Review and Meta-analysis. J Surg Res 2022;276:221-34. [Crossref] [PubMed]
4. Schuurmans J, Goslings JC, Schepers T. Operative management versus non-operative management of rib fractures in flail chest injuries: a systematic review. Eur J Trauma Emerg Surg 2017;43:163-8. [Crossref] [PubMed]
5. Yahn CA, McNally AP, Deivert K, et al. Outcomes of Trauma Patients with Flail Chest and Surgical Rib Stabilization. Am Surg 2022;88:810-2. [Crossref] [PubMed]
6. Pieracci FM, Schubl S, Gasparri M, et al. The Chest Wall Injury Society Recommendations for Reporting Studies of Surgical Stabilization of Rib Fractures. Injury 2021;52:1241-50. [Crossref] [PubMed]
7. Forrester JD, Bauman ZM, Doben AR, et al. Chest wall injury centers-how we did it. J Thorac Dis 2021;13:6104-7. [Crossref] [PubMed]
8. Madelyn F, Brynn S, Erika TL, et al. Educational assessment of intrathoracic and extrathoracic surgical stabilization of rib fractures. Injury 2023;54:63-9. [Crossref] [PubMed]
9. Delman AM, Turner KM, Ammann AM, et al. A method for identifying the learning curve for the surgical stabilization of rib fractures. J Trauma Acute Care Surg 2022;93:743-9. [Crossref] [PubMed]
10. Gao AY, Whitrock JN, Goodman MD, et al. The Next Generation: Surgeon Learning Curve in a Mature Operative Rib Management Program. J Surg Res 2024;301:461-7. [Crossref] [PubMed]
11. Toale C, O’Byrne A, Morris M, et al. Defining operative experience targets in surgical training: A systematic review. Surgery 2022;172:1364-72. [Crossref] [PubMed]
12. Prins JTH, Leasia K, Sauaia A, et al. A decade of surgical stabilization of rib fractures: the effect of study year on patient selection, operative characteristics, and in-hospital outcome. Injury 2022;53:1637-44. [Crossref] [PubMed]
13. Niziolek G, Goodman MD, Makley A, et al. Early results after initiation of a rib fixation programme: A propensity score matched analysis Injury 2022;53:137-44. [Crossref] [PubMed]
14. Baker JE, Skinner M, Heh V, et al. Readmission rates and associated factors following rib cage injury. J Trauma Acute Care Surg 2019;87:1269-76. [Crossref] [PubMed]
15. Schoenfeld DA, Bernard GR. ARDS Network. Statistical evaluation of ventilator-free days as an efficacy measure in clinical trials of treatments for acute respiratory distress syndrome. Crit Care Med 2002;30:1772-7. [Crossref] [PubMed]
16. Yehya N, Harhay MO, Curley MAQ, et al. Reappraisal of Ventilator-Free Days in Critical Care Research. Am J Respir Crit Care Med 2019;200:828-36. [Crossref] [PubMed]
17. Young A, Miller JP, Azarow K. Establishing learning curves for surgical residents using Cumulative Summation (CUSUM) Analysis. Curr Surg 2005;62:330-4. [Crossref] [PubMed]
18. Nicolas JD, Huang R, Teitelbaum EN, et al. Constructing Learning Curves to Benchmark Operative Performance of General Surgery Residents Against a National Cohort of Peers. J Surg Educ 2020;77:e94-e102. [Crossref] [PubMed]