Abstract
Objective
To develop an ensemble deep learning model (DLM) predicting anterior cruciate ligament (ACL) tears from lateral knee radiographs and to evaluate its diagnostic performance.
Materials and methods
In this study, 1433 lateral knee radiographs (661 with ACL tear confirmed on MRI, 772 normal) from two medical centers were split into training (n = 1146) and test sets (n = 287). Three single DLMs respectively classifying radiographs with ACL tears, abnormal lateral femoral notches, and joint effusion were developed. An ensemble DLM predicting ACL tears was developed by combining the three DLMs via stacking method. The sensitivities, specificities, and area under the receiver operating characteristic curves (AUCs) of the DLMs and three radiologists were compared using McNemar test and Delong test. Subgroup analysis was performed to identify the radiologic features associated with the sensitivity.
Results
The sensitivity, specificity, and AUC of the ensemble DLM were 86.8% (95% confidence interval [CI], 79.9–92.0%), 89.4% (95% CI, 83.4–93.8%), and 0.927 (95% CI, 0.891–0.954), achieving diagnostic performance comparable with that of a musculoskeletal radiologist (P = 0.193, McNemar test; P = 0.131, Delong test). The AUC of the ensemble DLM was significantly higher than those of non-musculoskeletal radiologists (P = 0.043, P < 0.001). The sensitivity of the DLM was higher than that of the radiologists in the absence of an abnormal lateral femoral notch or joint effusion.
Conclusion
The diagnostic performance of the ensemble DLM in predicting lateral knee radiographs with ACL tears was comparable to that of a musculoskeletal radiologist.
Similar content being viewed by others
Abbreviations
- ACL:
-
Anterior cruciate ligament
- DLM:
-
Deep learning model
- AUC:
-
Area under the receiver operating characteristic curve
- CI:
-
Confidence interval
References
Murrell GA, Maddali S, Horovitz L, Oakley SP, Warren RF. The effects of time course after anterior cruciate ligament injury in correlation with meniscal and cartilage loss. Am J Sports Med. 2001;29(1):9–14.
Granan L-P, Bahr R, Lie SA, Engebretsen L. Timing of anterior cruciate ligament reconstructive surgery and risk of cartilage lesions and meniscal tears: a cohort study based on the Norwegian National Knee Ligament Registry. Am J Sports Med. 2009;37(5):955–61.
Lubowitz JH, Bernardini BJ, Reid JB III. Current concepts review: comprehensive physical examination for instability of the knee. Am J Sports Med. 2008;36(3):577–94.
Pao DG. The lateral femoral notch sign. Radiology. 2001;219(3):800–1.
Grimberg A, Shirazian H, Torshizy H, Smitaman E, Chang EY, Resnick D. Deep lateral notch sign and double notch sign in complete tears of the anterior cruciate ligament: MR imaging evaluation. Skeletal Radiol. 2015;44(3):385–91.
Nakauchi M, Kurosawa H, Kawakami AJ. Abnormal lateral notch in knees with anterior cruciate ligament injury. J Orthop Sci. 2000;5(2):92–5.
Frobell R, Le Graverand M-P, Buck R, et al. The acutely ACL injured knee assessed by MRI: changes in joint fluid, bone marrow lesions, and cartilage during the first year. Osteoarthr Cartil. 2009;17(2):161–7.
Yoon AP, Lee Y-L, Kane RL, Kuo C-F, Lin C, Chung KC. Development and Validation of a Deep Learning Model Using Convolutional Neural Networks to Identify Scaphoid Fractures in Radiographs. JAMA Netw Open. 2021;4(5):e216096–e216096.
Park HS, Jeon K, Cho YJ, et al. Diagnostic Performance of a New Convolutional Neural Network Algorithm for Detecting Developmental Dysplasia of the Hip on Anteroposterior Radiographs. Korean J Radiol. 2021;22(4):612.
Nam JG, Park S, Hwang EJ, et al. Development and validation of deep learning–based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology. 2019;290(1):218–28.
Ganaie MA, Hu M, Tanveer M, Suganthan PN. Ensemble deep learning: A review. arXiv. 2021; arXiv:2104.02395. https://arxiv.org/abs/2104.02395. Accessed 4 Oct 2021.
Sanders TL, Maradit Kremers H, Bryan AJ, et al. Incidence of anterior cruciate ligament tears and reconstruction: a 21-year population-based study. Am J Sports Med. 2016;44(6):1502–7.
Yoon JP, Chang CB, Yoo JH, et al. Correlation of magnetic resonance imaging findings with the chronicity of an anterior cruciate ligament tear. J Bone Joint Surg Am. 2010;92(2):353–60.
Yoon JP, Yoo JH, Chang CB, et al. Prediction of chronicity of anterior cruciate ligament tear using MRI findings. Clin Orthop Surg. 2013;5(1):19–25.
Hall FM. Radiographic diagnosis and accuracy in knee joint effusions. Radiology. 1975;115(1):49–54.
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: 2017 IEEE Int Conf Comput Vis. IEEE; 2017: 618–26. https://doi.org/10.1109/ICCV.2017.74
Crawshaw M. Multi-task learning with deep neural networks: A survey. arXiv. 2020; arXiv:2009.09796. https://arxiv.org/abs/2009.09796. Accessed 11 May 2022.
Zhou Y, Chen H, Li Y, et al. Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images. Med Image Anal. 2021;70:101918.
Wu B, Zhou Z, Wang J, Wang Y. Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction. IEEE 15th Int Symp Biomed Imaging. 2018; 1109–13, https://doi.org/10.1109/ISBI.2018.8363765.
Hussein S, Cao K, Song Q, Bagci U. Risk stratification of lung nodules using 3D CNN-based multi-task learning. In: 2017 Int Conf Inf Process Med Imaging. Springer Verlag; 2017: 249–60. https://doi.org/10.1007/978-3-319-59050-9_20
Xing J, Chen C, Lu Q, et al. Using BI-RADS stratifications as auxiliary information for breast masses classification in ultrasound images. IEEE J Biomed Health Inform. 2020;25(6):2058–70.
Herbst E, Hoser C, Tecklenburg K, et al. The lateral femoral notch sign following ACL injury: frequency, morphology and relation to meniscal injury and sports activity. Knee Surg Sports Traumatol Arthrosc. 2015;23(8):2250–8.
Abreu MR, Chung CB, Trudell D, Resnick D. Hoffa’s fat pad injuries and their relationship with anterior cruciate ligament tears: new observations based on MR imaging in patients and MR imaging and anatomic correlation in cadavers. Skeletal Radiol. 2008;37(4):301–6.
Guenoun D, Le Corroller T, Amous Z, et al. The contribution of MRI to the diagnosis of traumatic tears of the anterior cruciate ligament. Diagn Interv Imaging. 2012;93(5):331–41.
Calmbach WL, Hutchens M. Evaluation of patients presenting with knee pain: Part I. History, physical examination, radiographs, and laboratory tests. Am Fam Physician. 2003;68(5):907–12.
Sternbach GL. Evaluation of the knee. J Emerg Med. 1986;4(2):133–43.
Acknowledgements
The authors thank Kangwhi Lee and Jung Oh Lee for data analysis.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
Approval from the Institutional Review Board was obtained, and in keeping with the policies for a retrospective review, informed consent was not required.
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Supplementary file1
(DOCX 14.5 KB)
Supplementary file2
(DOCX 14.2 KB)
Rights and permissions
About this article
Cite this article
Kim, D.H., Chai, J.W., Kang, J.H. et al. Ensemble deep learning model for predicting anterior cruciate ligament tear from lateral knee radiograph. Skeletal Radiol 51, 2269–2279 (2022). https://doi.org/10.1007/s00256-022-04081-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00256-022-04081-x