Abstract
Longitudinal imaging forms an essential component in the management and follow-up of many medical conditions. The presence of lesion changes on serial imaging can have significant impact on clinical decision making, highlighting the important role for automated change detection. Lesion changes can represent anomalies in serial imaging, which implies a limited availability of annotations and a wide variety of possible changes that need to be considered. Hence, we introduce a new unsupervised anomaly detection and localisation method trained exclusively with serial images that do not contain any lesion changes. Our training automatically synthesises lesion changes in serial images, introducing detection and localisation pseudo-labels that are used to self-supervise the training of our model. Given the rarity of these lesion changes in the synthesised images, we train the model with the imbalance robust focal Tversky loss. When compared to supervised models trained on different datasets, our method shows competitive performance in the detection and localisation of new demyelinating lesions on longitudinal magnetic resonance imaging in multiple sclerosis patients. Code for the models will be made available at https://github.com/toson87/MSChangeDetection.
This paper was partially supported by an Avant Doctor in Training Research Scholarship and the Australian Research Council through grants DP180103232 and FT190100525.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abraham, N., Khan, N.M.: A novel focal Tversky loss function with improved attention U-Net for lesion segmentation. arXiv:1810.07842 (2018)
Achanta, R., Shaji, A., Smith, K., et al.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012)
Altay, E.E., Fisher, E., Jones, S.E., et al.: Reliability of classifying multiple sclerosis disease activity using magnetic resonance imaging in a multiple sclerosis clinic. JAMA Neurol. 70, 338 (2013)
Aslani, S., Dayan, M., Storelli, L., et al.: Multi-branch convolutional neural network for multiple sclerosis lesion segmentation. NeuroImage 196, 1–15 (2019)
Birenbaum, A., Greenspan, H.: Multi-view longitudinal CNN for multiple sclerosis lesion segmentation. Eng. Appl. Artif. Intell. 65, 111–118 (2017)
Bosc, M., Heitz, F., Armspach, J.P., et al.: Automatic change detection in multimodal serial MRI: application to multiple sclerosis lesion evolution. NeuroImage 20(2), 643–656 (2003)
Bu, S., Li, Q., Han, P., et al.: Mask-CDNet: a mask based pixel change detection network. Neurocomputing 378, 166–178 (2020)
Daudt, R.C., Saux, B.L., Boulch, A.: Fully convolutional Siamese networks for change detection. arXiv:1810.08462 (2018)
Denner, S., Khakzar, A., Sajid, M., et al.: Spatio-temporal learning from longitudinal data for multiple sclerosis lesion segmentation. arXiv:2004.03675 (2020)
DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv:1708.04552 (2017)
Dufresne, E., Fortun, D., Kumar, B., et al.: Joint registration and change detection in longitudinal brain MRI. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 104–108. IEEE (2020)
Garyfallidis, E., Brett, M., Amirbekian, B., et al.: Dipy, a library for the analysis of diffusion MRI data. Front. Neuroinform. 8, 8 (2014)
Hofmanninger, J., Prayer, F., Pan, J., Röhrich, S., Prosch, H., Langs, G.: Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem. Eur. Radiol. Exp. 4(1), 1–13 (2020). https://doi.org/10.1186/s41747-020-00173-2
Huang, X., Shan, J., Vaidya, V.: Lung nodule detection in CT using 3D convolutional neural networks. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) (2017)
Isensee, F., Schell, M., Pflueger, I., et al.: Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain Mapp. 40(17), 4952–4964 (2019)
Kayalibay, B., Jensen, G., van der Smagt, P.: CNN-based segmentation of medical imaging data. arXiv:1701.03056 (2017)
Khelifi, L., Mignotte, M.: Deep learning for change detection in remote sensing images: comprehensive review and meta-analysis. IEEE Access 8, 126385–126400 (2020)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv:1312.6114 (2013)
Krüger, J., Opfer, R., Gessert, N., et al.: Fully automated longitudinal segmentation of new or enlarged multiple sclerosis lesions using 3D convolutional neural networks. NeuroImage: Clin. 28, 102445 (2020)
Lee, C.Y., Xie, S., Gallagher, P., et al.: Deeply-supervised nets. arXiv:1409.5185 (2014)
Li, H., Jiang, G., Zhang, J., et al.: Fully convolutional network ensembles for white matter hyperintensities segmentation in MR images. NeuroImage 183, 650–665 (2018)
Lowekamp, B.C., Chen, D.T., Ibáñez, L., Blezek, D.: The design of SimpleITK. Front. Neuroinform. 7, 45 (2013)
Lundervold, A.S., Lundervold, A.: An overview of deep learning in medical imaging focusing on MRI. Zeitschrift für Medizinische Physik 29, 102–127 (2019)
McKinley, R., Wepfer, R., Grunder, L., et al.: Automatic detection of lesion load change in multiple sclerosis using convolutional neural networks with segmentation confidence. NeuroImage: Clin. 25, 102104 (2020)
McNamara, C., Sugrue, G., Murray, B., MacMahon, P.J.: Current and emerging therapies in multiple sclerosis: implications for the radiologist, part 1–mechanisms, efficacy, and safety. AJNR 38, 1664–1671 (2017)
Patel, N., Horsfield, M.A., Banahan, C., et al.: Detection of focal longitudinal changes in the brain by subtraction of MR images. AJNR 38, 923–927 (2017)
Plassard, A.J., Davis, L.T., Newton, A.T., et al.: Learning implicit brain MRI manifolds with deep learning. arXiv:1801.01847 (2018)
Radke, R.J., Andra, S., Al-Kofahi, O., Roysam, B.: Image change detection algorithms: a systematic survey. IEEE Trans. Image Process. 14(3), 294–307 (2005)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. arXiv:1505.04597 (2015)
Àlex, R., Wattjes, M.P., Tintoré, M., et al.: MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis–clinical implementation in the diagnostic process. Nat. Rev. Neurol. 11, 471–482 (2015)
Russakovsky, O., Deng, J., Su, H., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)
Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3D fully convolutional deep networks. arXiv:1706.05721 (2017)
Schlemper, J., Oktay, O., Schaap, M., et al.: Attention gated networks: learning to leverage salient regions in medical images. Med. Image Anal. 53, 197–207 (2019)
Schmidt, P., Pongratz, V., Küster, P., et al.: Automated segmentation of changes in FLAIR-hyperintense white matter lesions in multiple sclerosis on serial magnetic resonance imaging. NeuroImage: Clin. 23, 101849 (2019)
Sepahvand, N.M., Arnold, D.L., Arbel, T.: CNN detection of new and enlarging multiple sclerosis lesions from longitudinal MRI using subtraction images. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 127–130 (2020)
Snaauw, G., Gong, D., Maicas, G., et al.: End-to-end diagnosis and segmentation learning from cardiac magnetic resonance imaging. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 802–805. IEEE (2019)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
Yun, S., Han, D., Oh, S.J., et al.: CutMix: regularization strategy to train strong classifiers with localizable features. arXiv:1905.04899 (2019)
Yushkevich, P.A., Piven, J., Hazlett, H.C., et al.: User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3), 1116–1128 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
To, MS., Sarno, I.G., Chong, C., Jenkinson, M., Carneiro, G. (2021). Self-Supervised Lesion Change Detection and Localisation in Longitudinal Multiple Sclerosis Brain Imaging. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham. https://doi.org/10.1007/978-3-030-87234-2_63
Download citation
DOI: https://doi.org/10.1007/978-3-030-87234-2_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87233-5
Online ISBN: 978-3-030-87234-2
eBook Packages: Computer ScienceComputer Science (R0)
