Abstract
In order to scan for or monitor retinal diseases, OCT is a useful diagnostic tool that allows to take high-resolution images of the retinal layers. For the aim of fully automated, semantic segmentation of OCT images, both graph based models and deep neural networks have been used so far. Here, we propose to interpret the semantic segmentation of 2D OCT images as a sequence alignment task. Splitting the image into its constituent OCT scanning lines (A-Modes), we align an anatomically justified sequence of labels to these pixel sequences, using dynamic time warping. Combining this dynamic programming approach with learned convolutional filters allows us to leverage the feature extraction capabilities of deep neural networks, while at the same time enforcing explicit guarantees in terms of the anatomical order of layers through the dynamic programming. We investigate both the solitary training of the feature extraction stage, as well as an end-to-end learning of the alignment. The latter makes use of a recently proposed, relaxed formulation of dynamic time warping, that allows us to backpropagate through the dynamic program to enable end-to-end training of the network. Complementing these approaches, a local consistency criterion for the alignment task is investigated, that allows to improve consistency in the alignment of neighbouring A-Modes. We compare this approach to two state of the art methods, showing favourable results.
H.M. was supported by TUM International Graduate School of Science and Engineering (IGSSE). H.M. and S.F. were supported by the ICL-TUM Joint Academy of Doctoral Studies (JADS) program. N.N. was partially supported by U.S. National Institutes of Health under grant number 1R01EB025883-01A1.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aumann, S., Donner, S., Fischer, J., Müller, F.: Optical Coherence Tomography (OCT): principle and technical realization. In: Bille, J.F. (ed.) High Resolution Imaging in Microscopy and Ophthalmology, pp. 59–85. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-16638-0_3
BenTaieb, A., Hamarneh, G.: Topology aware fully convolutional networks for histology gland segmentation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 460–468. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_53
Chiu, S.J., Allingham, M.J., Mettu, P.S., Cousins, S.W., Izatt, J.A., Farsiu, S.: Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. Biomed. Opt. Express 6(4), 1172–1194 (2015). https://doi.org/10.1364/BOE.6.001172
Chiu, S.J., Li, X.T., Nicholas, P., Toth, C.A., Izatt, J.A., Farsiu, S.: Automatic segmentation of seven retinal layers in sdoct images congruent with expert manual segmentation. Opt. Express 18(18), 19413–19428 (2010)
Cuturi, M., Blondel, M.: Soft-DTW: a differentiable loss function for time-series. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 894–903. PMLR, International Convention Centre, Sydney, Australia, 06–11 Aug 2017
Duan, W., et al.: A generative model for oct retinal layer segmentation by groupwise curve alignment. IEEE Access 6, 25130–25141 (2018)
Fang, L., Cunefare, D., Wang, C., Guymer, R.H., Li, S., Farsiu, S.: Automatic segmentation of nine retinal layer boundaries in oct images of non-exudative amd patients using deep learning and graph search. Biomed. Opt. Express 8(5), 2732–2744 (2017)
Giorgino, T.: Computing and visualizing dynamic time warping alignments in R: the DTW package. J. Stat. Softw. 31, 1–24 (2009)
He, Y., et al.: Topology guaranteed segmentation of the human retina from oct using convolutional neural networks. arXiv preprint arXiv:1803.05120v1 (2018)
He, Y., et al.: Towards topological correct segmentation of macular oct from cascaded fcns. In: Cardoso, M.J., et al. (eds.) Fetal, Infant and Ophthalmic Medical Image Analysis, pp. 202–209. Springer International Publishing, Cham (2017)
Kugelman, J., et al.: Automatic choroidal segmentation in oct images using supervised deep learning methods. Sci. Rep. 9(1), 1–13 (2019)
Kugelman, J., Alonso-Caneiro, D., Read, S.A., Vincent, S.J., Collins, M.J.: Automatic segmentation of oct retinal boundaries using recurrent neural networks and graph search. Biomed. Opt. Express 9(11), 5759–5777 (2018)
Lee, K., Niemeijer, M., Garvin, M.K., Kwon, Y.H., Sonka, M., Abramoff, M.D.: Segmentation of the optic disc in 3-d oct scans of the optic nerve head. IEEE Trans. Med. Imaging 29(1), 159–168 (2010). https://doi.org/10.1109/TMI.2009.2031324
Liu, W., Sun, Y., Ji, Q.: Mdan-unet: multi-scale and dual attention enhanced nested u-net architecture for segmentation of optical coherence tomography images. Algorithms 13(3), 60 (2020)
Mensch, A., Blondel, M.: Differentiable dynamic programming for structured prediction and attention. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 3462–3471. PMLR, Stockholmsmässan, Stockholm Sweden (10–15 Jul 2018)
Guru Pradeep Reddy, T., et al.: Retinal-layer segmentation using dilated convolutions. In: Chaudhuri, B.B., Nakagawa, M., Khanna, P., Kumar, S. (eds.) Proceedings of 3rd International Conference on Computer Vision and Image Processing. AISC, vol. 1022, pp. 279–292. Springer, Singapore (2020). https://doi.org/10.1007/978-981-32-9088-4_24
Romero-Aroca, P., Baget-Bernaldiz, M., Pareja-Rios, A., Lopez-Galvez, M., Navarro-Gil, R., Verges, R.: Diabetic macular edema pathophysiology: Vasogenic versus inflammatory. J. Diabetes Res. 2016, 2156273 (2016). https://doi.org/10.1155/2016/2156273
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. arxiv 2015. arXiv preprint arXiv:1505.04597 (2015)
Roy, A.G., et al.: Relaynet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed. Opt. Express 8(8), 3627–3642 (2017)
Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26(1), 43–49 (1978). https://doi.org/10.1109/TASSP.1978.1163055
Shah, M., Grabocka, J., Schilling, N., Wistuba, M., Schmidt-Thieme, L.: Learning dtw-shapelets for time-series classification. In: Proceedings of the 3rd IKDD Conference on Data Science, 2016. CODS ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2888451.2888456, https://doi.org/10.1145/2888451.2888456
Tran, A., Weiss, J., Albarqouni, S., Faghi Roohi, S., Navab, N.: Retinal layer segmentation reformulated as OCT language processing. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 694–703. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_67
Wei, H., Peng, P.: The segmentation of retinal layer and fluid in sd-oct images using mutex dice loss based fully convolutional networks. IEEE Access 8, 60929–60939 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Maier, H., Faghihroohi, S., Navab, N. (2021). A Line to Align: Deep Dynamic Time Warping for Retinal OCT Segmentation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_67
Download citation
DOI: https://doi.org/10.1007/978-3-030-87193-2_67
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87192-5
Online ISBN: 978-3-030-87193-2
eBook Packages: Computer ScienceComputer Science (R0)
