Skip to main content
SpringerLink
Log in

Fully Automated Patch-Based Image Restoration: Application to Pathology Inpainting

  • Conference paper
  • First Online:
Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries (BrainLes 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10154))

Abstract

Pathology can have an important impact on MRI analysis. Specifically, white matter hyper-intensities, tumours, infarcts, etc., can influence the results of various image analysis techniques such as segmentation and registration. Several algorithms have been proposed for image inpainting and restoration, mainly in the context of Multiple Sclerosis lesions. These techniques commonly rely on a set of manually segmented pathological regions for inpainting. Rather than relying on prior segmentations for image restoration, we present a combined segmentation and inpainting algorithm for multimodal images. The proposed method is based on an iterative collaboration between two patch-based techniques, PatchMatch and Non-Local Means, where the former is used to estimate the most probable location of the pathological outliers and the latter to gradually fill the segmented areas with the most plausible multimodal texture. We demonstrate that the proposed method is able to automatically restore multimodal intensities in pathological regions within the context of Multiple Sclerosis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lladó, X., Oliver, A., Cabezas, M., Freixenet, J., Vilanova, J.C., Quiles, A., Valls, L., Ramió, L., Rovira, A.: Segmentation of multiple sclerosis lesions in brain MRI: a review of automated approaches. Inf. Sci. 186(1), 164–185 (2012)

    Article  Google Scholar 

  2. Gordillo, N., Montseny, E., Sobrevilla, P.: State of the art survey on MRI brain tumor segmentation. Magn. Reson. Imaging 31(8), 1426–1438 (2013)

    Article  Google Scholar 

  3. Rekik, I., Allassonnière, S., Carpenter, T.K., Wardlaw, J.M.: Medical image analysis methods in MR/CT-imaged acute-subacute ischemic stroke lesion: segmentation, prediction and insights into dynamic evolution simulation models. A critical appraisal. NeuroImage Clin. 1(1), 164–178 (2012)

    Article  Google Scholar 

  4. Ceccarelli, A., Jackson, J., Tauhid, S., Arora, A., Gorky, J., Dell’Oglio, E., Bakshi, A., Chitnis, T., Khoury, S., Weiner, H., Guttmann, C., Bakshi, R., Neema, M.: The impact of lesion in-painting and registration methods on voxel-based morphometry in detecting regional cerebral gray matter atrophy in multiple sclerosis. Am. J. Neuroradiol. 33(8), 1579–1585 (2012)

    Article  Google Scholar 

  5. Govindarajan, K.A., Datta, S., Hasan, K.M., Choi, S., Rahbar, M.H., Cofield, S.S., Cutter, G.R., Lublin, F.D., Wolinsky, J.S., Narayana, P.A., MRI Analysis Center at Houston, T.C.I.G.: Effect of in-painting on cortical thickness measurements in multiple sclerosis: a large cohort study. Hum. Brain Mapp. 36(10), 3749–3760 (2015)

    Google Scholar 

  6. Sdika, M., Pelletier, D.: Nonrigid registration of MS brain images using lesion inpainting for morphometry or lesion mapping. Hum. Brain Mapp. 30(4), 1060–1067 (2009)

    Article  Google Scholar 

  7. Chard, D.T., Jackson, J.S., Miller, D.H., Wheeler-Kingshott, C.: Reducing the impact of white matter lesions on automated measures of brain gray and white matter volumes. J. Magn. Reson. Imaging 32(1), 223–228 (2010)

    Article  Google Scholar 

  8. Battaglini, M., Jenkinson, M., De Stefano, N.: Evaluating and reducing the impact of white matter lesions on brain volume measurements. HBM 33(9), 2062–71 (2012)

    Article  Google Scholar 

  9. Guizard, N., Nakamura, K., Coupe, P., Arnold, D.L., Collins, D.L.: Non-local MS MRI lesion inpainting method for image processing. In: The endMS Conference (2013)

    Google Scholar 

  10. Prados, F., Cardoso, M.J., MacManus, D., Wheeler-Kingshott, C.A.M., Ourselin, S.: A modality-agnostic patch-based technique for lesion filling in multiple sclerosis. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 781–788. Springer, Cham (2014). doi:10.1007/978-3-319-10470-6_97

    Google Scholar 

  11. Guizard, N., Nakamura, K., Coupé, P., Vladimir, S., Fonov, V., Arnold, D., Collins, D.: Non-local means inpainting of MS lesions in longitudinal image processing. Front. Neurosci. 9, 456 (2015)

    Article  Google Scholar 

  12. Prados, F., Cardoso, M.J., Kanber, B., Ciccarelli, O., Kapoor, R., Wheeler-Kingshott, C.A.G., Ourselin, S.: A multi-time-point modality-agnostic patch-based method for lesion filling in multiple sclerosis. NeuroImage 139, 376–384 (2016)

    Article  Google Scholar 

  13. Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The generalized patchmatch correspondence algorithm. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 29–43. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15558-1_3

    Chapter  Google Scholar 

  14. Shi, W., Caballero, J., Ledig, C., Zhuang, X., Bai, W., Bhatia, K., Marvao, A.M.S.M., Dawes, T., O’Regan, D., Rueckert, D.: Cardiac image super-resolution with global correspondence using multi-atlas PatchMatch. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8151, pp. 9–16. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40760-4_2

    Chapter  Google Scholar 

  15. Ta, V.-T., Giraud, R., Collins, D.L., Coupé, P.: Optimized PatchMatch for near real time and accurate label fusion. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8675, pp. 105–112. Springer, Cham (2014). doi:10.1007/978-3-319-10443-0_14

    Google Scholar 

  16. Giraud, R., Ta, V.T., Papadakis, N., Manjon, J.V., Collins, D.L., Coupe, P.: An optimized patchmatch for multi-scale and multi-feature label fusion. NeuroImage 124(Pt. A), 770–782 (2016)

    Google Scholar 

  17. Prados, F., Cardoso, M.J., Cawley, N., Ciccarelli, O., Wheeler-Kingshott, C.A., Ourselin, S.: Multi-contrast patchmatch algorithm for multiple sclerosis lesion detection. In: ISBI 2015 - Longitudinal MS Lesion Segmentation Challenge, pp. 1–2 (2015)

    Google Scholar 

  18. Xu, Z., Bagci, U., Seidel, J., Thomasson, D., Solomon, J., Mollura, D.J.: Segmentation based denoising of PET images: an iterative approach via regional means and affinity propagation. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 698–705. Springer, Cham (2014). doi:10.1007/978-3-319-10404-1_87

    Google Scholar 

  19. Boyes, R.G., Gunter, J.L., Frost, C., Janke, A.L., Yeatman, T., Hill, D.L.G., Bernstein, M.A., Thompson, P.M., Weiner, M.W., Schuff, N., Alexander, G.E., Killiany, R.J., DeCarli, C., Jack, C.R., Fox, N.C.: Intensity non-uniformity correction using N3 on 3-T scanners. NeuroImage 39(4), 1752–1762 (2008)

    Google Scholar 

  20. Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  21. Cardoso, M.J., Wolz, R., Modat, M., Fox, N.C., Rueckert, D., Ourselin, S.: Geodesic information flows. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7511, pp. 262–270. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33418-4_33

    Chapter  Google Scholar 

Download references

Acknowledgments

FP, BK and SO are funded by the National Institute for Health Research University College London Hospitals Biomedical Research Centre (NIHR BRC UCLH/UCL High Impact Initiative-BW.mn.BRC10269). SO receives funding from the EPSRC (EP/H046410/1, EP/J020990/1, EP/K005278), the MRC (MR/J01107X/1) and the NIHR Biomedical Research Unit (Dementia) at UCL. This work was also supported by the Medical Research Council, the UK Multiple Sclerosis Society (grant 892/08) and the Brain Research Trust.

Author information

Authors and Affiliations

  1. Translational Imaging Group, Centre for Medical Image Computing (CMIC), University College London, London, WC1E 6BT, UK

    Ferran Prados, M. Jorge Cardoso, Baris Kanber & Sébastien Ourselin

  2. NMR Research Unit, Queen Square MS Centre, UCL Institute of Neurology, London, WC1B 5EH, UK

    Ferran Prados, Niamh Cawley, Baris Kanber, Olga Ciccarelli & Claudia A. M. Gandini Wheeler-Kingshott

  3. Brain MRI 3T Centre, C. Mondino National Neurological Institute, Pavia, Italy

    Claudia A. M. Gandini Wheeler-Kingshott

  4. Dementia Research Centre, UCL Institute of Neurology, London, WC1N 3BG, UK

    M. Jorge Cardoso & Sébastien Ourselin

Authors
  1. Ferran Prados
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Jorge Cardoso
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Niamh Cawley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Baris Kanber
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Olga Ciccarelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Claudia A. M. Gandini Wheeler-Kingshott
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sébastien Ourselin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ferran Prados .

Editor information

Editors and Affiliations

  1. University of Zurich, Istituto Italiano di Tecnologia, Genoa, Italy

    Alessandro Crimi

  2. TU München, Computer Science, Munich, Germany

    Bjoern Menze

  3. Medical Informatics, University of Lübeck, Lübeck, Germany

    Oskar Maier

  4. Surgical Technology and Biomechanics, Universität Bern, Bern, Switzerland

    Mauricio Reyes

  5. Department of Medicine, University of Cambridge, Cambridge, United Kingdom

    Stefan Winzeck

  6. Institute of Medical Informatics, University of Lübeck, Lübeck, Germany

    Heinz Handels

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Prados, F. et al. (2016). Fully Automated Patch-Based Image Restoration: Application to Pathology Inpainting. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Winzeck, S., Handels, H. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2016. Lecture Notes in Computer Science(), vol 10154. Springer, Cham. https://doi.org/10.1007/978-3-319-55524-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-55524-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-55523-2

  • Online ISBN: 978-3-319-55524-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation