Skip to main content

Advertisement

SpringerLink
Log in

Medical Image Retrieval Using Multi-Texton Assignment

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

In this paper, we present a multi-texton representation method for medical image retrieval, which utilizes the locality constraint to encode each filter bank response within its local-coordinate system consisting of the k nearest neighbors in texton dictionary and subsequently employs spatial pyramid matching technique to implement feature vector representation. Comparison with the traditional nearest neighbor assignment followed by texton histogram statistics method, our strategies reduce the quantization errors in mapping process and add information about the spatial layout of texton distributions and, thus, increase the descriptive power of the image representation. We investigate the effects of different parameters on system performance in order to choose the appropriate ones for our datasets and carry out experiments on the IRMA-2009 medical collection and the mammographic patch dataset. The extensive experimental results demonstrate that the proposed method has superior performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Madabhushi A, Agner S, Basavanhally A, Doyle S, Lee G: Computer-aided prognosis: Predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data. Comput Med Imaging Graph 35(7–8):506–514, 2011

    Article  PubMed  Google Scholar 

  2. Zaidi H, Vees H, Wissmeyer M: Molecular PET/CT imaging-guided radiation therapy treatment planning. Acad Radiol 16(9):1108–1133, 2009

    Article  PubMed  Google Scholar 

  3. Marcus C, Ladam-Marcus V, Cucu C, Bouché O, Lucas L, Hoeffel C: Imaging techniques to evaluate the response to treatment in oncology: Current standards and perspectives. Crit Rev Oncol Hematol 72(3):217–238, 2009

    Article  CAS  PubMed  Google Scholar 

  4. Greenspan H, Pinhas AT: Medical image categorization and retrieval for PACS using the GMM-KL framework. IEEE Trans Inf Technol Biomed 11(2):190–202, 2007

    Article  PubMed  Google Scholar 

  5. Dhara AK, Mukhopadhyay S, Dutta A, Garg M, Khandelwal N: Content-based image retrieval system for pulmonary nodules: assisting radiologists in self-learning and diagnosis of lung cancer. J Digit Imaging 30(1):63–77, 2017

    Article  PubMed  Google Scholar 

  6. Camlica Z, Tizhoosh HR, Khalvatid F: Medical image classification via SVM using LBP features from saliency-based folded data. In: Proc IEEE Int Conf Mach Learn Appl, 2015. p. 128–132.

  7. Tang Q, Liu Y, Liu H: Medical image classification via multiscale representation learning. Artif Intell Med 79:71–78, 2017

  8. Lin Z, Brandt J: A local bag-of-features model for large-scale object retrieval. In: Proc Eur Conf Comput Vis, 2010, p. 294–308

  9. Lehmann TM, Deselaers T, Schubert H, Guld MO, Thies C, Fischer B, Spitzer K: IRMA–a content based approach to image retrieval in medical applications. In: IRMA Int Conf, 5033: 911–912, 2006.

  10. Puzicha J, Hofmann T, Buhmann JM: Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. In: Proc IEEE Conf Comput Vis Pattern Recognit, 1997, pp 267–272

  11. Arandjelovic R, Zisserman A: Three things everyone should know to improve object retrieval. In: Proc IEEE Conf Comput Vis Pattern Recognit, 2012, pp 2911–2918

  12. Kumar A, Kim J, Cai W, Fulham M, Feng D: Content-based medical image retrieval: A survey of applications to multidimensional and multimodality data. J Digit Imaging 26(6):1025–1039, 2013

    Article  PubMed  PubMed Central  Google Scholar 

  13. Gletsos M, Mougiakakou SG, Matsopoulos GK, Nikita KS, Nikita AS, Kelekis D: A computer-aided diagnostic system to characterize CT focal liver lesions: design and optimization of a neural network classifier. IEEE Trans Info Technol Biomed 7(3):153–162, 2003

    Article  Google Scholar 

  14. Lee C, Chen S, Tsai H, Chung P, Chiang Y: Discrimination of liver diseases from CT images based on Gabor filters. In: Proc IEEE Int Symp Biomed Comput-based Med Syst, 2006

  15. Nanni L, Brahnam S, Lumini A: A very high performing system to discriminate tissues in mammograms as benign and malignant. Expert Syst Appl 39(2):1968–1971, 2012

    Article  Google Scholar 

  16. Manik V, Andrew Z: A statistical approach to material classification using image patch exemplars. IEEE Trans Pattern Anal Mach Intell 31(11):2032–2047, 2009

    Article  Google Scholar 

  17. Jégou H, Douze M, Schmid C: Improving bag-of-features for large scale image search. Int J Comput Vis 87(3):316–336, 2010

    Article  Google Scholar 

  18. Galaro J, Judkins AR, Ellison D, Baccon J, Madabhushi A: An integrated texton and bag of words Cclassifier for identifying anaplastic medulloblastomas. In: Conf Proc IEEE Eng Med Biol Soc, 2011, pp 3443–3446

  19. Riaz F, Silva FB, Ribeiro MD, Coimbra MT: Invariant Gabor Texture Descriptors for Classification of Gastroenterology Images. IEEE Trans Biomed Eng 59(10):2893–2904, 2012

    Article  PubMed  Google Scholar 

  20. Li Y, Chen H, Rohde GK, Yao C, Cheng L: Texton analysis for mass classification in mammograms. Pattern Recognit Lett 52:87–93, 2015

    Article  Google Scholar 

  21. Fernández-Carrobles MM, Bueno G, Déniz O, Salido J, García-Rojo M, Gonzández-López L: Frequential versus spatial colour textons for breast TMA classification. Comput Med Imaging Graph 42:25–37, 2015

    Article  PubMed  Google Scholar 

  22. van Gemert JC, Veenman CJ, Smeulders AWM, Geusebroek J: Visual word ambiguity. IEEE Trans Pattern Anal Mach Intell 32(7):1271–1283, 2010

    Article  PubMed  Google Scholar 

  23. Julesz B: Textons, the elements of texture perception, and their interactions. Nature 290:91–97, 1981

    Article  CAS  PubMed  Google Scholar 

  24. Leung T, Malik J: Representing and recognizing the visual appearance of materials using three-dimensional textons. Int J Comput Vis 43(1):29–44, 2001

    Article  Google Scholar 

  25. Tang Q, Sang N, Liu H: Contrast-dependent surround suppression models for contour detection. Pattern Recogn 60:51–61, 2016

    Article  Google Scholar 

  26. Xie J, Zhang L, You J, Shiu S: Effective texture classification by texton encoding induced statistical features. Pattern Recogn 48(2):447–457, 2015

    Article  Google Scholar 

  27. Shotton J, Winn J, Rother C, Criminisi A: Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81:2–23, 2009

    Article  Google Scholar 

  28. Li X, Williams S, Bottema MJ: Constructing and applying higher order textons: Estimating breast cancer risk. Pattern Recogn 47(3):1375–1382, 2014

    Article  Google Scholar 

  29. Petroudi S, Brady M: Breast density characterization using texton distributions. In: Conf Proc IEEE Eng Med Biol Soc, 2011, pp 5004–5007

  30. Zhang L, Fisher M, Wang W: Retinal vessel segmentation using multi-scale textons derived from keypoints. Comput Med Imaging Graph 45:47–56, 2015

    Article  CAS  PubMed  Google Scholar 

  31. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y: Locality-constrained linear coding for image classification. In: Proc IEEE Conf Comput Vis Pattern Recognit, 2010, pp 3360–3367

  32. Zhang P, Wee C, Nieghammer M, Shen D, Yap P: Large deformation image classification using generalized locality-constrained linear coding. In: Proc Int Conf Med Image Comput Comput Assist Interv (MICCAI), 2013, pp 292–299

  33. Yang J, Yu K, Gong Y, Huang T: Linear spatial pyramid matching using sparse coding for image classification. In: Proc IEEE Conf Comput Vis Pattern Recognit, 2009, pp 1794–1801

  34. Lazebnik S, Schmid C, Ponce J: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proc. IEEE Conf. Comput. Vis. Pattern Recogn 2: 2169–2178, 2006.

  35. He K, Zhang X, Ren S, Sun J: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916, 2015

    Article  PubMed  Google Scholar 

  36. Meij E, Trieschnigg D, Rijke MD, Kraaij W: Conceptual language models for domain-specific retrieval. Inform Process Manag 46(4):448–469, 2010

    Article  Google Scholar 

  37. Tommasi T, Caputo B, Welter P, Guld MO, Deserno TM: Overview of the CLEF 2009 medical image annotation track. In: CLEF 2009 Workshop. Lect Notes Comput Sci 6242(85–93), 2010

  38. Lehmann TM, Schubert H, Keysers D, Kohnen M, Wein BB: The IRMA code for unique classification of medical images. In: Proc SPIE-Med Imaging, 5033:440–451, 2003

  39. Rahman MM, Desai BC, Bhattacharya P: Medical image retrieval with probabilistic multiclass support vector machine classifiers and adaptive similarity fusion. Comput Med Imaging Graph 32(2):95–108, 2008

    Article  PubMed  Google Scholar 

  40. Jiang W, Er G, Dai Q, Gu J: Similarity based online feature selection in content-based image retrieval. IEEE Trans Image Process 15(3):702–712, 2006

    Article  PubMed  Google Scholar 

  41. Avni U, Greenspan H, Konen E, Sharon M, Goldberger J: X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words. IEEE Trans Med Imaging 30(3):733–746, 2011

    Article  PubMed  Google Scholar 

  42. Liu X, Tizhoosh HR, Kofman J: Generating binary tags for fast medical image retrieval based on convolutional nets and radon transform. In: Proc IEEE Int Joint Conf Neural Networks, 2016, pp 2872–2878

  43. Unay D, Soldea O, Ozogür-Akyuz S, Cetin M, Ercil A: Medical image retrieval and automatic annotation: VPA-SABANCI at imageCLEF 2009. Working Notes for CLEF 2009 Workshop, 2009.

  44. de Oliveira JEE, Lopes APB, Camara-Chavez G, de Araújo A, Deserno TM, Mammo SVD: A content-based image retrieval system using a reference database of mammographies. In: Proc IEEE Int Symp Biomed Comput-based Med Syst, 2009, pp 1–4

  45. Crammer K, Singer Y: On the algorithmic implementation of multiclass kernel-based vector machines. J Mach Learn Res 2:265–292, 2001

    Google Scholar 

  46. Pinho E, Godinho T, Valente F, Costa C: A multimodal search engine for medical imaging studies. J Digit Imaging 30(1):39–48, 2017

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The main image dataset used in this study is courtesy of the IRMA Group, Aachen, Germany, http://irma-project.org. This work was supported by the Fundamental Research Funds for the Central Universities (CZP17033).

Author information

Authors and Affiliations

  1. South Central University for Nationalities, College of Biomedical Engineering, Wuhan, 430074, People’s Republic of China

    Qiling Tang

  2. Huibei Key Laboratory for Medical Information Analysis and Tumor Treatment, Wuhan, 430074, People’s Republic of China

    Jirong Yang

  3. Key Laboratory of Congnitive Science, State Ethnic Affairs Commission, Wuhan, 430074, People’s Republic of China

    Xianfu Xia

Authors
  1. Qiling Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jirong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianfu Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiling Tang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Q., Yang, J. & Xia, X. Medical Image Retrieval Using Multi-Texton Assignment. J Digit Imaging 31, 107–116 (2018). https://doi.org/10.1007/s10278-017-0017-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-017-0017-z

Keywords

Search

Navigation