Abstract
Object
We investigate the use of relevance feedback (RFb) and the inclusion of expert knowledge to reduce the semantic gap in content-based image retrieval (CBIR) of mammograms.
Materials and methods
Tests were conducted with radiologists, in which their judgment of the relevance of the retrieved images was used with techniques of query-point movement to incorporate RFb. The measures of similarity of images used for CBIR were based upon textural characteristics and the distribution of density of fibroglandular tissue in the breast. The features used include statistics of the gray-level histogram, texture features based upon the gray-level co-occurrence matrix, moment-based features, measures computed in the Radon domain, and granulometric measures. Queries for CBIR with RFb were executed by three radiologists. The performance of CBIR was measured in terms of precision of retrieval and a measure of relevance-weighted precision (RWP) of retrieval.
Results
The results indicate improvement due to RFb of up to 62% in precision and 39% in RWP.
Conclusion
The gain in performance of CBIR with RFb depended upon the BI-RADS breast density index of the query mammographic image, with greater improvement in cases of mammograms with higher density.
Similar content being viewed by others
References
Müller H, Michoux N, Bandon D and Geissbuhler A (2004). A review of content-based image retrieval systems in medical applications-clinical benefits and future directions. Int J Med Inform 73: 1–23
El-Naqa I, Yang Y, Galatsanos NP, Nishikawa RM and Wernick MN (2004). A similarity learning approach to content-based image retrieval: application to digital mammography. IEEE Trans Med Imaging 23: 1233–1244
Li Q, Li F, Shiraishi J, Katsuragawa S, Sone S and Doi K (2003). Investigation of new psychophysical measures for evaluation of similar images on thoracic computed tomography for distinction between benign and malignant nodules. Med Phys 30: 2584–2593
Tourassi GD, Eltonsy NH, Graham J, Floyd CE and Elmaghraby AS (2005). Feature and knowledge based analysis for reduction of false positives in the computerized detection of masses in screening mammography. Conf Proc IEEE Eng Med Biol Soc 6: 6524–6527
Kinoshita SK, Azevedo-Marques PM, Rodrigues JAH, Rangayyan RM and Pereira RR (2007). Content-based retrieval of mammograms using visual features related to breast density patterns. J Digit Imaging 20: 172–190
Azevedo-Marques PM, Honda MH, Rodrigues JAH, Santos RR, Traina AJM, Traina Júnior C and Bueno JM (2002). Recuperação de Imagem Baseada em Conteúdo: Uso de Atributos de Textura para Caracterização de Microcalcificações Mamográficas. Rev Bras Radiol 35: 93–98
Haralick RM (1979). Statistical and structural approaches to texture. Proc IEEE 1979: 786–804
Alto H, Rangayyan RM, Desautels JEL (2005) Content-based retrieval and analysis of mammographic masses. J Electron Imaging 14:023016:1–17
Muramatsu C, Li Q, Suzuki K, Schmidt RA, Shiraishi J, Newstead GM and Doi K (2005). Investigation of psychophysical measure for evaluation of similar images for mammographic masses: preliminary results. Med Phys 32: 2295–2304
Tao EY, Sklansky J (1996) Analysis of mammograms aided by database of images of calcifications and textures. In: Medical imaging: image processing. Proceedings of SPIE, pp 988–995
Ornes CJ, Valentino DJ, Yoon H-J, Eisenman JI, Sklansky J (2001) Search engine for remote database-aided interpretation of digitized mammograms. In: Siegel El, Huang HK (eds) Medical imaging: PACS and integrated medical information systems: design and evaluation, pp 132–137
El-Naqa I, Yang Y, Galatsanos NP, Wernick MN (2002) Content-based image retrieval for digital mammography. In: Yongyi Y (eds) IEEE proceedings of the international conference on image processing, Rochester, pp 141–144
Nakagawa T, Hara T, Fujita H, Iwase T, Endo T (2002) Image retrieval system of mammographic masses by using local pattern matching technique. In: Peitgen H-O (ed) Digital mammography. Proceedings of the IWDM 2002 Bremen, Germany. Springer, New York, pp 562–565
Nakagawa T, Hara T, Fujita H, Iwase T, Endo T (2002) Development of a computer-aided sketch system for mammograms. In: Peitgen H-O (ed) Digital mammography. Proceedings of the IWDM 2002 Bremen, Germany. Springer, New York, pp 581–583
Baeza-Yates RA and Ribeiro-Neto BA (1999). Modern information retrieval. Addison-Wesley, Wokingham
Ortega-Binderberger M and Mehrotra S (2003). Relevance feedback in multimedia databases. In: Furht, B and Marques, O (eds) Handbook of video databases—design and applications, pp 511–536. CRC Press, Boca Raton
Zhou XS and Huang TS (2003). Relevance feedback in image retrieval: A comprehensive review. Multimed Syst 8: 536–544
ACR (2003) BI-RADS® —mammography, 4th edn. American College of Radiology, Reston
Haralick RM, Shanmugam K and Dinstein I (1973). Textural features for image classification. IEEE Trans Syst Man Cybern 3: 610–621
Gonzalez RC and Woods RE (2007). Digital image processing. Prentice Hall, Englewood Cliffs
Ramm AG and Katsevich AI (1996). The radon transform and local tomography. CRC Press, Boca Raton
Hu M-K (1962). Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8: 179–187
Traina C Jr, Traina AJM, Faloutsos C and Seeger B (2002). Fast indexing and visualization of metric datasets using slim-trees. IEEE Trans Knowl Data Eng 14: 244–260
Rocchio JJ (1971). Relevance feedback in information retrieval. In: Cliffs, E (eds) The SMART retrieval system: experiments in automatic document processing, pp 313–323. Prentice Hall, Englewood Cliffs
Traina A, Marques J, Traina C (2006) Fighting the semantic gap on CBIR systems through new relevance feedback techniques. In: CBMS ’06: proceedings of the 19th IEEE international symposium on computer-based medical systems. IEEE Computer Society, Salt Lake City, pp 881–886
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
de Azevedo-Marques, P.M., Rosa, N.A., Traina, A.J.M. et al. Reducing the semantic gap in content-based image retrieval in mammography with relevance feedback and inclusion of expert knowledge. Int J CARS 3, 123–130 (2008). https://doi.org/10.1007/s11548-008-0154-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-008-0154-4