Skip to main content

Advertisement

SpringerLink
Log in

Designing user interfaces to enhance human interpretation of medical content-based image retrieval: application to PET-CT images

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose Content-based image retrieval (CBIR) in medicine has been demonstrated to improve evidence-based diagnosis, education, and teaching. However, the low clinical adoption of CBIR is partially because the focus of most studies has been the development of feature extraction and similarity measurement algorithms with limited work on facilitating better understanding of the similarity between complex volumetric and multi-modality medical images. In this paper, we present a method for defining user interfaces (UIs) that enable effective human user interpretation of retrieved images.

Methods We derived a set of visualisation and interaction requirements based on the characteristics of modern volumetric medical images. We implemented a UI that visualised multiple views of a single image, displayed abstractions of image data, and provided access to supplementary non-image data. We also defined interactions for refining the search and visually indicating the similarities between images. We applied the UI for the retrieval of multi-modality positron emission tomography and computed tomography (PET-CT) images. We conducted a user survey to evaluate the capabilities of our UI.

Results Our proposed method obtained a high rating (\(\ge \)4 out of 5) in the majority of survey questions. In particular, the survey responses indicated the UI presented all the information necessary to understand the retrieved images, and did so in an intuitive manner.

Conclusion Our proposed UI design improved the ability of users to interpret and understand the similarity between retrieved PET-CT images. The implementation of CBIR UIs designed to assist human interpretation could facilitate wider adoption of medical CBIR systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Abbreviations

CBIR:

Content-based image retrieval

CT:

Computed tomography

PACS:

Picture archiving and communications systems

PET:

Positron emission tomography

PET-CT:

Combined positron emission tomography and computed tomography

References

  1. Müller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications–clinical benefits and future directions. Int J Med Inform 73(1):1–23

    Google Scholar 

  2. Müller H, Zhou X, Depeursinge A, Pitkanen M, Iavindrasana J, Geissbuhler A (2007) Medical visual information retrieval: state of the art and challenges ahead. In: IEEE ICME, pp 683– 686

  3. Müller H, Kalpathy-Cramer J, Caputo B, Syeda-Mahmood T, Wang F (2010) Overview of the first workshop on medical content—based retrieval for clinical decision support at MICCAI 2009. In: LNCS, vol 5853, pp 1–17

  4. Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A, Korenblum D, Greenspan H, Ma Y, Rubin DL (2010) Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256(1): 243–252

    Google Scholar 

  5. Müller H, Rosset A, Garcia A, Vallée J-P, Geissbuhler A (2005) Benefits of content-based visual data access in radiology. Radiographics 25(3):849–858

    Article  PubMed  Google Scholar 

  6. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal 22(12):1349–1380

    Article  Google Scholar 

  7. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5: 1–5:60

    Google Scholar 

  8. Long LR, Antani S, Deserno TM, Thoma GR (2009) Content-based image retrieval in medicine: retrospective assessment, state of the art, and future directions. Int J Health Info Syst Inform 4(1):1–16

    Article  Google Scholar 

  9. Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimed Comput 2(1):1–19

    Article  Google Scholar 

  10. Deserno T, Antani S, Long R (2009) Ontology of gaps in content-based image retrieval. J Digit Imaging 22:202–215

    Article  PubMed  Google Scholar 

  11. Shyu C-R, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS (1999) ASSERT: a physician-in-the-loop content-based retrieval system for HRCT image databases. Comput Vis Image Underst 75(1–2):111–132

    Article  Google Scholar 

  12. Quellec G, Lamard M, Cazuguel G, Cochener B, Roux C (2010) Wavelet optimization for content-based image retrieval in medical databases. Med Image Anal 14(2):227–241

    Article  PubMed  CAS  Google Scholar 

  13. Quellec G, Lamard M, Bekri L, Cazuguel G, Roux C, Cochener B (2010) Medical case retrieval from a committee of decision trees. IEEE Trans Inf Technol B 14(5):1227–1235

    Article  Google Scholar 

  14. Quellec G, Lamard M, Cazuguel G, Roux C, Cochener B (2011) Case retrieval in medical databases by fusing heterogeneous information. IEEE Trans Med Imaging 30(1):108–118

    Article  PubMed  Google Scholar 

  15. Townsend DW, Beyer T, Blodgett TM (2003) PET/CT scanners: a hardware approach to image fusion. Semin Nucl Med 33(3):193–204

    Article  PubMed  Google Scholar 

  16. Judenhofer MS, Catana C, Swann BK, Siegel SB, Jung W-I, Nutt RE, Cherry SR, Claussen CD, Pichler BJ (2007) PET/MR images acquired with a compact MR-compatible PET detector in a 7-T magnet. Radiology 244(3):807–814

    Article  PubMed  Google Scholar 

  17. Blodgett TM, Meltzer CC, Townsend DW (2007) PET/CT: form and function. Radiology 242(2):360–385

    Article  PubMed  Google Scholar 

  18. Deserno T, Güld M, Plodowski B, Spitzer K, Wein B, Schubert H, Ney H, Seidl T (2008) Extended query refinement for medical image retrieval. J Digit Imaging 21:280–289

    Google Scholar 

  19. Hsu W, Antani S, Long LR, Neve L, Thoma GR (2009) SPIRS: a web-based image retrieval system for large biomedical databases. Int J Med Inform 78(Supplement 1):S13–S24

    Article  PubMed  Google Scholar 

  20. Kumar A, Haraguchi D, Kim J, Wen L, Eberl S, Fulham M, Feng DD (2011) A query and visualisation interface for a PET-CT image retrieval system. Int J Comput Assist Radiol Surg 6(Supplement 1):69–69

    Google Scholar 

  21. Kumar A, Kim J, Feng D, Fulham M (2012) Graph-based retrieval of multi-modality medical images: a comparison of representations using simulated images. In: IEEE Symp CBMS, pp 1–6

  22. Kumar A, Kim J, Wen L, Feng D (2012) A graph-based approach to the retrieval of volumetric PET-CT lung images. In: IEEE EMBC, pp 5408–5411

  23. Hu S, Hoffman EA, Reinhardt JM (2001) Automatic lung segmentation for accurate quantitation of volumetric X-ray CT images. IEEE Trans Med Imaging 20(6):490–498

    Article  PubMed  CAS  Google Scholar 

  24. Bradley J, Thorstad WL, Mutic S, Miller TR, Dehdashti F, Siegel BA, Bosch W, Bertrand RJ (2004) Impact of FDG-PET on radiation therapy volume delineation in non-small-cell lung cancer. Int J Radiat Oncol Biol Phys 59(1):78–86

    Article  PubMed  Google Scholar 

  25. Petrakis EGM (2002) Design and evaluation of spatial similarity approaches for image retrieval. Image Vis Comput 20(1):59–76

    Article  Google Scholar 

  26. Neuhaus M, Riesen K, Bunke H (2006) Fast suboptimal algorithms for the computation of graph edit distance. In: LNCS, vol 4109, pp 163–172

  27. Tory M, Moller T (2004) Human factors in visualization research. IEEE Trans Vis Comput Graph 10(1):72–84

    Article  PubMed  Google Scholar 

  28. Wilson ML (2011) Search user interface design. Synth Lect Info Concepts Retr Serv 3(3):1–143

    CAS  Google Scholar 

  29. Bunke H (1999) Error correcting graph matching: on the influence of the underlying cost function. IEEE Trans Pattern Anal 21(9):917–922

    Article  Google Scholar 

  30. Abramoff MD, Magelhaes PJ, Ram SJ (2004) Image processing with ImageJ. Biophot Int 11(7):36–42

    Google Scholar 

  31. O’Madadhain J, Fisher D, White S, Boey Y (2003) The JUNG (java universal network/graph) framework. http://jung.sourceforge.net. Last updated: 24/01/2010, last accessed: 12/03/2013

  32. Nielsen J, Landauer TK (1993) A mathematical model of the finding of usability problems. In: Proc. INTERACT ’93 CHI ’93 Human Fact Comput Sys, pp 206–213

  33. Law EL-C, Roto V, Hassenzahl M, Vermeeren APOS, Kort J. (2009) Understanding, scoping and defining user experience: a survey approach. In: Proc. SIGCHI Conf. Human Factors, Comput. Syst., pp 719–728

  34. Aisen AM, Broderick LS, Winer-Muram H, Brodley CE, Kak AC, Pavlopoulou C, Dy J, Shyu C-R, Marchiori A (2003) Automated storage and retrieval of thin-section CT images to assist diagnosis: system description and preliminary assessment. Radiology 228(1):265–270

    Article  PubMed  Google Scholar 

Download references

Conflict of Interest

None.

Author information

Authors and Affiliations

  1. Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Sydney, Australia

    Ashnil Kumar, Jinman Kim, Lei Bi, Michael Fulham & Dagan Feng

  2. Department of Molecular Imaging, Royal Prince Alfred Hospital, Sydney, Australia

    Michael Fulham

  3. Sydney Medical School, University of Sydney, Sydney, Australia

    Michael Fulham

  4. Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China

    Dagan Feng

Authors
  1. Ashnil Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinman Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Fulham
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dagan Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashnil Kumar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kumar, A., Kim, J., Bi, L. et al. Designing user interfaces to enhance human interpretation of medical content-based image retrieval: application to PET-CT images. Int J CARS 8, 1003–1014 (2013). https://doi.org/10.1007/s11548-013-0896-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-013-0896-5

Keywords

Search

Navigation