Skip to main content
SpringerLink
Log in

Multimedia Information Retrieval from Ophthalmic Digital Archives

  • Chapter
Health Monitoring and Personalized Feedback using Multimedia Data

Abstract

During a medical examination, clinicians build a health record containing all available information about their patient. A promising way to support their decisions is to retrieve similar patient records from a medical archive. Confronted to similar cases, clinicians may confirm or revise their decisions by analogy reasoning. In order to retrieve patient records, two challenges need to be addressed. First, how to characterize complex elements in patient records (images, videos, etc.)? Second, how to combine heterogeneous elements in these records (demographic and clinical data, images, videos, etc.) in order to define clinically-relevant similarity metrics? After a short review of content-based image, video or health record retrieval techniques, this chapter presents the solutions we have developed for two applications in ophthalmology: computer-aided retinal diagnosis and computer-aided eye surgery. Medical archives are a great asset to develop the medical decision supports of tomorrow. Thanks to major advances in information retrieval, network data storage (cloud), with related topics such as security, virtually any medical decision problem can benefit from information stored in medical archives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://medgift.hevs.ch/.

  2. 2.

    http://ganymed.imib.rwth-aachen.de/irma/index_en.php.

  3. 3.

    www.medalytix.com.

  4. 4.

    www.diagnos.ca/cara.

  5. 5.

    www.eyediagnosis.net.

  6. 6.

    www.hubbletelemedical.com.

  7. 7.

    visionquest-bio.com/eyestar-tm.html.

  8. 8.

    http://teleophta.fr.

  9. 9.

    http://reseau-ophdiat.aphp.fr.

References

  1. Abouelenien, M., Wan, Y., & Saudagar, A. (2012). Feature and decision level fusion for action recognition. In Proceedings of International Conference on Computing, Communications and Networking Technologies (ICCCNT) (pp. 1–7).

    Google Scholar 

  2. Amores, J. (2013). Multiple instance classification: Review, taxonomy and comparative study. Artificial Intelligence, 201, 81–105.

    Article  MathSciNet  Google Scholar 

  3. André, B., Vercauteren, T., Buchner, A. M., Shahid, M. W., Wallace, M. B., & Ayache, N. (2010). An image retrieval approach to setup difficulty levels in training systems for endomicroscopy diagnosis. In Proceedings of Medical Image Computing and Computer Assisted Interventions (MICCAI) (pp. 480–487).

    Google Scholar 

  4. André, B., Vercauteren, T., Buchner, A. M., Wallace, M. B., & Ayache, N. (2012). Learning semantic and visual similarity for endomicroscopy video retrieval. IEEE Transactions on Medical Imaging, 31(6), 1276–1288.

    Article  Google Scholar 

  5. André, B., Vercauteren, T., Wallace, M. B., Buchner, A. M., & Ayache, N. (2010). Endomicroscopic video retrieval using mosaicing and visual words. In Proceedings of IEEE International Symposium on Biomedical Imaging (ISBI) (pp. 1419–1422).

    Google Scholar 

  6. Avni, U., Greenspan, H., Konen, E., Sharon, M., Goldberger, J. (2011). X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words. IEEE Transactions on Medical Imaging, 30(3), 733–746.

    Article  Google Scholar 

  7. Bettadapura, V., Schindler, G., Ploetz, T., & Essa, I. (2013). Augmenting bag-of-words: Data-driven discovery of temporal and structural information for activity recognition. In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (pp. 2619–2626).

    Google Scholar 

  8. Bichindaritz, I. (2006). Mémoire: A framework for semantic interoperability of case-based reasoning systems in biology and medicine. Artificial Intelligence in Medicine, 36(2), 177–192.

    Article  Google Scholar 

  9. Bichindaritz, I., & Marling, C. (2006). Case-based reasoning in the health sciences: What’s next? Artificial Intelligence in Medicine, 36(2), 127–135.

    Article  Google Scholar 

  10. Blum, T., Feussner, H., & Navab, N. (2010). Modeling and segmentation of surgical workflow from laparoscopic video. In Proceedings of Medical Image Computing and Computer Assisted Interventions (MICCAI) (pp. 400–407).

    Google Scholar 

  11. Bruno, E., Moenne-Loccoz, N., & Marchand-Maillet, S. (2008). Design of multimodal dissimilarity spaces for retrieval of video documents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(9), 1520–1533.

    Article  Google Scholar 

  12. Cauvin, J. M., Le Guillou, C., Solaiman, B., Robaszkiewicz, M., Le Beux, P., & Roux, C. (2003). Computer-assisted diagnosis system in digestive endoscopy. IEEE Transactions on Information Technology, 7(4), 256–262.

    Article  Google Scholar 

  13. Chatzichristofis, S. A., Iakovidou, C., Boutalis, Y., Marques, O. (2013). Co.Vi.Wo.: Color visual words based on non-predefined size codebooks. IEEE Transactions on Cybernetics, 43(1), 192–205.

    Article  Google Scholar 

  14. Decencière, E., Cazuguel, G., Zhang, X., et al. (2013). TeleOphta: Machine learning and image processing methods for teleophthalmology. IRBM, 34(2), 196–203.

    Article  Google Scholar 

  15. Douze, M., Jégou, H., Schmid, C. (2010). An image-based approach to video copy detection with spatio-temporal post-filtering. IEEE Transactions on Multimedia, 12(4), 257–266.

    Article  Google Scholar 

  16. Droueche, Z., Lamard, M., Cazuguel, G., Quellec, G., Roux, C., & Cochener, B. (2011). Content-based medical video retrieval based on region motion trajectories. In Proceedings of International Federation for Medical and Biological Engineering (IFMBE) (pp. 622–625).

    Google Scholar 

  17. Droueche, Z., Lamard, M., Cazuguel, G., Quellec, G., Roux, C., & Cochener, B. (2012). Motion-based video retrieval with application to computer-assisted retinal surgery. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBS) (pp. 4962–4965).

    Google Scholar 

  18. Dyana, A., Subramanian, M. P., & Das, S. (2009). Combining features for shape and motion trajectory of video objects for efficient content based video retrieval. In Proceedings of International Conference on Advances in Pattern Recognition (ICAPR) (pp. 113–116).

    Google Scholar 

  19. Gao, H. P., & Yang, Z. Q. (2010). Content based video retrieval using spatiotemporal salient objects. In Proceedings of International Petroleum Technology Conference (IPTC) (pp. 689–692).

    Google Scholar 

  20. Haro, B. B., Zappella, L., & Vidal, R. (2012). Surgical gesture classification from video data. In Proceedings of Medical Image Computing and Computer Assisted Interventions (MICCAI) (pp. 34–41).

    Google Scholar 

  21. Haux, R. (2006). Health information system: Past, present, and future. International Journal of Medical Informatics, 75(3–4), 268–281.

    Article  Google Scholar 

  22. Hoi, S. C. H., & Lyu, M. R. (2007). A multimodal and multilevel ranking framework for content-based video retrieval. In Proceedings of International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1225–1228)

    Google Scholar 

  23. Hu, W., Xie, D., Fu, Z., Zeng, W., & Maybank, S. (2007). Semantic-based surveillance video retrieval. IEEE Transactions on Image Processing, 16(4), 1168–1181.

    Article  MathSciNet  Google Scholar 

  24. Ji, R., Duan, L. Y., Chen, J., Xie, L., Yao, H., & Gao, W. (2013). Learning to distribute vocabulary indexing for scalable visual search. IEEE Transactions on Multimedia, 15(1), 153–166.

    Article  Google Scholar 

  25. Juan, K., & Cuiying, H. (2010). Content-based video retrieval system research. In Proceedings of International Conference on Computer Science and Information Technology (ICCSIT) (pp. 701–704).

    Google Scholar 

  26. Lalys, F., Riffaud, L., Bouget, D., & Jannin, P. (2012). A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Transactions on Biomedical Engineering, 59(4), 966–976.

    Article  Google Scholar 

  27. Liu, Z., Li, H., Zhou, W., Zhao, R., & Tian, Q. (2014). Contextual hashing for large-scale image search. IEEE Transactions on Image Processing, 23(4), 1606–1614.

    Article  MathSciNet  Google Scholar 

  28. Mansencal, B., Benois-Pineau, J., Vieux, R., & Domenger, J. (2012). Search of objects of interest in videos. In Proceedings of Content-Based Multimedia Indexing (CBMI) (pp. 1–6).

    Google Scholar 

  29. Müller, H., Michoux, N., Bandon, D., & Geissbuhler, A. (2004). A review of content-based image retrieval systems in medical applications—clinical benefits and future directions. International Journal of Medical Informatics, 73(1), 1–23.

    Article  Google Scholar 

  30. Müller, H., Seco de Herrera, A. G., Kalpathy-Cramer, J., Fushman, D. D., Antani, S., & Eggel, I. (2012). Overview of the ImageCLEF 2012 medical image retrieval and classification tasks. In Conference and Labs of the Evaluation Forum (CLEF) 2012 working notes.

    Google Scholar 

  31. Naturel, X., & Gros, P. (2008). Detecting repeats for video structuring. Multimedia Tools and Applications, 38(2), 233–252.

    Article  Google Scholar 

  32. Niemeijer, M., van Ginneken, B., Cree, M. J., et al. (2010). Retinopathy online challenge: Automatic detection of microaneurysms in digital color fundus photographs. IEEE Transactions on Medical Imaging, 29(1), 185–195.

    Article  Google Scholar 

  33. Pan, W., Coatrieux, G., Cuppens, N., Cuppens, F., & Roux, C. (2010). An additive and lossless watermarking method based on invariant image approximation and haar wavelet transform. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBS) (pp. 4740–4743).

    Google Scholar 

  34. Patel, B. V., Deorankar, A. V., & Meshram, B. B. (2010). Content based video retrieval using entropy, edge detection, black and white color features. In Proceedings of International Conference on Chemical Engineering and Technology (ICCET) (pp. 272–276).

    Google Scholar 

  35. Perner, P. (Ed.). (2008). Case-based reasoning on images and signals. Studies in Computational Intelligence (Vol. 73). Heidelberg: Springer.

    Google Scholar 

  36. Pires, R., Jelinek, H. F., Wainer, J., Goldenstein, S., Valle, E., & Rocha, A. (2013). Assessing the need for referral in automatic diabetic retinopathy detection. IEEE Transactions on Biomedical Engineering, 60(12), 3391–3398.

    Article  Google Scholar 

  37. Quantin, C., Cohen, O., Riandey, B., & Allaert, F. A. (2007). Unique patient concept: A key choice for european epidemiology. International Journal of Medical Informatics, 76(5–6), 419–426.

    Article  Google Scholar 

  38. Quellec, G., Charrière, K., Lamard, M., Droueche, Z., Roux, C., & Cochener, B. (2014). Real-time recognition of surgical tasks in eye surgery videos. Medical Image Analysis, 18(3), 579–590.

    Article  Google Scholar 

  39. Quellec, G., Lamard, M., Abràmoff, M. D., Decencière, E., Lay, B., & Erginay, A. (2012). A multiple-instance learning framework for diabetic retinopathy screening. Medical Image Analysis, 16(6), 1228–1240.

    Article  Google Scholar 

  40. Quellec, G., Lamard, M., Bekri, L., Cazuguel, G., Roux, C., & Cochener, B. (2010). Medical case retrieval from a committee of decision trees. IEEE Transactions on Information Technology in Biomedicine,14(5), 1227–1235.

    Google Scholar 

  41. Quellec, G., Lamard, M., Cazuguel, G., Bekri, L., Daccache, W., & Roux, C. (2011). Automated assessment of diabetic retinopathy severity using content-based image retrieval in multimodal fundus photographs. Investigative Ophthalmology and Visual Science, 52(11), 8342–8348.

    Article  Google Scholar 

  42. Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2009). Multimodal information retrieval based on DSmT. Application to computer aided medical diagnosis. In F. Smarandache & J. Dezert (Eds.), Advances and applications of DSmT for information fusion III, chap. 18 (pp. 471–502). Ann Harbor: American Research Press.

    Google Scholar 

  43. Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2010). Adaptive nonseparable wavelet transform via lifting and its application to content-based image retrieval. IEEE Transactions on Image Processing, 19(1), 25–35.

    Article  MathSciNet  Google Scholar 

  44. Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2010). Wavelet optimization for content-based image retrieval in medical databases. Medical Image Analysis, 14(2), 227–241.

    Article  Google Scholar 

  45. Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2012). Fast wavelet-based image characterization for highly adaptive image retrieval. IEEE Transactions on Image Processing, 21(4), 1613–1623.

    Article  MathSciNet  Google Scholar 

  46. Quellec, G., Lamard, M., Cazuguel, G., Roux, C., & Cochener, B. (2008). Multimodal medical case retrieval using dezert-smarandache theory with a priori knowledge. In Proceedings of International Federation for Medical and Biological Engineering (IFMBE) (pp. 716–719).

    Google Scholar 

  47. Quellec, G., Lamard, M., Cazuguel, G., Roux, C., & Cochener, B. (2011). Case retrieval in medical databases by fusing heterogeneous information. IEEE Transactions on Medical Imaging, 30(1), 108–118.

    Article  Google Scholar 

  48. Quellec, G., Lamard, M., Cochener, B., & Cazuguel, G. (2014). Real-time segmentation and recognition of surgical tasks in cataract surgery videos. IEEE Trans Med Imaging, 33(12), 2352–2360.

    Article  Google Scholar 

  49. Quellec, G., Lamard, M., Cochener, B., Droueche, Z., Lay, B., & Chabouis, A. et al. (2012). Studying disagreements among retinal experts through image analysis. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBS) (pp. 5959–5962).

    Google Scholar 

  50. Quellec, G., Lamard, M., Cochener, B., Roux, C., & Cazuguel, G. (2012). Comprehensive wavelet-based image characterization for content-based image retrieval. In Proceedings of the Conference on Content-Based Multimedia Indexing (CBMI).

    Google Scholar 

  51. Quellec, G., Lamard, M., Droueche, Z., Cochener, B., Roux, C., & Cazuguel, G. (2013). A polynomial model of surgical gestures for real-time retrieval of surgery videos. In Lecture Notes in Computer Science: Vol. 7723. Proceedings MCBR-CDS (pp. 10–20).

    Google Scholar 

  52. Ren, R., & Collomosse, J. (2012). Visual sentences for pose retrieval over low-resolution cross-media dance collections. IEEE Transactions on Multimedia, 14(6), 1652–1661.

    Article  Google Scholar 

  53. Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. R. (2011). ORB: An efficient alternative to SIFT or SURF. In Proceedings of IEEE International Conference on Computer Vision (ICCV) (pp. 2564–2571).

    Google Scholar 

  54. Safran, C., Bloomrosen, M., Hammond, W. E., et al. (2007). Toward a national framework for the secondary use of health data: An american medical informatics association white paper. Journal of the American Medical Informatics Association, 14(1), 1–9.

    Article  Google Scholar 

  55. Sivic, J., Russell, B. C., Efros, A. A., Zisserman, A., & Freeman, W. T. (2005). Discovering objects and their location in images. In Proceedings of IEEE International Conference on Computer Vision (ICCV) (pp. 370–377).

    Google Scholar 

  56. Strat, S. T., Benoit, A., & Lambert, P. (2013). Retina enhanced SIFT descriptors for video indexing. In Proceedings of the Conference on Content-Based Multimedia Indexing (CBMI) (pp. 201–206).

    Google Scholar 

  57. Sweldens, W. (1998). The lifting scheme: A construction of second generation wavelets. SIAM Journal on Mathematical Analysis, 29(2), 511–546.

    Article  MATH  MathSciNet  Google Scholar 

  58. Syeda-Mahmood, T., Ponceleon, D., & Yang, J. (2005). Validating cardiac echo diagnosis through video similarity. In Proceedings of ACM Multimedia (pp. 527–530).

    Google Scholar 

  59. Tao, L., Elhamifar, E., Khudanpur, S., Hager, G. D., & Vidal, R. (2012). Sparse hidden markov models for surgical gesture classification and skill evaluation. In Proceedings of Information Processing in Computer-Assisted Interventions (IPCAI) (pp. 167–177).

    Google Scholar 

  60. Tao, L., Zappella, L., Hager, G. D., & Vidal, R. (2013). Surgical gesture segmentation and recognition. In Lecture Notes in Computer Science: Vol. 8151 (pp. 339–46).

    Google Scholar 

  61. Tsikrika, T., Kludas, J., & Popescu, A. (2012). Building reliable and reusable test collections for image retrieval: The Wikipedia task at ImageCLEF. IEEE Multimedia, 19(3), 24–33.

    Article  Google Scholar 

  62. Tutac, A. E., Cretu, V. I., & Racoceanu, D. (2010). Spatial representation and reasoning in breast cancer grading ontology. In Proceedings of International Joint Conference on Computational Cybernetics and Technical Informatics (ICCC-CONTI) (pp. 89–94).

    Google Scholar 

  63. Vieux, R., Benois-Pineau, J., Domenger, J. P. (2012). Content based image retrieval using bag of regions. In Proceedings of Multimedia Modeling (MMM) (pp. 507–517).

    Google Scholar 

  64. Xu, D., & Chang, S. F. (2008). Video event recognition using kernel methods with multilevel temporal alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11), 1985–1997.

    Article  Google Scholar 

  65. Yang, Y., & Newsam, S. (2013). Geographic image retrieval using local invariant features. IEEE Transactions on Geoscience Remote Sensing, 51(2), 818–832.

    Article  Google Scholar 

  66. Yuan, C., Li, X., Hu, W., Ling, H., & Maybank, S. J. (2014) Modeling geometric-temporal context with directional pyramid co-occurrence for action recognition. IEEE Transactions on Image Processing, 23(2), 658–672.

    Article  MathSciNet  Google Scholar 

  67. Zappella, L., Béjar, B., Hager, G., & Vidal, R. (2013). Surgical gesture classification from video and kinematic data. Medical Image Analysis, 17(7), 732–745.

    Article  Google Scholar 

  68. Zheng, L., & Wang, S. (2013). Visual phraselet: refining spatial constraints for large scale image search. IEEE Signal Processing Letters, 20(4), 391–394.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Inserm, UMR 1101 LaTIM, SFR ScInBioS, F-29200, Brest, France

    Gwenolé Quellec

  2. Université de Bretagne Occidentale, F-29200, Brest, France

    Mathieu Lamard

  3. CHRU Brest, Service d’Ophtalmologie, F-29200, Brest, France

    Béatrice Cochener

  4. Institut Mines-Telecom, Telecom Bretagne, UEB, Dpt ITI, F-29200, Brest, France

    Guy Cazuguel

Authors
  1. Gwenolé Quellec
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathieu Lamard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Béatrice Cochener
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guy Cazuguel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gwenolé Quellec .

Editor information

Editors and Affiliations

  1. Centre for Research and Technology, Hellas, Information Technologies Institute, Thermi, Thessaloniki, Greece

    Alexia Briassouli

  2. Laboratoire Bordelais de Recherche en Informatique/University Bordeaux, Talence, France

    Jenny Benois-Pineau

  3. Computer Science Department, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

    Alexander Hauptmann

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Quellec, G., Lamard, M., Cochener, B., Cazuguel, G. (2015). Multimedia Information Retrieval from Ophthalmic Digital Archives. In: Briassouli, A., Benois-Pineau, J., Hauptmann, A. (eds) Health Monitoring and Personalized Feedback using Multimedia Data. Springer, Cham. https://doi.org/10.1007/978-3-319-17963-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-17963-6_6

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-17962-9

  • Online ISBN: 978-3-319-17963-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation