Skip to main content
SpringerLink
Log in

Wearable Augmented Reality Optical See Through Displays Based on Integral Imaging

  • Conference paper
  • First Online:
Wireless Mobile Communication and Healthcare (MobiHealth 2016)

Abstract

In the context of Augmented Reality (AR), industrial pioneers and early adopters have considered the wearable optical see-through (OST) displays as proper and effective tools in applications spanning from manufacturing and maintenance up to the entertainment field and the medical area, because they provide the user with an egocentric viewpoint maintaining the quality of the visual perception of the real world.

The common OST displays paradigm entails intrinsic perceptual conflicts owing to mismatched accommodation between real 3D world and virtual 2D images projected over semitransparent surfaces. Such paradigm is suitable for augmenting the reality with simple virtual elements (models, icons or text), but various shortcomings remain in case of complex virtual contents. The major shortcoming is due to the tedious and error prone calibration methods required to obtain geometrical consistency, pivotal in many of the aforementioned fields of application. These shortcomings are due to the intrinsic incompatibility between the nature of the 4D light field, related to the real world, and the nature of the virtual content, rendered as a 2D image.

In this paper we describe a radical rethinking of the wearable OST displays paradigm by generating, through integral imaging technique, the virtual content as a light field, in order to overcome the typical limitations of the traditional approach. This paper describes the hardware components and an innovative rendering strategy in more details in respect to a previous work. Furthermore we report early results with the implementation of the integral imaging display using a lens array instead of a pinhole array.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Badiali, G., Ferrari, V., Cutolo, F., Freschi, C., Caramella, D., Bianchi, A., Marchetti, C.: Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J. Craniomaxillofac. Surg. 42, 1970–1976 (2014)

    Article  Google Scholar 

  2. Kersten-Oertel, M., Jannin, P., Collins, D.L.: The state of the art of visualization in mixed reality image guided surgery. Comput. Med. Imaging Graph. 37, 98–112 (2013)

    Article  Google Scholar 

  3. Cutolo, F., Badiali, G., Ferrari, V.: Human-PnP: ergonomic AR interaction paradigm for manual placement of rigid bodies. In: Linte, C.A., Yaniv, Z., Fallavollita, P. (eds.) AE-CAI 2015. LNCS, vol. 9365, pp. 50–60. Springer, Cham (2015). doi:10.1007/978-3-319-24601-7_6

    Chapter  Google Scholar 

  4. Rankin, T.M., Slepian, M.J., Armstrong, D.G.: Augmented reality in surgery. In: Latifi, R., Rhee, P., Gruessner, W.G.R. (eds.) Technological Advances in Surgery, Trauma and Critical Care, pp. 59–71. Springer, New York (2015)

    Chapter  Google Scholar 

  5. Ferrari, V., Cutolo, F., Calabrò, E.M., Ferrari, M.: HMD video see through AR with unfixed cameras vergence. In: International Symposium on Mixed and Augmented Reality – ISMAR (2014)

    Google Scholar 

  6. Cutolo, F., Parchi, P.D., Ferrari, V.: Video see through AR head-mounted display for medical procedures. In: International Symposium on Mixed and Augmented Reality – ISMAR (2014)

    Google Scholar 

  7. Ferrari, V., Megali, G., Troia, E., Pietrabissa, A., Mosca, F.: A 3-D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans. Biomed. Eng. 56, 2627–2633 (2009)

    Article  Google Scholar 

  8. Meola, A., Cutolo, F., Carbone, M., Cagnazzo, F., Ferrari, M., Ferrari, V.: Augmented reality in neurosurgery: a systematic review. Neurosurg. Rev. 1–12 (2016). doi:10.1007/s10143-016-0732-9 [epub ahead of print]

  9. Abe, Y., Sato, S., Kato, K., Hyakumachi, T., Yanagibashi, Y., Ito, M., Abumi, K.: A novel 3D guidance system using augmented reality for percutaneous vertebroplasty. J. Neurosurg. Spine 19, 492–501 (2013)

    Article  Google Scholar 

  10. Ferrari, V., Ferrari, M., Mosca, F.: Video see-through in the clinical practice. In: 1st International Workshop on Engineering Interactive Computing Systems for Medicine and Health Care, EICS4Med, vol. 727, pp. 19–24 (2011)

    Google Scholar 

  11. Ferrari, V.: Letter to the editor on “designing a wearable navigation system for image-guided cancer resection surgery”. Ann. Biomed. Eng. 42, 2600–2601 (2014)

    Article  Google Scholar 

  12. Cutolo, F., Carbone, M., Parchi, P.D., Ferrari, V., Lisanti, M., Ferrari, M.: Application of a new wearable augmented reality video see-through display to aid percutaneous procedures in spine surgery. In: Paolis, L.T., Mongelli, A. (eds.) AVR 2016. LNCS, vol. 9769, pp. 43–54. Springer, Cham (2016). doi:10.1007/978-3-319-40651-0_4

    Google Scholar 

  13. Parrini, S., Cutolo, F., Freschi, C., Ferrari, M., Ferrari, V.: Augmented reality system for freehand guide of magnetic endovascular devices. In: Conference Proceedings of IEEE Engineering in Medicine and Biology Society 2014, pp. 490–493 (2014)

    Google Scholar 

  14. Sielhorst, T., Feuerstein, M., Navab, N.: Advanced medical displays: a literature review of augmented reality. J. Disp. Technol. 4, 451–467 (2008)

    Article  Google Scholar 

  15. Holloway, R.L.: Registration error analysis for augmented reality. Presence Teleop. Virt. 6, 413–432 (1997)

    Article  Google Scholar 

  16. Benton, S.A.: Selected Papers on Three-Dimensional Displays. SPIE Optical Engineering Press, Bellingham (2001)

    Google Scholar 

  17. Rolland, J.P., Cakmakci, O.: The past, present, and future of head-mounted display designs, pp. 368–377 (2005)

    Google Scholar 

  18. Holliman, N.S., Dodgson, N.A., Favalora, G.E., Pockett, L.: Three-dimensional displays: a review and applications analysis. IEEE Trans. Broadcast. 57, 362–371 (2011)

    Article  Google Scholar 

  19. Mukawa, H., Akutsu, K., Matsumura, I., Nakano, S., Yoshida, T., Kuwahara, M., Aiki, K., Ogawa, M.: Distinguished paper: a full color eyewear display using holographic planar waveguides. SID Int. Symp. Dig. Tech. Papers 39, 89–92 (2008)

    Article  Google Scholar 

  20. Rolland, J.P., Holloway, R.L., Fuchs, H.: A comparison of optical and video see-through head-mounted displays. Telemanipulator Telepresence Technol. 2351, 293–307 (1994)

    Article  Google Scholar 

  21. Tuceryan, M., Genc, Y., Navab, N.: Single-point active alignment method (SPAAM) for optical see-through HMD calibration for augmented reality. Presence Teleop. Virt. 11, 259–276 (2002)

    Article  Google Scholar 

  22. Genc, Y., Tuceryan, M., Navab, N.: Practical solutions for calibration of optical see-through devices. In: International Symposium on Mixed and Augmented Reality, Proceedings, pp. 169–175 (2002)

    Google Scholar 

  23. Gilson, S.J., Fitzgibbon, A.W., Glennerster, A.: Spatial calibration of an optical see-through head-mounted display. J. Neurosci. Meth. 173, 140–146 (2008)

    Article  Google Scholar 

  24. Kellner, F., Bolte, B., Bruder, G., Rautenberg, U., Steinicke, F., Lappe, M., Koch, R.: Geometric calibration of head-mounted displays and its effects on distance estimation. IEEE Trans. Vis. Comput. Graph. 18, 589–596 (2012)

    Article  Google Scholar 

  25. Plopski, A., Itoh, Y., Nitschke, C., Kiyokawa, K., Klinker, G., Takemura, H.: Corneal-imaging calibration for optical see-through head-mounted displays. IEEE Trans. Vis. Comput. Graph. 21, 481–490 (2015)

    Article  Google Scholar 

  26. Levoy, M.: Light fields and computational imaging. Computer 39, 46–55 (2006)

    Article  Google Scholar 

  27. Hua, H., Javidi, B.: A 3D integral imaging optical see-through head-mounted display. Opt. Express 22, 13484–13491 (2014)

    Article  Google Scholar 

  28. Ferrari, V., Calabrò, E.M.: Wearable light field optical see-through display to avoid user dependent calibrations: a feasibility study. In: SAI Computing Conference (2016)

    Google Scholar 

  29. Liao, H., Inomata, T., Sakuma, I., Dohi, T.: 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans. Biomed. Eng. 57, 1476–1486 (2010)

    Article  Google Scholar 

  30. Stern, A., Javidi, B.: Three-dimensional image sensing, visualization, and processing using integral imaging. Proc. IEEE 94, 591–607 (2006)

    Article  Google Scholar 

  31. Lippmann, G.: Epreuves reversibles donnant la sensation du relief. J. Phys. Theor. Appl. 7, 821–825 (1908)

    Article  Google Scholar 

  32. Park, S.-G., Yeom, J., Jeong, Y., Chen, N., Hong, J.-Y., Lee, B.: Recent issues on integral imaging and its applications. J. Inf. Disp. 15, 37–46 (2014)

    Article  Google Scholar 

  33. Wang, J., Suenaga, H., Liao, H., Hoshi, K., Yang, L., Kobayashi, E., Sakuma, I.: Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation. Comput. Med. Imaging Graph. 40, 147–159 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. EndoCAS, University of Pisa, Via Paradisa, 2, 56124, Pisa, Italy

    Emanuele Maria Calabrò, Fabrizio Cutolo, Marina Carbone & Vincenzo Ferrari

  2. Department of Information Engineering, University of Pisa, Via Caruso 15, 56122, Pisa, Italy

    Vincenzo Ferrari

Authors
  1. Emanuele Maria Calabrò
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabrizio Cutolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marina Carbone
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vincenzo Ferrari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabrizio Cutolo .

Editor information

Editors and Affiliations

  1. Polytechnic University of Milan, Milan, Italy

    Paolo Perego

  2. Polytechnic University of Milan, Milan, Italy

    Giuseppe Andreoni

  3. Institute of Molecular Bioimaging and Physiology, Segrate, Italy

    Giovanna Rizzo

Rights and permissions

Reprints and permissions

Copyright information

© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Cite this paper

Calabrò, E.M., Cutolo, F., Carbone, M., Ferrari, V. (2017). Wearable Augmented Reality Optical See Through Displays Based on Integral Imaging. In: Perego, P., Andreoni, G., Rizzo, G. (eds) Wireless Mobile Communication and Healthcare. MobiHealth 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 192. Springer, Cham. https://doi.org/10.1007/978-3-319-58877-3_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-58877-3_44

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-58876-6

  • Online ISBN: 978-3-319-58877-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation