Skip to main content
SpringerLink
Log in

3D Shape Perception, Models of

  • Living reference work entry
  • First Online:
Encyclopedia of Computational Neuroscience

Definition

A 3D shape percept is the psychophysical response induced from a visual stimulus. The percept is the result of (partially) unknown neural processes within the visual system. Psychophysical tests of performance have mainly involved humans, and most relevant neurophysiological research (to date) has been performed on primates. The challenges are that (i) many different visual areas can be involved in producing a shape percept; and (ii) a single image rarely constrains the potential shape possibilities to a unique percept. Therefore, the visual system (and our computational models) must exploit assumptions to restrict the range and type of solutions. Since most models seek to return a single and, to some extent, veridical 3D shape percept, understanding both the assumptions needed and the mechanisms of computation are key components of research.

Detailed Description

An Ill-posed Inverse Problem

When a 3D object must be recovered from a single image of the object, the inverse...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Adato Y, Vasilyev Y, Zickler T, Ben-Shahar O (2010) Shape from specular flow. IEEE Trans Pattern Anal Mach Intell 32(11):2054–2070

    Article  Google Scholar 

  • Agin GJ, Binford TO (1976) Computer description of curved objects. IEEE Trans Comput 4:439–449

    Article  Google Scholar 

  • Barron JT, Malik J (2012) Shape, albedo, and illumination from a single image of an unknown object. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR), Providence, pp 334–341. https://doi.org/10.1109/CVPR.2012.6247693

  • Barron JL, Fleet DJ, Beauchemin SS (1994) Performance of optical flow techniques. Int J Comput Vis 12(1):43–77

    Article  Google Scholar 

  • Belhumeur P, Kriegman D (1998) What is the set of images of an object under all possible illumination conditions? Int J Comput Vis 28:1–16

    Article  Google Scholar 

  • Biederman I (1987) Recognition-by-components: a theory of human image understanding. Psychol Rev 94:115–147

    Article  CAS  Google Scholar 

  • Binford TO (1971) Visual perception by computer. In: Proceedings of the IEEE conference on systems and control, Miami

    Google Scholar 

  • Blum H (1973) Biological shape and visual science (part I). J Theor Biol 38(2):205–287

    Article  CAS  Google Scholar 

  • Breton P, Zucker SW (1996) Shadows and shading flow fields. In: IEEE computer society conference on computer vision and pattern recognition, 1996. Proceedings CVPR’96, San Francisco, 1996, IEEE, pp 782–789

    Google Scholar 

  • Britten KH, Shadlen MN, Newsome WT, Movshon JA (1992) The analysis of visual motion: a comparison of neuronal and psychophysical performance. J Neurosci 12(12):4745–4765

    Article  CAS  Google Scholar 

  • Brooks FP Jr (1995) The mythical man-month (anniversary ed.). Addison-Wesley Longman Publishing Co., Boston

    Google Scholar 

  • Bülthoff HH, Edelman S (1992) Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proc Natl Acad Sci 89(1):60–64

    Article  Google Scholar 

  • Collett M (2010) How desert ants use a visual landmark for guidance along a habitual route. Proc Natl Acad Sci 107(25):11638–11643

    Article  CAS  Google Scholar 

  • Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 6:681–685

    Article  Google Scholar 

  • Cormack LK, Czuba TB, Knll J, Huk AC (2017) Binocular mechanisms of 3d motion processing. Annu Rev Vis Sci 3(1):297–318. PMID: 28746813

    Article  Google Scholar 

  • Cutting JE (1986) Perception with an eye for motion, vol 1. Mit Press, Cambridge, MA

    Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), vol 1. IEEE Computer Society, Washington, DC, pp 886–893. ISBN:0-7695-2372-2. https://doi.org/10.1109/CVPR.2005.177

  • Deift P, Sylvester J (1981) Some remarks on the shape-from-shading problem in computer vision. J Math Anal Appl 84(1):235–248

    Article  Google Scholar 

  • Dickinson SJ, Pizlo Z (2013) Shape perception in human and computer vision: an interdisciplinary perspective. Springer Publishing Company, Incorporated. ISBN:1447151941

    Google Scholar 

  • Dickinson SJ, Pentland A, Rosenfeld A (1992) 3-d shape recovery using distributed aspect matching. IEEE Trans Pattern Anal Mach Intell 14(2):174–198

    Article  Google Scholar 

  • Eigen D, Puhrsch C, Fergus R (2014) Depth map prediction from a single image using a multi-scale deep network. In: Proceedings of the 27th international conference on neural information processing systems – vol 2, NIPS’14. MIT Press, Cambridge, MA, pp 2366–2374

    Google Scholar 

  • Faisman A, Langer MS (2013) Qualitative shape from shading, highlights, and mirror reflections. J Vis 13(5):10

    Article  Google Scholar 

  • Farley Norman J, Todd JT, Phillips F (1995) The perception of surface orientation from multiple sources of optical information. Percept Psychophys 57(5):629–636

    Article  Google Scholar 

  • Fleming RW, Torralba A, Adelson EH (2004) Specular reflections and the perception of shape. J Vis 4(9):10

    Article  Google Scholar 

  • Fleming R, Rice D-H, Bulthoff H (2011) Estimation of 3D shape from image orientations. Proc Natl Acad Sci 108(51):20438–20443. https://doi.org/10.1073/pnas.1114619109

  • Geman S, Graffigne C (1986) Markov random field image models and their applications to computer vision. In: Proceedings of the international congress of mathematicians, Berkeley

    Google Scholar 

  • Gibson JJ (1950) The perception of the visual world. Houghton Mifflin, Oxford

    Google Scholar 

  • Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin, Boston

    Google Scholar 

  • Gibson JJ (1979) The ecological approach to visual perception. Houghton Mifflin, Boston

    Google Scholar 

  • Gibson JJ, Cornsweet J (1952) The perceived slant of visual surfaces: optical and geographical. J Exp Psychol 44(1):11

    Article  CAS  Google Scholar 

  • Guzmán A (1968) Decomposition of a visual scene into three-dimensional bodies. In: Proceedings of the December 9–11, 1968, fall joint computer conference, part I, New York, AFIPS ‘68 (Fall, part I). ACM, pp 291–304

    Google Scholar 

  • Hoffman DD, Singh M, Prakash C (2015) The interface theory of perception. Psychon Bull Rev 22(6):1480–1506

    Article  Google Scholar 

  • Horn BKP (1970) Shape from shading: a method for obtaining a smooth opaque object from one view. PhD thesis

    Google Scholar 

  • Horn B (1986) Robot vision. The MIT Press, Cambridge, MA

    Google Scholar 

  • Horn B, Brooks M (1989) Shape from shading. The MIT Press, Cambridge, MA

    Google Scholar 

  • Howard IP, Rogers BJ (1995) Binocular vision and stereopsis. Oxford University Press, Oxford

    Google Scholar 

  • Hubel DH (1988) Eye, brain, and vision, vol 22. Scientific American Library, distributed by W. H. Freeman & Co., New York, 240pp

    Google Scholar 

  • Johansson G (1973) Visual perception of biological motion and a model for its analysis. Percept Psychophys 14(2):201–211

    Article  Google Scholar 

  • Khang B-G, Koenderink JJ, Kappers AML (2007) Shape from shading from images rendered with various surface types and light fields. Perception 36(8):1191–1213. PMID: 17972483

    Article  Google Scholar 

  • Kim J, Marlow PJ, Anderson BL (2014) Texture-shading flow interactions and perceived reflectance. J Vis 14(7):1

    Article  Google Scholar 

  • Kimia BB (2003) On the role of medial geometry in human vision. J Physiol Paris 97(2–3):155–190

    Article  Google Scholar 

  • Knill DC, Richards W (1996) Perception as Bayesian inference. Cambridge University Press, New York

    Google Scholar 

  • Koenderink JJ (1984) What does the occluding contour tell us about solid shape? Perception 13(3):321–330

    Article  CAS  Google Scholar 

  • Koenderink J (2011) Vision as a user interface

    Google Scholar 

  • Koenderink JJ, Van Doorn AJ (1980a) Photometric invariants related to solid shape. J Mod Opt 27(7):981–996

    Google Scholar 

  • Koenderink JJ, Van Doorn AJ (1980b) Photometric invariants related to solid shape. Opt Acta: Int J Optics 27(7):981–996

    Article  Google Scholar 

  • Koenderink JJ, Van Doorn AJ, Kappers AM (1992) Surface perception in pictures. Percept Psychophys 52(5):487–496

    Article  CAS  Google Scholar 

  • Koenderink J, van Doorn A, Wagemans J (2015) Part and whole in pictorial relief. Iperception 6(6):2041669515615713. https://doi.org/10.1177/2041669515615713

  • Koffka K (1955) Principles of Gestalt psychology. International library of psychology, philosophy, and scientific method. Routledge & K. Paul. https://books.google.com/books?id=KJkAAAAAMAAJ

  • Kolers PA (2013) Aspects of motion perception: international series of monographs in experimental psychology, vol 16. Elsevier. https://www.elsevier.com/books/aspects-of-motion-perception/kolers/978-0-08-016843-2

  • Kunsberg B, Zucker SW (2018) Critical contours: an invariant linking image flow with salient surface organization. SIAM J Imag Sci 11(3):1849–1877

    Article  Google Scholar 

  • Langer MS, Zucker SW (1994) Shape-from-shading on a cloudy day. J Opt Soc Am A 11(2):467–478

    Article  Google Scholar 

  • Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  CAS  Google Scholar 

  • Li G, Zucker S (2010) Differential geometric inference in surface stereo. IEEE Trans Pattern Anal Mach Intell 32:72–86

    Article  CAS  Google Scholar 

  • Liu F, Shen C, Lin G, Reid ID (2015) Learning depth from single monocular images using deep convolutional neural fields. CoRR abs/150207411. http://arxiv.org/abs/1502.07411, https://dblp.org/rec/bib/journals/corr/LiuSLR15

  • Logothetis NK, Sheinberg DL (1996) Visual object recognition. Annu Rev Neurosci 19(1):577–621. PMID: 8833455

    Article  CAS  Google Scholar 

  • Logothetis NK, Pauls J, Poggio T (1995) Shape representation in the inferior temporal cortex of monkeys. Curr Biol 5(5):552–563

    Article  CAS  Google Scholar 

  • Longuet-Higgins HC (1981) A computer algorithm for reconstructing a scene from two projections. Nat Neurosci 293:133–135

    Google Scholar 

  • Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the international conference on computer vision, vol 2. ICCV'99. IEEE Computer Society, Washington, DC, p 1150. ISBN:0-7695-0164-8, http://dl.acm.org/citation.cfm?id=850924.851523

  • Mach E (1965) On the physiological effect of spatially distributed light stimuli. Translation in Ratliff F, Mach Bands: Quantitative studies on neural networks in the retina

    Google Scholar 

  • Marr D (1982) Vision: a computational investigation into the human representation and processing of visual information. Henry Holt and Co., New York

    Google Scholar 

  • Metzger W, Senckenbergische Naturforschende Gesellschaft (1936) Gesetze des Sehens. Frankfurt am Main: W. Kramer & Co.

    Google Scholar 

  • Mooney S, Anderson B (2014) Specular image structure modulates the perception of three-dimensional shape. Curr Biol 24(22):2737–2742

    Article  CAS  Google Scholar 

  • Norman D (2002) The design of everyday things. Basic Books, Inc., New York. ISBN:9780465067107

    Google Scholar 

  • Pizlo Z (2008) 3D shape: its unique place in visual perception. The MIT Press. http://mitpress.mit.edu/catalog/item/default.asp?ttype=2\&\#38;tid=12396

    Google Scholar 

  • Pizlo Z, Li Y, Sawada T, Steinman RM (2014) Making a machine that sees like us. Oxford University Press

    Google Scholar 

  • Poggio T, Edelman S (1990) A network that learns to recognize three-dimensional objects. Nature 343:236–266

    Article  Google Scholar 

  • Poggio T, Torre V, Koch C (1985) Computational vision and regularization theory. Nature 317:314–319

    Article  CAS  Google Scholar 

  • Pylyshyn ZW (1986) Computation and cognition: toward a foundation for cognitive science. The MIT Press, Cambridge, MA

    Google Scholar 

  • Ramachandran VS (1988) Perception of shape from shading. Nature 331(6152):163

    Article  CAS  Google Scholar 

  • Reichardt W (1987) Evaluation of optical motion information by movement detectors. J Comp Physiol A 161(4):533–547

    Article  CAS  Google Scholar 

  • Rosenfeld A, Kak AC (1982) Digital image processing, vol 2, 2nd edn. Academic, New York

    Google Scholar 

  • Saxena A, Chung SH, Ng AY (2005) Learning depth from single monocular images. In: NIPS 18. MIT Press

    Google Scholar 

  • Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1):7–42

    Article  Google Scholar 

  • Seitz SM, Curless B, Diebel J, Scharstein D, Szeliski R (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the 2006 I.E. computer society conference on computer vision and pattern recognition – vol 1, CVPR ‘06. IEEE Computer Society, Washington, DC, pp 519–528

    Google Scholar 

  • Serre T, Wolf L, Bileschi S, Riesenhuber M, Poggio T (2007) Robust object recognition with cortex-like mechanisms. IEEE Trans Pattern Anal Mach Intell 3:411–426

    Article  Google Scholar 

  • Shepard RN, Metzler J (1971) Mental rotation of three-dimensional objects. Science 171(3972):701–703

    Article  CAS  Google Scholar 

  • Siddiqi K, Pizer S (2008) Medial representations: mathematics, algorithms and applications, 1st edn. Springer Publishing Company, Incorporated. ISBN:1402086571

    Google Scholar 

  • Stevens KA (1983) Slant-tilt: the visual encoding of surface orientation. Biol Cybern 46(3):183–195

    Article  CAS  Google Scholar 

  • Sun P, Schofield AJ (2012) Two operational modes in the perception of shape from shading revealed by the effects of edge information in slant settings. J Vis 12(1):12

    Article  Google Scholar 

  • Tanaka K (1996) Inferotemporal cortex and object vision. Annu Rev Neurosci 19(1):109–139

    Article  CAS  Google Scholar 

  • Todd JT, Norman JF (2018) The visual perception of metal. J Vis 18:3–9

    Article  Google Scholar 

  • Tomasi C, Kanade T (1992) Shape and motion from image streams under orthography: a factorization method. Int J Comput Vis 9(2):137–154

    Article  Google Scholar 

  • Ullman S (1979) The interpretation of structure from motion. Proc R Soc Lond B: Biol Sci 203(1153):405–426. https://doi.org/10.1098/rspb.1979.0006

  • Ullman S (1980) Against direct perception. Behav Brain Sci 3(3):373–381

    Article  Google Scholar 

  • Ullman S (1989) Aligning pictorial descriptions: an approach to object recognition. Cognition 32(3):193–254

    Article  CAS  Google Scholar 

  • Vergne R, Barla P, Fleming R, Granier X (2012) Surface flows for image-based shading design. ACM Trans Graph 31(3):1–9

    Article  Google Scholar 

  • von Helmholtz H (1962) Helmholtz’s treatise on physiological optics. Dover Publications, New York. http://www.worldcat.org/title/helmholtzs-treatise-on-physiological-optics/oclc/8110065

  • Waltz D (1975) Understanding line drawings of scenes with shadows. In: The psychology of computer vision. McGraw-Hill

    Google Scholar 

  • Warren WH (2012) Does this computational theory solve the right problem? Marr, Gibson, and the goal of vision. Perception 41(9):1053–1060. PMID: 23409371

    Article  Google Scholar 

  • Witkin A (1980) Shape from contour. PhD thesis

    Google Scholar 

  • Witkin A (1981) Recovering surface shape and orientation from texture. Artif Intell 17(1–3):17–45

    Article  Google Scholar 

  • Wu J, Wang Y, Xue T, Sun X, Freeman WT, Tenenbaum JB (2017) MarrNet: 3D shape reconstruction via 2.5D sketches. CoRR. http://arxiv.org/abs/1711.03129, https://dblp.org/rec/bib/journals/corr/abs-1711-03129

  • Yamane Y, Carlson ET, Bowman KC, Wang Z, Connor CE (2008) A neural code for three-dimensional object shape in macaque inferotemporal cortex. Nat Neurosci 11:1352–1360. https://doi.org/10.1038/nn.2202

  • Yamins DL, Dicarlo JJ (2016) Using goal-driven deep learning models to understand sensory cortex. Nat Neurosci 19(3):356

    Article  CAS  Google Scholar 

  • Yuille A, Kersten D (2006) Vision as bayesian inference: analysis by synthesis? Trends Cogn Sci 10(7):301–308

    Article  Google Scholar 

  • Zhang R, Tsai P, Cryer JE, Shah M (1999) Shape from shading: a survey. IEEE Trans Pattern Anal Mach Intell 21:690–706

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Applied Mathematics, Brown University, Providence, RI, USA

    Benjamin Kunsberg

  2. Department of Computer Science, Yale University, New Haven, CT, USA

    Steven W. Zucker

Authors
  1. Benjamin Kunsberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steven W. Zucker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Kunsberg .

Editor information

Editors and Affiliations

  1. Atlanta, GA, USA

    Dieter Jaeger

  2. Department of Biomedical Engineering, Florida International University, Miami, FL, USA

    Ranu Jung

Section Editor information

  1. Institute for Brain Sciences, Brown University, Providence, RI, USA

    Thomas Serre

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Science+Business Media, LLC, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Kunsberg, B., Zucker, S.W. (2018). 3D Shape Perception, Models of. In: Jaeger, D., Jung, R. (eds) Encyclopedia of Computational Neuroscience. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7320-6_100661-1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7320-6_100661-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-7320-6

  • Online ISBN: 978-1-4614-7320-6

  • eBook Packages: Springer Reference Biomedicine and Life SciencesReference Module Biomedical and Life Sciences

Publish with us

Policies and ethics

Search

Navigation