Skip to main content
SpringerLink
Log in

A Robust Analog VLSI Motion Sensor Based on the Visual System of the Fly

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Sensing visual motion gives a creature valuable information about its interactions with the environment. Flies in particular use visual motion information to navigate through turbulent air, avoid obstacles, and land safely. Mobile robots are ideal candidates for using this sensory modality to enhance their performance, but so far have been limited by the computational expense of processing video. Also, the complex structure of natural visual scenes poses an algorithmic challenge for extracting useful information in a robust manner. We address both issues by creating a small, low-power visual sensor with integrated analog parallel processing to extract motion in real-time. Because our architecture is based on biological motion detectors, we gain the advantages of this highly evolved system: A design that robustly and continuously extracts relevant information from its visual environment. We show that this sensor is suitable for use in the real world, and demonstrate its ability to compensate for an imperfect motor system in the control of an autonomous robot. The sensor attenuates open-loop rotation by a factor of 31 with less than 1 mW power dissipation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adelson, E.H. and Bergen, J.R. 1985. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am., A 2:284-299.

    Google Scholar 

  • Borst, A. 1990. How do flies land? BioScience, 40:292-299.

    Google Scholar 

  • Borst, A. and Egelhaaf, M. 1989. Principles of visual motion detection. Trends Neurosci., 12:297-306.

    Google Scholar 

  • Borst, A. and Egelhaaf, M. 1990. Direction selectivity of blowfly motion-sensitive neurons is computed in a two-stage process. Proc. Natl. Acad. Sci. USA, 87:9363-9367.

    Google Scholar 

  • Collett, T.S. 1980. Some operating rules for the optomotor system of a hoverfly during voluntary flight. J. Comp. Physiol., 138:271-282.

    Google Scholar 

  • Delbrück, T. 1993. Silicon retina with correlation-based, velocity-tuned pixels. IEEE Trans. Neural Networks, 4:529-541.

    Google Scholar 

  • Delbrück, T. and Mead, C.A. 1996. Analog VLSI phototransduction by continuous-time, adaptive, logarithmic photoreceptor circuits. CNS Memo 30, California Institute of Technology.

  • Dong, D.W. and Atick, J.J. 1995. Statistics of natural time-varying images. Network, 6:345-358.

    Google Scholar 

  • Douglas, R., Mahowald, M., and Mead, C. 1995. Neuromorphic analogue VLSI. Ann. Rev. Neurosci., 18:255-281.

    Google Scholar 

  • Duchon, A.P., Kaelbling, L.P., and Warren, W.H. 1998. Ecological robotics. Adaptive Behavior, 6:473-507.

    Google Scholar 

  • Egelhaaf, M. 1985. On the neuronal basis of figure-ground discrimination by relative motion in the visual system of the fly: II. figure-detection cells, a new class of visual interneurones. Biol. Cybern., 52:195-209.

    Google Scholar 

  • Egelhaaf, M. 1987. Dynamic properties of two control-systems underlying visually guided turning in house-flies. J. Comp. Physiol., A 161:777-783.

    Google Scholar 

  • Egelhaaf, M. and Borst, A. 1989. Transient and steady-state response properties of movement detectors. J. Opt. Soc. Am., A 6:116-127.

    Google Scholar 

  • Egelhaaf, M. and Borst, A. 1993. A look into the cockpit of the fly: visual orientation, algorithms, and identified neurons. J. Neurosci., 13:4563-4574.

    Google Scholar 

  • Egelhaaf, M., Borst, A., and Reichardt, W. 1989. Computational structure of a biological motion-detection system as revealed by local detector analysis in the fly's nervous system. J. Opt. Soc. Am., A 6:1070-1087.

    Google Scholar 

  • Etienne-Cummings, R. and Van der Spiegel, J. 1996. Neuromorphic vision sensors. Sensors and Actuators, A 56:19-29.

    Google Scholar 

  • Field, D.J. 1987. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am., A 4:2379-2394.

    Google Scholar 

  • Franceschini, N., Pichon, J.M., and Blanes, C. 1992. From insect vision to robot vision. Phil. Trans. R. Soc., B 337:283-294.

    Google Scholar 

  • Gibson, J.J. 1950. The Perception of the Visual World., Houghton-Mifflin: Boston.

    Google Scholar 

  • Götz, K.G. 1975. The optomotor equilibrium of the Drosophila navigation system. J. Comp. Physiol., 99:187-210.

    Google Scholar 

  • Haag, J. and Borst, A. 1997. Encoding of visual motion information and reliability in spiking and graded potential neurons. J. Neurosci., 17:4809-4819.

    Google Scholar 

  • Harrison, R.R. and Koch, C. 1998. An analog VLSI model of the fly elementary motion detector. In Advances in Neural Information Processing Systems 10. M.I. Jordan, M.J. Kearns, and S.A. Solla (Eds), MIT Press: Cambridge, Mass., pp. 880-886.

    Google Scholar 

  • Hassenstein, B. and Reichardt, W. 1956. Systemtheoretische Analyse der Zeit-, Reihenfolgen-, und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers, Chlorophanus. Z. Naturforsch., 11b:513-524.

    Google Scholar 

  • Hatsopoulos, N., Gabbiani, F., and Laurent, G. 1995. Elementary computation of object approach by a wide-field visual neuron. Science, 270:1000-1003.

    Google Scholar 

  • Hausen, K. and Egelhaaf, M. 1989. Neural mechanisms of visual course control in insects. In Facets of Vision, D.G. Stavenga and R.C. Hardie (Eds.), Berlin: Springer-Verlag.

    Google Scholar 

  • Higgins, C., Deutschmann, R., and Koch, C. 1999. Pulse-based 2D motion sensors. IEEE Trans. Circuits and Systems II, 46:677-687.

    Google Scholar 

  • Huber, S.A. 1997. Studies of the visual orientation behavior in flies using the artificial life approach. Ph.D. Thesis, Universität Tübingen, Germany.

    Google Scholar 

  • Kandel, E.R., Schwartz, J.H., Jessell, T.M. 1991. Principles of Neural Science, Appleton & Lange: Norwalk, Conn.

    Google Scholar 

  • Kern, R., Egelhaaf, M., and Srinivasan, M.V. 1997. Edge detection by landing honeybees: behavioural analysis and model simulations of the underlying mechanism. Vis. Research, 37:2103-2117.

    Google Scholar 

  • Kimmerle, B., Warzecha, A.-K., and Egelhaaf, M. 1997. Object detection in the fly during simulated translatory flight. J. Comp. Physiol., A 181:247-255.

    Google Scholar 

  • Krapp, H.G. and Hengstenberg, R. 1996. Estimation of self-motion by optic flow processing in single visual interneurons. Nature, 384:463-466.

    Google Scholar 

  • Lewis, M.A. 1998. Visual navigation in a robot using zig-zag behavior. In Advances in Neural Information Processing Systems 10, M.I. Jordan, M.J. Kearns, and S.A. Solla (Eds.), MIT Press: Cambridge, Mass., pp. 822-828.

    Google Scholar 

  • Matthies, L., Gat, E., Harrison, R., Wilcox, B., Volpe, R., and Litwin, T. 1995. Mars microrover navigation: performance evaluation and enhancement. Autonomous Robots, 2:291-311.

    Google Scholar 

  • Mead, C. 1989. Analog VLSI and Neural Systems, Reading, Mass.: Addison-Wesley.

    Google Scholar 

  • Moini, A., Bouzerdoum, A., Eshraghian, K., Yakovleff, A., Nguyen, X.T., Blanksby, A., Beare, R., Abbott, D., and Bonger, R.E. 1997. An insect vision-based motion detection chip. IEEE J. Solid State Circuits, 32:279-284.

    Google Scholar 

  • O'Carroll, D.C., Bidwell, N.J., Lauglin, S.B., and Warrant, E.J. 1996. Insect motion detectors matched to visual ecology. Nature, 382:63-66.

    Google Scholar 

  • Reichardt, W., Poggio, T., and Hausen, K. 1983. Figure-ground discrimination by relative movement in the visual system of the fly, part II: towards the neural circuitry. Biol. Cybern., 46:1-30.

    Google Scholar 

  • Reichardt, W. and Egelhaaf, M. 1988. Properties of individual movement detectors as derived from behavioral experiments on the visual system of the fly. Biol. Cybern., 58:287-294.

    Google Scholar 

  • Ruderman, D.L. and Bialek, W. 1994. Statistics of natural images: scaling in the woods. Phys. Rev. Lett., 73:814-817.

    Google Scholar 

  • Sarpeshkar, R., Kramer, J., Indiveri, G., and Koch, C. 1996. Analog VLSI architectures for motion processing: from fundamental limits to system applications. Proc. of the IEEE, 84:969-987.

    Google Scholar 

  • Single, S. and Borst, A. 1998. Dendritic integration and its role in computing image velocity. Science, 281:1848-1850.

    Google Scholar 

  • Srinivasan, M.V., Chahl, J.S., and Zhang, S.W. 1997. Robot navigation by visual dead-reckoning: inspiration from insects. Intl. J. Patt. Recog. and Art. Intelligence, 11:35-47.

    Google Scholar 

  • Warzecha, A.-K. and Egelhaaf, M. 1996. Intrinsic properties of biological motion detectors prevent the optomotor control system from getting unstable. Phil. Trans. R. Soc., B 351:1579-1591.

    Google Scholar 

  • Weckström, M., Juusola, M., and Laughlin, S.B. 1992. Presynaptic enhancement of signal transients in photoreceptor terminals in the compound eye. Proc. R. Soc. Lond., B 250:83-89.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computation and Neural Systems Program, MS 139-74, California Institute of Technology, Pasadena, CA, 91125, USA

    Reid R. Harrison & Christof Koch

Authors
  1. Reid R. Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christof Koch
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Harrison, R.R., Koch, C. A Robust Analog VLSI Motion Sensor Based on the Visual System of the Fly. Autonomous Robots 7, 211–224 (1999). https://doi.org/10.1023/A:1008916202887

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008916202887

Search

Navigation