Abstract
Highly-interconnected networks of nonlinear analog neurons are shown to be extremely effective in computing. The networks can rapidly provide a collectively-computed solution (a digital output) to a problem on the basis of analog input information. The problems to be solved must be formulated in terms of desired optima, often subject to constraints. The general principles involved in constructing networks to solve specific problems are discussed. Results of computer simulations of a network designed to solve a difficult but well-defined optimization problem-the Traveling-Salesman Problem-are presented and used to illustrate the computational power of the networks. Good solutions to this problem are collectively computed within an elapsed time of only a few neural time constants. The effectiveness of the computation involves both the nonlinear analog response of the neurons and the large connectivity among them. Dedicated networks of biological or microelectronic neurons could provide the computational capabilities described for a wide class of problems having combinatorial complexity. The power and speed naturally displayed by such collective networks may contribute to the effectiveness of biological information processing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, P.W.: Ch. 2. In: Basic notions of condensed matter physics. Menlo Park: Benjamin Cummings 1984
Arbib, M.A.: Artificial intelligence and brain theory: unities and diversities. Ann. Biomed. Eng. 3, 238–274 (1975)
Ballard, D.H.: Cortical connections and parallel processing: structure and function. Behav. Brain Sci. (1985, in press)
Ballard, D.H., Hinton, G.E., Sejnowski, T.J.: Parallel visual computation. Nature 306, 21–26 (1983)
Feldman, J.A., Ballard, D.H.: Connectionist models and their properties. Cog. Sci. 6, 205–254 (1982)
Garey, M.R., Johnon, D.S.: Computers and intractability. New York: W.H. Freeman 1979
Gelperin, A.E., Hopfield, J.J., Tank, D.W.: The logic of Limax learning. In: Model neural networks and behavior. Selverston, A., ed.: New York: Plenum Press 1985
Goldman-Rakic, P.S.: Modular organization of the prefrontal cortex. Trends Neurosci, 7, 419–424 (1984)
Gross, D.J., Mezard, M.: The simplest spin glass. Nucl. Phys. FS B 240, 431–452 (1984)
Hinton, G.E., Sejnowski, T.J.: Optimal perceptual inference. In: Proc. IEEE Comp. Soc. Conf. Comp. Vision & Pattern Recog., pp. 448–453, Washington, D.C., June 1983
Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)
Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA 81, 3088–3092 (1984)
Huskey, H.D., Korn, G.A.: Computer Handbook, pp. 1–4. New York: McGraw Hill 1962
Ikeuchi, K., Horn, B.K.P.: Numerical shape from shading and occluding boundaries. Artif. Intell. 17, 141–184 (1981)
Jackson, W.A.: Analog computation, pp. 5–8. New York: McGraw Hill 1960
Julesz, B.: In: Foundations of cycleopean vision. Chicago: University of Chicago Press 1971
Julesz, B.: Textons, the elements of texture perception, and their interactions. Nature 290, 91–97 (1981)
Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)
Koch, C., Poggio, T., Torre, V.: Nonlinear interactions in a dendritic tree: Localization, timing, and role in information processing. Proc. Natl. Acad. Sci. USA 80, 2799 (1983)
Lawler, E.L., Lenstra, A., Rinnooy Kan, H. G., Eds.: The traveling salesman problem. New York: John Wiley (in press)
Lin, S., Kernighan, B.W.: An effective heuristic algorithm for the traveling-salesman problem. Oper. Res. 21, 498–516 (1973)
Marr, D.: Vision. New York: W. H. Freeman 1982
Mead, C., Conway, L.: Introduction to VLSI systems, pp. 12,265. Reading, MA: Addison-Wesley 1980
Poggio, T., Koch, C.: An analog model of computation for the illposed problems of early vision. M.I.T. Artif. Intell. Lab. A.I. Memo No. 783, 1984
Poggio, T., Torre, V.: Ill-posed problems in early vision. Proc. Roy. B. (1985, in press)
Sebestyn, G.S.: Decision-making processes in pattern recognition. New York: McMillan 1962
Shepherd, G.M.: Microcircuits in the nervous system. Sci. Am. 238, 92–103 (1978)
Terzopoulos, D.: Multilevel reconstruction of visual surfaces: variational principles and finite element representations. In: Multi-resolution image processing and analysis. Rosenfeld, A., ed.. Berlin, Heidelberg, New York: Springer 1984
Thouless, D.J., Anderson, P.W., Palmer, R.G.: Solution of “solvable model of a spin glass”. Phil. Mag. 35, 593–601 (1977)
Tomovic, R., Karplus, W.J.: High speed analog computers, p. 4. New York: Wiley 1962
Wannier, G.H.: Statistical physics, p. 330ff. New York: Wiley 1966
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hopfield, J.J., Tank, D.W. “Neural” computation of decisions in optimization problems. Biol. Cybern. 52, 141–152 (1985). https://doi.org/10.1007/BF00339943
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00339943