Abstract
This paper explains the various models of learning in artificial neural networks which are appropriate for implementation as analog VLSI circuits and systems. We do not cover the wider topic of analog VLSI neural networks in general, but restrict the presentation to circuits which perform in situ learning. Both supervised and unsupervised learning models are included.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ackley, D.H., Hinton, G.E. and Sejnowski, T.J. “A learning algorithm for Boltzmann machines”, Cognitive Science, Vol. 9, pp.147–169, 1985.
Alspector, J., Allen, R.B., Jayakumar, A., Zeppenfeld, T., and Meir, R., “Relaxation networks for large supervised learning problems”, in Advances in Neural Information Processing Systems 3, D. S. Touretzky, J. Moody, and R. Lippmann, editors, San Mateo, CA: Morgan Kaufmann, pp. 1015–1021, Apr. 1991.
Arima, Y., Mashiko, K., Okada, K., Yamada, T., Maeda, A., Notani, H., Kondoh, H., and Kayano, S., “A 336-Neuron, 28K Synapse, Self-Learning Neural Network Chip with Branch-Neuron-Unit Architecture”, IEEE J. Solid St. Ccts., Vol. 26, pp. 1637–1644, Nov. 1991.
Becker, S. and Hinton, G. E., “Self-Organizing Neural Network that Discovers Surfaces in Random-Dot Stereograms”, Nature, Vol. 355, pp. 161–163, 1992.
Card, H.C., Schneider, C.R., and Schneider, R.S., “Learning Capacitive Weights in Analog CMOS Neural Networks”, J. VLSI Signal Proc, in press.
Card, H. C, Schneider, C, and Moore, W.R., “Hebbian plasticity in MOS synapses”, IEE Proc. J., Vol. 138, pp. 13–16, Feb. 1991.
Carpenter, G.A. and Grossberg, S., “ART-2: Self Organization of Stable Category Recognition Codes for Analog Input Patterns”, Appi. Optics, Vol., 26, pp. 4919–4930, 1987.
Durbin, R. and Willshaw, D., “An Analog Approach to the Travelling Salesman Problem using an Elastic Net Method”, Nature, Vol. 326, pp. 689–691, 1987.
Hebb, D.O., Organization of Behaviour. New York: John Wiley, 1949.
Hertz, J., Krogh, A., and Palmer, R. G., Introduction to the Theory of Neural Computation. Menlo Park, CA: Addison-Wesley, 1991.
Hinton, G. E., “Connectionist learning procedures”, Artificial Intelligence, Vol. 40, pp. 185–234, 1989.
Hochet, B., Peiris, V., Abdo, S., and Declercq, M.J., “Implementation of a Learning Kohonen Neuron based on a new Multilevel Storage Technique”, IEEE J. Solid St. Ccts., Vol. 26, pp. 262–267, Mar 1991.
Holler, M., Tarn, S., Castro, H., and Benson, R., “An electrically trainable artificial neural network with 10240 floating gate synapses”, Proc. IJCNN-89, Part II, pp, 191–196, 1989.
Hopfield, J., “Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons”, Proc. Natl. Acad. Sci., Vol. 81, pp. 3088–3092, 1984.
Kohonen, T., “The Self Organizing Map”, Proc. IEEE t Vol. 78, pp. 1464–1480, Sept 1990.
Lazzaro, J., Ryckebusch, S., Mahowald, M.A., and Mead, C.A., “Winner-Take-All Networks of O(N) Complexity”, in Advances in Neural Information Processing Systems I, D.S. Touretzky, editor, San Mateo, CA: Morgan Kaufmann, pp. 703–711, 1989.
Linares-Barranco, R., Sanchez-Sinencio, E., Rodriguez-Vazquez, A., and Huertas, J.L., “A Modular T-Mode Design Approach for Analog Neural Network Hardware Implementations”, IEEE J. Solid St. Ccts., Vol. 27, pp. 701–713, 1992.
Linsker, R., “Self Organization in a Perceptual Network”, IEEE Computer, Vol. 21, pp. 105–117, 1988.
Mahowald, M. and Douglas, R., “A Silicon Neuron”, Nature, Vol. 354, pp. 515–518, Dec 1991.
Mann, J. R. and Gilbert, S., “An Analog Self-Organizing Neural Network Chip”, in Advances in Neural Information Processing Systems I, D.S. Touretzky, editor, San Mateo, CA: Morgan Kaufmann, pp. 739–747, 1990.
Maundy, B. and El-Masry, E., “Switched Capacitor Neural Networks using Pulse Based Arithmetic”, Elect. Lett., Vol. 26, pp. 1118–1119, 1990.
Mead, CA., Analog VLSI and Neural Systems. Reading: Addison-Wesley, 1988.
Moody, J. and Darken, C.J., “Fast Learning in Networks of Locally Tuned Processing Units”, Neural Computation, Vol. 1, pp. 281–294, 1989.
Movellan, J. R., “Contrastive Hebbian Learning in the Continuous Hopfield Model”, Connectionist Models: Proceedings of the 1990 Summer School, D.S. Touretzky et al, editors, pp. 10–17, 1990.
Murray, A. F., Del Corso, D., and Tarassenko, L., “Pulse Stream VLSI Neural Networks Mixing Analog and Digital Techniques”, IEEE Trans, on Neural Networks, Vol. 2, pp. 193–204, Mar. 1991.
Nowlan, S.J., and Hinton, G.E., “Evaluation of Adaptive Mixtures of Competing Experts”, in Advances in Neural Information Processing Systems 3, D. S. Touretzky, J. Moody, and R. Lippmann, editors, San Mateo, CA: Morgan Kaufmann, pp. 774–780, 1991.
Oja, E., “Neural Networks, Principal Components, and Subspaces”, Int. J. Neural Systems, Vol. 1, pp. 61–68, 1989.
Peterson, C. and Anderson, J.R., “A mean field theory learning algorithm for neural networks”, Complex Systems, Vol. 1, pp. 995–1019, 1987.
Raffel, J., Mann, J., Berger, R., Soares, A., and Gilbert, S., “A Generic Architecture for Wafer Scale Neuromorphic Systems”, Proc. IEEE Int. Conf. Neural Networks, Vol. III, pp. 501–513, 1987.
Rumelhart, D.E., Hinton, G.E. and Williams, R.W., “Learning Representations by Back Propagating Errors”, Nature, Vol. 323, pp.533–536, 1986.
Sage, J.P., Thompson, K., and Withers, R.S., “An Artificial Neural Network Integrated Circuit based on MNOS/CCD Principles”, In AIP Conf Proc. 151, Neural Networks for Computing, Snowbird, ed. J.S. Denker, American Institute of Physics, New York, pp. 381–385, 1986.
Sanger, T.D., “Optimal Unsupervised Learning in a Single Layer Linear Feedforward Neural Network”, Neural Networks, Vol. 2, pp. 459–473, 1989.
Schneider, C. and Card, H.C., “Analog CMOS Contrastive Hebbian Networks”, SPIE Proc. 1709, Applications of Artificial Neural Networks III, Orlando, Florida, Apr 21–24, pp. 726–735, 1992.
Schneider, C. R. and Card, H. C., “Analog CMOS Hebbian Synapses”, Elec. Lett., Vol 27, pp. 785–786, 1991.
Shoemaker, P.A., Lagnado, I., and Shimabukuro, R., “Artificial neural network implementation with floating gate MOS devices”, in Hardware Implementations of Neuron Nets and Synapses, NSF/ONR Workshop, San Diego, CA, P. Mueller, editor, pp.114 – 119, Jan 14–15, 1988.
Tomlinson, M.S., Walker, D.J., and Sivilotti, M.A. (1990), “A Digital Neural Network Architecture for VLSI”, Int. Joint Conf. on Neural Networks, San Diego, Vol. II, pp. 545–550, 1990.
Tsay, S.W. and Newcomb, R.W., “VLSI Implementation of ARTI Memories”, IEEE Trans. Neural Networks, Vol. 2, pp. 214–221, 1991.
Von der Malsburg, G, “Self Organization of Orientation Selective Cells in Striate Cortex”, Kybernetik, Vol. 14, pp. 85–100, 1973.
Walker, M., Hasler, P., and Akers, L., “A CMOS Neural Network for Pattern Association”, IEEE Micro, pp. 68–74, Oct. 1989.
Watola, D., Gembala, D., and Meador, J., “Competitive Learning in Asynchronous Pulse Density Integrated Circuits”, IEEE Int. Symp. on Ccts. and Systems, San Diego, CA, May 10–13, 1992.
Willshaw, DJ. and von der Malsburg, G, “How Patterned Neural Connections can be Set Up by Self Organization”, Proc. Royal Soc. London, Vol. B194, pp. 431–445, 1976.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer Science+Business Media New York
About this chapter
Cite this chapter
Card, H.C. (1994). Analog VLSI Neural Learning Circuits — A Tutorial. In: Delgado-Frias, J.G., Moore, W.R. (eds) VLSI for Neural Networks and Artificial Intelligence. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-1331-9_1
Download citation
DOI: https://doi.org/10.1007/978-1-4899-1331-9_1
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4899-1333-3
Online ISBN: 978-1-4899-1331-9
eBook Packages: Springer Book Archive