Skip to main content
SpringerLink
Log in

Artificial neural networks for pattern recognition

  • Published:
Sadhana Aims and scope Submit manuscript

Abstract

This tutorial article deals with the basics of artificial neural networks (ANN) and their applications in pattern recognition. ANN can be viewed as computing models inspired by the structure and function of the biological neural network. These models are expected to deal with problem solving in a manner different from conventional computing. A distinction is made between pattern and data to emphasize the need for developing pattern processing systems to address pattern recognition tasks. After introducing the basic principles of ANN, some fundamental networks are examined in detail for their ability to solve simple pattern recognition tasks. These fundamental networks together with the principles of ANN will lead to the development of architectures for complex pattern recognition tasks. A few popular architectures are described to illustrate the need to develop an architecture specific to a given pattern recognition problem. Finally several issues that still need to be addressed to solve practical problems using ANN approach are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abu-Mostafa Y S, St. Jaques J M 1985 Information capacity of the Hopfield model.IEEE Trans. Inf. Theor. 31: 461–464

    Article  MATH  Google Scholar 

  • Ackley D M, Hinton G E, Sejnowski T J 1985 A learning algorithm for Boltzmann machines.Cogn. Sci. 9: 147–169

    Article  Google Scholar 

  • Ahalt S C, Krishnamurthy A K, Chen P, Melton D E 1990 Competitive learning algorithms for vector quantization.Neural Networks 3: 277–290

    Article  Google Scholar 

  • Aleksander I, Morton H 1990An introduction to neural computing (London: Chapman and Hall)

    Google Scholar 

  • Bhat N, McAvoy T 1989 Use of neural nets for dynamic modelling and control of chemical process systems.Proc. Am. Autom. Contr. Conf., Pittsburgh, PA, pp. 1342–1348

  • Bienenstock E, von der Malsburg Ch 1987 A neural network for the retrieval of superimposed connection patterns.Euro. Phys. Lett. 3: 1243–1249

    Article  Google Scholar 

  • Bottou L, Soulie F F, Blanchet P, Lienard J S 1990 Speaker independent isolated digit recognition: multilayer perceptrons vs. dynamic time warping.Neural Networks 3: 436–465

    Article  Google Scholar 

  • Carpenter G A 1989 Neural network models for pattern recognition and associative memory.Neural networks 2: 138–152

    Google Scholar 

  • Carpenter G A, Grossberg S 1987 art2: Self-organization of stable category recognition codes for analog input patterns.Appl. Opt. 26: 4919–4930

    Google Scholar 

  • Carpenter G A, Grossberg S 1988 The art of adaptive pattern recognition by a self-organizing neural network.IEEE Comput. 21: 77–88

    Google Scholar 

  • Cohen M, Grossberg S 1983 Absolute stability of global pattern formation and parallel storage by competitive neural networks.IEEE Trans. Syst., Man Cybern. SMC-13: 815–825

    MathSciNet  Google Scholar 

  • Cole R, Fanty M, Muthuswamy Y, Gopalakrishna M 1992 Speaker-independent recognition of spoken English letters.Proc. Int. Joint Conf. Neural Networks, San Diego, ca

  • Collins E, Ghosh S, Scotfield C L 1988 An application of a multiple neural network learning system to emulation of mortgage underwriting judgements.IEEE Int. Conf. Neural Networks (Piscataway, NJ: IEEE Press) 2: 459–466

    Chapter  Google Scholar 

  • Cybenko G 1989 Continuous value neural networks with two hidden layers are sufficient.Math. Control. Signal Syst. 2: 303–314

    Article  MATH  MathSciNet  Google Scholar 

  • Desai M S 1990Noisy pattern retrieval using associative memories. MSEE thesis, University of Louisville, Kentucky

    Google Scholar 

  • Deuker J, Schwartz D, Wittner B, Solla S, Howard R, Jackel L, Hopfield J 1987 Large automatic learning, rule extraction, and generalization.Complex Syst. 1: 877–922

    Google Scholar 

  • Dotsenko V S 1988 Neural networks: translation-, rotation- and scale invariant pattern recognitionJ. Phys. A21: L783-L787

    MathSciNet  Google Scholar 

  • Dutta S, Shekkar S 1988 Bond rating: a non-conservative application of neural networks.IEEE Int. Conf. Neural Networks (Piscataway, NJ: IEEE Press) 2: 443–450

    Chapter  Google Scholar 

  • Freeman J A, Skupura D M 1991Neural network algorithms, applications and programming techniques (New York: Addison-Wesley)

    Google Scholar 

  • Fukushima K 1975 Cognitron: A self-organizing multilayer neural network.Biol. Cybern. 20: 121–136

    Article  Google Scholar 

  • Fukushima K 1988 A neural network for visual pattern recognition.IEEE Comput. 21: 65–75

    MathSciNet  Google Scholar 

  • Fukushima K 1991 Handwritten alphanumeric character recognition by the neocognitron.IEEE Trans. Neural Networks 2: 355–365

    Article  Google Scholar 

  • Fukushima K, Miyake S 1982 Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position.Pattern Recogn. 15: 455–469

    Article  Google Scholar 

  • Geman S, Geman D 1984 Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.IEEE Trans. Pattern Anal. Machine Intell. PAMI-6: 721–741

    Google Scholar 

  • Gorman R, Sejnowski T 1988 Learned classification of sonar targets using a massively parallel network.IEEE Trans. Acoust. Speech Signal Process. 36: 1135–1140

    Article  MATH  Google Scholar 

  • Grossberg S 1969 Some networks that can learn, and reproduce any number of complicated space-time patterns.Int. J. Math. Mech. 19: 53–91

    MATH  MathSciNet  Google Scholar 

  • Grossberg S 1980 How does a brain build a cognitive code?Psychol. Rev. 87: 1–51

    Article  Google Scholar 

  • Grossberg S 1982Studies of mind & brain (Boston: Reidel)

    MATH  Google Scholar 

  • Grossberg S 1988 Nonlinear neural networks: Principles, mechanisms, and architecture.Neural networks 1: 17–61

    Article  Google Scholar 

  • Handleman D H, Lane S H, Gelfand J J 1990 Integrating neural networks and knowledge-based systems for intelligent robotic control.IEEE Control Syst. Mag. 10(3): 77–87

    Article  Google Scholar 

  • Hassoun M H 1989 Dynamic heteroassociative memories.Neural Networks 2: 275–287

    Article  Google Scholar 

  • Hebb D 1949Organization of the behaviour (New York: Wiley)

    Google Scholar 

  • Hecht-Nielson R 1987 Counterpropagation networks.Appl. Opt. 26: 4979–4984

    Google Scholar 

  • Hecht-Nielson R 1990Neurocomputing (Reading, MA: Addison-Wesley)

    Google Scholar 

  • Hertz J, Krogh A, Richard G P 1991Introduction to the theory of neural computation (New York: Addison-Wesley)

    Google Scholar 

  • Hinton G E, Sejnowski T J 1986 Learning and relearning in Boltzmann machines. InParallel distributed processing: Explorations in the microstructure of cognition (eds) D E Rumelhart, J L McClelland (Cambridge, MA: MIT Press) 1: 282–317

    Google Scholar 

  • Hodgkin A L, Huxley A F 1952 A quantitative description of membrane current and its application to conduction and excitation in nerve.J. Physiol. 117: 500–544

    Google Scholar 

  • Hopfield J J 1982 Neural networks and physical systems with emergent collective computational capabilities.Proc. Natl. Acad. Sci. (USA) 79: 2554–2558

    Article  MathSciNet  Google Scholar 

  • Hopfield J J, Tank D W 1985 Neural computation of decisions in optimization problems.Biol. Cybern. 52: 141–154

    MATH  MathSciNet  Google Scholar 

  • Huang Z, Kuh A 1992 A combined self-organizing feature map and multilayer perceptron for isolated word recognition.IEEE Trans. Signal Process. 40: 2651–2657

    Article  Google Scholar 

  • Hush D R, Horne B G 1993 Progress in supervised neural networks.IEEE Signal Process. Mag. 10: 8–39

    Article  Google Scholar 

  • Kamp Y, Hasler M 1990Recursive neural networks for associative memory (Chichester: John Wiley & Sons)

    MATH  Google Scholar 

  • Kennedy M P, Chau L O 1988 Neural networks for nonlinear programming.IEEE Trans. Circuits Syst. CAS-35: 554–562

    Article  Google Scholar 

  • Kirkpatrick S, Gelatt C D Jr, Vecchi M P 1983 Optimization by simulated annealing.Science 220: 671–680

    Article  MathSciNet  Google Scholar 

  • Kohonen T 1988 An introduction to neural computing.Neural Networks 1: 3–16

    Article  Google Scholar 

  • Kohonen T 1989Self-organization and associative memory (3rd edn) (Berlin: Springer-Verlag)

    Google Scholar 

  • Kohonen T 1990 The self-organizing map.Proc. IEEE 78: 1464–1480

    Article  Google Scholar 

  • Konishi M, Otsuka Y, Matsuda K, Tamura N, Fuki A, Kadoguchi K 1990 Application of a neural network to operation guidance in a blast furnace.3rd European Seminar on Neural Computing: The Marketplace, London

  • Kosko B 1988 Bidirectional associative memories.IEEE Trans. Syst., Man Cybern. 18: 49–60

    Article  MathSciNet  Google Scholar 

  • Kosko B 1990 Unsupervised learning in noise.IEEE Trans. Neural Networks 1: 44–57

    Article  Google Scholar 

  • Kosko B 1992Neural networks and fuzzy systems (Englewood Cliffs, NJ: Prentice-Hall)

    MATH  Google Scholar 

  • Krzyzak A, Dali W, Yuen C Y 1990 Unconstrained handwritten character classification using modified back propagation model. InFrontiers in handwriting recognition (ed.) C Y Suen (Montreal: CENPARMI)

    Google Scholar 

  • Kung S Y, Hwang J N 1989 Neural network architectures for robotic applications.IEEE Trans. Robotics Autom. 5: 641–657

    Article  Google Scholar 

  • Kuperstein M, Wang J 1990 Neural controller for adaptive movements with unforeseen payloads.IEEE Trans. Neural Networks 1(1): 137–142

    Article  Google Scholar 

  • Lang K J, Waibel A H, Hinton G E 1990 A time-delay neural network architecture for isolated word recognition.Neural Networks 3 (1): 23–44

    Article  Google Scholar 

  • LeCun Y, Boser B, Denker J S, Henderson D, Howard R E, Hubbard W, Jackel L D 1989 Back propagation applied to handwritten zip code recognition.Neural Comput. 1:541–551

    Google Scholar 

  • Lippmann R P 1987 An introduction to computing with neural nets.IEEE Trans. Acoust. Speech Signal Process. Mag. (April): 4–22

  • Lippmann R P 1989a Review of neural networks for speech recognition.Neural Comput. 1(1): 1–38

    Article  Google Scholar 

  • Lippmann R P 1989b Pattern classification using neural networks.IEEE Commun. Mag. (Nov): 47–64

  • Lisboa P G P 1992Neural networks current applications (London: Chapman & Hall)

    MATH  Google Scholar 

  • Maa C Y, Chin C, Shanblatt M A 1990 A constrained optimization neural net techniques for economic power dispatch.Proc. 1990 (New York: IEEE Press)

    Google Scholar 

  • Marcus A, van Dam A 1991 User-interface developments for the nineties.IEEE Comput. 24: 49–57

    Google Scholar 

  • McCulloch W S, Pitts W 1943 A logical calculus of the ideas immanent in nervous activity.Bull. Math. Biophys. 5: 115–133

    Article  MATH  MathSciNet  Google Scholar 

  • Michel A N, Farrell J A 1990 Associative memories via artificial neural networks.IEEE Control Syst. Mag. (April): 6–17

  • Minsky M, Papert S A 1988Perceptron (Cambridge, MA: MIT Press)

    Google Scholar 

  • Muller B, Reinhardt J 1990Neural networks: An introduction (Berlin: Springer-Verlag)

    Google Scholar 

  • Murakami K, Aibara T 1987 An improvement on the Moore-Penrose generalized inverse associative memory.IEEE Trans. Syst. Man. Cybern. SMC 17: 699–706

    Article  MathSciNet  Google Scholar 

  • Naidu S R, Zafiriou E, McAvoy T J 1990 Use of neural networks for sensor failure detection in a control system.IEEE Control Syst. Mag. 10 (3): 49–55

    Article  Google Scholar 

  • Nasrabadi N M, King R A 1988 Image coding using vector quantization: A review.IEEE Trans. Commun. 36: 957–971

    Article  Google Scholar 

  • Naylor J, Li K P 1988 Analysis of a neural network algorithm for vector quantization of speech parameters.Neural Networks 1 (Suppl): 310

    Google Scholar 

  • Pal S K, Mitra S 1992 Multilayer perceptron, fuzzy sets, and classification.IEEE Trans. Neural Networks 3: 683–697

    Article  Google Scholar 

  • Raghu P P, Chouhan H M, Yegnanarayana B 1993 Multispectral image classification using neural network.Proc. Natl. Conf. on Neural Networks, Anna University, Madras (NCNN): 1–10

    Google Scholar 

  • Rauch H E, Winarske T 1988 Neural networks for routing communications traffic.IEEE Control Syst. Mag. (April): 26–31

  • Reggia J A, Suttonn G G 1988 III. Self-processing networks and their biomedical implications.Proc. IEEE 76: 680–692

    Article  Google Scholar 

  • Rosenblatt F 1958 A probabilistic model for information storage and organization in the brain.Psychol. Rev. 65: 386–408

    Article  MathSciNet  Google Scholar 

  • Rosenblatt F 1962Principles of neurodynamics (Washington, DC: Spartan)

    MATH  Google Scholar 

  • Rumelhart D, McClelland J 1986Parallel distributed processing: Explorations in the microstructure of cognition (Boston: MIT Press) vol. 1

    Google Scholar 

  • Rumelhart D E, Zipser D 1986 Feature discovery by competitive learning.Parallel and distributed processing (eds) J L McClelland, D E Rumelhart 1: 151-193

  • Scalia F, Marconi L, Ridella S, Arrigo P, Mansi C, Mela G S 1988 An example of back propagation: diagnosis of dyspepsia.Ist IEE Conf. Neural Networks (IEE Conf. Publ.) 313: 332–540

    Google Scholar 

  • Schalkoft R 1992Pattern recognition — Statistical, structural and neural approaches (New York: John Wiley & Sons)

    Google Scholar 

  • Seibert M, Waxman A 1989 Spreading activation layers, visual saccades, and invariant representations for neural pattern recognition systems.Neural Networks 2: 9–27

    Article  Google Scholar 

  • Sejnowski T, Rosenberg C 1987 Parallel networks that learn to pronounce English text.Complex Syst. 1: 145–168

    MATH  Google Scholar 

  • Shea P M, Lin V 1989 Detection of explosives in checked airline baggage using an artificial neural system.Int. Joint. Conf. on Neural Networks 2: 31–34

    Article  Google Scholar 

  • Simpson K P 1990Artificial neural systems (New York: Pergamon)

    Google Scholar 

  • Simpson K P 1992Foundations of neural networks in artificial neural networks (eds) Edgar Sanchez-Sinencio, Clifford Lau (New York: IEEE Press)

    Google Scholar 

  • Szu H 1986 Fast simulated annealing. InNeural networks for computing (ed.) J S Denker (New York: Snowbird)

    Google Scholar 

  • Tagliarini G A, Page E W 1988 A neural network solution to the concentrator assignment problem.Neural information processing systems (ed.) D Z Anderson (New York: Am. Inst. Phys.)

    Google Scholar 

  • von der Malsburg Ch 1973 Self-organization of orientation sensitive cells in the striate cortex.Kybernetik 14: 85–100

    Article  Google Scholar 

  • Waibel A 1989 Modular construction of time-delay neural networks for speech recognition.Neural Comput. 1: 39–46

    Google Scholar 

  • Wasserman P D 1988 Combined backpropagation/cauchy machine.Neural networks: Abstracts of the first INNS Meeting, Boston (Elmsford, NY: Pergamon) 1: 556

    Google Scholar 

  • White H 1988 Economic prediction using neural networks: the case of IBM daily stock returns.Neural networks: Abstracts of the First INNS Meeting, Boston (Elmsford, NY: Pergamon) 1: 451–458

    Google Scholar 

  • Widrow B, Hoff M E 1960 Adaptive switching circuits.IRE WESCON Convention Record (4): 96–104

  • Willshaw D J, von der Malsburg Ch 1976 How patterned neural connections can be set up by self-organization.Proc. R. Soc. London B194: 431–445

    Google Scholar 

  • Zurada J M 1992Introduction to artificial neural systems (St. Paul, MN: West)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology, 600 036, Madras, India

    B Yegnanarayana

Authors
  1. B Yegnanarayana
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B Yegnanarayana.

Additional information

This paper is mostly a consolidation of work reported by several researchers in the literature, some of which is cited in the references. The author has borrowed several ideas and illustrations from the references quoted in this paper.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yegnanarayana, B. Artificial neural networks for pattern recognition. Sadhana 19, 189–238 (1994). https://doi.org/10.1007/BF02811896

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02811896

Keywords

Search

Navigation