Architectures: An Introduction

Abstract

Proposed new neural net architectures and theoretical approaches reflect the multidisciplinary nature of the research community. The very diverse tools used for carrying out theoretical investigations in neural nets go far beyond those discussed in this book, and cover a very wide range of subjects. A brief review of examples of some of the research in the different areas may be of benefit to the reader interested in pursuing the theme. Predicate logic has played a significant part, especially in establishing the limitations of simpler neural nets in the famous book of Minsky and Papert [1]. Extensions of the capabilities of more complex neural nets have involved work in functional analysis of the mappings generated by multilayered non-linear networks. An example of work of this type in function approximation may be seen in Poggio [2]. The nonlinear dynamics of many physical systems has given birth to a number of optimisation techniques — simulated annealing for example [3] — and provides a basis for the study of dynamical response nets with feedback. Many nets are based on principles known from probability and statistics, and the performance of neural net classifiers in particular is related to that of optimal statistical classifiers [4]. Much of the original interest in neural nets was generated by new knowledge of the animal brain acquired by neuroscientists, and the reverse engineering approach to neural net design is based firmly in biology.