Improving Learning and Generalization in Neural Networks through the Acquisition of Multiple Related Functions

Abstract

This paper presents evidence from connectionist simulations providing support for the idea that forcing neural networks to learn several related functions together results in both improved learning and better generalization. More specifically, if a neural network employing gradient descent learning is forced to capture the regularities of many semi-correlated sources of information within the same representational substrate, it then becomes necessary for it to only represent hypotheses that are consistent with all the cues provided. When the different sources of information are sufficiently correlated the number of candidate solutions will be reduced through the development of more efficient representations. To illustrate this, the paper draws briefly on research in the neural network engineering literature, while focusing on recent work on the segmentation of speech using connectionist networks. Finally, some implications for language acquisition of the present approach are discussed.