Abstract
The performance of on-line training is compared with off-line or batch training using an unrealizable learning task. In naive off-line training this task shows a tendency to strong overfitting on the other hand its optimal training scheme is known. In the regime, where overfitting occurs, on-line training outperforms batch training quite easily. Asymptotically, off-line training is better but if the learning rate is chosen carefully on-line training remains competitive.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
M. Biehl, and H. Schwarze (1995), ”Learning by on-line gradient descent”, J. Phys. A 28 p.643–656.
S. Bös (1995), ”Avoiding overfitting by finite temperature learning and cross-validation”, in ICANN'95, edited by EC2 & Cie, Vol.2, p.111–116.
S. Bös (1996), ”Optimal weight decay in a perceptron”, this volume.
D. Saad, and S. Solla (1995), Workshop at NIPS'95, see World-Wide-Web page: http://neural-server.aston.ac.uk/nips95/workshop.html and references therein.
J.W. Kim, and H. Sompolinsky (1995) ”On-line Gibbs-learning”, preprint.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bös, S. (1996). Learning curves of on-line and off-line training. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds) Artificial Neural Networks — ICANN 96. ICANN 1996. Lecture Notes in Computer Science, vol 1112. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61510-5_19
Download citation
DOI: https://doi.org/10.1007/3-540-61510-5_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61510-1
Online ISBN: 978-3-540-68684-2
eBook Packages: Springer Book Archive