Abstract
It is empirically known that most incremental learning systems are order dependent, i.e. provide results that depend on the particular order of the data presentation. This paper aims at uncovering the reasons behind this, and at specifying the conditions that would guarantee order independence. It is shown that both an optimality and a storage criteria are sufficient for ensuring order independence. Given that these correspond to very strong requirements however, it is interesting to study necessary, hopefully less stringent, conditions. The results obtained prove that these necessary conditions are equally difficult to meet in practice.
Besides its main outcome, this paper provides an interesting method to transform an history dependent bias into an history independent one.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Cornuéjols A. (1989): “An Exploration into Incremental Learning: the INFLUENCE System”, in Proc.of the 6th Intl. Conf. on Machine Learning. Ithaca, June 29–July 1, 1989, pp. 383–386.
Daley & Smith (1986): “On the Complexity of Inductive Inference”. Information and Control. 69, pp. 12–40, 1986.
Doyle J. (1979): “A truth maintenance system”. Artificial Intelligence. 12, 231–272, 1979.
Fisher, Xu & Zard (1992): “Ordering Effects in COBWEB and an Order-Independent Method”. To appear in Proc. of the 9th Int. Conf. on Machine Learning. Aberdeen, June 29–July 1st, 1992.
Goldman & Kearns (1991): “On the Complexity of Teaching”. Proc. of COLT'91. Santa Cruz, Aug. 5–7 1991, pp.303–314.
Haussler D. (1988): “Quantifying inductive bias: AI learning algorithms and Valiant's learning framework”. Artificial Intelligence. 36, 177–222.
Hirsh H. (1990): “Incremental Version-Space Merging”. Proc. of the 7th Int. Conf. on Machine Learning. Univ. of Austin, Texas, June 21–23, 1990, pp.330–338.
Ling (1991): “Inductive Learning from Good Examples”. In Proc. of the 12th Int. Joint Conf. on Artif. Intel. (IJCAI-91), pp.751–756.
MacGregor J. (1988): “The Effects of Order on Learning Classifications by Example: Heuristics for Finding the Optimal Order”, Artificial Intelligence, vol.34, pp.361–370, 1988.
Mitchell T. (1978): Version Spaces: an Approach to Concept Learning. PhD Thesis, Stanford, December 1978.
Mitchell T. (1982): “Generalization as Search”. Artificial Intelligence, vol.18, pp.203–226, 1982.
Natarajan B. (1991): Machine Learning. A Theoretical Approach. Morgan Kaufmann, 1991.
Pearl J. (1988): Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
Porat & Feldman (1991): “Learning Automata from Ordered Examples”. Machine Learning, 7, pp.109–138. 1991.
Rivest & Sloan (1988): “Learning Complicated Concepts Reliably and Usefully”. In Proc. of COLT'88 (Workshop on Computational Learning Theory), Cambridge 1988, pp.69–79.
Shinohara & Miyano (1990): “Teachability in Computational Learning”, in Proc. of the Workshop on Algorithmic Learning Theory. 1990, pp.247–255.
Salzberg, Delcher, Heath & Kasif (1991): “Learning with a helpful teacher”. Proc. of the IJCAI-91. pp.705–711.
Utgoff P. (1989): “Incremental Induction of Decision Trees”. Machine Learning. 4,161–186,(1989).
Van Lehn (1987): “Learning one subprocedure per lesson”. Artificial Intelligence. 31 (1), pp.1–40, january 1987.
Winston (1970): “Learning structural descriptions from examples”. AI-TR-231. MIT. Artificial Intelligence Laboratory, Cambridge, MA, 1970.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1993 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cornuéjols, A. (1993). Getting order independence in incremental learning. In: Brazdil, P.B. (eds) Machine Learning: ECML-93. ECML 1993. Lecture Notes in Computer Science, vol 667. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-56602-3_137
Download citation
DOI: https://doi.org/10.1007/3-540-56602-3_137
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-56602-1
Online ISBN: 978-3-540-47597-2
eBook Packages: Springer Book Archive