Abstract
This paper describes our work on the use of anticipation in Learning Classifier Systems (LCS) applied to Markov problems. We present YACS1, a new kind of Anticipatory Classifier System. It calls upon classifiers with a [Condition], an [Action] and an [Effect] part. As in the traditional LCS framework, the classifier discovery process relies on a selection and a creation mechanism. As in the Anticipatory Classifier System (ACS), YACS looks for classifiers which anticipate well rather than for classifiers which propose an optimal action. The creation mechanism does not rely on classical genetic operators but on a specialization operator, which is explicitly driven by experience. Likewise, the action qualities of the classifiers are not computed by a classical bucket-brigade algorithm, but by a variety of the value iteration algorithm that takes advantage of the effect part of the classifiers.
This paper presents the latent learning process of YACS. The description of the reinforcement learning process is focussed on the problem induced by the joint use of generalization and dynamic programming methods.
YACS stands for “Yet Another Classifier System”
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bellman, R. E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ.
Booker, L., Goldberg, D. E., and Holland, J. H. (1989). Classifier systems and genetic algorithms. Artificial Intelligence, 40(1–3):235–282.
Butz, M. V., Goldberg, D. E., and Stolzmann, W. (2000a). Introducing a genetic generalization pressure to the anticipatory classifier system part i: Theoretical approach. In Proceedings of the 2000 Genetic and Evolutionary Computation Conference (GECCO 2000).
Butz, M. V., Goldberg, D. E., and Stolzmann, W. (2000b). Investigating generalization in the anticipatory classifier system. In Proceedings of the Sixth International Conference on Parallel Problem Solving from Nature.
Butz, M. V. and Stolzmann, W. (1999). Action-planning in anticipatory classifier sytems. In Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Program.
Cliff, D. and Ross, S. (1994). Adding memory to ZCS. Adaptive Behavior, 3(2):101–150.
Dorigo, M. (1994). Genetic and non-genetic operators in alecsys. Evolutionary Computation, 1(2):151–164.
Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning. Addison Wesley.
Holland, J. H., Holyoak, K. J., Nisbett, R. E., and Thagard, P. R. (1986). Induction. MIT Press.
Lanzi, P. L. (1998). Adding memory to XCS. In Proceedings of the IEEE Conference on Evolutionary Computation (ICEC98). IEEE Press.
Lanzi, P. L. (1999). An analysis of generalization in the XCS classifier system. Evolutionary Computation, 2(7):125–149.
Lanzi, P. L. (2000). Toward optimal performance in classifier systems. Evolutionary Computation Journal. in print.
McCallum, R. A. (1996). Learning to use selective attention and short-term memory. In Maes, P., Mataric, M., Meyer, J.-A., Pollack, J., and Wilson, S. W., (Eds.), Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, pages 315–324, Cambridge, MA. MIT Press.
Riolo, R. L. (1991). Lookahead planning and latent learning in a classifier system. In Meyer, J.-A. and Wilson, S. W., (Eds.), From annimals to animats: Proceedings of the First International Conference on Simulation of Adaptative Behavior, pages 316–326, Cambridge, MA. MIT Press.
Sigaud, O. (2000). Using classifier systems as adaptive expert systems for control. In Stolzmann, W., Lanzi, P.-L., and Wilson, S. W., (Eds.), LNCS: New trends in Classifier Systems. Springer-Verlag.
Stolzmann, W. (1998). Anticipatory classifier systems. In Koza, J., Banzhaf, W., Chellapilla, K., Deb, K., Dorigo, M., Fogel, D., Garzon, M., Goldberg, D., Iba, H., and Riolo, R., (Eds.), Genetic Programming. Morgan Kaufmann Publishers, Inc., San Francisco, CA.
Stolzmann, W. (1999). Latent learning in khepera robots with anticipatory classifier systems. In Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Program.
Sutton, R. S. and Barto, A. (1998). Reinforcement Learning: An Introduction. MIT Press.
Watkins, C. J. (1989). Learning with delayed rewards. PhD thesis, Psychology Department, University of Cambridge, England.
Wilson, S. W. (1994). ZCS, a zeroth level classifier system. Evolutionary Computation, 2(1):1–18.
Wilson, S. W. (1995). Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149–175.
Witkowski, C. M. (1999). Integrating unsupervised learning, motivation and action selection in an a-life agent. In Floreano, D., Mondada, F., and Nicoud, J.-D., (Eds.), 5th European Conference on Artificial Life (ECAL-99), pages 355–364, Lausanne. Springer.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gérard, P., Sigaud, O. (2001). YACS: Combining Dynamic Programming with Generalization in Classifier Systems. In: Luca Lanzi, P., Stolzmann, W., Wilson, S.W. (eds) Advances in Learning Classifier Systems. IWLCS 2000. Lecture Notes in Computer Science(), vol 1996. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44640-0_5
Download citation
DOI: https://doi.org/10.1007/3-540-44640-0_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42437-6
Online ISBN: 978-3-540-44640-8
eBook Packages: Springer Book Archive