Abstract
After two papers comparing ATNoSFERES with XCSM, a Learning Classifier System with internal states, this paper is devoted to a comparison between ATNoSFERES and ACS (an Anticipatory Learning Classifier System). As previously, we focus on the way perceptual aliazing problems encountered in non-Markov environments are solved with both kinds of systems. We shortly present ATNoSFERES, a framework based on an indirect encoding Genetic Algorithm which builds finite-state automata controllers, and we compare it with ACS through two benchmark experiments. The comparison shows that the difference in performance between both system depends on the environment. This raises a discussion of the adequacy of both adaptive mechanisms to particular subclasses of non-Markov problems. Furthermore, since ACS converges much faster than ATNoSFERES, we discuss the need to introduce learning capabilities in our model. As a conclusion, we advocate for the need of more experimental comparisons between different systems in the Learning Classifier System community.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. University of Michigan Press, Ann Arbor (1975)
Lanzi, P.L.: Learning Classifier Systems from a Reinforcement Learning Perspective. Technical report, Dip. di Elettronica e Informazione, Politecnico di Milano (2000)
Sutton, R.S., Barto, A.G.: Reinforcement Learning, an introduction. MIT Press, Cambridge (1998)
Lanzi, P.L.: An Analysis of the Memory Mechanism of XCSM. In: Koza, J.R., et al. (eds.) Proceedings of the Third Annual Conference on Genetic Programming, University of Wisconsin, Madison, Wisconsin, USA, Morgan Kaufmann, San Francisco (1998)
Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary Computation 3(2), 149–175 (1995)
Tomlinson, A., Bull, L.: CXCS. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) Learning Classifier Systems: from Foundations to Applications, pp. 194–208. Springer, Heidelberg (2000)
Landau, S., et al.: A comparison between ATNoSFERES and XCSM. In: GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, New York, pp. 926–933. Morgan Kaufmann, San Francisco (2002)
Landau, S., et al.: Further Comparison between ATNoSFERES and XCSM. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2003. LNCS (LNAI), vol. 2661, Springer, Heidelberg (2003)
Landau, S., Picault, S.: ATNoSFERES: a Model for Evolutive Agent Behaviors. In: Proceedings of the AISB’01 Symposium on Adaptive Agents and Multi-Agent Systems (2001)
Métivier, M., Lattaud, C.: Anticipatory Classifier System using Behavioral Sequences in Non-Markov Environments. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2003. LNCS (LNAI), vol. 2661, pp. 143–163. Springer, Heidelberg (2003)
Picault, S., Landau, S.: Ethogenetics and the Evolutionary Design of Agent Behaviors. In: Callaos, N., Esquivel, S., Burge, J. (eds.) Proceedings of the 5th World Multi-Conference on Systemics, Cybernetics and Informatics (SCI’01), vol. III, pp. 528–533 (2001)
Woods, W.A.: Transition Networks Grammars for Natural Language Analysis. Communications of the Association for the Computational Machinery 13(10), 591–606 (1970)
Holland, J.H., Reitman, J.S.: Cognitive Systems based on adaptive algorithms. Pattern Directed Inference Systems 7(2), 125–149 (1978)
Robertson, G.G., Riolo, R.L.: A tale of two classifier systems. Machine Learning 3, 139–159 (1988)
Smith, R.E.: Memory exploitation in learning classifier systems. Evolutionary Computation 2(3), 199–220 (1994)
Cliff, D., Ross, S.: Adding memory to ZCS. Adaptive Behavior 3(2), 101–150 (1994)
Lanzi, P.L., Wilson, S.W.: Toward optimal classifier system performance in non-markov environments. Evolutionary Computation 8(4), 393–418 (2000)
Lin, L.J., Mitchell, T.M.: Memory approaches to reinforcement learning in non-markovian domains. Technical Report CMU-CS-92-138, Carnegie Mellon University, School of Computer Science (1992)
McCallum, R.A.: Reinforcement Learning with Selective Perception and Hidden State. PhD thesis, University of Rochester, Rochester, NY (1995)
Tomlinson, A., Bull, L.: A zeroth level corporate classifier system. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 1999. LNCS (LNAI), vol. 1813, pp. 306–313. Springer, Heidelberg (2000)
Stolzmann, W.: Latent Learning in Khepera Robots with Anticipatory Classifier Systems. In: Wu, A.S. (ed.) Proceedings of the 1999 Genetic and Evolutionary Computation Conference (GECCO’99), pp. 290–297 (1999)
Wiering, M., Schmidhuber, J.: HQ-Learning. Adaptive Behavior 6(2), 219–246 (1997)
Sun, R., Sessions, C.: Multi-agent reinforcement learning with bidding for segmenting action sequences. In: Meyer, J.A., et al. (eds.) From Animals to Animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, Paris, pp. 317–324. MIT Press, Cambridge (2000)
Meuleau, N., et al.: Learning finite-state controllers for partially observable environments. In: Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 427–436. AAAI Press, Menlo Park (1999)
Hansen, E.A.: Finite Memory Control of Partially Observable Systems. PhD thesis, University of Massachusetts, Amherst, MA (1998)
Stolzmann, W.: Anticipatory Classifier Systems. In: Koza, J.R., et al. (eds.) Proceedings of the Third Annual Conference on Genetic Programming, University of Wisconsin, Madison, Wisconsin, USA, pp. 658–664. Morgan Kaufmann, San Francisco (1998)
Gérard, P., Stolzmann, W., Sigaud, O.: YACS: a new Learning Classifier System with Anticipation. In: Journal of Soft Computing (2001)
Gérard, P., Meyer, J.A., Sigaud, O.: Combining latent learning with dynamic programming. European Journal of Operation Research, to appear (2003)
Sigaud, O., Gérard, P.: Contribution au problème de la sélection de l’action en environnement partiellement observable (In French). In: Drogoul, A., Meyer, J.A. (eds.) Intelligence Artificielle Située, pp. 129–146. Hermès, Paris (1999)
Grefenstette, J.J.: Lamarckian learning in multi-agent environments. In: Belew, R., Booker, L. (eds.) Proceedings of the Fourth International Conference on Genetic Algorithms, pp. 303–310. Morgan Kaufmann, San Mateo (1991)
Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.): IWLCS 2003. LNCS (LNAI), vol. 2661. Springer, Heidelberg (2003)
Koza, J.R., et al. (eds.): Proceedings of the Third Annual Conference on Genetic Programming, University of Wisconsin, Madison, Wisconsin, USA. Morgan Kaufmann, San Francisco (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Landau, S., Sigaud, O., Picault, S., Gérard, P. (2007). An Experimental Comparison Between ATNoSFERES and ACS. In: Kovacs, T., Llorà, X., Takadama, K., Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds) Learning Classifier Systems. IWLCS IWLCS IWLCS 2003 2004 2005. Lecture Notes in Computer Science(), vol 4399. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71231-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-71231-2_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71230-5
Online ISBN: 978-3-540-71231-2
eBook Packages: Computer ScienceComputer Science (R0)