Abstract
Nowadays, video games conforms a huge industry that is always developing new technology. In particular, artificial intelligence techniques have been used broadly in the well-known non-player characters (NPC) given the opportunity to users to feel video games more real. This paper proposes the usage of the MaxQ-Q hierarchical reinforcement learning algorithm in non-player characters in order to increase the experience of the user in terms of naturalness. A case study of an NPC with the proposed artificial intelligence based algorithm in a first personal shooter video game was developed. Experimental results show that this implementation improves naturalness from the user’s point of view. In addition, the proposed MaxQ-Q based algorithm in NPCs allow to programmers a robust way to give artificial intelligence to them.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Botea, A., Herbrich, R., Graepel, T.: Video Games and Artificial Intelligence. Microsoft Research Cambridge, Sydney (2008)
Diebold, F.X., Kilia, L.: Measuring predictability: Theory and macroeconomic applications. Journal of Applied Econometrics, 16, 675–669 (2001)
Dietterich, T.: Hierarchical reinforcement learning with MAXQ value function decomposition. Journal of Artificial Intelligence Research 13, 227–303 (2000)
Gemrot, J., Kadlec, R., Bída, M., Burkert, O., Píbil, R., Havlíček, J., Zemčák, L., Šimlovič, J., Vansa, R., Štolba, M., Plch, T., Brom, C.: Pogamut 3 can assist developers in building AI (Not only) for their videogame agents. In: Dignum, F., Bradshaw, J., Silverman, B., van Doesburg, W. (eds.) Agents for Games and Simulations. LNCS, vol. 5920, pp. 1–15. Springer, Heidelberg (2009)
Herbich, R., Hatton, M., Tipping, M.: Mixture model for motion lines in a virtual reality environment. Technical Report US Patent 7358973 B2, Microsoft Corporation (2013)
Isla, D.: Building a better battle. In: Game Developers Conference, San Francisco (2008)
Kluwer, T., Xu, F., Adolphs, P., Uszkoreit, H.: Evaluation of the komparse conversational non-player characters in a commercial virtual world. In: International Conference on Language Resources and Evaluation, number 3535-3542, Istanbul (2012)
Llargues, J., Peralta, J., Arrabales, R., Gonzalez, M., Cortez, P., Lopez, A.: Artificial intelligence approaches for the generation and assessment of believable human-like behaviour in virtual characters. Expert Systems With Applications 41(15), 7281–7290 (2014)
Mikkulainen, R.: Creating intelligent agents in games. In: The Bridge, pp. 5–13 (2006)
Mitchell, T.: Machine Learning. McGraw Hill (1997)
Parr, R., Russell, S.: Reinforcement learning with hierarchies of machines. In: Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems, pp. 1043–1049. MIT Press, Cambridge (1997)
Sutton, R., Precup, D., Singh, S.: Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112, 181–211 (1999)
Taylor, A.: HQ-DoG: Hierarchical q-learning in domination games. Master’s thesis, The University of Georgia (August. 2012)
Wooldridge, M.: An Introduction to Multi-Agent Systems. John Wiley & Sons (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Ponce, H., Padilla, R. (2014). A Hierarchical Reinforcement Learning Based Artificial Intelligence for Non-Player Characters in Video Games. In: Gelbukh, A., Espinoza, F.C., Galicia-Haro, S.N. (eds) Nature-Inspired Computation and Machine Learning. MICAI 2014. Lecture Notes in Computer Science(), vol 8857. Springer, Cham. https://doi.org/10.1007/978-3-319-13650-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-13650-9_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13649-3
Online ISBN: 978-3-319-13650-9
eBook Packages: Computer ScienceComputer Science (R0)