Skip to main content
SpringerLink
Log in

Deep learning for procedural content generation

  • S. I : Neural Networks in Art, sound and Design
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Procedural content generation in video games has a long history. Existing procedural content generation methods, such as search-based, solver-based, rule-based and grammar-based methods have been applied to various content types such as levels, maps, character models, and textures. A research field centered on content generation in games has existed for more than a decade. More recently, deep learning has powered a remarkable range of inventions in content production, which are applicable to games. While some cutting-edge deep learning methods are applied on their own, others are applied in combination with more traditional methods, or in an interactive setting. This article surveys the various deep learning methods that have been applied to generate game content directly or indirectly, discusses deep learning methods that could be used for content generation purposes but are rarely used today, and envisages some limitations and potential future directions of deep learning for procedural content generation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. https://github.com/AIDungeon/AIDungeon.

  2. https://github.com/amidos2006/gym-pcgrl.

  3. https://www.facebook.com/Petalz-238904402867390/.

  4. http://gar.eecs.ucf.edu/.

  5. http://www.hedgewars.org/.

  6. https://artbreeder.com/.

References

  1. Abdal R, Qin Y, Wonka P (2019) Image2StyleGAN: how to embed images into the StyleGAN latent space? In: Proceedings of the IEEE International Conference on Computer Vision, pp 4432–4441

  2. Ammanabrolu P, Cheung W, Tu D, Broniec W, Riedl MO (2020) Bringing stories alive: generating interactive fiction worlds. In: Proceedings of the sixteenth annual AAAI conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020)

  3. Awiszus M, Schubert F, Rosenhahn B (2020) TOAD-GAN: coherent style level generation from a single example. In: Proceedings of the sixteenth annual AAAI conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020)

  4. Bontrager P, Togelius J (2020) Fully differentiable procedural content generation through generative playing networks. arXiv preprint arXiv:200205259

  5. Bontrager P, Roy A, Togelius J, Memon N, Ross A (2018) DeepMasterPrints: generating masterprints for dictionary attacks via latent variable evolution. In: 2018 IEEE 9th International Conference on Biometrics Theory. Applications and Systems (BTAS). IEEE, pp 1–9

  6. Briot JP, Hadjeres G, Pachet F (2019) Deep learning techniques for music generation, vol 10. Springer, Berlin

    Google Scholar 

  7. Browne C, Maire F (2010) Evolutionary game design. IEEE Trans Comput Intell AI Games 2(1):1–16

    Article  Google Scholar 

  8. Camilleri E, Yannakakis GN, Dingli A (2016) Platformer level design for player believability. In: 2016 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, pp 1–8

  9. Camilleri E, Yannakakis GN, Liapis A (2017) Towards general models of player affect. In: 2017 seventh international conference on Affective Computing and Intelligent Interaction (ACII). IEEE, pp 333–339

  10. Chen Z, Amato C, Nguyen THD, Cooper S, Sun Y, El-Nasr MS (2018) Q-deckrec: a fast deck recommendation system for collectible card games. In: 2018 IEEE conference on Computational Intelligence and Games (CIG), pp 1–8. https://doi.org/10.1109/CIG.2018.8490446

  11. Colton S (2008) Creativity versus the perception of creativity in computational systems. In: AAAI spring symposium: creative intelligent systems, vol 8

  12. Cook M, Colton S, Raad A, Gow J (2013) Mechanic miner: reflection-driven game mechanic discovery and level design. In: European conference on the applications of evolutionary computation. Springer, pp 284–293

  13. Cook M, Colton S, Gow J (2016) The angelina videogame design system—part I. IEEE Trans Comput Intell AI Games 9(2):192–203

    Article  Google Scholar 

  14. Dahlskog S, Togelius J, Nelson MJ (2014) Linear levels through n-grams. In: Proceedings of the 18th International Academic MindTrek Conference: Media Business, Management, Content & Services, pp 200–206

  15. Davoodi O, Ashtiani M, Rajabi M (2020) An approach for the evaluation and correction of manually designed video game levels using deep neural networks. Comput J. https://doi.org/10.1093/comjnl/bxaa071

    Article  Google Scholar 

  16. De Kegel B, Haahr M (2020) Procedural puzzle generation: a survey. IEEE Trans Games 12(1):21–40

    Article  Google Scholar 

  17. Delarosa O, Dong H, Ruan M, Khalifa A, Togelius J (2020) Mixed-initiative level design with RL brush. arXiv preprint arXiv:200802778

  18. Dhariwal P, Jun H, Payne C, Kim JW, Radford A, Sutskever I (2020) Jukebox: a generative model for music. arXiv preprint arXiv:200500341

  19. Dharna A, Togelius J, Soros L (2020) Coevolution of game levels and game-playing agents. In: Proceedings of the sixteenth annual AAAI conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020)

  20. Di Liello L, Ardino P, Gobbi J, Morettin P, Teso S, Passerini A (2020) Efficient generation of structured objects with constrained adversarial networks. arXiv preprint arXiv:200713197

  21. Donahue C, Lipton ZC, McAuley J (2017) Dance dance convolution. In: International conference on machine learning, pp 1039–1048

  22. Dormans J (2010) Adventures in level design: generating missions and spaces for action adventure games. In: Proceedings of the 2010 workshop on procedural content generation in games, pp 1–8

  23. Earle S (2019) Using fractal neural networks to play SimCity 1 and Conway’s Game of Life at variable scales. In: Proceedings of the Experimental AI in Games (EXAG) Workshop at AIIDE

  24. Ebert DS, Musgrave FK, Peachey D, Perlin K, Worley S (2003) Texturing & modeling: a procedural approach. Morgan Kaufmann, Burlington

    Google Scholar 

  25. Fadaeddini A, Majidi B, Eshghi M (2018) A case study of generative adversarial networks for procedural synthesis of original textures in video games. In: 2018 2nd National and 1st International Digital Games Research Conference: trends, technologies, and applications (DGRC). IEEE, pp 118–122

  26. Fang K, Zhu Y, Savarese S, Fei-Fei L (2020) Adaptive procedural task generation for hard-exploration problems. arXiv preprint arXiv:200700350

  27. Ferreira LN, Lelis LH, Whitehead J (2020) Computer-generated music for tabletop role-playing games. In: Proceedings of the sixteenth annual AAAI conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020)

  28. Fontaine M, Togelius J, Nikolaidis S, Hoover AK (2020) Covariance matrix adaptation for the rapid illumination of behavior space. In: Proceedings of the 2020 genetic and evolutionary computation conference

  29. Fontaine MC, Liu R, Togelius J, Hoover AK, Nikolaidis S (2020) Illuminating Mario scenes in the latent space of a generative adversarial network. arXiv preprint arXiv:200705674

  30. Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style. arXiv preprint arXiv:150806576

  31. Giacomello E, Lanzi PL, Loiacono D (2018) Doom level generation using generative adversarial networks. In: 2018 IEEE Games, Entertainment, Media Conference (GEM). IEEE, pp 316–323

  32. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems, vol 27. Curran Associates, Inc., Red Hook, pp 2672–2680

    Google Scholar 

  33. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press. http://www.deeplearningbook.org

  34. Greff K, Srivastava RK, Koutník J, Steunebrink BR, Schmidhuber J (2017) LSTM: a search space odyssey. IEEE Trans Neural Netw Learn Syst 28(10):2222–2232

    Article  MathSciNet  Google Scholar 

  35. Gudmundsson SF, Eisen P, Poromaa E, Nodet A, Purmonen S, Kozakowski B, Meurling R, Cao L (2018) Human-like playtesting with deep learning. In: 2018 IEEE conference on Computational Intelligence and Games (CIG). IEEE, pp 1–8

  36. Gutierrez J, Schrum J (2020) Generative adversarial network rooms in generative graph grammar dungeons for the Legend of Zelda. In: 2020 IEEE Congress on Evolutionary Computation (CEC). IEEE

  37. Guzdial M, Riedl M (2016) Game level generation from gameplay videos. In: Twelfth artificial intelligence and interactive digital entertainment conference

  38. Guzdial M, Long D, Cassion C, Das A (2017) Visual procedural content generation with an artificial abstract artist. In: Proceedings of ICCC computational creativity and games workshop

  39. Guzdial M, Liao N, Riedl M (2018) Co-creative level design via machine learning. In: Proceedings of the Experimental AI in Games (EXAG) workshop at AIIDE

  40. Guzdial M, Reno J, Chen J, Smith G, Riedl M (2018) Explainable PCGML via game design patterns. In: Proceedings of the Experimental AI in Games (EXAG) workshop at AIIDE

  41. Guzdial M, Liao N, Chen J, Chen SY, Shah S, Shah V, Reno J, Smith G, Riedl MO (2019) Friend, collaborator, student, manager: how design of an AI-driven game level editor affects creators. In: Proceedings of the 2019 CHI conference on human factors in computing systems, pp 1–13

  42. Guzdial MJ, Sturtevant N, Li B (2016) Deep static and dynamic level analysis: a study on infinite mario. In: Twelfth artificial intelligence and interactive digital entertainment conference

  43. Ha D, Eck D (2017) A neural representation of sketch drawings. arXiv preprint arXiv:170403477

  44. Hastings EJ, Guha RK, Stanley KO (2009) Automatic content generation in the galactic arms race video game. IEEE Trans Comput Intell AI Games 1(4):245–263

    Article  Google Scholar 

  45. Hochreiter S, Schmidhuber J (1997) LSTM can solve hard long time lag problems. In: Advances in neural information processing systems, pp 473–479

  46. Holmgård C, Liapis A, Togelius J, Yannakakis GN (2014) Evolving personas for player decision modeling. In: 2014 IEEE conference on computational intelligence and games. IEEE, pp 1–8

  47. Holmgard C, Green MC, Liapis A, Togelius J (2018) Automated playtesting with procedural personas with evolved heuristics. IEEE Trans Games 11(4):352–362

    Article  Google Scholar 

  48. Hong S, Kim S, Kang S (2019) Game sprite generator using a multi discriminator GAN. KSII Trans Internet Inf Syst 13(8):4255–4269

    Google Scholar 

  49. Hoover AK, Szerlip PA, Stanley KO (2014) Functional scaffolding for composing additional musical voices. Comput Music J 38(4):80–99

    Article  Google Scholar 

  50. Hoover AK, Cachia W, Liapis A, Yannakakis GN (2015) Audioinspace: exploring the creative fusion of generative audio, visuals and gameplay. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 101–112

  51. Hoover AK, Togelius J, Yannakis GN (2015) Composing video game levels with music metaphors through functional scaffolding. In: First computational creativity and games workshop. ACC

  52. Irfan A, Zafar A, Hassan S (2019) Evolving levels for general games using deep convolutional generative adversarial networks. In: 2019 11th Computer Science and Electronic Engineering (CEEC). IEEE, pp 96–101

  53. Isaksen A, Holmgård C, Togelius J (2017) Semantic hashing for video game levels. Game Puzzle Des 3(1):10–16

    Google Scholar 

  54. Jain R, Isaksen A, Holmgård C, Togelius J (2016) Autoencoders for level generation, repair, and recognition. In: Proceedings of the ICCC workshop on computational creativity and games

  55. Jin Y, Zhang J, Li M, Tian Y, Zhu H, Fang Z (2017) Towards the automatic anime characters creation with generative adversarial networks. CoRR arXiv:1708.05509

  56. Jordanous A (2012) A standardised procedure for evaluating creative systems: computational creativity evaluation based on what it is to be creative. Cogn Comput 4(3):246–279

    Article  Google Scholar 

  57. Karavolos D, Liapis A, Yannakakis G (2017) Learning the patterns of balance in a multi-player shooter game. In: Proceedings of the 12th international conference on the foundations of digital games, pp 1–10

  58. Karavolos D, Liapis A, Yannakakis GN (2018) Pairing character classes in a deathmatch shooter game via a deep-learning surrogate model. In: Proceedings of the 13th international conference on the Foundations of digital games, pp 1–10

  59. Karavolos D, Liapis A, Yannakakis GN (2018) Using a surrogate model of gameplay for automated level design. In: 2018 IEEE conference on Computational Intelligence and Games (CIG). IEEE, pp 1–8

  60. Karavolos D, Liapis A, Yannakakis GN (2019) A multi-faceted surrogate model for search-based procedural content generation. IEEE Trans Games. https://doi.org/10.1109/TG.2019.2931044

  61. Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2019) Analyzing and improving the image quality of stylegan. arXiv preprint arXiv:191204958

  62. Khalifa A, Bontrager P, Earle S, Togelius J (2020) PCGRL: procedural content generation via reinforcement learning. arXiv preprint arxiv:2001.09212

  63. Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint arXiv:13126114

  64. Kuang P, Luo D (2020) Conditional convolutional generative adversarial networks based interactive procedural game map generation. In: Future of information and communication conference. Springer, pp 400–419

  65. Kumaran V, Mott BW, Lester JC (2020) Generating game levels for multiple distinct games with a common latent space. In: Proceedings of the sixteenth annual AAAI conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020)

  66. Larsson S, Petri O (2016) Content evaluation of starcraft maps using neuroevolution. Dissertation. Retrieved from http://urn.kb.se/resolve?urn=urn:nbn:se:bth-11684

  67. Liang Y, Li W, Ikeda K (2019) Procedural content generation of rhythm games using deep learning methods. In: Joint international conference on entertainment computing and serious games. Springer, pp 134–145

  68. Liapis A, Yannakis GN (2016) Boosting computational creativity with human interaction in mixed-initiative co-creation tasks. In: Proceedings of the ICCC workshop on computational creativity and games

  69. Liapis A, Martínez HP, Togelius J, Yannakakis GN (2013) Transforming exploratory creativity with delenox. In: International conference on computational creativity

  70. Liapis A, Yannakakis GN, Togelius J (2013) Sentient sketchbook: computer-aided game level authoring. In: Proceedings of the 2013 ACM conference on foundations of digital games

  71. Liapis A, Yannakakis GN, Togelius J (2013) Sentient world: human-based procedural cartography. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 180–191

  72. Liapis A, Yannakakis GN, Togelius J (2014) Computational game creativity. In: ICCC

  73. Liapis A, Yannakakis GN, Nelson MJ, Preuss M, Bidarra R (2018) Orchestrating game generation. IEEE Trans Games 11(1):48–68

    Article  Google Scholar 

  74. Liu MY, Breuel T, Kautz J (2017) Unsupervised image-to-image translation networks. In: Advances in neural information processing systems, pp 700–708

  75. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV)

  76. Lopes P, Liapis A, Yannakakis GN (2015) Sonancia: sonification of procedurally generated game levels. In: ICCC

  77. Lucas SM, Volz V (2019) Tile pattern KL-divergence for analysing and evolving game levels. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-19. Association for Computing Machinery, New York, NY, USA, pp 170–178. https://doi.org/10.1145/3321707.3321781

  78. Makantasis K, Liapis A, Yannakakis GN (2019) From pixels to affect: a study on games and player experience. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, pp 1–7

  79. Martínez HP, Yannakakis GN (2014) Deep multimodal fusion: combining discrete events and continuous signals. In: Proceedings of the 16th international conference on multimodal interaction, pp 34–41

  80. Martinez HP, Bengio Y, Yannakakis GN (2013) Learning deep physiological models of affect. IEEE Comput Intell Mag 8(2):20–33

    Article  Google Scholar 

  81. Melhart D, Gravina D, Yannakakis GN (2020) Moment-to-moment engagement prediction through the eyes of the observer: PUBG streaming on twitch. In: Foundations of digital games

  82. Min W, Ha EY, Rowe J, Mott B, Lester J (2014) Deep learning-based goal recognition in open-ended digital games. In: Tenth artificial intelligence and interactive digital entertainment conference

  83. Mordvintsev A, Randazzo E, Niklasson E, Levin M (2020) Growing neural cellular automata. Distill 5:e23. https://doi.org/10.23915/distill.00023

    Article  Google Scholar 

  84. Mott J, Nandi S, Zeller L (2019) Controllable and coherent level generation: a two-pronged approach. In: Experimental AI in games workshop

  85. Park K, Mott BW, Min W, Boyer KE, Wiebe EN, Lester JC (2019) Generating educational game levels with multistep deep convolutional generative adversarial networks. In: 2019 IEEE Conference on Games (CoG). IEEE, pp 1–8

  86. Pease A, Colton S (2011) On impact and evaluation in computational creativity: a discussion of the turing test and an alternative proposal. In: Proceedings of the AISB symposium on AI and philosophy, vol 39

  87. Perez-Liebana D, Liu J, Khalifa A, Gaina RD, Togelius J, Lucas SM (2019a) General video game AI: a multitrack framework for evaluating agents, games, and content generation algorithms. IEEE Trans Games 11(3):195–214

    Article  Google Scholar 

  88. Perez-Liebana D, Lucas SM, Gaina RD, Togelius J, Khalifa A, Liu J (2019) General video game artificial intelligence. Morgan & Claypool Publishers. https://gaigresearch.github.io/gvgaibook/

  89. Perlin K (1985) An image synthesizer. ACM Siggraph Comput Graph 19(3):287–296

    Article  Google Scholar 

  90. Pfau J, Liapis A, Volkmar G, Yannakakis GN, Malaka R (2020) Dungeons & replicants: automated game balancing via deep player behavior modeling. In: Proceedings of the 2020 IEEE Conference on Games (CoG)

  91. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:151106434

  92. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. OpenAI Blog

  93. Risi S, Togelius J (2019) Increasing generality in machine learning through procedural content generation. arXiv preprint arXiv:1911.13071

  94. Risi S, Lehman J, D’Ambrosio DB, Hall R, Stanley KO (2015) Petalz: search-based procedural content generation for the casual gamer. IEEE Trans Comput Intell AI Games 8(3):244–255

    Article  Google Scholar 

  95. Roy A, Memon N, Ross A (2017) Masterprint: exploring the vulnerability of partial fingerprint-based authentication systems. IEEE Trans Inf Forensics Secur 12(9):2013–2025

    Article  Google Scholar 

  96. Sarkar A, Cooper S (2018) Blending levels from different games using LSTMs. In: Proceedings of the Experimental AI in Games (EXAG) workshop at AIIDE

  97. Sarkar A, Cooper S (2020) Sequential segment-based level generation and blending using variational autoencoders. arXiv preprint arXiv:200708746

  98. Sarkar A, Cooper S (2020) Towards game design via creative machine learning (GDCML). In: Proceedings of the 2020 IEEE Conference on Games (CoG)

  99. Sarkar A, Yang Z, Cooper S (2019) Controllable level blending between games using variational autoencoders. In: Proceedings of the Experimental AI in Games (EXAG) workshop at AIIDE

  100. Sarkar A, Summerville A, Snodgrass S, Bentley G, Osborn J (2020) Exploring level blending across platformers via paths and affordances. In: Sixteenth artificial intelligence and interactive digital entertainment conference

  101. Schaul T (2013) A video game description language for model-based or interactive learning. In: Proceedings of the IEEE conference on computational intelligence in games. IEEE Press, Niagara Falls

  102. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117. https://doi.org/10.1016/j.neunet.2014.09.003

    Article  Google Scholar 

  103. Schrum J, Gutierrez J, Volz V, Liu J, Lucas SM, Risi S (2020) Interactive evolution and exploration within latent level-design space of generative adversarial networks. In: Proceedings of the genetic and evolutionary computation conference. ACM

  104. Schrum J, Volz V, Risi S (2020) CPPN2GAN: combining compositional pattern producing networks and GANs for large-scale pattern generation. In: Proceedings of the genetic and evolutionary computation conference. ACM

  105. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681

    Article  Google Scholar 

  106. Scirea M, Eklund P, Togelius J, Risi S (2018) Evolving in-game mood-expressive music with metacompose. In: The audio mostly 2018 on sound in immersion and emotion, pp 1–8

  107. Serpa YR, Rodrigues MAF (2019) Towards machine-learning assisted asset generation for games: a study on pixel art sprite sheets. In: 2019 18th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames). IEEE, pp 182–191

  108. Shaham TR, Dekel T, Michaeli T (2019) Singan: learning a generative model from a single natural image. In: Proceedings of the IEEE international conference on computer vision, pp 4570–4580

  109. Shaker N, Yannakakis G, Togelius J (2010) Towards automatic personalized content generation for platform games. In: Sixth artificial intelligence and interactive digital entertainment conference

  110. Shaker N, Togelius J, Yannakakis GN, Weber B, Shimizu T, Hashiyama T, Sorenson N, Pasquier P, Mawhorter P, Takahashi G et al (2011) The 2010 Mario AI championship: level generation track. IEEE Trans Comput Intell AI Games 3(4):332–347

    Article  Google Scholar 

  111. Shaker N, Nicolau M, Yannakakis GN, Togelius J, O’neill M (2012) Evolving levels for Super Mario Bros using grammatical evolution. In: Computational intelligence and games. IEEE, pp 304–311

  112. Shaker N, Togelius J, Nelson MJ (2016) Procedural content generation in games. Springer, Berlin

    Book  Google Scholar 

  113. Shu T, Wang Z, Liu J, Yao X (2020) A novel CNET-assisted evolutionary level repairer and its applications to Super Mario Bros. In: 2020 IEEE Congress on Evolutionary Computation (CEC). IEEE

  114. Sirota J, Bulitko V, Brown MR, Hernandez SP (2019) Towards procedurally generated languages for non-playable characters in video games. In: 2019 IEEE Conference on Games (CoG). IEEE, pp 1–4

  115. Smith AM, Mateas M (2011) Answer set programming for procedural content generation: a design space approach. IEEE Trans Comput Intell AI Games 3(3):187–200

    Article  Google Scholar 

  116. Smith G, Whitehead J (2010) Analyzing the expressive range of a level generator. In: Proceedings of the 2010 workshop on procedural content generation in games, pp 1–7

  117. Snodgrass S, Ontañón S (2014) Experiments in map generation using Markov chains. In: Proceedings of the 9th conference on the foundations of digital games

  118. Snodgrass S, Ontanon S (2015) A hierarchical MDMC approach to 2D video game map generation. In: Eleventh artificial intelligence and interactive digital entertainment conference

  119. Snodgrass S, Ontanón S (2016) Controllable procedural content generation via constrained multi-dimensional Markov chain sampling. In: IJCAI, pp 780–786

  120. Snodgrass S, Ontanón S (2016b) Learning to generate video game maps using Markov models. IEEE Trans Comput Intell AI Games 9(4):410–422

    Article  Google Scholar 

  121. Snodgrass S, Sarkar A (2020) Multi-domain level generation and blending with sketches via example-driven BSP and variational autoencoders. In: Proceedings of the 15th international conference on the foundations of digital games

  122. Snodgrass S, Summerville A, Ontañón S (2017) Studying the effects of training data on machine learning-based procedural content generation. In: Thirteenth artificial intelligence and interactive digital entertainment conference

  123. Soares ES, Bulitko V (2019) Deep variational autoencoders for NPC behaviour classification. In: 2019 IEEE Conference on Games (CoG). IEEE, pp 1–4

  124. Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127

    Article  Google Scholar 

  125. Summerville A (2018) Expanding expressive range: evaluation methodologies for procedural content generation. In: Fourteenth artificial intelligence and interactive digital entertainment conference

  126. Summerville A, Mateas M (2016) Super Mario as a string: platformer level generation via LSTMs. In: International Joint Conference of DiGRA and FDG

  127. Summerville A, Guzdial M, Mateas M, Riedl MO (2016) Learning player tailored content from observation: platformer level generation from video traces using LSTMs. In: Twelfth artificial intelligence and interactive digital entertainment conference

  128. Summerville A, Mariño JR, Snodgrass S, Ontañón S, Lelis LH (2017) Understanding Mario: an evaluation of design metrics for platformers. In: Proceedings of the 12th international conference on the foundations of digital games, pp 1–10

  129. Summerville A, Snodgrass S, Guzdial M, Holmgård C, Hoover AK, Isaksen A, Nealen A, Togelius J (2018) Procedural content generation via machine learning (PCGML). IEEE Trans Games 10(3):257–270

    Article  Google Scholar 

  130. Summerville AJ, Mateas M (2016) Mystical tutor: a magic: the gathering design assistant via denoising sequence-to-sequence learning. In: Twelfth artificial intelligence and interactive digital entertainment conference

  131. Summerville AJ, Philip S, Mateas M (2015) MCMCTS PCG 4 SMB: Monte Carlo tree search to guide platformer level generation. In: Artificial intelligence and interactive digital entertainment

  132. Togelius J, Kastbjerg E, Schedl D, Yannakakis GN (2011) What is procedural content generation? Mario on the borderline. In: Proceedings of the 2nd international workshop on procedural content generation in games. ACM, p 3

  133. Togelius J, Yannakakis GN, Stanley KO, Browne C (2011b) Search-based procedural content generation: a taxonomy and survey. IEEE Trans Comput Intell AI Games 3(3):172–186

    Article  Google Scholar 

  134. Togelius J, Champandard AJ, Lanzi PL, Mateas M, Paiva A, Preuss M, Stanley KO (2013) Procedural content generation: goals, challenges and actionable steps. In: Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik

  135. Togelius J, Shaker N, Karakovskiy S, Yannakakis GN (2013b) The Mario AI championship 2009–2012. AI Mag 34(3):89–92

    Article  Google Scholar 

  136. Torrado RR, Bontrager P, Togelius J, Liu J, Perez-Liebana D (2018) Deep reinforcement learning for general video game AI. In: Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games (CIG). IEEE, pp 1–8

  137. Torrado RR, Khalifa A, Green MC, Justesen N, Risi S, Togelius J (2019) Bootstrapping conditional gans for video game level generation. arXiv preprint arXiv:1910.01603

  138. Treanor M, Blackford B, Mateas M, Bogost I (2012) Game-o-matic: generating videogames that represent ideas. In: Proceedings of the the third workshop on procedural content generation in games, pp 1–8

  139. Tsujino Y, Yamanishi R (2018) Dance dance gradation: a generation of fine-tuned dance charts. In: International conference on entertainment computing. Springer, pp 175–187

  140. Volz V, Schrum J, Liu J, Lucas SM, Smith A, Risi S (2018) Evolving mario levels in the latent space of a deep convolutional generative adversarial network. In: Proceedings of the genetic and evolutionary computation conference. ACM, pp 221–228

  141. Volz V, Justesen N, Snodgrass S, Asadi S, Purmonen S, Holmgård C, Togelius J, Risi S (2020) Capturing local and global patterns in procedural content generation via machine learning. In: Proceedings of the 2020 IEEE Conference on Games (CoG)

  142. Walton N (2019) AI Dungeon 2: creating infinitely generated text adventures with deep learning language models. https://pcc.cs.byu.edu/2019/11/21/ai-dungeon-2-creating-infinitely-generated-text-adventures-with-deep-learning-language-models/. Accessed 2 May 2020

  143. Wang T, Kurabayashi S (2020) Sketch2map: a game map design support system allowing quick hand sketch prototyping. In: Proceedings of the 2020 IEEE Conference on Games (CoG)

  144. Wong A, Wang GH (2017) Image\_retrieval\_demo: a demo for image retrieval. https://github.com/DoctorKey/image_retrieval_demo

  145. Wulff-Jensen A, Rant NN, Møller TN, Billeskov JA (2017) Deep convolutional generative adversarial network for procedural 3D landscape generation based on DEM. In: Interactivity, game creation, design, learning, and innovation. Springer, pp 85–94

  146. Yang Z, Sarkar A, Cooper S (2020) Game level clustering and generation using Gaussian mixture VAEs. In: Proceedings of the sixteenth annual AAAI conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2020). AAAI

  147. Yannakakis GN, Togelius J (2011) Experience-driven procedural content generation. IEEE Trans Affect Comput 2(3):147–161

    Article  Google Scholar 

  148. Yannakakis GN, Togelius J (2018) Artificial intelligence and games. Springer. http://gameaibook.org

  149. Yannakakis GN, Liapis A, Alexopoulos C (2014) Mixed-initiative co-creativity. In: Proceedings of the 9th conference on the foundations of digital games

  150. Yoo B, Kim KJ (2016) Changing video game graphic styles using neural algorithms. In: 2016 IEEE conference on Computational Intelligence and Games (CIG). IEEE, pp 1–2

  151. Yumer ME, Asente P, Mech R, Kara LB (2015) Procedural modeling using autoencoder networks. In: Proceedings of the 28th annual ACM Symposium on User Interface Software & Technology, UIST ’15. Association for Computing Machinery, New York, NY, USA, pp 109–118. https://doi.org/10.1145/2807442.2807448

  152. Zafar A, Irfan A, Sabir MZ (2019) Generating general levels using Markov chains. In: 2019 11th Computer Science and Electronic Engineering (CEEC). IEEE, pp 134–138

Download references

Acknowledgements

J. Liu was supported by the National Key R&D Program of China (Grant No. 2017YFC0804003), the National Natural Science Foundation of China (Grant No. 61906083), the Guangdong Provincial Key Laboratory (Grant No. 2020B121201001), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (Grant No. 2017ZT07X386), the Science and Technology Innovation Committee Foundation of Shenzhen (Grant No. JCYJ20190809121403553), the Shenzhen Science and Technology Program (Grant No. KQTD2016112514355531) and the Program for University Key Laboratory of Guangdong Province (Grant No. 2017KSYS008). S. Risi was supported by a Google Faculty Research award and a Sapere Aude: DFF-Starting Grant. A. Khalifa and J. Togelius acknowledge the financial support from National Science Foundation (NSF) award number 1717324 - “RI: Small: General Intelligence through Algorithm Invention and Selection”. G. N. Yannakakis was supported by European Union’s Horizon 2020 AI4Media (951911) and TAMED (101003397) projects. 

Author information

Authors and Affiliations

  1. Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China

    Jialin Liu

  2. Modl.ai, Copenhagen, Denmark

    Sam Snodgrass, Sebastian Risi, Georgios N. Yannakakis & Julian Togelius

  3. New York University, New York, USA

    Ahmed Khalifa & Julian Togelius

  4. IT University of Copenhagen, Copenhagen, Denmark

    Sebastian Risi

  5. Institute of Digital Games, University of Malta, Msida, Malta

    Georgios N. Yannakakis

  6. Technical University of Crete, Chania, Greece

    Georgios N. Yannakakis

Authors
  1. Jialin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sam Snodgrass
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmed Khalifa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sebastian Risi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Georgios N. Yannakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Julian Togelius
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julian Togelius.

Ethics declarations

Conflict of interest

The authors declare that they have a conflict of interest with modl.ai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Snodgrass, S., Khalifa, A. et al. Deep learning for procedural content generation. Neural Comput & Applic 33, 19–37 (2021). https://doi.org/10.1007/s00521-020-05383-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05383-8

Keywords

Search

Navigation