Abstract
Modern game engines such as Unity make prototyping and developing experiences for virtual and mixed reality contexts increasingly accessible and efficient, and their value has long been recognized by the scientific community as well. However, these game engines do not have the capabilities to control virtual embodied characters, situated in such environments, with the same expressiveness, flexibility, and generalizability, as offered by modern BML realizers that generate synchronized multimodal behavior from Behavior Markup Language (BML). We implemented a Unity embodiment bridge to the Articulated Social Agents Platform (ASAP) to combine the benefits of a modern game engine and a modern BML realizer. The challenges and solutions we report can help others integrate other game engines with BML realizers, and we end with a glimpse at future challenges and features of our implementation.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Bruijnes, M.: Believable suspect agents: response and interpersonal style selection for an artificial suspect. Ph.D. thesis, University of Twente, Enschede, October 2016. sIKS dissertation series no. 2016–39. http://doc.utwente.nl/101371/
Hartholt, A., Traum, D., Marsella, S.C., Shapiro, A., Stratou, G., Leuski, A., Morency, L.P., Gratch, J.: All together now: introducing the virtual human toolkit. In: 13th International Conference on Intelligent Virtual Agents, Edinburgh, UK. http://ict.usc.edu/pubs/All%20Together%20Now.pdf
Kopp, S., Krenn, B., Marsella, S., Marshall, A.N., Pelachaud, C., Pirker, H., Thórisson, K.R., Vilhjálmsson, H.: Towards a common framework for multimodal generation: the behavior markup language. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS, vol. 4133, pp. 205–217. Springer, Heidelberg (2006). doi:10.1007/11821830_17
Kopp, S., van Welbergen, H., Yaghoubzadeh, R., Buschmeier, H.: An architecture for fluid real-time conversational agents: integrating incremental output generation and input processing. Journal on Multimodal User Interfaces 8(1), 97–108 (2014)
Niewiadomski, R., Bevacqua, E., Mancini, M., Pelachaud, C.: Greta: an interactive expressive eca system. In: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pp. 1399–1400. International Foundation for Autonomous Agents and Multiagent Systems (2009)
Reidsma, D., Charisi, V., Davison, D., Wijnen, F., van der Meij, J., Evers, V., Cameron, D., Fernando, S., Moore, R., Prescott, T., Mazzei, D., Pieroni, M., Cominelli, L., Garofalo, R., De Rossi, D., Vouloutsi, V., Zucca, R., Grechuta, K., Blancas, M., Verschure, P.: The EASEL project: towards educational human-robot symbiotic interaction. In: Lepora, N.F.F., Mura, A., Mangan, M., Verschure, P.F.M.J.F.M.J., Desmulliez, M., Prescott, T.J.J. (eds.) Living Machines 2016. LNCS, vol. 9793, pp. 297–306. Springer, Cham (2016). doi:10.1007/978-3-319-42417-0_27
Reidsma, D., Van Welbergen, H.: Multimodal plan representation for adaptable bml scheduling. In: Autonomous Agents and Multi-Agent Systems. Citeseer (2013)
Reidsma, D., van Welbergen, H.: AsapRealizer in practice-a modular and extensible architecture for a bml realizer. Entertainment Computing 4(3), 157–169 (2013)
Shapiro, A.: Building a character animation system. In: Allbeck, J.M., Faloutsos, P. (eds.) MIG 2011. LNCS, vol. 7060, pp. 98–109. Springer, Heidelberg (2011). doi:10.1007/978-3-642-25090-3_9
van Welbergen, H., Yaghoubzadeh, R., Kopp, S.: AsapRealizer 2.0: The next steps in fluent behavior realization for ECAs. In: Bickmore, T., Marsella, S., Sidner, C. (eds.) IVA 2014. LNCS, vol. 8637, pp. 449–462. Springer, Cham (2014). doi:10.1007/978-3-319-09767-1_56
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kolkmeier, J., Bruijnes, M., Reidsma, D., Heylen, D. (2017). An ASAP Realizer-Unity3D Bridge for Virtual and Mixed Reality Applications. In: Beskow, J., Peters, C., Castellano, G., O'Sullivan, C., Leite, I., Kopp, S. (eds) Intelligent Virtual Agents. IVA 2017. Lecture Notes in Computer Science(), vol 10498. Springer, Cham. https://doi.org/10.1007/978-3-319-67401-8_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-67401-8_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67400-1
Online ISBN: 978-3-319-67401-8
eBook Packages: Computer ScienceComputer Science (R0)