Abstract
One of the central problems in developing a spoken dialogue system (SDS) is in how the system makes the decision of “what to say next” at any specific point in a conversation. This selection of an appropriate action is the core problem of dialogue management (DM), and it depends on having a representation of the conversational context at each decision point. This context information could consist of, for example, what information has already been conveyed in the dialogue, what the user has said in the preceding utterance (according to a speech recogniser), and the length of the dialogue so far. Making decisions regarding what to say next has been approached in a variety of ways.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
See www.voicexml.org.
- 2.
Note that a common misunderstanding is that the Markov property constrains the state to exclude the dialogue history. However, we can employ variables in the current state which explicitly represent features of the history.
- 3.
This similarity measure is known as a linear kernel, see the discussion in [16].
- 4.
- 5.
References
Ai, H., Tetreault, J., Litman, D.: Comparing user simulation models for dialog strategy learning. In: Proc. of the North American Meeting of the Association of Computational Linguistics (NAACL), pp. 1–4. Rochester, New York, USA, April 2007
Asher, N., Lascarides, A., Lemon, O., Guhe, M., Rieser, V., Muller, P., Afantenos, S., Benamara, F., Vieu, L., Denis, P., Paul, S., Keizer, S., Degremont, C.: Modelling Strategic Conversation: the STAC project. The 16th workshop on the semantics and Pragmatics of Dialogue (SeineDial’12). Paris, 2012
Bennett, C., Rudnicky, A.: The carnegie mellon communicator corpus. In: Proc. of the International Conference of Spoken Language Processing (ICSLP), 2002
Bohus, D., Langner, B., Raux, A., Black, A.W., Eskenazi, M., Rudnicky, A.L: Online supervised learning of non-understanding recovery policies. In: Proc. of the IEEE/ACL workshop on Spoken Language Technology (SLT), pp. 170–173. Aruba, December 2006
Bos, J., Klein, E., Lemon, O., Oka, T.: DIPPER: Description and formalisation of an Information-State Update dialogue system architecture. In: Proc. of the 4th SIGdial Workshop on Discourse and Dialogue, 2002
Dethlefs, N., Cuayáhuitl, H.: Hierarchical reinforcement learning and hidden markov models for task-oriented natural language generation. In: Proc. of 49th Annual Meeting of the Association for Computational Linguistics, 2011
Dietterrich, T.G.: Machine learning. Nature Encyclopedia of Cognitive Science, 2003
Doran, C., Aberdeen, J., Damianos, L., Hirschman, L.: Comparing several aspects of Human-Computer and Human-Human dialogues. In: Proc. of the 2nd SIGDIAL Workshop on Discourse and Dialogue, 2001
Eckert, W., Levin, E., Pieraccini, R.: User modeling for spoken dialogue system evaluation. In: Proc. of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 80–87. Santa Barbara, CA , USA, December 1997
Forbes-Riley, K., Litman, D.: Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system. Computer Speech and Language 25(1), 105–126 (2011)
Fraser, N.M., Gilbert, G.N.: Simulating speech systems. Computer Speech and Language 5, 81–99 (1991)
Gargett, A., Garoufi, K., Koller, A., Striegnitz, K.: The GIVE-2 corpus of giving instructions in virtual environments. In: Proceedings of the 7th Conference on International Language Resources and Evaluation (LREC), 2010
Gasic, M., Jurcicek, F., Keizer, S., Mairesse, F., Thomson, B., Yu, K., Young, S.: Gaussian processes for fast policy optimisation of a pomdp dialogue manager for a real-world task. In: Proceedings of SIGDIAL, 2010
Gasic, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Young, S.: Training and Evaluation of the HIS POMDP Dialogue System in Noise. In: Proc. of SIGdial Workshop on Discourse and Dialogue, 2008
Gesmundo, A., Henderson, J., Merlo, P., Titov, I.: A latent variable model of synchronous syntactic-semantic parsing for multiple languages. In: CoNLL 2009 Shared Task, Conf. on Computational Natural Language Learning, 2009
Henderson, J., Lemon, O., Georgila, K.: Hybrid Reinforcement / Supervised Learning of Dialogue Policies from Fixed Datasets. Computational Linguistics 34(4), 487–513 (2008)
Janarthanam, S., Lemon, O.: Learning to Adapt to Unknown Users: Referring Expression Generation in Spoken Dialogue Systems. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 69–78. Uppsala, Sweden, July 2010
Janarthanam, S., Hastie, H., Lemon, O., Liu, X.: ’The day after the day after tomorrow? ’ A machine learning approach to adaptive temporal expression generation: training and evaluation with real users. In: Proceedings of SIGDIAL, 2011
Jönsson, A., Dahlbäck, N.: Talking to a computer is not like talking to your best friend. In: Proc. of the Scandinavian Conference on Artificial Intelligence, 1988
Koehn, P.: Europarl: A parallel corpus for statistical machine translation. In: Proceedings of the MT Summit, 2005
Larsson, S., Traum, D.: Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering Special Issue on Best Practice in Spoken Language Dialogue Systems Engineering 6(3-4), 323–340 (2000)
Lemon, O., Pietquin, O.: Machine Learning for spoken dialogue systems. In: Proc. of the International Conference of Spoken Language Processing (Interspeech/ICSLP), pp. 2685–2688. Antwerp, Belgium, September 2007
Mairesse, F., Gašić, M., Jurčíček, F., Keizer, S., Thomson, B., Yu, K., Young, S.: Phrase-based statistical language generation using graphical models and active learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL ’10, pp. 1552–1561. Stroudsburg, PA, USA (2010) Association for Computational Linguistics
Moore, R.K., Morris, A.: Experiences collecting genuine spoken enquiries using woz techniques. In: Proc. 5th DARPA workshop on Speech and Natural Language, 1992
Orkin, J., Roy, D.: The restaurant game: Learning social behavior and language from thousands of players online. Journal of Game Development 3(1), 39–60 (2007)
Parent, G., Eskenazi, M.: Toward better crowdsourced transcription: Transcription of a year of the let’s go bus information system data. In: Proc. of the IEEE/ACL Spoken Language Technology (SLT), 2010
Pieraccini, R., Suendermann, D., Dayanidhi, K., Liscombe, J.: Are we there yet? research in commercial spoken dialog systems. In: Proceedings of TSD’09, pp. 3–13, 2009
Pietquin, O., Geist, M., Chandramohan, S., Frezza-Buet, H.: Sample-Efficient Batch Reinforcement Learning for Dialogue Management Optimization. ACM Transactions on Speech and Language Processing 7(3), 21 (2011)
Pietquin, O., Hastie, H., Janarthanam, S., Keizer, S., Putois, G., van der Plas, L.: D6.5 annotated data archive. Technical report, CLASSiC project deliverable, 2011
van der Plas, L., Merlo, P., Henderson, J.: Scaling up cross-lingual semantic annotation transfer. In: In Proceedings of ACL/HLT, 2011
Prommer, T., Holzapfel, H., Waibel, A.: Rapid simulation-driven Reinforcement Learning of multimodal dialog strategies in human-robot interaction. In: Proc. of the International Conference of Spoken Language Processing (Interspeech/ICSLP), pp. 1918–1924. Pittsburgh, Pennsylvania, USA, September 2006
Raux, A., Bohus, D., Langner, B., Black, A.W., Eskenazi, M.: Doing research on a deployed spoken dialogue system: One year of let’s go! experience. In: Proc. of Interspeech, 2006
Rieser, V.: Bootstrapping Reinforcement Learning-based Dialogue Strategies from Wizard-of-Oz data. PhD thesis, Saarbrueken Dissertations in Computational Linguistics and Language Technology, vol. 28, 2008
Rieser, V., Kruijff-Korbayová, I., Lemon, O.: A corpus collection and annotation framework for learning multimodal clarification strategies. In: Proc. of the 6th SIGdial Workshop on Discourse and Dialogue, pp. 97–106. Lisbon, Portugal, September 2005
Rieser, V., Lemon, O., Liu, X.: Optimising Information Presentation for Spoken Dialogue Systems. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1009–1018. Uppsala, Sweden, July 2010
Rieser, V., Lemon, O.: Learning effective multimodal dialogue strategies from Wizard-of-Oz data: Bootstrapping and evaluation. In: Proc. of the 21st International Conference on Computational Linguistics and 46th Annual Meeting of the Association for Computational Linguistics (ACL/HLT), pp. 638–646. Columbus, Ohio, USA, June 2008
Rieser, V., Lemon, O.: Learning and evaluation of dialogue strategies for new applications: Empirical methods for optimization from small data sets. Computational Linguistics 37(1), 2011
Rieser, V., Lemon, O.: Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems. In: Proc. of the Conference of European Chapter of the Association for Computational Linguistics (EACL), 2009
Rieser, V., Lemon, O.: Reinforcement Learning for Adaptive Dialogue Systems: A Data-driven Methodology for Dialogue Management and Natural Language Generation. Theory and Applications of Natural Language Processing, Springer (2011)
Schatzmann, J., Thomson, B., Weilhammer, K., Ye, H., Young, S.: Agenda-based user simulation for bootstrapping a POMDP dialogue system. In: Proc. of the North American Meeting of the Association of Computational Linguistics (NAACL), pp. 149–152. Rochester, New York, USA, April 2007
Schatzmann, J., Weilhammer, K., Stuttle, M., Young, S.: A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. Knowledge Engineering Review 21(2), 97–126 (2006)
Stent, A., Walker, M., Whittaker, S., Maloor, P.: User-tailored generation for spoken dialogue: an experiment. In: Proc. of ICSLP, 2002
Stuttle, M.N., Williams, J.D., Young, S.: A framework for dialogue data collection with a simulated ASR channel. In: Proc. of the International Conference of Spoken Language Processing (Interspeech/ICSLP), Jeju, South Korea, October 2004
Sutton, R., Barto, A.: Reinforcement Learning. MIT Press (1998)
Thomson, B., Young, S.: Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems. Computer Speech and Language 24(4), 562–588 (2010)
Tvarožek, J., Bieliková, M.: Wizard-of-oz-driven bootstrapping of a socially intelligent tutoring strategy. In: Bastiaens, T., Ebner, M. (eds.) Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2011, pp. 3635–3644. Lisbon, Portugal, June 2011, AACE
Walker, M., Kamm, C., Litman, D.: Towards developing general models of usability with PARADISE. Natural Language Engineering 6(3), 363–377 (2000)
Whittaker, S., Walker, M., Maloor, P.: Should i tell all? an experiment on conciseness in spoken dialogue. In: Proc. European Conference on Speech Processing (EUROSPEECH), 2003
Williams, J., Young, S.: Using Wizard-of-Oz simulations to bootstrap Reinforcement-Learning-based dialog management systems. In: Proc. of the 4th SIGDIAL Workshop on Discourse and Dialogue, pp. 135–139. Sapporo, Japan, July 2004
Young, S., Gasic, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K.: The Hidden Information State Model: a practical framework for POMDP-based spoken dialogue management. Computer Speech and Language 24(2), 150–174 (2009)
Young, S.: Probabilistic methods in spoken dialogue systems. Philosophical Trans Royal Society (Series A) 358(1769), 1389–1402 (2000)
Acknowledgements
The research leading to these results has received partial support from the European Community’s Seventh Framework Programme (FP7) under grant agreement no. 216594 (classic project), from the EPSRC, project no. EP/G069840/1, and from the European Community’s Seventh Framework Programme (FP7) under grant agreement no. 269427 (STAC project), under grant agreement no. 270019 (SpaceBook project), under grant agreement no. 270435 (james project), and under grant agreement no. 287615 (PARLANCE project).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media New York
About this chapter
Cite this chapter
Rieser, V., Lemon, O. (2012). Developing Dialogue Managers from Limited Amounts of Data. In: Lemon, O., Pietquin, O. (eds) Data-Driven Methods for Adaptive Spoken Dialogue Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4803-7_2
Download citation
DOI: https://doi.org/10.1007/978-1-4614-4803-7_2
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-4802-0
Online ISBN: 978-1-4614-4803-7
eBook Packages: Computer ScienceComputer Science (R0)