Abstract
Conversational agents that draw on the framework of academically productive talk (APT) have been lately shown to be effective in helping learners sustain productive forms of peer dialogue in diverse learning settings. Yet, literature suggests that more research is required on how learners respond to and benefit from such flexible agents in order to fine-tune the design of automated APT intervention modes and, thus, enhance agent pedagogical efficacy. Building on this line of research, this work explores the impact of a configurable APT agent that prompts peers to build on prior knowledge and logically connect their contributions to important domain concepts discussed in class. A total of 96 computer science students engaged in a dialogue-based activity in the context of a Human-Computer Interaction (HCI) university course. During the activity, students worked online in dyads to accomplish a learning task. The study compares three conditions: students who collaborated without any agent interference (control), students who received undirected agent interventions that addressed both peers in the dyad (U treatment), and students who received directed agent interventions addressing a particular learner instead of the dyad (D treatment). The results suggest that although both agent intervention methods can improve students’ learning outcomes and dyad in-task performance, the directed one is more effective than the undirected one in enhancing individual domain knowledge acquisition and explicit reasoning. Furthermore, findings show that the positive effect of the agent on dyad performance is mediated by the frequency of students’ contributions displaying explicit reasoning, while most students perceive agent involvement favorably.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adamson, D., & Rosé, C. P. (2013). Academically Productive Talk: One Size Does Not Fit All. In Artificial Intelligence in Education (AIED) 2013 Workshops Proceedings (p. 51–60).
Adamson, D., Ashe, C., Jang, H., Yaron, D., & Rosé, C. P, (2013). Intensification of group knowledge exchange with academically productive talk agents. In N. Rummel, M. Kapur, M. Nathan, & S. Puntambekar (Eds.), To See the World and a Grain of Sand: Learning across Levels of Space, Time, and Scale: CSCL 2013 Conference Proceedings (vol. 1, pp. 10–17).
Adamson, D., Dyke, G., Jang, H., & Rosé, C. P. (2014). Towards an agile approach to adapting dynamic collaboration support to student needs. International Journal of Artificial Intelligence in Education, 24(1), 92–124.
Asterhan, C. S., & Schwarz, B. B. (2016). Argumentation for learning: well-trodden paths and unexplored territories. Educational Psychologist, 51(2), 164–187.
Boeije, H. (2002). A purposeful approach to the constant comparative method in the analysis of qualitative interviews. Quality and Quantity, 36(4), 391–409.
Brandom, R. (1998). Making it explicit: reasoning, representing, and discursive commitment. Harvard University Press.
Cafaro, A., Glas, N., & Pelachaud, C. (2016). The Effects of Interrupting Behavior on Interpersonal Attitude and Engagement in Dyadic Interactions. In J. Thangarajah, K. Tuyls, C. Jonker, & S. Marsella (Eds.), 15th International Conference on Autonomous Agents and Multiagent Systems (pp. 911–920).
Chi, M. T. (2009). Active-constructive-interactive: a conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105.
Dillenbourg, P., & Tchounikine, P. (2007). Flexibility in macro-scripts for computer-supported collaborative learning. Journal of Computer Assisted Learning, 23(1), 1–13.
Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6, 241–252.
Dyke, G., Adamson, D., Howley, I., & Rosé, C. P. (2013). Enhancing scientific reasoning and discussion with conversational agents. Learning Technologies, IEEE Transactions on, 6(3), 240–247.
Fischer, F., Kollar, I., Mandl, H., & Haake, J. (Eds.) (2007). Scripting computer-supported communication of knowledge. Cognitive, computational and educational perspectives. New York: Springer.
Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56–66.
Goodman, B. A., Linton, F. N., Gaimari, R. D., Hitzeman, J. M., Ross, H. J., & Zarrella, G. (2005). Using dialogue features to predict trouble during collaborative learning. User Modeling and User-Adapted Interaction, 15(1–2), 85–134.
Gulz, A., Haake, M., Silvervarg, A., Sjödén, B., & Veletsianos, G. (2011). Building a social conversational pedagogical agent: design challenges and methodological approaches. In D. Perez-Marin & I. Pascual-Nieto (Eds.), Conversational Agents and Natural Language Interaction: Techniques and Effective Practices (pp. 128–155). IGI Global.
Harrer, A., McLaren, B. M., Walker, E., Bollen, L., & Sewall, J. (2006). Creating cognitive tutors for collaborative learning: steps toward realization. User Modeling and User-Adapted Interaction, 16(3–4), 175–209.
Hayes, A. F. (2013). An introduction to mediation, moderation, and conditional process analysis: a regression-based approach. New York, NY: Guilford Press.
Hmelo-Silver, C. E. (2013). Multivocality as a tool for design-based research. In D. D. Suthers, K. Lund, C. P. Rosé, C. Teplovs, & N. Law (Eds.), Productive multivocality in the analysis of group interactions (pp. 561–573). New York: Springer.
Howley, I., Kumar, R., Mayfield, E., Dyke, G., & Rosé, C. P. (2013). Gaining insights from sociolinguistic style analysis for redesign of conversational agent based support for collaborative learning. In D. Suthers, K. Lund, C. P. Rosé, C. Teplovs, & N. Law (Eds.), Productive multivocality in the analysis of group interactions (pp. 477–494) .Springer US
Huitt, W. (2011). Bloom et al.’s taxonomy of the cognitive domain. In Educational Psychology Interactive. Valdosta, GA: Valdosta State University.
Kumar, R., & Rosé, C. P. (2011). Architecture for building conversational agents that support collaborative learning. IEEE Transactions on Learning Technologies, 4(1), 21–34.
Liu, C. C., & Tsai, C. C. (2008). An analysis of peer interaction patterns as discoursed by on-line small group problem-solving activity. Computers & Education, 50(3), 627–639.
Ludvigsen, S., & Mørch, A. (2010). Computer-supported collaborative learning: basic concepts, multiple perspectives, and emerging trends. In B. McGaw, P. Peterson, & E. Baker (Eds.), The international encyclopedia of education 3rd edition, volume 5 (pp. 290–296). Oxford, UK: Elsevier.
Magnisalis, I., Demetriadis, S., & Karakostas, A. (2011). Adaptive and intelligent systems for collaborative learning support: a review of the field. IEEE Transactions on Learning Technologies, 4(1), 5–20.
Michaels, S., & O’Connor, C. (2013). Conceptualizing talk moves as tools: professional development approaches for academically productive discussion. In L. B. Resnick, C. Asterhan, & S. N. Clarke (Eds.), Socializing intelligence through talk and dialogue. Washington DC: American Educational Research Association.
Michaels, S., O’Connor, C., & Resnick, L. B. (2008). Deliberative discourse idealized and realized: accountable talk in the classroom and in civic life. Studies in Philosophy and Education, 27(4), 283–297.
Michaels, S., O’Connor, M. C., Hall, M. W., & Resnick L. B. (2010). Accountable Talk Sourcebook: For Classroom That Works. University of Pittsburgh Institute for Learning. Retrieved on May 1, 2016, from http://ifl.pitt.edu/index.php/download/index/ats.
Noroozi, O., Teasley, S., Biemans, H. A., Weinberger, A., & Mulder, M. (2013). Facilitating learning in multidisciplinary groups with transactive CSCL scripts. International Journal of Computer-Supported Collaborative Learning, 8(2), 189–223.
Oehl, M., & Pfister, H. R. (2010). E-collaborative knowledge construction in chat environments. E-Collaborative Knowledge Construction: Learning from Computer-Supported and Virtual Environments, 54–72.
Papadopoulos, P. M., Demetriadis, S., & Weinberger, A. (2013). ‘Make it explicit!’: improving collaboration through increase of script coercion. Journal of Computer Assisted Learning, 29(4), 383–398.
Preece, J., Sharp, H., & Rogers, Y. (2015). Interaction design: beyond human-computer interaction. John Wiley & Sons.
Resnick, L. B., Michaels, S., & O’Connor, C. (2010). How (well structured) talk builds the mind. In R. Sternberg & D. Preiss (Eds.), From genes to context: new discoveries about learning from educational research and their applications (pp. 163–194). New York: Springer.
Rus, V., D’Mello, S., Hu, X., & Graesser, A. (2013). Recent advances in conversational intelligent tutoring systems. AI Magazine, 34(3), 42–54.
Sionti, M., Ai, H., Rosé, C. P., & Resnick, L. (2012). A framework for analyzing development of argumentation through classroom discussions. In N. Pinkwart & B. McLaren (Eds.), Educational Technologies for Teaching Argumentation Skills (pp. 28–55). Bentham Science Publishers.
Slavin, R. E. (1992). When and why does cooperative learning increase achievement? Theoretical and empirical perspectives. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in cooperative groups. The theoretical anatomy of group learning (pp. 145–173). Cambridge: Cambridge University Press.
Sohmer, R., Michaels, S., O’Connor, M. C., & Resnick, L. (2009). Guided construction of knowledge in the classroom. In B. Schwarz, T. Dreyfus, & R. Hershkowitz (Eds.), Transformation of knowledge through classroom interaction (pp. 105–129). New York: Taylor and Francis.
Stahl, G. (2015). Computer-supported academically productive discourse. Socializing intelligence through academic talk and dialogue, 213–224. Retrieved on May 1, 2016 from http://gerrystahl.net/pub/lrdc2015.pdf.
Stahl, G., & Rosé, C. P. (2011). Group cognition in online teams. In E. Salas & S. M. Fiore (Eds.), Theories of team cognition: cross-disciplinary perspectives. New York, NY: Taylor & Francis.
Stahl, G., Cress, U., Ludvigsen, S., & Law, N. (2014). Dialogic foundations of CSCL. International Journal of Computer-Supported Collaborative Learning, 9(2), 117.
Tegos, S. (2016). Web-based conversational agents for collaborative learning support (Unpublished doctoral dissertation). Aristotle University of Thessaloniki, Thessaloniki, Greece.
Tegos, S., & Demetriadis, S. (in press). Conversational agents improve peer learning through building on prior knowledge. Educational Technology & Society.
Tegos, S., Demetriadis, S., & Karakostas, A. (2014). Conversational Agent to Promote Students’ Productive Talk: The Effect of Solicited vs. Unsolicited Agent Intervention. In IEEE 14th International Conference on Advanced Learning Technologies (pp. 72–76).
Tegos, S., Demetriadis, S., & Karakostas, A. (2015). Promoting academically productive talk with conversational agent interventions in collaborative learning settings. Computers & Education, 87, 309–325.
Vogel, F., Wecker, C., Kollar, I., & Fischer, F. (2016). Socio-Cognitive Scaffolding with Computer-Supported Collaboration Scripts: a Meta-Analysis. Educational Psychology Review, 1–35.
Webb, N. M. (2009). The teacher’s role in promoting collaborative dialogue in the classroom. British Journal of Educational Psychology, 79(1), 1–28.
Weinberger, A. (2011). Principles of transactive computer-supported collaboration scripts. Nordic Journal of Digital Literacy, 6(3), 189–202.
Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46(1), 71–95.
Weinberger, A., Stegmann, K., & Fischer, F. (2007). Knowledge convergence in collaborative learning: concepts and assessment. Learning and Instruction, 17(4), 416–426.
Wolf, M. K., Crosson, A. C., & Resnick, L. B. (2005). Accountable talk in reading comprehension instruction. Regents of the University of California.
Acknowledgments
We are appreciative of Fotini Bourotzoglou’s contribution to this work.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tegos, S., Demetriadis, S., Papadopoulos, P.M. et al. Conversational agents for academically productive talk: a comparison of directed and undirected agent interventions. Intern. J. Comput.-Support. Collab. Learn 11, 417–440 (2016). https://doi.org/10.1007/s11412-016-9246-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11412-016-9246-2