Skip to main content
SpringerLink
Log in

Learning Analytics for Smart Learning Environments: A Meta-Analysis of Empirical Research Results from 2009 to 2015

  • Living reference work entry
  • First Online:
Learning, Design, and Technology

Abstract

Although several qualitative analyses appeared in the domain of Learning Analytics (LA), a systematic quantitative analysis of the effects of the empirical research findings toward the development of more reliable Smart Learning Environments (SLE) is still missing. This chapter aims at preserving and enhancing the chronicles of recent LA developments as well as covering the abovementioned gap. The core question is where these two research areas intersect and how the significant LA research findings could be beneficial for guiding the construction of SLEs. This meta-analysis study synthesizes research on the effectiveness of LA and targets at determining the influence of its dimensions on learning outcomes so far. Sixty-six experimental and quasi-experimental papers published from 2009 through September 2015 in the domain of LA were coded and analyzed. Overall, the weighted random effects mean effect size (g) was 0.433 (p = 0.001). The collection was heterogeneous (Qt(66) = 78.47). Here, the results of the statistical and classification processes applied during the meta-analysis process are presented and the most important issues raised are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Agudo-Peregrina, Á. F., Iglesias-Pradas, S., Conde-González, M. Á., & Hernández-García, Á. (2015). Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Computers in Human Behavior, 31, 542–550.

    Article  Google Scholar 

  • Ali, L., Asadi, M., Gašević, D., Jovanović, J., & Hatala, M. (2013). Factors influencing beliefs for adoption of a learning analytics tool: An empirical study. Computers & Education, 62, 130–148.

    Article  Google Scholar 

  • Ali, L., Hatala, M., Gašević, D., & Jovanović, J. (2012). A qualitative evaluation of evolution of a learning analytics tool. Computers & Education, 58(1), 470–489.

    Article  Google Scholar 

  • Aramo-Immonen, H., Jussila, J., & Huhtamäki, J. (2015). Exploring co-learning behavior of conference participants with visual network analysis of Twitter data. Computers in Human Behavior, 51, 1154–1162.

    Article  Google Scholar 

  • Baalsrud Hauge, J., Stanescu, I. A., Moreno-Ger, P., Arnab, S., Lim, T., Serrano-Laguna, A., … Degano, C. (2015). Learning analytics architecture to scaffold learning experience through technology-based methods. International Journal of Serious Games, 2(1), 29–44.

    Google Scholar 

  • Baker, R. S. J., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.

    Google Scholar 

  • Barla, M., Bieliková, M., Ezzeddinne, A. B., Kramár, T., Šimko, M., & Vozár, O. (2010). On the impact of adaptive test question selection for learning efficiency. Computers & Education, 55(2), 846–857.

    Article  Google Scholar 

  • Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R. (2013). Assessment in and of serious games: An overview. Advances in Human-Computer Interaction, 2013, 1–11.

    Google Scholar 

  • Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief (pp. 1–57). Washington, DC: US Department of Education, Office of Educational Technology.

    Google Scholar 

  • Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2006). Comprehensive meta-analysis [computer software]. Englewood, NJ: Biostat. https://www.meta-analysis.com/

  • Cambruzzi, W. L., Rigo, S. J., & Barbosa, J. L. V. (2015). Dropout prediction and reduction in distance education courses with the learning analytics multitrail approach. Journal of Universal Computer Science, 21(1), 23–47.

    Google Scholar 

  • Chen, C.-M., & Chen, M.-C. (2009). Mobile formative assessment tool based on data mining techniques for supporting web-based learning. Computers & Education, 52(1), 256–273.

    Article  Google Scholar 

  • Cheung Kong, S., & Song, Y. (2015). An experience of personalized learning hub initiative embedding BYOD for reflective engagement in higher education. Computers & Education, 88, 227–240.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Cooper, H. (1989). Homework. White Plains, NY: Longman.

    Book  Google Scholar 

  • Cooper, H. (2010). Research synthesis and meta analysis: A step by step approach (Applied Social Research Methods Series 4th ed., Vol. 2). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Credé, M., & Niehorster, S. (2012). Adjustment to college as measured by the student adaptation to college questionnaire: A quantitative review of its structure and relationships with correlates and consequences. Educational Psychology Review, 24(1), 133–165.

    Article  Google Scholar 

  • D’Mello, S. K. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology, 105(4), 1082–1099.

    Article  Google Scholar 

  • Dejaeger, K., Goethals, F., Giangreco, A., Mola, L., & Baesens, B. (2012). Gaining insight into student satisfaction using comprehensible data mining techniques. European Journal of Operational Research, 218(2), 548–562.

    Article  Google Scholar 

  • Demmans Epp, C., Bull, S. (2015). Uncertainty representation in visualizations of learning analytics for learners: Current approaches and opportunities. TLT, 8(3), 242–260.

    Google Scholar 

  • Economides, A. A. (2009). Adaptive context-aware pervasive and ubiquitous learning. International Journal of Technology Enhanced Learning, 1(3), 169–192.

    Article  Google Scholar 

  • Feidakis, M., Daradoumis, T., Caballe, S., & Conesca, J. (2013). Measuring the impact of emotion awareness on elearning situations. Proceedings of the Seventh International Conference on Complex, Intelligent, and Software Intensive Systems.

    Google Scholar 

  • Ferguson, R. (2012). The state of learning analytics in 2012: A review and future challenges (Technical Report KMI-2012). Retrieved from http://kmi.open.ac.uk/publications/techreport/kmi-12-01

  • Fidalgo-Blanco, Á., Sein-Echaluce, M. L., García-Peñalvo, F. J., & Conde, M. Á. (2015). Using learning analytics to improve teamwork assessment. Computers in Human Behavior, 47, 149–156.

    Article  Google Scholar 

  • Fulantelli, G., Taibi, D., & Arrigo, M. (2015). A framework to support educational decision making in mobile learning. Computers in Human Behavior, 47, 50–59.

    Article  Google Scholar 

  • Giesbers, B., Rienties, B., Tempelaar, D., & Gijselaers, W. (2013). Investigating the relations between motivation, tool use, participation, and performance in an e-learning course using web-videoconferencing. Computers in Human Behavior, 29(1), 285–292.

    Article  Google Scholar 

  • Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.

    Article  Google Scholar 

  • Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.

    Google Scholar 

  • Guo, W. W. (2010). Incorporating statistical and neural network approaches for student course satisfaction analysis and prediction. Expert Systems with Applications, 37(4), 3358–3365.

    Article  Google Scholar 

  • Guruler, H., Istanbullu, A., & Karahasan, M. (2010). A new student performance analysing system using knowledge discovery in higher educational databases. Computers & Education, 55(1), 247–254.

    Article  Google Scholar 

  • Hernández-García, A., González-González, I., Jiménez-Zarco, A. I., & Chaparro-Peláez, J. (2015). Applying social learning analytics to message boards in online distance learning: A case study. Computers in Human Behavior, 47, 68–80.

    Article  Google Scholar 

  • Hwang, G.-J. (2014). Definition, framework and research issues of smart learning environments – a context-aware ubiquitous learning perspective. Smart Learning Environments Open Journal, 1(4), 1–14.

    Google Scholar 

  • Joksimovic, S., Gasevic, D., Kovanovic, V., Adesope, O., & Hatala, M. (2014). Psychological characteristics in cognitive presence of communities of inquiry: A linguistic analysis of online discussions. The Internet and Higher Education, 22, 1–10.

    Article  Google Scholar 

  • Joksimović, S., Gašević, D., Loughin, T. M., Kovanović, V., & Hatala, M. (2015). Learning at distance: Effects of interaction traces on academic achievement. Computers & Education, 87, 204–217.

    Article  Google Scholar 

  • Kim, S., Song, S.-M., & Yoon, Y.-I. (2011). Smart learning services based on smart cloud computing. Sensors, 11(8), 7835–7850.

    Article  Google Scholar 

  • Koper, R. (2014). Conditions for effective smart learning environments. Smart Learning Environments Open Journal, 1(5), 1–17.

    Google Scholar 

  • Kovanović, V., Gašević, D., Joksimović, S., Hatala, M., & Adesope, O. (2015). Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions. The Internet and Higher Education, 27, 74–89.

    Article  Google Scholar 

  • Lee, J., Zo, H., & Lee, H. (2014). Smart learning adoption in employees and HRD managers. British Journal of Educational Technology, 45(6), 1082–1096.

    Article  Google Scholar 

  • Leong, C.-K., Lee, Y.-H., & Mak, W.-K. (2012). Mining sentiments in SMS texts for teaching evaluation. Expert Systems with Applications, 39(3), 2584–2589.

    Article  Google Scholar 

  • Lin, C.-F., Yeh, Y., Hung, Y. H., & Chang, R. (2013). Data mining for providing a personalized learning path in creativity: An application of decision trees. Computers & Education, 68, 199–210.

    Article  Google Scholar 

  • Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. Educause Review Online, 46(5), 31–40.

    Google Scholar 

  • Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53(3), 950–965.

    Article  Google Scholar 

  • Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599.

    Article  Google Scholar 

  • Minović, M., Milovanović, M., Šošević, U., & Conde González, M. Á. (2015). Visualisation of student learning model in serious games. Computers in Human Behavior, 47, 98–107.

    Article  Google Scholar 

  • Moissa, B., Gasparini, I., & Kemczinski, A. (2015). A systematic mapping on the learning analytics field and its analysis in the massive open online courses context. International Journal of Distance Education Technologies, 13(3), 1–24.

    Article  Google Scholar 

  • Moridis, C. N., & Economides, A. A. (2009). Prediction of student’s mood during an online test using formula-based and neural network-based method. Computers & Education, 53(3), 644–652.

    Article  Google Scholar 

  • Moridis, C. N., & Economides, A. A. (2012). Affective learning: Empathetic agents with emotional facial and tone of voice expressions. IEEE Transactions on Affective Computing, 3, 260–272.

    Article  Google Scholar 

  • Muñoz-Merino, P. J., Ruipérez-Valiente, J. A., Alario-Hoyos, C., Pérez-Sanagustín, M., & Delgado Kloos, C. (2015). Precise effectiveness strategy for analyzing the effectiveness of students with educational resources and activities in MOOCs. Computers in Human Behavior, 47, 108–118.

    Article  Google Scholar 

  • Noh, K. S., Ju, S. H., & Jung, J. T. (2011). An exploratory study on concept and realization conditions of smart learning. Journal of Digital Policy & Management, 9(2), 79–88.

    Google Scholar 

  • Papamitsiou, Z., & Economides, A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.

    Google Scholar 

  • Richardson, J. T. E. (2012). The attainment of White and ethnic minority students in distance education. Assessment & Evaluation in Higher Education, 37(4), 393–408.

    Article  Google Scholar 

  • Rienties, B. & Rivers, B.A. (2014). Measuring and understanding learner emotions: Evidence and prospects. Learning Analytics Review, 1, 1–30, ISSN: 2057–7494.

    Google Scholar 

  • Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135–146.

    Article  Google Scholar 

  • Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12–27.

    Google Scholar 

  • Romero-Zaldivar, V.-A., Pardo, A., Burgos, D., & Kloos, C. D. (2012). Monitoring student progress using virtual appliances: A case study. Computers & Education, 58(4), 1058–1067.

    Article  Google Scholar 

  • Rosenthal, R. (1984). Meta-analytic procedures for social research. Newbury Park, CA: Sage.

    Google Scholar 

  • Scott, K., & Benlamri, R. (2010). Context-aware services for smart learning spaces. IEEE Transactions on Learning Technologies, 3(3), 214–227.

    Article  Google Scholar 

  • Serrano-Laguna, A., Torrente, J., Moreno-Ger, P., & Fernández-Manjón, B. (2012). Tracing a little for big improvements: Application of learning analytics and videogames for student assessment. Procedia Computer Science, 15, 203–209.

    Article  Google Scholar 

  • Shute, V. (2011). Stealth assessment in computer-based games to support learning. In Computer games and instruction (pp. 503–523), Charlotte, NC: Information Age.

    Google Scholar 

  • Spector, J. M. (2014). Conceptualizing the emerging field of smart learning environments. Smart Learning Environments, 1(1), 1–10.

    Article  Google Scholar 

  • Suthers, D. D., & Verbert, K. (2013). Learning analytics as a “middle space.” In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 1–4). New York, NY: ACM.

    Google Scholar 

  • Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015). Time will tell: The role of mobile learning analytics in self-regulated learning. Computers & Education, 89, 53–74.

    Article  Google Scholar 

  • Tanes, Z., Arnold, K. E., King, A. S., & Remnet, M. A. (2011). Using signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4), 2414–2422.

    Article  Google Scholar 

  • Tatar, D., Roschelle, J., Vahey, P., & Penuel, W. R. (2003). Handhelds go to school: Lessons learned. IEEE Computer, 36(9), 30–37.

    Article  Google Scholar 

  • Tempelaar, D. T., Niculescu, A., Rienties, B., Giesbers, B., & Gijselaers, W. H. (2012). How achievement emotions impact students’ decisions for online learning, and what precedes those emotions. The Internet and Higher Education, 15(3), 161–169.

    Article  Google Scholar 

  • Tempelaar, D. T., Rienties, B., & Giesbers, B. (2014). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167.

    Article  Google Scholar 

  • Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167.

    Article  Google Scholar 

  • Terzis, V., Moridis, C. N., & Economides, A. A. (2012). The effect of emotional feedback on behavioral intention to use computer based assessment. Computers & Education, 59, 710–721.

    Article  Google Scholar 

  • Traxler, J. (2007). Defining, discussing and evaluating mobile learning: The moving finger writes and having writ … The International Review of Research in Open and Distance Learning, 8(2), 1–12.

    Google Scholar 

  • van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers & Education, 79, 28–39.

    Article  Google Scholar 

  • van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2015). Teacher regulation of cognitive activities during student collaboration: Effects of learning analytics. Computers & Education, 90, 80–94.

    Article  Google Scholar 

  • Veletsianos, G., Collier, A., & Schneider, E. (2015). Digging Deeper into Learners’ Experiences in MOOCs: Participation in social networks outside of MOOCs, Notetaking, and contexts surrounding content consumption. British Journal of Educational Technology, 46(3), 570–587.

    Article  Google Scholar 

  • Verbert, K., Manouselis, N., Drachsler, H., & Duval, E. (2012). Dataset-driven research to support learning and knowledge analytics. Journal of Educational Technology & Society, 15(3), 133–148.

    Google Scholar 

  • Westera, W., Nadolski, R., Hummel, H., & Wopereis, I. (2008). Serious games for higher education: A framework for reducing design complexity. Journal of Computer-Assisted Learning, 24(5), 420–432.

    Article  Google Scholar 

  • Westera, W., Nadolski, R., & Hummel, H. (2014). Serious gaming analytics: What students’ log files tell us about gaming and learning. International Journal of Serious Games (to appear). http://journal.seriousgamessociety.org/

  • Xing, W., Wadholm, R., Petakovic, E., & Goggins, S. (2015). Group learning assessment: Developing a theory-informed analytics. Educational Technology & Society, 18(2), 110–128.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Interdepartmental Programme of Postgraduate Studies in Information Systems, University of Macedonia, 156 Egnatia Avenue, Thessaloniki, Greece

    Zacharoula Papamitsiou & Anastasios A. Economides

Authors
  1. Zacharoula Papamitsiou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anastasios A. Economides
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zacharoula Papamitsiou .

Editor information

Editors and Affiliations

  1. College of Info, Ste G150, Univ of North Texas, Denton, Texas, USA

    Michael J Spector

  2. School of Education, Virginia Tech, Blacksburg, Virginia, USA

    Barbara B Lockee

  3. School of Education, Baker University, Overland Park, Kansas, USA

    Marcus D. Childress

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this entry

Cite this entry

Papamitsiou, Z., Economides, A. (2016). Learning Analytics for Smart Learning Environments: A Meta-Analysis of Empirical Research Results from 2009 to 2015. In: Spector, M., Lockee, B., Childress, M. (eds) Learning, Design, and Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-17727-4_15-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-17727-4_15-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Online ISBN: 978-3-319-17727-4

  • eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education

Publish with us

Policies and ethics

Search

Navigation