Keywords

1 Introduction

A dashboard is a visual display of the most relevant information, which is consolidated and arranged on a single screen to be monitored at a glance and needed to achieve one or more objectives [1]. Siemens defined Learning Analytics (LA) as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” [2]. Whereas, the main question addressed by Erik Duval is about what exactly should be measured to get a deeper understanding of how learning takes place [3].

Therefore, a Learning Analytics dashboard often present information to resources used, time spent, social interactions, artefacts produced, and exercise and test results [4]. Thus, the students are able to monitor their learning efforts for reaching their intended learning outcomes more easily [5].

In this paper we discuss the design and development of Learning Analytics dashboards in the area Higher Education (HE). Our objectives are:

  1. 1.

    cover the demands of the different stakeholders,

  2. 2.

    maximize the mainstreaming potential and transferability to other contexts, and

  3. 3.

    make it available for everyone by developing in the path of Open Source.

The research concentrates on developing an appropriate concept to fulfil these objectives and finding a suitable technology stack. Therefore, we determine the capabilities and functionalities of the dashboard for the different stakeholders. This is of significant importance as it identifies which data can be collected, which feedback can be given, and which functionalities are provided.

The main demands of the stakeholders are the support of different data-sources as well as different types of data-sources. Therefore, we searched for appropriate technologies which support various kinds of sources. During this search, we focused on technologies which are available under an Open Source licenses to cover our third objective.

Further, we wanted to easily combine different technologies, so we decided to go with a modular architecture. This ensures that the stakeholders can choose between different software modules (e.g. they can choose to use proprietary software) for easy adjustment to their needs. We decided to use a design with three layers: the data collection, the search and information processing, and the data presentation. Additionally, this approach helped us with our second objective, the transferability to other contexts and the maximization of the mainstreaming potential.

The next section provides background about Learning Analytics and dashboards. The third section explains the development of our Learning Analytics dashboard as well as improvements made throughout the different versions. The last section discusses the restrictions and remarks on future work.

2 Background

The purpose of Learning Analytics is the understanding and optimization of learning and the environment in which it occurs trough measurement, collection, analysis, and reporting of data [6, 7]. Since Learning Analytics was first mentioned in the New Media Consortium (NMC) Horizon Report 2012, it has gained an increasing relevance [8]. Further, the Horizon Report 2013 defined Learning Analytics as one of the most important trends in technology-enhanced learning and teaching [9]. Learning Analytics evaluates user’s behavior in the context of teaching and learning. Further, it analysis and interprets this behavior to gain new insights to support stakeholders with models for improving learning and teaching, organization and decision making [2].

One of the main goals of Learning Analytics is the return of the gained knowledge to learners and teachers; thereby, optimizing their learning and teaching behavior, promoting the development of skills in the area, and to better understand education as well as the connected fields such as university business and marketing [10, 11].

The Learning Analytics Life Cycle, which was introduced in 2015 by Khalil and Ebner and shown in Fig. 1, consists of four parts. The learning environment, where stakeholders such as learner or teachers produce data. The big data, which consist of massive amounts of different datasets. The analytics, which include different analytical techniques. And the Act, where objectives are achieved to optimize the learning environment [12].

Fig. 1.
figure 1

Learning Analytics life cycle [12]

Full size image

In Higher Education, Learning Analytics has been proven to help universities in strategic areas such as finance, resource allocation and student success [13]. Therefore, the universities are collecting more data than ever before to maximize the strategic outcomes [13]. Additionally, universities use methods of Learning Analytics to obtain findings on the progress of students, predict future behaviors and recognize problems at an early stage [10]. However, ethical and legal issues of collecting and processing the data of students are still seen as a barrier by the institutions [14, 15].

Learning analytics is applied in many adaptive learning systems such as MOOC-platforms [16, 17]. The results of such analysis are demonstrated on Learning Analytics Dashboards for better comprehension and further recognition of ongoing activities. An example of in detail analysis is the work on one-digit multiplication problems [18,19,20] or even beyond [21, 22]. Taraghi et al. analyzed at the first step the most prevalent error types and the statistical correlations between them in one-digit multiplication problems [23]. In a second step, they carried out a detailed analysis to highlight the misconceptions of students that are of higher relevance providing hints at probable reasons behind them [24, 25]. The discovered learning paths and difficulty levels were also used to build learner profiles [26]. Last but not least they applied Bayesian models and probabilistic graphical models for modeling misconceptions [27]. According to Verbert et al., evaluating Learning Analytics dashboards is often complex and still little is known about the usefulness to solve real issues and needs of students and teachers [4].

3 Development of the Dashboard

This section discusses the development and implementation of our Learning Analytics dashboard. The following subsection describes the foundation of our development, provides an overview, and gives a short explanation of the technologies used.

3.1 Foundation of the Development

We used the Elastic StackFootnote 1 as foundation for our development. The Elastic Stack combines Logstash, Elasticsearch, and Kibana and provides a powerful platform for indexing, searching, and analyzing data. Additionally, each of these technologies is available under the Apache 2.0 Open Source LicenseFootnote 2.

Logstash has been proven to be a powerful data collection, enrichment, and transportation pipeline with connectors to common infrastructure for easy integration. It is designed to efficiently process log, event, and unstructured data sources for distribution into a variety of outputs [28].

ElasticsearchFootnote 3 provided the abilities to search and process the information. It is a popular and powerful distributed search and analytics engine. It is based on Apache LuceneFootnote 4 and designed for horizontal scalability, reliability, and easy management. It combines the speed of search with the power of analytics via a sophisticated, developer-friendly query language covering structured, unstructured, and time-series data [29].

For the data presentation and visualization we used KibanaFootnote 5, an open source data visualization platform that allows interaction through powerful graphics. It provides various visualizations, such as histograms or geomaps, which can be combined into custom dashboards [30].

Unfortunately, Elastic Stack lacks security features, which forced us to add Search GuardFootnote 6 to our concept. This Elasticsearch plugin offers encryption, authentication, and authorization. It builds on Search Guard SSL and provides pluggable authentication modules. Search Guard is an alternative to ES ShieldFootnote 7, and offers all basic security features for free. Search Guard is available under the Apache 2 Open Source License. An overview of the concept is shown in Fig. 2.

Fig. 2.
figure 2

Initial setup of our dashboard

Full size image

This initial setup of the technology stack handles multiple data-sources very well, has a good scalability, and presents a custom and dynamic dashboard with good and easy visualizations. Unfortunately, there are problems with handling complex queries and despite Elasticsearch is able to handle minor data-relationships, it by design does not support them very well. As a result, executing queries costs more resources and take more time. Further, Kibana cannot handle data-relationships at all. A workaround for this problem is to combine the linked data-sources in Logstash and insert this combined-data into Elasticsearch, such as adding the user-data to each entry of a Learning Management System (LMS) actions). This might work for small settings, but is very resource intensive. Additionally, many-to-many relationships cannot be modelled this way!

3.2 The Final Concept

Because of our self-imposed requirements, it is essential for our dashboard to handle data-relationships, that’s why we replaced the module for data presentation, Kibana with GrafanaFootnote 8, which is able to handle relationships. Similar to Kibana, Grafana is most commonly used for visualizing time series data and provides a powerful and elegant way to create, explore, and share dashboards. Additionally, Grafana comes with an own authentication layer [31].

With this configuration the data presentation layer was able to handle relationships between data, but the problems with complex queries remain as well as the relation-limitations of Elasticsearch.

In the next step, we tried to solve this problems by splitting up the gathering and the presentation of the information into two different technologies. Therefore, we used CodeIgnitorFootnote 9 in combination with Charts.jsFootnote 10 for the analysis and visualization of the data. On the one hand, CodeIgnitor is a powerful, light-weight PHP-framework, which consistently outperforms most of its competitors and responsible for the analysis of the data. On the other hand, Charts.js is a simple and flexible JavaScript charting library, which offers eight different ways to visualize the data, each of them animated and customizable. Both technologies are available under the open source MIT LicenseFootnote 11.

Additionally, we extended our technology stack by the powerful, object-relational database management system PostgreSQLFootnote 12 (short Postgres). This support for relational data gives Elasticsearch the liberty to concentrate on its benefits, the ability to handle Big Data. Postgres has earned a strong reputation for reliability, data integrity, and correctness. It is free and open-source software, released under the terms of the PostgreSQL LicenseFootnote 13 [32].

After this step, the setup was able to handle all relationships. Additionally, the new created layer was able to combine the queries and taking of the complexity of the queries themselves. Despite that, we lost the ability of creating custom dashboards in the frontend. This concept of the dashboard, which is currently in use, is shown in Fig. 3.

Fig. 3.
figure 3

Current/final setup of our dashboard

Full size image

4 Conclusion, Restrictions and Outlook

In this paper, we discussed the design and development of a Learning Analytics dashboard. We defined three objectives: the different demands of the stakeholders, the maximization of the mainstreaming potential, and the transferability to other contexts, and the development in the path of Open Source.

We fulfilled the demands of our stakeholders’ trough supporting various types of data sources. Further, we kept the mainstreaming potential and the transferability to other context at a maximum by using a modular architecture. Thereby, it is possible to replace any of those modules with proprietary software already in use at the different universities. Finally, we achieved our last objective by introducing fitting Open Source technologies for our concept. Additionally, we described in detail our way of finding these technologies and explained the pros and cons of the different technology stacks.

Initial results show that the data and analytics layering allows the usage of multiple data sets and analytics techniques in a single interface for both, visualizing and analyzing. It should be mentioned that our final concept complies with our current requirements. Through future changes of those requirements, it might be necessary to adapt our concept. We are currently working on the proof of concept for our implementation of the Learning Analytics dashboard in the context of a Small Private Online Course (SPOC) which we are going to publish soon.