1 Introduction

The rapid growth in computer technology has produced innovative communication technologies. These technologies allow for fast exchange of information and knowledge (Paragina et al. 2010). The utilization of Information and Communication Technology (ICT) in education has lead to new technology-enhanced learning settings. This can assist in shifting the education approach from knowledge transmission into a model of practical experimentation and interaction (Blezu, and Popa 2008). This has played a crucial role in emerging the so- called E-learning.

E-Learning can be broadly defined as an approach to facilitate teaching and learning based on computer and Internet technologies (Hodhod et al. 2010). This includes technology that supplements traditional classroom training with web-based components and learning environments. In this case, the educational process is experienced either entirely online (i.e. Distance Learning) or partly online (i.e. Blended Learning).

Literature in the last decade indicates the effectiveness of blended learning in engaging learners and achieving better quality of learning. There are varying definitions for the term “blended learning”. In this research, Graham’s definition is adopted, which considers blended learning systems to “combine face-to-face instruction with computer-mediated instruction” (Graham 2006).

When reviewing literature one notices that research on developing blended courses describe: the final product (i.e. the learning objects in the blended course), the technologies utilized in this course, and the evaluation of the effectiveness of the course after applying it on students. In order to develop a Learning Object (LO), a systematic development process is required. Atif and colleagues define learning objects as self-contained instructional units that include heterogeneous learning sources (text, presentation, audio, or video) or a combination of any of these media (Atif et al. 2003). According to research, the most common development process is the ADDIE model, which was originally known as Instructional System Development as reported in (Molenda 2003). The model is divided into five phases: Analysis, Design, Development, Implementation and Evaluation. There are other instructional design models which are based on the ADDIE model. For example, Bower suggests a learning system engineering cycle which consists of seven phases. One of these phases is a separate stage where technology is selected based on formal and non- formal evaluations; this is because of the continual change of features presented by learning technologies (Bower 2006).

On the other hand, user-centered approach focuses on the end user requirements rather than on the technological aspects of system artifacts. Branon et al. (2001) utilize this approach to emphasize the importance of having student feedback at early stages of the development process. Iverson and Colky (2004) implement the scenario–based design approach, which provides initial user requirements for the e-course as scenarios taken from users. Also, Chiang recognizes the need to identify the learning objectives of online material and to perform audience analysis. However, no details were given on how to choose the technology that satisfies the learning objectives (Chiang 2002).

The analysis phase is often well explained in literature. It includes processes such as audience analysis, course content analysis and resource analysis (Piskurich 2003; Kulvietiene and Sileikiene 2006). On the contrary, the design phase is usually based on the experience of the instructional designer. The literature provides, for example, design guidelines extracted from: proven theories and experiments of the Instructional Design (ID), and the Human Computer Interaction (HCI) disciplines. However, there is a lack of systematic methods that can be used during the design phase to choose the best technology to implement the required LO. In general, the design process of LOs is difficult due to the following factors:

  • The technologies used to implement the LO are changing rapidly.

  • The designer should be aware of ID theories in order to achieve the desired learning objectives from the LO.

This paper proposes a systematic process approach to design a blended course starting with the traditional (face-to-face) course objectives. Furthermore, the paper presents an approach for evaluating the effectiveness of the selected technologies in the designed course. The evaluation measures the effectiveness in achieving the course objectives in terms of three dimensions: the format used to present information, the interaction level and the collaboration type. This evaluation is performed before starting the implementation stage, hence reducing the “trial and error” development cost.

The remainder of this paper is structured as follows: Section 2 provides background information related to Instructional Design theories that have been utilized in this research. The details of the proposed design process and evaluation approach are explained in Section 3. Section 4 presents a case study, where the proposed approach is applied to a System Analysis and Design course. Section 5 discusses the presented approach in relation to other existing design and evaluation approaches. Finally, the conclusions and future work are presented in Section 6.

2 Background

According to (Ally 2008), “the development of effective online learning material should be based on proven learning theories”. Bower proposed the following three components of “technology-based” learning design: content design, activity design and technology design (Bower 2008). This research adopts these components in order to establish a systematic design process for developing a blended course. Thus, three taxonomies have been adopted in the proposed approach: (1) Bloom Taxonomy is utilized for representing learning objectives of the course content; (2) Redeker Taxonomy is employed to classify the activity component; (3) Guerra Scale is used to classify available learning tools and technologies. The next sections provide brief descriptions of Bloom Taxonomy, Redeker Taxonomy and Guerra Scale.

2.1 Bloom taxonomy

Bloom Taxonomy (BT) was presented in 1956 by Benjamin Bloom (Bloom and Krathwohl 1956). The taxonomy is based on the behaviorist theory and it classifies the learning behavior into three domains: psychomotor, attitudinal and cognitive. In the cognitive domain, Bloom classifies the thinking process and the learning objectives into six levels that range from lower order thinking skills (LOTS) to higher order thinking skills (LOTS). Bloom’s taxonomy was then revised in 2001 by Lorin Anderson. The revised levels of thinking from simple to complex are: Remember, Understand, Apply, Analyze, Evaluate, and Create (Anderson and Krathwohl 2001).

Recently, Andrew Churches updated the revised Bloom’s taxonomy to account for new behaviors and actions that emerged with digital learning (Churches 2008). In Bloom’s digital taxonomy, collaboration has been added as an element that supports the learning process.

2.2 Redeker taxonomy

A lot of research provides evidence that students’ learning is improved by interaction (Redeker 2003; Ally 2008; Bower 2008). Redeker’s taxonomy classifies LOs into three types: (1) Receptive: where the learner is consuming information. (2) Internally interactive: where the learner interacts with the LO. (3) Cooperative: where the learner is required to perform communicative activities among other learners (Redeker 2003).

2.3 Guerra scale

The list of available technologies that can be utilized to develop LOs is rapidly expanding. Furthermore, the features provided by any of these technologies are changing; making it difficult to classify a particular system or tool into a specific category. This research utilizes Guerra Scale (Guerra and Heffernan 2004), which is developed by the American Society for Training and Development. The scale outlines the range of online content on a scale from one to ten based on several factors such as increased interactivity, complexity of development and functionality. Figure 1 shows the 10 levels of the Guerra Scale.

Fig. 1
figure 1

Guerra scale adopted from (Guerra and Heffeman 2004)

Full size image

3 Design process for developing blended courses

3.1 Introduction to Bloom-Redeker-Guerra mapping model

The proposed design process merges the three components of learning design suggested by Bower: content design, activity design, and technology design. Consequently, three taxonomies were chosen (as mentioned earlier) to represent these components. Each taxonomy has different levels of measuring the component. Thus, a mapping model is needed to merge the three components. In this research, a model called Bloom-Redeker-Guerra (B-R-G) has been devised based on ID learning theories to combine the three taxonomies as shown in Fig. 2.

Fig. 2
figure 2

B-R-G Model

Full size image

The B-R-G mapping model as shown in Table 1 has distributed the Bloom’s levels and the Guerra levels on the three Redeker interaction levels. The mapping model recognizes that in order to achieve Bloom’s low levels of thinking (Remember and Understand), only receptive learning is required, which can be supported by technologies in the Guerra Scale levels 1, 2, 4 and 5.

Table 1 B-R-G mapping model
Full size table

The Apply and Analyze learning objectives are better achieved through interactive environments. Interaction is supported by technologies in levels 3, 6, 8 and 10 of the Guerra Scale. Level 8 in the scale- Simulation- and level 10- Virtual Reality- can be designed to work on two modes: a stand alone mode and a collaborative mode. For this level of interaction, only the stand alone mode is required, where the learner interacts with the LO.

Finally, the high thinking levels-Analyze, Evaluate and Create- are better achieved through a collaborative environment. Technologies classified in the Guerra Scale levels 7, 8, 9 and 10 support these high thinking levels. Technologies in levels 8 and 10 in this case should support the collaboration mode. The “analysis” learning objective is repeated in both the second and the third Redeker levels. This is because the authors believe that the analysis can be supported by both interaction types and technologies. Table 2 shows examples of the most commonly used e-learning technologies that have been classified using the B-R-G mapping model.

Table 2 Examples of e-learning technologies mapped using the B-R-G Model
Full size table

3.2 The instructional design process of a blended course

Generally, during the development of a blended course, the course objectives and syllabus are determined during the analysis process. Then, the course content is divided into modules and the objectives of each module are identified. The realization of the module objectives produces a set of LOs. Figure 3 illustrates the main activities of the proposed design process, which consists of the following steps:

Fig. 3
figure 3

Design process activity diagram

Full size image
  1. 1.

    The process starts with the course objectives of the traditional (face-to-face) course. Thus, modules of the course and their objectives are identified.

  2. 2.

    For each module, BT is used to classify each module objective into the six thinking levels.

  3. 3.

    For each module, the B-R-G mapping model is used to propose a number of alternative technologies that can be used to implement the LOs. The designer may select one of the technologies that support this learning objective based on other constraints such as expertise in the technology, budget, and availability of the technology.

  4. 4.

    The designer may document the results of each iteration in a Design Sheet as shown in Table 3. The sheet reports: the module name, module objectives, BT level of the module objective and the proposed technology of the LOs.

    Table 3 Design sheet example
    Full size table
  5. 5.

    When all modules of the course are designed, the proposed course design is evaluated based on the approach that will be illustrated in the next section.

One should note that in order to achieve the desired module objectives, more than one LO is sometimes needed. For instance, in Table 3, Power Point and Flash technologies have been utilized in implementing two separate LOs to achieve the objectives of the second module.

3.3 The evaluation process of the designed blended course

The aim of the evaluation process in the proposed approach is to assess the effectiveness of the technologies used to implement the LOs. The idea of evaluating technologies utilized in e-learning before the development stage is not unique. For instance, Andrew Churches presented the Bloom’s Digital Taxonomy with a set of rubrics. Each rubric guides the usage of a digital learning technology to achieve Bloom’s learning levels (Churches 2008).

The proposed evaluation process is based on giving three weights for each chosen technology corresponding to the three Redeker interaction levels. In other words, the effectiveness of the selected technologies of the designed blended course has three dimensions that relate to the receptive, interactive and collaborative weights of the proposed LOs. The weights are chosen based on human computer interaction principles and theories. The three dimensions are:

  1. 1.

    The receptive dimension is based on the ‘multimedia principle’, which states that people learn more effectively from words and pictures than from words alone.

  2. 2.

    The interactive dimension is based on the ‘modality principle’, which states that presenting some of the instructional content in a visual mode and other parts of the material in an auditory mode can lead to more effective learning (Bower 2008).

  3. 3.

    The collaborative dimension is based on a study by Neale and colleagues, which proposes that proper levels of communication and coordination among groups can achieve effective learning (Neale et al. 2004). The scales of the three dimensions are shown in Table 4.

    Table 4 The scale of the three dimensions
    Full size table

The input of the evaluation process is the design sheet produced during the design process. The evaluation process proceeds as follows:

  1. 1.

    Identify the scale values for each technology in the suggested design. For instance Power Point slides that include text and graphics get a score of 2 in the receptive scale, 0 for the interactive scale and 0 for the collaborative scale.

  2. 2.

    Then, the summation of all the receptive scores for all the technologies of the LOs of the course is calculated.

  3. 3.

    Similarly, the summation of the interactive scores and the collaborative scores are calculated.

  4. 4.

    The percentage of the total receptive score to the total course weight is calculated. In the same way, the percentages of the total interactive and collaborative scores to the total course weight are calculated. Figure 4 summarizes the proposed evaluation process.

    Fig. 4
    figure 4

    The algorithm of evaluating the effectiveness of the technologies of the designed blended course

    Full size image

In the next section, a demonstration will be made to show how the design and evaluation process was applied during the development of a System Analysis and Design blended course.

4 Case study

The proposed design process has been applied on the System Analysis and Design-SAD. The course is offered to third-year students at the Faculty of Information Technology. It has been taught for the last 15 years and is divided into modules with clear objectives. The course is already blended on a minimum scale, because the course instructors use Power Point slides that contain course material and these slides are available to the students using the school's learning management system.

During the design phase, each module objectives are classified according to BT. Then, the B-R-G mapping model is used to obtain a set of selected technologies of LOs. Table 5 shows the design sheet for some of the course modules and two alternative designs using different technologies for implementing the suggested LOs.

Table 5 Part of the design sheet for SAD course
Full size table

The weight scales are used to evaluate the two suggested designs. Table 6 shows the weights for SAD-Design-1, which contains five proposed LOs. While, Table 7 shows the weights for SAD-Design-2 which contains six proposed LOs.

Table 6 The weights for SAD-design-1
Full size table
Table 7 The weights for SAD-design 2
Full size table

The results of the evaluation process show that the second design is more interactive, less receptive and less collaborative than the first suggested design. This result will allow the designer and the course expert to estimate the technologies capabilities before implementing any LO.

5 Discussion

5.1 Discussion of the design process

In general, the design process of any product is a complex process that involves making decisions on a number of aspects concerning the final product. At the high level design activities, decisions can be as broad as defining the goals and objectives of the final product and selecting the right technology to implement it. At the low level design activities, more specific details of the product functionalities and interfaces are determined. All the different design methodologies try to provide a set of activities or guidelines that address all or part of these decision making process to ensure the quality and/or the cost of the final product. For example, in scenario-based design, scenarios that describe the expected interactions between the users and the system are utilized to make sure that the designer achieves the user vision through the proposed solution. Meanwhile, participatory design actively involves stakeholders in the analysis and design stages to make sure that the user views and needs are well represented and satisfied by the final product. However, there is a lack in literature in addressing the question of how to choose the suitable e-learning technology that can better achieve the required objectives of the blended course.

The proposed approach in this study has attempted to address this particular decision related to technology selection within the design process. We argue that our model can be embedded with the design process of most of the design approaches. For instance, the proposed approach is suitable with the procedural nature of the ADDIE model, as described in Section 3.2. Also, the proposed approach can be utilized within the scenario-based model. For example in Iverson and Colky (2004), the authors present a five stages e-learning design process. The fourth stage is “deliver and facilitate” which follows the “creating scenarios” stage. In the “deliver and facilitate” stage, the process of choosing the delivery technology was left undetermined. This is where the proposed approach can be utilized. In this case, verbs in the scenario can be extracted and mapped to BT using keywords suggested by (Churches 2008).

5.2 Discussion of the evaluation process

In Section 3.2, after identifying the BT level of the required module objective, the level is mapped to one of Redeker’s interaction levels as well as to several suitable technologies. As explained earlier, there are other factors that will be used to make the final technology selection decision, which adds more dimensions and complexity to the selection decision. Thus, this evaluation process can assist the designer to have a preliminary assessment of the capabilities of the selected technologies to implement the LOs.

The three dimensional assessment utilized in the proposed evaluation process reflects the degree by which a particular technology supports the different levels of knowledge. As reported in (Ally 2008) and (Churches 2008) there are three levels of knowledge: knowing-what (facts), Knowing-how (processes) and knowing-why (higher-level thinking and contextual thinking). Hence, the weights in the proposed evaluation process were selected based on instructional design learning theories that support these levels. The receptive weight is based on the Behaviorist and Cognitive theories, which believe that facts should be provided in sequential and different presentation modes to facilitate the learning process. The interactive weight ensures that the suggested LO satisfies the constructivist theory in keeping learners active doing meaningful activities in order to achieve high-level knowledge processing. The collaboration weight is based on the socio- constructivist theory, which encourages collaborative and cooperative learning process (Ally 2008).

Iverson and Colky (2004) have adopted similar weights in their scenario-based e-learning design model for adult learning. Iverson and Colky base their design model on four practices: collaborative, constructive, contextual and meta cognitive. The first three practices are included in the proposed weights in this study; however the meta-cognitive criteria is less likely to be achieved by undergraduate learners, because of lack of in-depth critical thinking skills (Gruenbaum 2010).

The proposed weights in this approach are evaluating the effectiveness of the technology in implementing the LO. They are not designed to evaluate the LO, since this is usually done later in the development life cycle either when the LO is developed and implemented or at least when a prototype is produced. The evaluation of the LO is usually based on a number of usability evaluation techniques such as Heuristics evaluation or end-user evaluation (Ssemugabi, and Villiers 2007), which is outside the scope of this paper.

6 Conclusions

The lack of a clear systematic process to choose e-learning technology in designing blended courses led the authors to devise a design approach. An important aspect of this approach is that it blends with any e-learning development process. In this work, ID learning theories and taxonomies have been utilized to represent the three components of the technology-based learning design. A B-R-G mapping model has been proposed to assist developers to transform the contents and objectives of a traditional course into a number of suggested LOs for the blended course. This approach takes into consideration the learning objectives of the course while integrating technology with course content; thus assisting in improving the learning process.

In addition, an evaluation process has been proposed to measure the effectiveness of the suggested technologies of the designed blended course in terms of: course content format, interaction and collaboration in the course.

The approach has been applied to the System Analysis and Design course as a case study. Future work will perform an empirical study in order to evaluate the effectiveness and usability of different technologies in designing LOs. More specifically, an experiment will be designed to evaluate different modules of the System Analysis and Design course implemented using different e-learning technologies. The results of the experiment will assist in evaluating the weights of the evaluation process with user evaluation results.