Keywords

1 Introduction

The evaluation of digital artifacts and systems with users is fundamental to guarantee their effectiveness and efficiency not only in terms of usability but also in the user experience (UX) during the interaction. The literature provides a range of assessment methods/techniques with users that can vary in approach (qualitative and quantitative) and nature (behavioral and attitudinal) [1]. Methods/techniques with a quantitative approach evaluate users’ attitudes and/or behavior in the interaction with artifacts and systems in an indirect, usually remote, way. On the other hand, those with a qualitative approach directly evaluate possible attitudes and/or behavior of the users when using/interacting with digital artifacts/systems. This may occur in laboratory condition (e.g., interaction testing, interviews) or in real context of use (e.g., ethnographic field studies; diary/camera studies). One of the advantages of the qualitative approach is to enable more in-depth verification of limitations and shortcomings of digital artifacts and systems with users.

Among the qualitative techniques, the interaction test is highlighted in this article because it allows the direct observation of the use of digital artifacts and systems through key tasks selected to be performed by the users. The selection of these tasks is usually based upon previous evaluations with experts, such as heuristic evaluation and technical inspection. The results of expert evaluation make it possible to foreseen difficulties that users may have when using digital artifacts and systems which should be verified in the interaction test.

Aspects of the graphic interface design may jeopardize the use of digital artifacts when they do not either meet design principles or promote a pleasant UX. This is particularly relevant in the health education field, given the need to train health professionals through digital artifacts, such as e-books. Considering this, the present article shows the results of an interaction test of the SaiteBooker authoring tool for e-books, which was developed by the Open University of the Unified Health System (UNA-SUS) of the Federal University of Maranhão, Brazil.

1.1 Interaction Design and Graphic Interface of Digital Artifacts

A good interaction design of digital artifacts and systems led them to be usable, effective and easy to learn, so as to provide users with a pleasant experience. A good interaction design prevents negative aspects of user experience (UX), such as frustration and annoyance, while at the same time, may promote positive attitudes/behavior, such as enjoyment and engagement [2]. For that, the graphic interface should be a facilitator of the interaction in the use of digital artifacts/systems. In this sense, the design of icons, the composition and visualization of interface elements should ease interaction. A good interaction design is only possible with a good graphic interface design.

When developing a graphic interface, some design principles must be taken into account to achieve effectiveness in usability and UX in the use of digital artifacts. Among the principles of interface design the following are highlighted here: visibility, action feedback, constraints to delimit interaction, interface consistency, and social signifier [2, 3]. The former principle regards the interface elements that must be visible to be easily perceived by users. In this sense, graphic aspects such as color, size, simplicity of form, organization and hierarchy of the elements are paramount to promote visualization of icons, buttons and menus in the interface. The more visible the elements of the graphic interface are, the more users will know how to interact with them, using their functions. The principle of action feedback assures users that the action was actually performed, allowing them to proceed in using the artifact/system. The principle of constraints aims at preventing interactions that may lead users to unwanted paths/actions during the interaction. In this sense, it predicts errors in the interaction. Thus, it establishes the forms of interaction which are beneficial to users. The principle of consistency ensures that there is no variation in the way the interface presents its elements and their functions. Finally, the principle of social signifier refers to elements of the interface design that ‘give clues’ to users about the functions they convey, i.e., what they are for (e.g., icons and buttons).

The principles abovementioned are in alignment with the literature on heuristics for digital artifacts and systems [3,4,5,6,7]. Among them, it is worth mentioning the heuristics for interface design proposed by Nielsen in the 1990’s [8], but still in use since they aim at the design of intuitive graphic interfaces and systems. The author emphasizes that the system should employ words/language and concepts from users’ repertoire to facilitate interaction (Match between system and the real world); users should have control and freedom in interacting with the graphic interface as for instance undo/redo an action (User control and freedom). The graphic interface should also present elements easily recognizable by users (Recognition rather than recall), and the system must be flexible and efficient to accommodate the demands of both experienced and novice users (Flexibility and efficiency of use).

Therefore, heuristics and design principles aim for an optimal user experience (UX) when they interact with digital artifacts and systems. In this sense, it is worth mentioning the well-known UX Honeycomb diagram proposed by Morville [9], which is composed of seven key-qualities of digital artifacts/systems: to be useful, usable, desirable, accessible, credible, findable and valuable (Fig. 1). Some of these qualities are extrinsic to users, i.e., refer to the artifact/system itself, whereas others are intrinsic to users. These are related to users’ emotions and expectations which must be fulfilled by the artifact/system. In this sense, usable, useful, accessible and findable can be considered as extrinsic or objective qualities since they are in the domain of the artifact/system. On the other hand, desirable, credible and valuable would be the intrinsic or subjective qualities as they are in the domain of the users. Accordingly, for an optimal UX, an artifact/system should have a good usability (usable); provide the conditions to be used by people with disabilities (accessible); display easy-to-find elements/functionalities in the graphic interface (findable), make it possible to users to perform the intended tasks (useful), meet the aesthetic expectations of users (desirable), and promote trust and credibility (credible). As a result, the users will value the digital artifact/system (valuable). In this sense, it is possible to say that an artifact/system will be desirable, credible and valuable as long as it is useful, usable, accessible and (its elements are) findable.

Fig. 1.
figure 1

UX Honeycomb diagram by Morville [9] with indication of the intrinsic and extrinsic qualities proposed here.

Full size image

Considering that UX aspects are related to how users behave when interacting with artifacts and systems, it is important to understand users’ information behavior. Among the types of information behavior acknowledged in the literature, the information seek, information search and information use are highlighted here, as they are pertinent to users’ experience with digital artifacts/systems [5, 10]. Information seek regards satisfying users’ need to reach a goal, such as creating an e-book in a health topic. Information search regards the resources used to achieve a goal, such as looking for an authoring tool to create e-books. Finally, information use regards the physical and mental acts to utilize the acquired information, incorporating individual’s existing knowledge, such as employing previous experience with digital artifacts/systems to use an e-book authoring tool.

The seek and search behaviors are related to the need for cognition. Individuals aim at finding order and meaning in the environment, that is expressed as a need to know and a desire to be informed [10]. Users receive and shape information from their cognitive perspective, aligned to their needs and motivation. They also seek to engage and enjoy agile information processing when interacting with digital artifacts/systems [5].

Based on users’ information behavior, Lonsdale, Lonsdale and Lim [5] propose the Online Information Processing Model (Fig. 2) which considers three stages: (1) information need; (2) information seeking (users start a task, users continue the tasks and users find and extract information) and (3) information use. According to this model, users employ reading and navigation strategies when starting the task, and their task performance is influenced by factors the amount, quality, complexity and accessibility of information, and the time available to read the information. These factors may result in negative consequences, such as cognitive overload and strain, poor engagement, attention, retention and recall.

Fig. 2.
figure 2

The online information processing model based on Lonsdale, Lonsdale and Lim [5].

Full size image

However, these undesirable results can be minimized, or even prevented, by employing information design principles and good practices [5, 6] to promote a positive UX. For example, grouping content units in the graphic interface; limiting typographic fonts and colors; balancing texts and images may decrease users’ cognitive overload when interacting with digital artifacts/systems [5, 12]. Thus, graphic interface as well as interaction design are keys to an optimal UX. This is particularly important in the use of e-book authoring tools in the health field, since some content can be complex, requiring the inclusion of various educational resources (e.g., videos, animations about medical procedures). The following sessions briefly present the authoring tool for e-books in the health field, and the interaction test that evaluated its effectiveness in Brazil.

1.2 SaiteBooker: An Authoring Tool for E-Books

Several studies have investigated the efficacy of e-books in communicating content, pointing out their advantages and disadvantages, as well as readers’ preference for printed books or e-books [7, 10, 13,14,15,16,17,18,19,20]. Among the advantages of e-books, the literature points: their flexibility in the inclusion and updating of content; the possibility of adding hypermedia resources, videos and animations; and the financial and environmental gains. These regards the cost of producing an e-book that is lesser than of a printed book, and the fact that its production does not generate paper and ink consumption [17].

However, some disadvantages are mentioned in the literature regarding possible discomforts in reading on screen, problems in the use of e-books caused by drawbacks in the interface design, in navigation and interaction design [21]. In order to improve the development of e-books, recommendations have been made, such as the inclusion of hyperlinks in the topics of the table of contents, adjustment of font size to meet individual’s preferences/needs, and the inclusion of search resources [10]. Moreover, the employment of visual metaphors from the printed book (e.g., turn page) are suggested to facilitate readers’ acceptance of e-books [10].

In Brazil, the use of e-books has grown in the continuing education of health professionals with the creation of the Open University of the Unified Health System (UNA-SUS) in 2008. It is a network of 36 public universities that has offered more than 60 e-courses to 1,097,330 medical professionals. To ease the production of e-books on health education, the UNA-SUS of the Federal University of Maranhão (UFMA) developed the SaiteBooker, a collaborative and open-source authoring tool for e-books. The tool is structured in five main areas:

  1. 1.

    The project area to enter general information about the e-books project;

  2. 2.

    The theme area to create the layout features of the e-book (e.g., color, font size, line spacing) and/or to choose from a range of themes available (layout default);

  3. 3.

    The Media area to upload, store and add images to an ongoing project;

  4. 4.

    The Canvas area to typeset the e-book pages and;

  5. 5.

    The Export area to upload the created e-book.

To verify the effectiveness of the SaiteBooker, studies have been conducted with experts on technology and information design in Brazil, being: (a) a technical inspection followed by heuristic evaluation and FIP technique (frequency, impact and persistence) to hierarchize problems according to degree of severity attributed to them; and (b) hands-on workshops with developers and healthcare professionals. The results of each evaluation enabled adjustments in the authoring tool, generating improved versions for testing. The last stage of evaluation was the interaction test with healthcare professionals, which is described next.

2 Interaction Test of the SaiteBooker Authoring Tool

The interaction test was carried out to verify the usability and interactivity of the SaiteBooker authoring tool through 19 tasks to be performed by the participants. The tasks were selected based on the results of previous evaluations, and they regard: login and register users; create and rename an e-book project; create and edit pages and texts; copy and paste content; add images and videos; preview the e-book project in different screen sizes (laptop, cell phone and tablet); browse the e-book pages (forward/backwards); unpack files and share the project link.

A simultaneous interview with the interaction test was conducted, followed by a satisfaction questionnaire. The interview made it possible to learn about participants’ attitudes when performing the tasks; and to gather their opinions and suggestions to improve the tool. The research dimensions and metrics considered in the interaction testing are shown in Table 1.

Table 1. Dimensions and metrics used for the interaction test.
Full size table

The results were analyzed qualitatively, however figures were considered in order to identify possible trends in the results.

2.1 Participants, Material and Procedures

A total of 39 health professionals voluntarily took part in the interaction test. The material used were laptops presenting the SaiteBooker authoring tool, printed protocols in A4 sheet for the interview, satisfaction questionnaire and the Free and Informed Consent Term to be signed by the participants. The interaction test was performed individually and in isolation by each participant. While carrying on the 19 tasks, participants were asked questions regarding the tasks in the simultaneous interview. After the interaction test/interview, a satisfaction questionnaire was delivered to the participants. Their responses and reactions were recorded in writing.

3 Results and Discussion

Most participants were female (N = 31), and the main characteristics of their profile (occurrences of responses) were: age range of 26 to 35 years (N = 17); expertise in the field of psychology (N = 10), and experience in the area of health between 5 and 15 years (N = 14).

The results of the task performance indicate that the majority of participants (N = 34) completed the 19 tasks satisfactorily (N = 548 out of 704 completions). In general, the interface of the SaiteBooker tool was positively assessed on the organization and size of its graphic elements, hierarchy, color and simplicity of the layout. Incorrect actions have not occurred during the test, since the system prevented interactions that could lead participants to unwanted paths/actions while performing the tasks. Most of the icons and buttons were easily located on the interface, and their functions were in most cases understood.

As for satisfaction with the SaiteBooker tool, participants’ responses were positive about the organization and aesthetic aspects of its graphic interface (N = 20). They also considered the tool interesting (N = 21) and visually enjoyable (N = 20). In general, participants declared themselves very satisfied with the tool, and would recommend the SaiteBooker to other health professionals (N = 19).

However, problems were detected regarding interaction and navigation. A total of 106 attempts to activate non-clickable elements of the interface occurred during interaction with the tool. Doubts about how to carry on some tasks led participants to interrupt the task for not knowing how to proceed (N = 158 task interruptions). As a consequence of these problems, several participants requested the researcher’s help to complete the tasks (N = 145 requests). The most difficult tasks were adding images and videos to the pages of the e-book project; copy and paste content from external links; editing text; and return to pages to insert content into the e-book project. The results lead to consider the following aspects as the main causes of the problems found by the participants:

  • Excess of actions to use a function

  • Inappropriate labels (wording) for icons

  • Lack of feedback regarding action completion

  • Poor differentiation between clickable and non-clickable elements

  • Use of non-familiar graphic representation for icons

  • Lack of visibility of functions to edit texts (hidden menu)

An example of the excess of actions to use functions of the tool is the insertion of texts (e.g., headings, captions). For that, participants should click on an icon of the menu and drag it to the editable area of the screen (Fig. 3). These actions were not easily inferred by the participants, and therefore they found inserting texts difficult during the interaction test. The actions of clicking and dragging the icons to add text in a page may have required a greater cognitive effort of the participants. This may be due to inserting text in other text editors (e.g., Word) can be done by just positioning the cursor on the editable area and then click. Moreover, participants could not promptly find the icons for adding and/or editing headings/titles. This was because participants were not acquainted with icons with the letter ‘H’ to convey these concepts. Thus, they had to search for the labels of the icons to be able to perform the task, what may have required a greater cognitive effort.

Fig. 3.
figure 3

Inclusion of headings in the page of an e-book by clicking and dragging the icons

Full size image

Another example of drawbacks in the interaction test of the SaiteBooker regards the lack of visibility of the text editing functions (e.g., alignment, type size) which were in a hidden menu (Fig. 4). To view this menu, participants should pass the mouse over the area next to the inserted text. This was also not easily inferred by the participants, making finding the menu difficult. A possible reason for that could be it is not common for text editing features to be in hidden menus.

Fig. 4.
figure 4

Hidden menu with text editing features.

Full size image

Regarding the lack of feedback in action completion, this has particularly affected the task of inserting images and videos in the pages of the e-book project. Participants expressed their doubts about the completion of this task during the interviews, leading to an insecure behavior.

Analyzing the results in light of the literature, the SaiteBooker authoring tool is in line with some principles and recommendations of the literature, but in dissonance with others. The tool met the following principles: constraints to delimit interaction; user control and freedom in interaction and flexibility of use [8]. It is also possible to say that the graphic interface of the SaiteBooker met users’ aesthetic expectations and goals, thus, possessing the qualities of being desirable and useful [9]. Nevertheless, the results also indicate that the SaiteBooker tool has not fully met the following principles: visibility (hidden menu), action feedback (insert images/videos), interface consistency (clickable elements) and social signifiers (clues to infer actions) [3, 4]. This may have affected UX regarding the ease of finding the elements of the interface (findable) and the tool usability (usable).

With respect to the online information processing model [5], the results allow to infer that the stages of information need, information seeking (users start a task and users continue the tasks) and information use occurred satisfactorily during the interaction test. However, with regard ‘find and extract information’ at the stage of information seeking, cognitive overload and strain may have occurred in the use of the SaiteBooker tool. This seems to be due to the tool’s system and interface design have not fully taken into account participants’ previous experience with other digital artifacts/systems regarding certain tasks (e.g., text inclusion and editing). This is in dissonance with the principle of ‘Match between system and the real world’ [8], in which language and concepts of users’ repertoire must be employed in the design of digital artifacts and systems.

4 Conclusions and Final Considerations

Although the qualitative nature of the interaction test does not allow generalizations, some conclusions can be drawn based on the results. Overall, the healthcare professionals were satisfied with the SaiteBooker authoring tool and had use it adequately, since the difficulties they encountered regarded certain tasks only. Similarly, in the scope of human information behavior, most participants seem to have processed information satisfactorily, although problems were found at the stage of information seeking.

In terms of UX, the SaiteBooker tool was useful and desirable, despite not fully possessing the extrinsic UX qualities of usable and findable. Since the SaiteBooker was developed by the UNA-SUS/UFMA (Ministry of Health of Brazil), it is possible to conclude that the healthcare professionals would consider it a credible and valuable tool.

The problems identified in the interaction test made it possible to improve the SaiteBooker authoring tool and to propose requirements for future digital projects of UNA-SUS/UFMA. An example of the improvements made is shown in Fig. 5 regarding editing and adding texts in the title page, for that, now users just have to position the cursor directly on the marked fields.

Fig. 5.
figure 5

Improvements made in the graphic interface of the SaiteBooker based upon the outcomes of the interaction testing.

Full size image

The results of the interaction test aligned with the literature [1, 3, 6, 9, 20] also made it possible to propose recommendations for the design of the graphic interface of authoring tools. Those referring to clickable elements and presentation of the information are shown below:

Recommendations for clickable elements of the interface

  1. 1.

    Clearly identify clickable elements/areas;

  2. 2.

    Signal the links that have just been activated;

  3. 3.

    The icons should be self-explanatory in relation to their function;

  4. 4.

    Maintain consistency in icons/buttons (same icon, same purpose);

  5. 5.

    Use unambiguous terms for icons/buttons’ labels;

  6. 6.

    Use icons and buttons familiar to users;

  7. 7.

    Display menus, icons and buttons in the same location.

Recommendations for presenting information

  1. 1.

    Present amount of information adequate to memory retention.

  2. 2.

    Position information logically on the screen, following the order of tasks to be performed.

  3. 3.

    Related information should appear on the same screen and not on different screens connected by links.

  4. 4.

    Present the information on the screen in a concise and graphically simple way.

  5. 5.

    Maintain consistency in the presentation of information.

  6. 6.

    Displays error messages in a different way from the other contents.

  7. 7.

    Present messages of confirmation (feedback) in a different way from the other contents.