Keywords

1 Introduction

The research in MAR applications is effervescent. There is a vast interest in this field, particularly because of the recent technological evolution of smartphones. Many researchers try to steer their studies towards the adoption of AR technologies in contexts linked to cultural heritage. Some analyze the opportunities and the risks of using AR in tourism and history [1]. E.g., in [2], authors present an overview of the main challenges and prospects of using AR to enhance the tourism experience.

Others focus on specific test cases and implement custom AR solutions to solve particular problems [3]. Although still in the experimental phase, outdoor AR is targeted by studies such as [4]. In [5], authors make use of multimodal interfaces for offering a customizable experiences that can attract a broader spectrum of users in the case of indoor and outdoor cultural heritage sites. Considering the purpose of broadening the results and the contribution to science, some researchers present theoretical models of mobile augmented reality acceptance questionnaires, as it is in the case of the urban heritage tourism [6]. Their model evaluates AR applications through seven dimensions that should be, according to them, always considered in research: information quality, system quality, financial aspects, personal innovativeness, recommendations, risks and facilitating conditions. In [7], it is proposed a wearable augmented reality system for experiencing outdoor cultural heritage. The main issues with the system are the inconsistent tracking and the low brightness levels, especially when exposed to sunlight.

One interesting study which comes close to the subject of this manuscript is presented in [8], where authors investigate the use of a MAR application for cultural heritage, based on the Android platform. The proposed application superimposed virtual ancient elements on real scenes, thus creating the illusion of travelling in the past.

Sometimes, AR and VR technologies are mixed together to obtain a better user experience. Such is the case of the MAUS Museum [9]. Maintaining this idea, in [10], the authors studied the best approach to apply AR technologies to present the Egyptian cultural heritage. By choosing the best combination of techniques, users and sites, the appropriate AR display can be determined. The authors identified the ease of use and the technological and cultural accessibility as the main requirements of an AR application. A better user experience and an increased satisfaction are also the key elements pursued in [11]. Authors identified three factors that can maximize the tourist’s satisfaction of using an AR application for heritage site: technology readiness, the visual and the situational factor.

In [12], researchers combine the concept of tourist binoculars with the AR technology, in order to enhance to interaction. However, tourists and their wellbeing were targeted from almost 20 years ago in [13], where authors implement and archeo-guide based on an incipient form of AR. Closer to the present, in [14] it is presented a mobile outdoor AR application for city visualization. What is close to the subject of this study, a set of guidelines was compiled and presented. This approach is beneficial for the scientific community, as guidelines can be adopted and used from the initial stages of the research. Continuing on this idea, a survey on AR technologies, applications and limitations is presented in [15].

Sometimes, the little details are the things impress people the most. This concept was behind the study presented in [16], where authors developed an AR application to help visitors explore hidden features of cultural heritage artifacts.

After extensively investigating the state of the art, we can conclude that AR technologies are today widely used and marketed. Most of the literature focuses on applications (museum guides, building virtual and augmented installations and exhibitions, and so on) instead of analyzing the influence of this technology over the behavior of travelers. In this paper, we present 3 test cases where we have used different types of MAR applications, the methodology behind the undergone experiments and the participants’ characteristics. We apply the HARUS questionnaire presented in [17] to find out the details behind the manipulability and the comprehensibility of the applications. We compile based on these experiences several guidelines useful for anyone who wants to build an outdoor MAR application.

2 Materials and Methods

2.1 Procedure

We have constructed 3 different applications, for 3 different locations. The first scenario targeted the Etruscan tomb from Cecina, Italy. We have reconstructed in AR the entry, the ceiling and the central pillar of the tomb, and presented the application to the visitors of the museum. The second scenario was designed for the central square in front of the Colloseum from Rome, Italy. The year 2017 was the year which marked 2000 years from the death of Ovid, the famous roman poet. In order to celebrate this, we have built a MAR application which presented Ovid wandering around the Colloseum, reciting one of his famous poems. The third and last application was implemented in order to raise awareness about a lost monument from Brasov, Romania: the Reformed Church. The monument, a symbol of eclectic architecture which was built in 1893, was destroyed by the communist regime in order to make space for a state hotel (see Fig. 1).

Fig. 1.
figure 1

Pictures from all 3 applications, from Rome, Italy (top), Cecina, Italy (bottom-right) and Brasov, Romania (bottom-left)

Full size image

Inspecting an intangible monument (a catholic church destroyed decades ago) was a good chance to measure the advantages and the limitations of the AR technology.

During all tests from all 3 scenarios, we investigate how much time tourists spend using the MAR system, as compared to exploring the museums/locations without a digital guide. We measure the “fatigue” resulted from using the interface (e.g. brought by holding the tablet, or playing with the UI) and analyze perceptual issues based on a questionnaire, such as the tracking stability, the content quality and the display-induced cognitive load.

2.2 Participants

8 participants tested the MAR application with the Etruscan tomb, 63 participants took part in the study outside the Colloseum and 21 participants inspected the lost Reformed Church from Brasov. The details about all participants and presented in Table 1.

Table 1. Individual characteristics of participant for each location
Full size table

2.3 Questionnaire

We have used a slightly modified version of the HARUS questionnaire from [17] to assess the manipulability and the comprehensibility of the 3 MAR applications (Table 2):

Table 2. The questionnaire
Full size table

3 Results

The questionnaire evaluates manipulability and comprehensibility. Each is characterized by a set of eight items and participants used a 7 point Likert-type scale (1 – Strongly disagree, 7 – Strongly agree) to rate them. The questionnaire contains both positive and negative worded items, as such we first inverted the results for the negative scale items for all the users. The new scores were then converted to a range of 0 to 6 and presented graphically in Fig. 2.

Fig. 2.
figure 2

Boxplot showing the results for manipulability and comprehensibility

Full size image

We have computed separately the average time spent on using the AR handheld devices in selected locations (by tracking the time of each session inside the application, see Fig. 3) and the strong and the weak points of the application, by an open answer question.

Fig. 3.
figure 3

Box plots showing the time spent (in sec) using the AR application for each scenario

Full size image

4 Guidelines for Outdoor MAR Applications Designed for the Cultural Heritage Field

  • 1. MAR applications need to be built for specific contexts.

Contexts are usually different. Each monument, each place, has its own particularities. Given the high variation in history, scenarios and objectives, it is hard to build an application flexible enough to cover the entire spectrum of use cases.

  • 2. The historical data/context needs to be resumed well.

Because the presentation space in MAR applications is limited, special care should be put in retrieving and resuming the historical data.

  • 3. Overcome limitations with good user interfaces and high quality models and textures.

The technology readiness level will never offer a full satisfaction. Even the best smartphones still encounter issues such as the glare effect, the limited brightness and the relatively small screen. Some of these problems can be augmented with a good implementation of the AR content, including well-rendered textures and intuitive interfaces.

  • 4. Dynamic AR content and storytelling work better than simple 3D models display.

We’ve found that people are much more attracted by moving 3D objects, and retain much more information if it is served as a story instead of a billboard. MAR applications allow their creators to be as creative as possible. Implementing engaging AR scenarios will attract users with a multiplying effect.

  • 5. Offer means of interaction, options and alternatives

Some people like to read historical data, while others prefer to listen it from a synthesized voice. Some people like to use the gyroscope of the handheld devices to see the AR content, while others prefer the classic swipe. Make sure the MAR application offers various ways of interaction. Moreover, people like to rate things. A good idea is to offer the option of rating the content or the application itself.

  • 6. Build simple User Interfaces

Users appreciate simple and intuitive user interfaces, similarly to those found online. MAR applications should use a standard well-known UI, which share icons and button position with many of the mainstream applications.

  • 7. Use quality devices

Many participants, especially those under 40, pay much attention to the capabilities of the handheld device they received for the trials. A state-of-the-art smartphone or phablet was always much more appreciated than an older model, even though the user interface remained unchanged.

5 Conclusions

This paper presents the results of a study undergone by a total 92 participants, almost equality spread among age levels, which tested 3 different MAR applications in 3 different locations. We used the diversity of the setup in order to find a common ground on which we’ve built a set of guidelines useful for anyone who is interesting in building an outdoor MAR application, with a particular focus in the cultural heritage field.