Keywords

1 Introduction

1.1 ACC and Usability

Automation systems, such as cleaning robots and automated driving systems, are becoming very popular in recent years because of the technological advances in this century. An automation system is a technology that autonomously behaves on behalf of humans [1]. In particular, the development and the prevalence of highly advanced automated systems have been remarkable. The Society of Automotive Engineers International [2] defines six levels of automation from complete manual driving to fully automated driving. In this year (i.e., 2018), automation systems at level 1 are being commercially produced. A level 1 system in a vehicle can sometimes assist the human driver with either steering the vehicle or with braking and accelerating it but not both simultaneously. In level 2 systems, the vehicle can itself actually control both the steering and the braking and accelerating simultaneously under some circumstances; however, these systems are still in their nascent stage.

Adaptive cruise control (ACC) is a system that can assist the human driver with braking and accelerating, and this is a level 1 system. The ACC senses the vehicle in front by using a radar sensor and controls the vehicle speed by keeping a certain distance between the two vehicles; the distance is determined by the driver. Also, when the vehicle in front is not sensed, the ACC maintains the speed set by the driver. Although the ACC has already been developed, usability evaluation is essential for these automated systems [3].

Usability tests are performed to develop and improve home electric appliances and information technology devices. Usability is defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [4]. Effectiveness is “the accuracy and completeness with which users achieve specified goals.” Efficiency is “the resources expended in relation to the accuracy and completeness with which users achieve goals.” Satisfaction is “the freedom from discomfort and positive attitudes toward the use of the product” [4].

A usability test is an evaluation method of artifacts and interfaces [5]. In the objective evaluation of usability, the measurable indexes should be determined according to the definitions of effectiveness, efficiency, and satisfaction; subsequently, human behaviors are measured based on the indexes [6]. For example, effectiveness, efficiency, and satisfaction could respectively be measured by the percentage of errors that occur during a task, the task completion time, and the frequency of using an artifact. Moreover, the subjective evaluation of usability is performed throughout the user experience, and this gives the designers helpful information for improving the user experience [6]. The subjective evaluation is usually done by responding to questionnaires and interviews.

Automation systems have features that cognitive artifacts, such as computers, do not have [3]. Therefore, it is difficult to evaluate the usability of the systems based on the traditional elements of usability, i.e., effectiveness, efficiency, and satisfaction. Maehigashi et al. [3] developed a usability evaluation questionnaire for automation systems. In this questionnaire, the following three new elements were added to evaluate the automation system usability: understandability, discomfort, and motivation. Understandability is the comprehensibility of the intentions of automation systems. Discomfort is the unpleasant feeling associated with behaviors of the automation systems; these are independent of task performance. Motivation is the desire of users to perform tasks by themselves.

1.2 Purpose

The purpose of this study is to investigate the factors that determine the usability evaluation of ACC. This investigation is important for understanding the features of a human–automation system interaction and for improving the usability of an ACC.

The factors to be investigated were driving control and driving style factors (see Fig. 1). The driving factor was divided into the following three categories: ACC driving control, ACC-driver driving control, and driver driving control. The ACC driving control factor indicated the ACC’s driving control or the results of the control when the ACC was activated. The ACC-driver driving control factor referred to the driver’s driving control or the results of the control when the ACC was activated. Also, the driver driving control factor expressed the driver’s driving control or the results of the control when the ACC was deactivated.

Fig. 1.
figure 1

Three types of driving control factors and driving style factor.

Full size image

Moreover, previous studies about the usability evaluation showed that the personal traits of users influence the usability evaluation of various kinds of systems [7]. Therefore, we decided to investigate the driving style factor because the driver’s driving style was considered to influence the usability evaluation of an ACC.

Most of the previous studies for ACCs used driving simulators e.g., [8]. A few studies used real cars with ACCs e.g., [9]. Furthermore, very few studies collected detailed driving data and discussed the ACC. In this study, we conducted an experiment with an actual car that was equipped with an ACC and investigated the usability evaluation of an ACC based on detailed driving data.

2 Experiment

In the experiment, the participants drove a Toyota Prius car with an ACC, which was called a radar cruise control system. The driving data was collected using the Vector CANcardXL interface. We collected 215 types of driving data at 60 Hz while driving; this included data, such as the activation and the deactivation of the ACC, the vehicle speed, the steering angle, the accelerator opening degree, and the brake hydraulic pressure. Moreover, to measure a driver’s driving style, we used a driving style questionnaire that comprised 18 questions about eight elements [10]. To assess the usability of the ACC, we used a usability questionnaire for the automation system that comprised 18 questions about six elements [3].

2.1 Participants and Procedure

Twenty persons (13 males and 7 females) participated in this experiment. The mean age was 41.95 years (ranging from 21 to 55 years). The mean experience as a driver was 21.95 years (ranging from 2 to 34 years). All the participants drove on a daily basis; however, they did not have any experience in driving with an ACC.

First, the participants answered the driving style questionnaire and were informed about the driving course. The driving course comprised six driving sections (see Fig. 2). The total length of the course was about 80 km. Next, the functions and the operation method of the ACC were explained to the participants. They were instructed to use the ACC as much as possible during driving. After that, they started to drive.

Fig. 2.
figure 2

Driving course

Full size image

While driving, the experimenter was seated in the front passenger seat. When there was no vehicle ahead, the ACC maintained a speed of 100 km/h, which was set up by participants at the beginning of each section. When there was a vehicle ahead, the participants freely chose a preferred distance from three options as the distance between their car and the vehicle ahead. Furthermore, the participants were allowed to ask questions about the operations of the ACC anytime during the driving.

At the end of each section, the participants stopped the car and evaluated the usability of the ACC by answering the usability questionnaire. After a 5 min break, they restarted their drive. After driving all the six sections, they were interviewed about their interaction with the ACC. Figure 3 shows a still from the video recordings during the driving experiment.

Fig. 3.
figure 3

Still from video recordings during driving. Camera 1 monitored the road ahead. Camera 2 monitored the driver. Camera 3 monitored the road behind. Camera 4 monitored the ACC dashboard indicator, which displayed the status of the ACC.

Full size image

2.2 Result

Analysis Method.

In each section, the average driving time was 518.36 s, while the average time during ACC driving control was 404.19 s. Also, the average time during ACC-driver driving control was 21.19 s. Finally, the average time of the driver driving control was 92.97 s.

To investigate the determinative factors, we conducted a multiple regression analysis. The ACC is a system that controls acceleration and braking; therefore, we chose 20 independent variables related to accelerating and braking from the driving data for analysis (Table 1). To detect multicollinearity, we calculated variance inflation factor (VIF) from the correlation coefficients. As a result, the VIF between the number of accelerations and the number of decelerations in the ACC driving control factor was more than 10 (VIF = 65.21). Therefore, the number of decelerations was eliminated from the data; consequently, 19 independent variables were used for the analyses. Also, eight out of the 19 independent variables, which were the average scores of each element of the driving style questionnaire, were extracted from the answers. We calculated the average scores of the answers to the two questions for each element in the driving style questionnaire.

Table 1. Independent variables for three types of driving control factors and driving style factor and the corresponding explanations of the variables.
Full size table

The dependent variables were the average scores of the answers to the three questions for each element in the usability questionnaire. Finally, we conducted multiple regression analyses of all the independent variables for each of the dependent variables (Table 2).

Table 2. Results of the multiple regression analyses. I, II, III, IV, V, and VI represent effectiveness, efficiency, satisfaction, understandability, discomfort, and motivation respectively. The values indicate the standard regression coefficients. +p < .10, *p < .05, **p < .01, ***p < .001
Full size table

In addition, 20 participants drove in six sections; therefore, we used 120 sampled data (20 participants × 6 sections) for the analyses. However, the driving style questionnaire was answered once for each participant. Hence, based on the assumption that the participants’ driving styles were consistent, we used the average score of each element for each participant six times for the analyses.

Influence on Each Usability Element.

First, the results of the analyses showed that regarding the driving control factor, the ACC and the ACC-driver driving control factors influenced the usability evaluation. However, the driver driving control factor did not influence the usability evaluation. The behaviors related to the ACC influenced the evaluation because the participants evaluated the ACC. Therefore, these results are considered to be valid. Moreover, the driver’s driving style factor influenced the evaluation of all the usability elements. The influence on each usability element is given below.

Effectiveness.

The participants who had scrupulous and stable driving tendencies and who naturally worried or were anxious about accidents evaluated the effectiveness of the ACC as high. These driving styles are considered to be related to safe driving. In fact, the ACC could keep a consistent distance between the car and the vehicle ahead and handle dangerous interruptions. The participants could drive safely with the ACC; therefore, the participants who had safe driving styles were assumed to provide a high evaluation of the effectiveness of the ACC. In the effectiveness evaluation, only the driver’s driving style factor influenced the usability evaluation.

Efficiency.

The participants who had a tendency of frequently stepping on the accelerator and had a short driving time in ACC-driver driving control evaluated the efficiency of the ACC as high. The frequent and short intervention in the ACC driving control improved the evaluation; therefore, the ease of intervention could be considered to influence the efficiency. The participants who easily intervened in the ACC driving control provided a high evaluation of the efficiency.

Satisfaction.

The participants who had impatient driving tendencies but were stable and were naturally anxious evaluated the ACC satisfaction level as high. The driving styles of drivers who were stable and naturally anxious also influenced the effectiveness. However, in evaluating the satisfaction, there was an influence of the impatient driving style. Impatient driving is related to actively overtaking the vehicle ahead and closing the distance with the vehicle ahead [10]. Therefore, the impatient driving style was considered to be related to the activeness of driving. In reality, when the vehicle ahead moved out of the range of the ACC radar sensor, such as when it changed the lanes, the ACC rapidly accelerated to the preset speed of 100 km/h at once. The participants could drive with safety and activeness using the ACC; therefore, the participants whose driving style was related to safety and activeness provided a high evaluation of the satisfaction level. In the evaluation of the ACC, safety as well as activeness is important.

Understandability.

The participants evaluated the understandability as high because the ACC showed numerous accelerations (or decelerations) and a high value for the rapid braking acceleration. These would become high because the ACC adjusted the vehicle speed according to the surrounding environmental changes. Therefore, when the drivers apparently recognized the ACC driving control, the understandability is assumed to increase. In their interviews after the driving experience, the three participants clearly stated that when the ACC controlled the driving, they tended to make dangerous interruptions and rapid decelerations to understand the deceleration control of the ACC.

Discomfort.

The discomfort was evaluated as low because of the small variance of the intervehicular distance in the ACC-driver driving control. The participants adjusted the vehicle position by intervening in the ACC driving; therefore, the variance of the intervehicular distance was considered to be small, and the discomfort in the ACC decreased. In other words, the discomfort associated with using the ACC decreased with the participants’ direct adjustments of the vehicle positions.

Motivation.

The independent variables that negatively influence the motivation tended to positively influence the other usability elements. The large variance in the intervehicular distance increased the efficiency and comfort, but it decreased the motivation. Moreover, the impatient driving style increased the satisfaction and understandability, but it decreased the motivation. These results indicated that the motivation for the drivers to drive by themselves would decrease because the ACC is useful.

3 Insight for Human-Automation System Interaction

First, the experimental results showed that the driver’s driving style influenced the usability evaluation of the ACC. Rasmussen et al. [7] experimentally indicated that the driver’s personality traits influence the usability evaluation of various kinds of systems. They showed that users with different personality traits or different interests tended to use different strategies for the same system; consequently, they evaluated the effectiveness, efficiency, and satisfaction differently. Our results were consistent with those of the previous studies for the usability evaluation of the ACC, i.e., the driver’s driving style influenced the usability evaluation. These results indicated that it is important to understand the results of the usability evaluation for automation systems by considering the personality traits. If there was no such consideration, the results of the usability evaluation could be misunderstood.

Next, the results of this study showed the influence on the usability evaluation, which could not be observed when cognitive artifacts were used. In using cognitive artifacts, such as computers, the users subjectively performed tasks, and the artifacts supported their activities [11]. In such situations, the ease of obtaining information inputs and outputs and the information processing speed and accuracy influence the usability evaluation of the artifacts [6].

However, in using automation systems, the systems subjectively conduct tasks, and the users monitor their activities and intervene in the system activities whenever necessary. In this study, the understandability results revealed that if the users recognize the system activities, the understandability of the system would increase. It is important for the users to clearly recognize the system activities in using automation systems. Furthermore, regarding the intervention in the system activities, the results about the evaluations of the efficiency and the discomfort revealed that the ease of use and the adjustability of the system activities influence the usability evaluation. Therefore, the efficiency and the discomfort would be enhanced if the users easily and directly adjusted the system activities even when the system does not behave according to the users’ intentions. These results are specific for interactions with automation systems, and similar results could not be observed while using cognitive artifacts.

Finally, previous studies have pointed out the issue of disuse atrophy, which causes reduced human ability by overreliance on automation systems [12]. The results of the experiment indicated that because the ACC is useful, the motivation for the drivers to drive by themselves would decrease. The participants who consider driving as a status symbol evaluated the motivation as high; therefore, the participants who considered driving as a valuable activity are motivated to drive by themselves. These results indicate that the users were influenced by the types of activities that they perceived as valuable and were motivated to perform by themselves. Disuse atrophy might be prevented by letting the users subjectively perform such activities or by only supporting the activities using the automation systems.

4 Suggestions for ACC Design

In this section, we discuss the principle of the ACC design by considering the experimental results and the interviews. From the experimental results, we can see that the ease of intervention in the ACC driving control influenced the evaluation of efficiency. Also, the adjustability of the vehicle position influenced the evaluation of the discomfort. To enhance efficiency and reduce the discomfort, we could improve the setup methods of the vehicle speed and the distance between the vehicles in the ACC driving control.

First, in regard to the setup methods of the ACC driving speed, the participants had to keep the control lever up or down until the speed was set as intended. In the interviews after the driving, some participants mentioned the setup method as “It was difficult to control the lever,” “The position of the lever was hard to find,” and “It takes too much time to control the lever.” To overcome the difficulty of setting up the vehicle speed, the lever could be replaced by a button or a switch so that the efficiency would be enhanced.

Next, for adjusting the intervehicular distances in the ACC driving control, the participants had to push a button to choose one of the three different distances. In the interviews, some participants made the following statements about the distance: “I would like to take the distance further” and “I would like to make a more minor adjustment.” The discomfort was assumed to decrease by making it possible to choose a distance from the more multiple distance levels and by making additional minor adjustments.

Moreover, the recognition of the ACC driving control influenced the understandability. Also, in the interviews, some participants described their understandability as follows: “In the beginning, I felt fear because I did not know how the ACC would behave, but I gradually understood the behavior and did not feel any fear” and “I understood the ACC behavior late in the driving.” To overcome this, we could display the state of the ACC driving control on the monitor in future; this would enable drivers to understand early the manner in which the ACC behaves. Also, drivers could experience using an ACC with a driving simulator; this would help them understand the ACC functioning before using it for real-life driving.

5 Conclusion

In this study, we investigated the determinative factors for the usability evaluation of an ACC. The results revealed that driving styles influenced all the usability elements. Also, the usability evaluation was not only influenced by the ACC driving control factor but also by the ACC-driver driving control factor. Based on our investigations into the human–automation system interactions, we have provided suggestions for improving the ACC design to enhance usability.