Keywords

1 Introduction

This article presents lessons learnt from the project, implementation and evaluation of a wheelchair virtual simulator operated by a noninvasive and low cost brain-computer interface (BCI). A BCI measures the brain activity related to the user’s intention and translates into control signals, which are detected and decoded by applications [1–5]. The noninvasive BCIs can base themselves on electroencephalogram signals (EEG), device which distributes electrodes within the scalp and through them makes the electrophysiological brain activity log [1, 4, 6]. In this article, the user moves a virtual wheelchair through a blink of an eye. Therefore, the portable EEG was chosen, Neurosky Mindwave (MW) [7], which allows identifying a blink of an eye and through the strength, defines if they were voluntary or involuntary.

In this sense, [8] developed a noninvasive BCI based on motion envisioned engines, capable of controlling in a real environment, an unmanned aerial vehicle. In the training phase, this study also used a real vehicle simulator.

This is justified, since “simulation is the process of designing a computational model of a real system and conducting experiments with this model in order to understand its behavior and/or evaluate strategies for its operation”.

In addition, the simulation can be used when the real system does not exist yet as part of the real model planning [9]. [10] already investigated the potential use of the teaching simulations in situations where natural demonstrations were impossible to be carried out or potentially dangerous, as in a case of accidents. These factors contribute to our motivation in developing the simulator.

The simulator was developed using the game engine Unity3D. There are two scenarios, wheelchair control and drive system, detecting possible collisions system. In the simulator, the wheelchair speed is increased while it moves frontwards, being considered its weight, wheels friction with the ground and, any unevenness existing in the existing scenarios. The scenarios have targets, which appear on a predetermined manner, they may be collected by the users.

The evaluation’s goal was to establish the EEG MW and the wheelchair’s simulator usability. Nevertheless, the following questions were set, related to the usability goals.

  • Be effective and efficient: the Simulator must allow the wheelchair to be conducted through a blink of an eye, recognized by a BCI. Questions: Is the user able to conduct the wheelchair efficiently, and collect the objects which are in a route, in the shortest time possible? Is the use of EEG MW efficient to recognize voluntary and involuntary blinks?

  • Be safe on the use: the Simulator detects a possible wheelchair collision in the scenario and it disables the command which would take to such collision. Question: Was it necessary the use of the collision system?

  • Be useful: the Simulator has commands which allow using the wheelchair, as well as indicators of location of the next object to be collected. Question: does the wheelchair drive and control systems allow the user to move in a desired manner in the best way possible?

  • Be easy to learn how to use it: the Simulator has simple commands to move the wheelchair and spin it round, which are triggered by a BCI. Questions: Was it easy to use the BCI to conduct the wheelchair? Was it easy to learn how to conduct the wheelchair and collect the objects?

  • Be easy to remember how to be used: there was a stage of experimentation/BCI adjustment to recognize the simulator and experiment the EEG MW. Question: Is it important a user training phase, so the user interacts with BCI as well as with the wheelchair’s conducting system, before its use per se?

The Simulator’s evaluation was performed with users in a controlled environment who had to collect objects which appeared on the scenarios. Two experiments were performed, they varied in type and size of samples, implementation stages (BCI adjustment, simulator training, object collection, object stage), time for collecting (free or determined), wheelchair control system (forward, right, left; or forward, right, left, rear), collision system (forward, right, left; or forward, right, left, rear), measure of strength which identifies a voluntary blink (fixed or customizable). These variations aimed to reduce negative aspects of the use experience identified on Experiment 1 as well as tried to improve the positive aspects to Experiment 2.

To obtain efficiency and effectiveness measures, all participants received the same tasks in each experiment. Data collection took part from the use and questionnaires observation. There were also recording logs of user’s actions in which with the rest of the collected data enabled to answer questions related to the simulators and EEG MW usability as well as discuss about the participants felt regarding these resources.

There were positive results regarding the time used to perform tasks, decrease error rate, high acceptance of the wheelchair moving systems, reference of routes and collision prevention. However, the observation of the users behavior, confirmed by some given opinions showed that the use of the BCI did not respond to some participants commands, with such needed accuracy, which caused discomfort and dissatisfaction with BCI and with the manner of the commands collection to interact with the Simulator.

These issues, with the detail of project’s stages, implementation and Simulator’s evaluation, virtual wheelchair simulator, discussion of associated projects and lessons learnt are presented and discussed in this paper.

2 Related Papers

There are different tasks that relate the BCI use to control the wheelchair. Some studies use real wheelchairs [11–13] and others do it through virtual wheelchair [14, 15] in a computing environment. There are studies in which the chair’s control is by the blink of an eye [16], movement’s imagination [17] and selective attention [18]. There are systems that use mechanisms to prevent accidents involving wheelchairs [19]. Some studies use the BCI calibration sessions [20]. In addition to these, they are related to the wheelchair’s control through BCI and BCI hybrid use [21, 22].

3 The Simulator

3.1 Applied Technologies

It was chosen the portable EEG NeuroSky Mindwave (MW) [7], which has an electrode disposed in the prefrontal region, Fp1 in the 10–20 pattern and a reference electrode in the ear clip. Whereas it is unlikely possible to modulate the attention and relaxation levels with precision and control, it was decided to use the blink of an eye as a way to move and rotate the wheelchair in the Simulator. The MW is capable of measuring the strength with which it is carried out a blink and pass on this information on integral values, which may vary from 1 to 255, respectively from a light blink to a strong blink, or an involuntary blink to a voluntary blink. Within the simulator system, it is possible to configure a threshold to aim a voluntary blink. As an engine to the application, it was used Unity 3D, which besides being free, has a large user community. In order to model objects, it was used Autodesk 3DS Max, which features easy integration with Unity 3D.

For this study it was developed two scenarios:

  • Scenario A1: consists of a room with four sides and three creeping obstacles (Fig. 1A and B).

  • Scenario A2: it is based on an open recreational area of the university campus (Fig. 1C and D). It has a “mesh” object, which prevents the wheelchair to collide with other objects while being displaced or rotated.

Fig. 1.
figure 1

Scenarios

Full size image

To carry out the tasks in the scenarios, it was instantiated objects of the “Target” type, which appear in serial form inasmuch they are collected. The Simulator features an “Arrow” object, which indicates the direction of the next target to be collected, as these are not always in the active viewing area of the scenario.

The wheelchair was modeled as a reference [23]. There are two central wheels, which are responsible for the torque force that moves the chair, two rear wheels that support the central wheels with the directional control, and two front wheels, which help in the balance of the chair. In the wheelchair implementation it was defined its weight in addition to the approximate weight of a person between 18 and 34 years [24], in a total of 105 kg.

The virtual wheelchair has two maximum speed, 3.5 km/h and 7 km/h. The chair’s speed is increased while it is moved forward. The speed increase also considers the wheelchair’s total weight, the friction of the wheels with the ground and the existing gaps in the ground.

From the based command definition at the blink of an eye it was developed an arrows-based control system, which allows the user to control the wheelchair (Fig. 2). This follows a scanning system classified as simple, automatic and independent [25]. In this way, the scanning starts automatically and the moving and rotation arrows are highlighted by an interval of time, going to the next arrow, if it has not been chosen by the user. When the user blinks, the arrow is chosen and carries out its action immediately. To stop it, the user should blink again. The control system starts until the user selects the next action.

Fig. 2.
figure 2

Target indication arrow and scanning system.

Full size image

As error prevention, it was developed a sensors system to avoid collisions of the wheelchair with the scenario objects. Three sensors were included in each active side of the wheelchair. When a sensor detects a possible collision, the motion of the wheelchair is interrupted and the actions that led to this collision are disabled. Returning to a state of security, the controls are reactivated.

4 Usability Evaluation

4.1 Experiment 1

Profile of the Participants.

The sample was intentional and the experiment was carried out with students from a discipline of the 7th semester of an undergraduate course of Feevale University. The sample consisted of four students aged between 22 and 25 years, all male, healthy, who have not had previous contact with BCIs.

Data Collecting Tools.

It was observed the use, application of questionnaires and the events record logs. The log stores the date of usage, execution starting time, the scenario that is being executed (A1 or A2), maximum speed (3.5 km/h or 7 km/h), targeted collection time, total number of avoided collisions and ending time execution.

Methodology.

The experiment involves a period of Training and a Test phase, in that order. The objectives of the training phase were to experiment the EEG MW control modes and interact with the virtual wheelchair in the scenario A1. The test phase aimed to collect the targets of scenario A2 through the BCI usage.

In the training phase, there was an explanation of the functioning of the EEG MW and of the Simulator as well, and the participants had to collect 4 targets in the shortest possible time with the wheelchair’s maximum speed at 3.5 km/h.

The test phase consists of two sessions, with no time limit, with intervals of two minutes between them. The task was to collect 8 targets in the shortest possible time with the wheelchair’s maximum speed at 3.5 km/h in Session 1 and 7 km/h in Session 2. The targets were placed in the same positions in the two sessions. It was configured the blink threshold amounting to 75 for all users at all stages.

Findings.

To analyze the results it was defined a better time, which consisted in the shortest time by target, taking into account the participants logs. Figure 3 shows the graph of the participants performance in collecting the targets in Sessions 1 and 2 of the Test phase.

Fig. 3.
figure 3

Experiment 1: participants performance in Sessions 1 and 2 (Color figur online)

Full size image

From this experiment, it can be noted that:

  • (Be effective and efficient in the use) The scanning system which moves the wheelchair proved to be efficient, although there were difficulties related to the modulation of the blink of an eye. It is worth to emphasize that even though there is the blink threshold calibration, the EEG MW algorithm (and not of the Simulator) may encounter problems in detecting the blink as voluntary or involuntary. Still, there have been speed increasing in collecting the targets between the sessions, which may be due to the increasing of the wheelchair’s speed, of the training in the BCI use, in the applicant’s scanning system usage, in the prior knowledge of the scenario and location of the targets.

  • (Be safe in the usage) The collision system was activated by 2 participants who had recognition difficulties of their voluntary blink by the EEG MW.

  • (Be of good usage) There was no problem in the scanning system usage. There were suggestions to include a back arrow to facilitate maneuvers with the wheelchair.

  • (Be easy to learn) The performance improvement between the sessions may indicate a facility to learn how to use the Simulator, which can be due to the Training stage or the sequential collection targets. Despite the initial difficulties to interact with the EEG MW, the participants indicated that the Simulator is easy to use intuitively.

  • (Be easy to remember how to use) Users remembered how they should use the scanning system, how to prevent the activation of the collision system and how to adjust the blink. However, it cannot be observed if the improvement in the time was given by the training with EEG MW, knowledge of the scenario and targets location and/or continuous usage of the wheelchair’s Simulator.

In order to evaluate the need for a training phase and the refinement of the scanning system, it was carried out the Experiment 2.

4.2 Experiment 2

Profile of the Participants.

The sample was composed of nine volunteers, healthy, between the ages of 19 to 42, being that eight were male and one was female, without prior knowledge with the BCI.

Methodology.

Experiment 2 differs from Experiment 1 mainly due to the substitution of the Training phase by the EEG MW Calibration phase for voluntary blinks training without using the Simulator scenarios. There have also been changes in the scanning and in the motion systems of the virtual wheelchair, which now has a fourth arrow to allow the chair to be moved backwards. The test phase was similar to Experiment 1.

Findings.

Data were treated and represented similarly to Experiment 1. Figure 4 presents the participants’ performance graph to collect the targets in Sessions 1 and 2 of the Test phase.

From the findings, the following questions are answered:

  • In order to have a better user performance, it is necessary to have a training phase in the wheelchair’s Simulator or is it satisfactory to have a BCI calibration phase? This experiment showed that the continuous usage of the Simulator and consequently of the EEG MW, brings improvement in the participants performance. When comparing the data collected from the initial two experimental sessions of the two experiments, it is clear that there was no gain in the performance that there has been training.

  • A customization of the blink of an eye threshold would facilitate the Simulator’s usage instead of using a fixed value? Yes, in general, participants reported that they had ease of use after understand the pattern of the blinks. However, some participants reported difficulties of interaction, even though they have calibrated the EEG.

  • Backwards favors the replacement of the chair in the scenario to replace the use of the rotating arrows? Yes, the inclusion of this arrow allowed reducing the amount of avoided collisions and of the time while conducting the chair to collect the targets.

Fig. 4.
figure 4

Experiment 2: participants performance in Sessions 1 and 2 (Color figur online)

Full size image

5 Final Considerations

This paper presented the design, implementation and evaluation of a wheelchair simulator. To interact with the simulator it was chosen the EEG MW and the commands were activated by the blink of an eye. Two experiments with different samples were carried out. The studies showed that it is important to have an EEG calibration phase in order that the users can establish which blinks should be recognized as voluntary and involuntary. The Simulator interface to conduct the wheelchair was based on a scanning system, which did not present difficulties to the participants. It is noteworthy that in the second experiment the wheelchair could be moved backwards, which optimized the chair conduction process.

From this study, the following issues have been identified for future work:

  • Customization: the escalation time between the Scanning System arrows must be configured by the user. It is also considered the configuration possibilities for scenarios, targets, total time limit for collection and time target for collection.

  • Vision mode of the virtual camera: the third-person perspective should be changed to the first person.

  • User actions registration: for a better analysis, it is planned the threshold registration, the path traveled by the chair, being possible to check the collisions and the time that the chair moved around the scenario and the speed in which the wheelchair was at the moment.

  • Improve the collision system in order to recognize objects which are not only at the sensors height.

  • Carry out an evaluation with real wheelchairs users and include other interaction possibilities, besides the blink of an eye.

  • Disengage the Simulator’s control, making it independent from the control interface. Currently, the simulator is functional only with the use of MW. To expand the possibilities of simulator’s usage, a communication protocol control will be created, being possible the use different types of control interface.