Keywords

1 Introduction

While the visual world gets more connected and data gets more complex, visual presentation is an effective means to convey information. Problems arise if participants cannot access the presented information, in particular blind people. Non-visual access to graphics is often provided by a verbal description, which is not always a satisfying solution. To create an alternative textual description can be challenging for some graphics, for example maps, complex diagrams or images containing some kind of graphical notation. In addition, verbal descriptions can be incomplete or can contain wrong interpretations. This can change the meaning of an image or can lead to misunderstanding its intention. For this reason, a combination of a description and a transcribed tactile representation should be provided for important graphical data as it improves comprehension by blind people [1]. For some photographs or art it is hard to create even a suitable tactile counterpart, access to them remains to be limited.

Creating tactile graphics is also not an easy task, especially if the creator has only little experience. In a survey in German speaking countries, we found that only 5 % of the 78 blind and visually impaired participants generally prefer image descriptions over tactile graphics [2]. Conversely, most blind people wish to get more tactile graphics as they not only allow for an independent exploration, but also for a better spatial comprehension of the content. However, a basic requirement for an effective reading is a sufficient quality of tactile graphics. In most institutions it is common to check some of their produced materials by visually impaired colleagues to ensure a good quality [2]. In the conventional production process it is necessary to produce a digitally mastered tactile graphic for every change again to show it to a blind reader. In contrast, we use a refreshable tactile pin-matrix device to include a blind user in early stages of the production process within a collaborative workstation.

2 Tactile Graphics

Tactile graphics allow readers to feel presented structures, graphic elements or Braille text, through their tactile perception. Several different production methods exist. They all differentiate in their ability of presenting tactile structures. Differing features are, for example, possible resolution, the ability of allowing different raise levels, stability, speed and costs for reproducibility. While microcapsule paper and matrix embossers can produce tactile graphics from digital data without big additional effort, reliefs and other production methods need some manual post-processing or long production times. In this paper we focus on digital production methods, such as normal printing and embossing. Tactile embossers allow resolutions form 10 dpi for Braille embossers up to 20 dpi with different height levels for matrix embossers. Microcapsule paper is able to present detailed tactile structures printed with a normal printer on a special paper that rises on dark areas while heated under a special lamp.

Instead of conventional static production media refreshable two-dimensional tactile displays can be used to interactively present tactile graphical and textual information. For instance, the BrailleDis 7200 [3] allows to show content on a 120 × 60 pins large display area as binary tactile image with a resolution of about 10 dpi. Because of its zooming and panning possibilities, theoretically, each type of graphic can be shown on the pin device. However, there are some limitations. For instance, it is hard to handle graphics that use colors or different height levels. Very fine-grained structures, rounded objects and non-orthogonal lines can also lead to problems in their recognizability on such low-resolution devices. Reducing the complexity is a main requirement for tactile graphics; therefore these kinds of structures should be avoided in any case.

2.1 Tools for Preparing Tactile Graphics

The creation of a tactile graphic is not the challenging part of a transcription process. Almost every source image can be directly transformed into a grayscale or binary image which can then be produced as tactile version, for example, on microcapsule paper or with a matrix-embosser. The main problem is the readability and recognition of objects. Structures and shapes often get lost due to the low resolution. Therefore, a transcriber needs to prepare the image with respect to simplification and readability.

Several drawing tools exist to support transcribers. They contain functions for filling objects with different tactile patterns or allow redrawing image parts in a way that they fit into the embossing matrix of the related embosser. Some of them also apply graphic filters for grayscale or binary transformation as well as for finding and highlighting the outlines of objects (e.g. PictureBrailleFootnote 1). Creating Braille labels is another task for transcribers. The TGA system [4], for example, supports the translation of text in images by semi automatic OCR-mechanisms. However, it seems that mostly standard office or image processing software is used [2].

In contrast, suppliers for tactile graphic production hardware often deliver their own production software. For example, the audio-tactile graphic exploration system IVEOFootnote 2 comes with a touch-sensitive image frame and its own tactile graphic production software for creating scalable vector graphic (SVG) images with the feature to add a two-level description to graphic elements. Technically, SVG is utilized for annotating a title and description of tactile objects.

2.2 Tactile Graphics by Blind People

Drawing graphics without vision is difficult. Blind pupils learn to create drawings by different techniques. Most of them are based on handicraft techniques or combine tactile primitives to more complex structures [2]. Creating freehand drawings can be done by scratching tactile perceptible structures into special foils or wood plates. These are all analog methods and, therefore, hard to reproduce. The digital creation of a drawing by a blind user is more complex and with limited tactile feedback. Often Braille characters form drawings or command languages define graphical structures. All analog techniques have the disadvantage that an error correction is almost impossible for the creator. For the digital methods this is also very exhausting because it needs a lot of mental imagination.

2.3 Quality of Tactile Graphics

The dissemination of tactile graphics among blind people lacks behind their importance. Due to lack of training many blind users have difficulties in recognizing complex tactile graphics [5]. Reasons may be, for instance, the overloading of a graphic with too much information, the reduced distinctiveness among different objects as well as the limited preservation of orientation [2]. To minimize such problems some kind of quality management for tactile graphics is important.

There are several guidelines (see [6]), which should improve the quality of tactile materials. However, such guidelines are often complex and general, which complicates their practical application - especially for non-expert sighted persons who commonly produce tactile graphics [2]. We propose the inclusion of blind users into the creation process. Therefore, we have developed a collaborative workstation as described in the following section.

The main purpose of our system is to allow a simplified inclusion of blind users in the production process as well as to support inexperienced sighted users. Besides, the system can also be used for sketching difficult concepts with and for a blind user, for instance in learning or working scenarios. First user observations in a collaborative graphic production scenario have shown that it is not effective to let the blind user follow the creation of a graphic from the beginning. It is more effective to include the blind user after a first draft of the graphic is produced. The blind reviewer can than check the graphic and make some annotations and changes if necessary. The sighted user can also ask for specific problems which can lead to further discussions and improvements of the resulting tactile graphic.

3 Collaborative Workstation for Tactile Graphics

Compared to cooperation, where each individual works separately on his own part, collaboration requires direct interaction and discussions among the participants [7]. For an efficient collaboration between a sighted and a blind user, the visual and the non-visual workspace have to remain consistent and have to be synchronized in a multimodal sense. The Tangram collaborative workstation not only has to allow for the creation and modification of tactile and visualized graphics by both users. It also has to support the communication between the partners by focusing on objects of interest jointly. Beyond basic graphics operations, the workstation also has to include functions for annotation and quality management. In the following, the general architecture of the Tangram workstation as well as the mechanisms for supporting the collaborative production of tactile graphics by sighted and blind users are presented.

3.1 Architecture of the Tangram Workstation

The Tangram workstation consists of two parallel I/O workspaces connected to one computer (see Fig. 1). The sighted user uses mouse and standard keyboard as input and a monitor screen for the visual output. The blind user on the other hand uses the dynamic pin device BrailleDis 7200 for tactile output and its 28 buttons as well as its touch-sensitive display area as input device for keying in commands, Braille text and gestural input [3]. In addition, auditory feedback is given, consisting of sound and text-to-speech output. With the exception of non-visual drawing, all basic tasks for transcription can be mapped to interactive operations at which the two partners do not hinder each other while working on the same document independently and in parallel.

Fig. 1.
figure 1

Schematic representation of the Tangram workstation

Full size image

As basic drawing system the open source office suites OpenOfficeFootnote 3 and LibreOfficeFootnote 4 can be used. These programs are free to use, even in a commercial context, and produce vector graphics which can be exported to several different document formats. Multiple tools have been added to implement the working process for creating audio-tactile graphics.

For the blind user a special screen reader has been implemented that allows access to the accessible interface of the office suite as well as to the document object structure [8]. This enables the blind user to manipulate the graphical and textual document elements. The presentation of the graphical content is realized by rendering a tactile screenshot of the working area window on the pin device. Thereto, a configurable filter is applied to the screenshot, converting lighter points to lowered pins and darker points to raised ones. The user can freely explore the tactile screenshot by zooming and panning operations. Additional information is added to the screenshot image, such as a dotted frame resembling the size and position of the sheet to draw on within the working area. The blind user can retrieve further information on document objects by pointing at them while pressing a button.

The blind users’ input and output system is realized with our open-source framework BrailleIO [9]. This framework allows for a fast and easy implementation of a basic windowing system for two-dimensional tactile pin devices. Furthermore, the framework also allows the Tangram workstation software to be applied to other pin device types besides the BrailleDis 7200.

3.2 Supporting the Sighted User

Especially for sighted producers with little experiences, the provision of a specialized toolbar by the graphic editing software itself can reduce some of the deficits when designing tactile graphics on a visual screen. This can include the specification of predefined sets of filling patterns, line styles or elementary graphical primitives, such as arrows or tactile symbols. Moreover, an easier access to important properties of graphical objects, such as position, size and textual information (title and description), can also be helpful for enhancing the productivity of a sighted user. The implemented toolbar supports the following tasks (compare Fig. 2):

  • position properties: edit x and y position of the selected object.

  • size properties: specify width and height of the selected object.

  • keep ratio toggle: enable and disable preservation ratio of width and height.

  • unit selection: select measuring unit for presentation of position and size.

  • line styles: select a tactile perceivable line style (solid, dotted and dashed) for the selected object.

  • pattern styles: select a texture for filling the selected graphical object. We have tested the suitability and discriminability of the provided patterns for utilization with different tactile graphics technologies (microcapsule paper, matrix embossed printing and pin-matrix device) in an unpublished study with 12 blind users.

  • pattern-border spacing: add or remove white space between the filling pattern and the border of the selected object to increase distinctiveness and outline perception.

  • title and description dialog: open or hide a docking window for annotating title and description of graphical objects.

  • open Tangram workstation: start the Tangram workstation for the blind reviewer. Besides, document properties are adapted, such as standard colors and font style.

  • open check tool: open a dialog for reviewing tactile graphics based on our authoring guidelines.

Fig. 2.
figure 2

Toolbar for supporting the sighted user in producing tactile graphics

Full size image

3.3 Supporting the Blind User

The blind user manipulates graphic objects independently on the pin-matrix device by a rotating menu with five different editing modes. Input is realized by the buttons of the right cursor key pad of BrailleDis 7200 (see Fig. 3). If an element of the graphic is selected, the user can start it’s editing by pressing the center key. Initially, the move mode is activated. By pressing one of the four direction buttons the element is moved into the chosen direction. The current editing mode can be changed by pressing the center key iteratively. In this way an element can be moved, scaled, rotated, and its texture and line style can be changed. The assignment of functions to the cursor key pad buttons is dependent on the current mode (see Fig. 3). Therefore, the user doesn’t have to remember many button combinations. Furthermore, one hand may remain on the cursor key pad while the other hand can monitor the changes on the element in real time. Bi-manual operation has been found to be effective for most interaction tasks [10]. One drawback is the reduced efficiency due to the modal menu. Nevertheless, an informal discussion with four blind users showed that this is acceptable if there are not too many modes.

Fig. 3.
figure 3

Usage of the cursor keys within the five editing modes in the non-visual user interface

Full size image

Besides the editing of the appearance of graphic objects, the blind user also can annotate the tactile graphic, for instance, by adding a title and description for each element. Thereto, he can open a dialog in the detail region of the pin-matrix device, where reading and editing of Braille content is performed. Either a text field for the title or for the description is shown. The user can switch between these two by using the up or down button of the right cursor key pad. For writing, the Braille input keys on the BrailleDis are used. Finally, the data have to be saved by calling the save menu, which is realized by pushing the center button of the right cursor key pad. Using the left or right button the user can choose between the following menu options: save, not save or cancel. All options are presented in Braille, where the current selected option is underlined and also is spoken out.

3.4 Collaboration

The blind user can edit the graphic elements independently from the sighted user, as mentioned above. Therefore, it is necessary to have two separate foci: the standard GUI focus, which will be called mouse focus in the following, and the pin device focus which refers to the DOM objects of the drawing document. The latter will be called Braille focus. To enable an effective collaboration, the two users have to be aware of these focus concepts.

Allowing the blind user to follow his sighted partner’s activities, the selected GUI element can be highlighted on the pin device, and a follow mouse focus mode can be activated. In this mode, the element which is currently selected by the sighted user, is not only spoken by audio output, but also is signaled by a blinking pin frame and is shown in the center of the pin device. If the sighted user selects another element, the tactile output is refreshed automatically. In this way the blind user can easily track the mouse focus. If he wants to edit the element which has the mouse focus, he can set the Braille focus to that element and quit the follow mouse focus mode with a single button command. By this means, the sighted is able to show the blind user an element and ask him to do some improvements, for example.

On the other side, the blind user can also highlight the Braille focus for his sighted partner on the GUI to indicate the element he is currently editing. Therefore, a blinking overlay window is shown for some seconds above the corresponding element.

4 Conclusion and Outlook

A collaborative workstation for producing tactile graphics was implemented. This allows blind users to actively take part in the transcription process for tactile graphics from early stages. As they are the target user group of the materials created, this should increase the quality of the resulting images. First prototyping tests with four blind users showed that the editing functionalities on the pin device were rated as useful and intuitive. Nevertheless, a high learning curve was mentioned. The focus concepts were rated as understandable, but the effective usage in collaboration with a sighted partner needs a lot of training, practice and coordination.

As the next step a real scenario user study with pairs of blind and sighted subjects reworking a previously transcribed tactile graphic will be conducted. Thereby, we will be able to assess usability and accessibility of the Tangram workstation’s approach to collaboration. It already became clear that blind reviewers would like to do some changes in existing tactile graphics. In our studies, we want to find out if materials produced by professionals will be improved by a blind user, too.