Keywords

1 Introduction

One of the most popular setups of Tangible User Interfaces (TUI) is the tangible tabletop. This setup allows groups of people to simultaneously touch and manipulate a shared space and thus supports collaboration in different kinds of tasks [1]. To date, a variety of application scenarios have been implemented, such as landscape modelling [2], urban planning [3], musical performance [4], or logistics training [5].

Various interaction techniques have been proposed up to now for such application scenarios. While the first approaches had exploited the paradigm of direct manipulation (e.g. [6]), more recent works propose technical solutions for creating generic, reusable interaction objects, which are inspired by well-known GUI widgets (e.g., [7, 8]). Patten et al. propose a technique that allows dual hand input, requiring the concurrent use of two pucks [9]. Finally, we can find physical handles that are able to alter their position or shape (e.g., [10, 11]), with the aim to provide haptic feedback.

In this paper, we report on the design and implementation of a new interaction technique to support multiple users in their specifying and manipulating individual choices on a tangible tabletop. While previous work has focussed on improving individual interactions on a tangible tabletop, this paper presents a solution which facilitates group interactions. The aim of our work is to explore the designs of the tangibles to facilitate the potential interactions in a collaborative setting.

2 Methodology

The interaction technique has been developed in a participatory design approach as part of a research and development project dealing with the use of TUIs to support children in exploring and understanding the effects of their daily life decisions onto the emissions of CO2 and the increase in global average temperature.

The TUI was designed in a multidisciplinary design group across several competencies, covering Software Engineering, Interaction Design, Graphic Design, and Pedagogy. In this group we iteratively designed and the widgets, the visual feedback, the selected parameters of daily life decisions, the underlying equations describing the effects onto the environment, and the scenario of use as part of one-day workshops on Climate Change. Intermediate designs were first visualized as sketches, and then implemented as low-fidelity prototypes that were improved in each iteration. Each of these was then discussed and refined collaboratively in the design team. Two of the preliminary versions were further tested with several groups of children (Fig. 1).

Fig. 1.
figure 1

The Tangible Voting technique has been developed in four iterations

Full size image

In total, we created four different prototypes. The first was set up as semi-functional prototype, to test the technical feasibility of the interaction technique. For the second prototype, we used magnets and a poster on a magnetic board, in order to validate the Tangible Voting technique in a scenario of discussing Climate Change with children. The third prototype was fully functional and operated on the TUI. For the fourth and final version, we improved the design of the different components based on observations collected during the evaluation of the previous version (Table 1).

Table 1. Prototypes generated during the participatory design process
Full size table

3 The Tangible Voting Interaction Technique

Global Warming is a complex phenomenon influenced by a high number of parameters. Human activities are considered to be a significant cause of this change. Concentrations of greenhouse gases are increasing, with carbon dioxide (CO2) being the largest contributor. Based on these scientific facts, we agreed that the learning goals of the TUI are to understand:

  • Which human activities have the most negative impact on the climate.

  • The children’s individual lifestyle having a negative impact on the climate.

  • The individual person having little impact; changing the climate is a group effort.

In order to allow individual children to reflect upon their own lifestyle and explore the impact of the group, we were looking for an interaction technique that takes into account individual choices of the group members. We realized that this could not be done in the traditional way with tangibles imitating knobs or sliders.

Inspired by RunaroundFootnote 1, the popular gameshow, we created an interaction technique that allows groups of users to individually provide input to questions with a set of predefined answers, while still being able to differentiate between multiple users’ inputs. Subsequently, and due to the physical nature of the interface and the persistent nature of the interface components, users are able to consult previously given answers - either individual answers or group consensus - at any time, and adapt their individual inputs freely and concurrently.

The physical design consists of an enclosing with multiple, non-overlapping zones, one for each possible answer. Each of the group members can add a provided token into one of the zones in order to indicate their selected input. The system then counts the number of tokens in each zone and calculates the distribution.

In the following, we describe the major components related to the design of the Tangible Voting technique in the context of the Climate Change scenario. We will describe the design decisions regarding object shape and visual feedback in particular.

3.1 Object Shape

The original layout from the game show which inspired the Tangible Voting technique had three separate zones aligned on one axis. As the typical widget design we are working with features a marker, we chose to align the different input zones in a circular manner with the marker hidden in their centre (see Fig. 2 left). This makes the widget resemble a pie chart. We used a diameter of 15 cm and three zones. The enclosing is physical, with its bottom supported by a thin transparent plastic film to allow users to lift, drop, and reposition the widget without impacting votes that have already been cast. We embedded the widget label indicating the question on a flat, vertical board (third prototype) first, then on a cylinder placed in the centre of the widget (fourth prototype). The rationale was that such a 3D label may facilitate reading the text from a lower perspective and be memorized more easily compared to a flat, horizontal label.

Fig. 2.
figure 2

The final design of the Tangible Voting widget (left); detection via reacTIVision (right).

Full size image

Users cast their votes using small tokens (Ø 1,5 cm). While for the first version we chose small cylinders as tokens, we decided to use pawns for the third prototype. Their simple shape was expected to facilitate correct orientation during placement as well as grasping and positioning them. For the fourth prototype we decided to use different colours for each child, to allow them to better retrace their individual answer and identify with their choice.

3.2 Visual Feedback

We decided to provide feedback for the question asked, the three answers, and the current impact of the already provided answers by making use of the hybrid nature of the TUI, i.e. capitalizing on the possibilities of the digital and the physical nature of the widget. The question was indicated on a 3D cylinder in the centre of the widget. The answers, which exclusively related to quantities, were displayed as digital icons projected in the centre of each zone. A short text describing each answer was engraved on the physical border next to each zone. Feedback of the current status was, on one hand, provided by the pawns. We expected users to be able to quickly approximate their number, which was confirmed by the test sessions. To visualize the result of the distribution, we added a bar as visual cue on top of the widget which showed the impact of already cast votes.

4 Implementation

The implementation of the Tangible Voting technique was done for the Climate Change application developed in Java using TULIP [12] a software framework for implementing widgets on TUI and developed in-house. The framework allows to define the physical qualities of the widget, such as handles, identifiers, and dimensions, and to link it with digital components such as different types of visualizations. The framework hides much of the complexity related to the connection to the Computer Vision Framework, reacTIVision in this instance, and handles the receipt of protocol messages such as TUIO [13]. It will drive the changes of the interface as well.

To implement required widgets, we first defined a set of questions and answers as well as the underlying model we wanted to influence with the inputs. For each widget, we created three zones, one per answer. We defined the shape of the zones and added images and text to visualize the answers. With the look and feel defined, the input zones as well as any feedback was then tied to the underlying model using the Observer pattern.

To detect the position of the tokens, we added a round white dot underneath each token. This could be recognised as cursor in the reacTIVision framework (see Fig. 2 right). In each frame, we counted the number of tokens per zone, and compared it with their number in the previous frame in order to recognise token placements and removals. To detect the position and orientation of the physical enclosing(s), we added a small fiducial marker recognized by reacTIVision.

5 Evaluation

First insights regarding the functionality and the usability of the implemented interaction technique were collected as part of a case study with 27 groups of 3–9 children aged 8–10 years. Each session around the tabletop was animated by a pedagogue and a researcher and lasted about 30 min. Two video cameras were installed to capture speech and bodily interactions after consent was given. At the end of the session, a group interview was conducted, asking children questions about what they learned and how they liked it. While detailed analysis of the collected data will be published in a forthcoming publication, this paper provides a preliminary evaluation of the functionality of the Tangible Voting technique, based on collected direct observations.

The course of action adopted in the sessions consisted of three phases. In a first phase, children were asked to answer questions defined by eight widgets lying next to the interactive surface. The questions turned around the children’s daily life, for instance, the kind of food they were eating, or the way they travel to school. For each of these questions, three different answers were provided by the widgets, with each having a different impact on the climate model: either low, medium, high, or none at all.

The children were each provided with a cup of coloured tokens. The questions were then asked by a moderator, one by one. The moderator read the question on the widget which they placed onto the tabletop. Each child then answered the question by placing one of their tokens in the respective zone, then sliding the widget to their neighbour. While continuing to answer the questions, the children saw the levels of CO2 and temperature rise. The input approach was quickly adopted by each child, and answering questions using the pawns and the physical zones was done naturally and without further explanation regarding the input method itself, allowing the participants to focus on the questions and the impact.

In the second phase, children were asked to individually reflect on whether they would be willing to change their own lifestyle in order to try and alleviate their negative impact on the climate. Children then changed, if they were so inclined, the position of their tokens in some of the widgets. As in the first phase, this was a straightforward interaction, which did not require any further explanation. To reach the different widgets, the children either moved around the table, or asked someone else to slide the widget closer to them.

In the third phase children were asked to explore in detail multiple “what-if” scenarios provided by the moderator on four of the widgets, in order to understand the impact of each of the parameters. The children freely moved the pawns and observed the effects. After their exploration, the moderator provided a concrete goal, expressed as a target CO2 value for the children to reach. To answer this question, children needed to reposition the pawns until the value was reached. We observed that, during this highly collaborative phase, children were rather possessive of their pawns and preferred to make adjustments to their previous choices on their own. After being encouraged by the moderators, they then started to work simultaneously in subgroups, each group focussing on one widget. Only when the value had been sufficiently approached, children started to coordinate their actions and focused on one widget at a time, methodologically manipulating individual pawns in order to obtain the exact value.

In this last phase, two usability problems were observed. As the manipulation of the widgets is based on removing and adding pawns, changing the position of a pawn required the application to interpret two actions. Therefore, when changing a position, the calculated outcome subsequently showed two different values. One value indicated the result when the pawn was removed, then a second indicated the result when the pawn was placed at a new position. This was slightly confusing for the children and they began asking questions as to why the CO2 emissions were constantly changing.

A second issue dealt with the coordination of manual activity and visual feedback. Changing the position of pawn inside a widget could not be done blindly and required the children to look at their hands. Therefore, they were not able to look at the projected level of CO2 to understand whether it was increasing or decreasing. This resulted in confusion regarding to whether the manipulation had changed anything. Some of the groups then distributed roles regarding who would take care of the manipulation of the pawns and who would observe the results.

In order to address these problems, we suggest enhancing the provided feedback. For instance, to enhance retracing the modifications of the output, we will explore the possibility to display the previous value in the visualisation of the output. To avoid confusion due to frequent changes in output, we suggest providing more feedback related to the validity of configurations. For instance, the output could adopt a different transparency if a pawn is momentarily lifted and turning opaque again once the pawn is set on the board again.

6 Conclusion and Future Work

In this paper, we have presented a new technique for groups to interact on tangible tabletops. Tangible Voting uses a physical enclosing with several separated zones, as well as multiple small tokens that can be distributed in these zones to specify the individual choices of the group.

A limitation of the current implementation is that the system does not differentiate between the pawns of each user. Tracking individual pawns could allow for new possibilities regarding logging of interactions and related feedback, a topic to be investigated in future work.

The instantiation of the technique in the context of a pedagogical workshop about climate change targeting children 8–10 years has shown that the Tangible Voting technique supported the planned group activities towards the three expected learning goals well. During phase one and two, the children reflected upon their individual lives and indicated their own answers. On the other hand, in the third phase the users were able to collaboratively modify the distribution of the whole group, thus creating a collaborative learning situation where the impact could be analysed and understood.

While the activities of answering questions and modifying their own answers were straightforward for all children, the conduction of small experiments in order to understand the relation between the parameters required a higher mental effort. To better support the users in these types of tasks, we suggest enriching the provided feedback.

The case study conducted with the Tangible Voting interaction technique has provided us with a large amount of video data of a scenario consisting of both individual and collaborative activities. We believe that this interaction technique is particularly interesting for the analysis of social interactions as it allows a high variety of usage patterns. In future work, we will analyse the collected video data, with the aim of enhancing the understanding of how users interact with tangible resources in a collaborative setting.