Keywords

1 Introduction

In recent years, directly printing circuits on paper with low cost, fast and accessible technology has been studied and commercialized [5]. By combining the features of paper and the interactive properties of electronics, a variety of applications that go beyond the conventional uses of paper are emerging. One important feature of paper is that it can easily be cut and folded into many shapes, as can be seen in paper-craft and origami. In our research, we have combined this feature of paper with the conductive ink printing technique to produce interactive devices (e.g., Fig. 1) that are new uses of paper.

Fig. 1.
figure 1

A room layout design application in which the user can move the interactive pop-up card interface on the table to manipulate the furniture layout in a room.

Full size image

Among the methods of using paper to construct shapes, we focused on the pop-up card, which is used in greeting cards and childrens picture books. There are several attractive aspects of pop-up cards: (1) it is easy to use them to make 3D shapes by using everyday tools and materials such as scissors, cutters, and paper; (2) the cost of creating a pop-up card is low; (3) A pop-up card can quickly change from flat to shaped, a feature that is important for its users as entertainment.

In our research, we devised a method to build a tangible user interface (TUI) by combining pop-up cards with the conductive ink printing technique without the user having to calculate the electronic circuit layout on the crease pattern of the pop-up card. The interactive property of this paper is that it can recognize whether and where the user touched the pop-up card and present such information to the user. We use electrostatic capacitance to recognize the touch and place electronic elements (e.g., microcontroller, LEDs) to handle and output the interactive information. In order to add interactive properties to paper, it is necessary to calculate the circuit layout on the paper to connect the touched place and the output. The user can connect electronic elements (e.g., LEDs) to microcontrollers and other sensors because the whole surface of the pop-up card is connected to the base sheet. However, calculating a circuit layout on a pop-up card with a complex 3D shape can be frustrating for a novice.

To make such calculations easier on paper with crease patterns, we developed software that calculates the circuit layout between user-specified start and end points on the crease pattern of the pop-up card [8]. The circuit layout is made to have the shortest path by considering the thickness of the layout, the space between circuit lines, and that the layout should not crossover cut lines. The layout data is then converted into data for printing.

The interactive pop-up cards made with our method can be used as greeting cards with a TUI expressing messages or music by touching the pop-up or be manipulated for various purposes, e.g., as miniatures that can be placed on a table top for room design applications (Fig. 1). In a user trial, we found that even novice could use our software to build an interactive pop-up card and application with a pop-up card interface. We also devised our own example applications to demonstrate the effectiveness of our concept.

2 Related Work

2.1 Paper with Electronics

A number of studies have tried combining paper with electronics to produce interactive media. Kawahara et al. presented an approach to inkjet printing flexible circuits on paper using commodity inkjet printers; it has been commercialized in the form of AgICFootnote 1. Olberding et al. proposed a printed capacitive multi-touch sensor that is robust against cuts, damages and removed areas [9]. Kato et al. proposed a striped pattern sticker called ExtensionSticker that allows a touch input to be transferred from an external source by simply attaching the sticker to a touch panel [4]. Koizumi et al. proposed “Animated Paper”, which combines paper, shape memory alloy (SMA), retro-reflective material, and copper foil. In Animated Paper, paper-craft actuated by the SMA can be made to move with a high-power laser pointer [6]. Qi et al. developed a dynamic interactive pop-up book that integrates pop-up mechanisms with thin, flexible, paper-based electronics [11]. Zhu et al. proposed “AutoGami”, which allows users to explore the design space of automated paper craft in a low-cost way [1]. Ramakers et al. proposed “PaperPulse”, which enables designers to produce standalone interactive paper artifacts by augmenting them with electronics [10]. Olberding et al. proposed “Foldios”, which combines the advantages of folding for thin, lightweight and shape-changing objects with the strengths of thin-film printed electronics for embedded sensing and output [3]. In our research, our contribution is to provide an automatically circuit layout calculation software for calculting circuit layout on complex pop-up card. Such software is expect to support user to design the circuit layout on complex pop-up card.

2.2 Support for Circuit Layouts

Many optimizations for supporting circuit layouts have been incorporated in standard libraries or tools in electronic circuit design software. The EagleFootnote 2 which is an easy graphical layout editor for circuit layout design, and its algorithm calculates optimal layouts among electronic components. There is also a research to support the circuit layout calculation on a particular model. Savage et al. imported the model shape by photograph and build a software to define the desired shape, layout, and type of touching sensitive areas, routing obstacles and calculate the circuit layout [12]. Comparing to these existing researches, we focus on the optimization for making circuit layout on pop-up card.

2.3 Support for Pop-up Card Fabrication

Several studies have been undertaken on making interactive pop-up cards. Mitani proposed a method and developed commercial software to design pop-up cards and export the crease patterns to be printed on a single sheet of paper [8]. Li et al. proposed an algorithm for generating pop-up paper architecture from a user-specified 3D model [7]. Iizuka et al. developed software to support pop-up card designs using multiple sheets of paper [2]. In our research, our motivation is to add interactive property to the pop-up card in order to give new experience to the user.

3 Software of Circuit Layout on Pop-up Card

First, we import the crease pattern of the pop-up card designed by Mitani’s software (Fig. 2(a)). The crease pattern is on one piece of paper with cut lines and folding lines (i.e., mountain and valley folding lines). The 3D object rises when the pop-up card is opened 90 degrees. Such pop-up card is widely known as the folding architecture whose feature is that it can be easily created. As there are various pieces of software for designing pop-up cards besides Mitani’s, we import the crease pattern, instead of the 3D shape of the pop-up card itself, into our system for general and further use of our software.

Fig. 2.
figure 2

Overview of our method: (a) a pop-up card designed by the user, (b) the user specifies the start and end points, (c) the circuit layout is calculated automatically, (d) data is converted into printable form, (e) the model made by user.

Full size image

Then, as shown in Fig. 2(b), our software automatically calculate and render the circuit layout when the user specifies the start and end points by clicking the mouse on crease pattern (discussed in Sect. 4). The resulting circuit layout is the shortest path avoiding cut lines (Figs. 2(c) and 3) and has space between it and different layouts (Fig. 3). The user can increase the thickness of the circuit layout to make it capable of containing capacitance-type sensors but should avoid interference from other circuits. Furthermore, the user can modify the circuit layout by dragging the start or end point, or even specifying a new middle point. Such modifications can be made in real-time and always maintain the constraints for calculating the circuit layout. When circuit layouts are calculated, the next one does not cross over the region of the previous one.

Finally, when the circuit layouts are completed, the user exports the crease pattern and the circuit layouts in PDF format and prints them separately (Fig. 2(d)). In our research, we used a printer to print the crease pattern with normal ink and the circuit layouts with conductive ink. The user then makes the pop-up card by cutting and folding the crease pattern and attaching electronic elements to add interactive properties on it (Fig. 2(e)).

Fig. 3.
figure 3

Individual circuits never crossover the cut lines, and the space separating the circuits is maintained.

Full size image

4 Implementing the Circuit Layout Software

In this section, we discuss the implementation of the circuit layout calculation. First, we discretize the crease pattern by using grids (Fig. 4 left) and distinguish the grids that are incapable of being part of the circuit layout because they contain cut lines or other circuits (Fig. 4 right). The user can adjust the size of the grid to approximate the crease pattern while controlling the search cost. Specifically, it is hard to represent a crease pattern when the grid is too big. On the other hand, it is difficult to modify one circuit layout in real time as the calculation cost rises drastically when the grid is too fine.

Fig. 4.
figure 4

Discrete crease pattern with grids (left), and grids distinguished as capable or incapable of being used in the circuit layout (right).

Full size image

Then, our software recognizes the grids which are capable of being circuit layout nodes as defined in graph theory. We applied Dijkstra’s algorithm to find the shortest layout between the user specified start and end points. The calculated layout satisfies the constraints, which the circuits should not crossover cut lines and maintain space between individual layouts.

Furthermore, the start or end point of one circuit layout can be moved by dragging with the mouse, and the constraints will be maintained. When we drag the end points (Fig. 5 left), the circuit layout around the end point stays parallel to the boundary of the pop-up card. Such circuit layouts are inconvenient for connecting outer electronic elements such as alligator clips. To refine the circuit layout, we insert a middle point in such a layout to divide one circuit layout into two parts and then find the shortest path of the two parts separately. The middle point can be moved by dragging and thus the user can move the middle point to let the circuit layout around end point be perpendicular to the boundary of the pop-up card (Fig. 5 right).

Fig. 5.
figure 5

Drag the end point (left), and insert one middle point to refine the circuit layout (right).

Full size image

Our circuit layout software can handle complex 3D models. Figure 6 illustrates an example of circuit layouts on a pop-up card, called Block Castle?, designed by the fourth author. Such a 3D model is complex, and the work of making the layouts manually is time-consuming and could soon become frustrating for novices. However, by using our software, we only took about three minutes to complete the circuit layout work.

Fig. 6.
figure 6

Example of circuit layouts on pop-up card.

Full size image

5 User Trial

We conducted a user trial on one novice to demonstrate that our system can help users build a TUI system. The procedure of the user trial is: first, the user make a pop-up card by the software of Mitani [8]. Second, the user creates circuit layouts using our software. Finally, the user prints the crease pattern and the circuit layouts and attaches the provided electronic elements to make an original interactive pop-up card. After the user trial, we interview the user to obtain the comments on our software.

The user successfully designed the pop-up card (Fig. 7(a)) and used our software to design the circuit layouts on the crease pattern (Fig. 7(b)). Figure 7(c) and (d) shows the user and his interactive pop-up card, respectively. The designed pop-up card is named Tower and could emit red light by LED when touching the edge of the card.

We interviewed the user after the trial. He said that, designing the pop-up card by myself is more interesting than selecting commercial interactive pop-up cards, because I can freely make the shape and design the circuit layouts to add interactive properties to it. The user also commented: the software can calculate the circuit layouts without me having to consider its constraints. On the other hand, the user complained, the software would be better if it had some patterns of electronic elements for calculating the circuit layout and the software would be better if it gave more details when it fails to find a path. Such comments will help us to refine our software.

Fig. 7.
figure 7

The pop-up card designed by the user in a user trial.

Full size image

6 Example Applications

In this section, we display two example applications. The first application (Fig. 1) is a TUI based on an interactive pop-up card interface displayed on the table. We export designs of objects such as desks, chairs, and stairs to the interactive pop-up card interface and place them on the table. The application recognizes the positions of pop-up cards on the table when a user touches it. Unique IDs made by different conductive layout patterns are assigned to the pop-up cards. When moving the interactive pop-up card interface on the table, the user can intuitively manipulate the furniture layout in a room. We also developed an application to play music on a pop-up card (Fig. 8). This application shows that our circuit layout software can handle curved cut lines on the pop-up card. We place the start point of the circuit layout on the middle of each stage. The application play notes corresponding to the location of the touched stage.

Fig. 8.
figure 8

A musical instrument application which plays a musical note corresponding to the location of the touched stage.

Full size image

7 Conclusion and Future Work

We proposed a method to combine the pop-up card, which can easily generate a 3D shape, with the conductive ink printing technique to build a TUI. To fulfill our purpose, we built software to calculate circuit layouts on the crease patterns of pop-up cards. We conducted a user trial on one novice and obtained feedback for refining our software. Finally, we displayed two applications to show the advantages of the TUI based on pop-up cards.

There are limitations to the method: (1) the pop-up card made of paper is rather flimsy. In the application (Fig. 8), the pop-up card was heavily deformed when it was touched by the user; (2) although pop-up cards can express 3D shapes, they can’t express whole surfaces; (3) the 3D shapes of pop-up cards can degenerate to 2.5D ones.

As future work, three aspects of our work can be improved: (i) inviting more participants for user test to evaluate the result in terms of entertainment or usability, (ii) using flexible and stretchable conductive ink as an alternative material, since we notice that the circuit layouts printed on the crease pattern were cut several times during the process of making the pop-up card, (iii) adding interactive properties not only for pop-up cards but also for paper-craft and origami.