Keywords

1 Introduction

1.1 Sketch and Sketch Support

A sketch is a drawing act where one roughly depicts objects such as a person and a landscape. Since anybody can start sketch if they only have paper and pen, it becomes a widely-practiced creative activity. Digital sketch also has become popular, along with a spread of high-performance smart devices with large screen. Since it is one of the basic skills in painting, sketching is often adopted as a training task for beginners.

Up to now, various sketch learning support systems for beginners have been studied and developed. Most of these systems provide users “tutorials and guides” according to typical drawing procedures. By using them, beginners can learn basic procedures to depict the object. For example, Fernquist et al. [1] developed a comprehensive sketch tutorial and assistance system that provides drawing step navigation, stroke guidance, stroke feedback, and automatic tutorial generation function. Soga et al. [2] developed a system that provides advices annotated in sketch area in advance according to user’s pen position. Soga et al. [3] also proposed a system to support drawing appropriate rough composition in the sketch process. This system requests users to draw circumscribed rectangle of motifs’ view on a paper and diagnoses its location, size, and aspect ratio. Similarly, Takagi et al. [4] proposed a system that compares completed user’s sketch image with motif data set by using feature extraction and diagnosing the balance. Nishizawa [5] proposed a support system to draw a balanced portrait based on correct understanding of the human body structure by providing overlaid semi-transparent skeleton model. Xie et al. [6] developed a system that automatically adjusts both outlines and shading strokes in real-time based on important features of a motif portrait image. Several studies proposed an environment for learners on which users can generate tips for learning by themselves. For example, Huang et al. [7] developed a system that users can practice drawing and share drawing process with other learners. They proposed a reflection workflow that allows learners to append annotation of learning points on shared drawing process before or after short practice.

1.2 Problems in Beginners’ Sketch

While various support systems had been proposed, few studies focused on the fundamental problems into which beginners often fall. In the training of sketch, an important issue for beginners is to learn how to see the motif object and draw it as they see, before learning the procedure to depict it. Beginners often see motif object inappropriately and face following problems.

The first problem is a difficulty to grasp the entirety of the object. Beginners often attempt to draw the specific part of object as accurate as possible from the beginning of the sketch [8]. As a result, the balance of the entire sketch often collapses. In the beginning of the sketch, it is necessary to roughly see the entire shape of the object to grasp its entire proportion, rather than focusing on the details of the parts.

The second problem is a difficulty to draw exactly what they saw caused by labeling of conceptual meaning to focused part. Beginners often draw the object as “what they know,” instead of “what they see.” Edwards [9] suggested two types of mode of human cognition; L-mode and R-mode. L-mode manages verbal, symbolic, and analytic thought while R-mode manages nonverbal, actual, and spatial thought. Edwards pointed out that L-mode is usually activated since our thought is often associated with language, and it frequently interferes drawing sketch. For example, when beginners draw sketch of person’s figure, L-mode expects to see motif based on conceptual human figure that they are familiar with. To be released from the curse of L-mode, a method to see a person’s figure upside down has been often employed. By using this method, they become able to see the figure as unfamiliar aggregate of lines, and they become able to draw it as they see it.

We assumed these problems are caused by beginners’ subjective way of seeing, i.e. excessive focus on detail of specific parts of motif object. In order to solve these problems, we propose a sketch learning support system VujaDessin. The system supports beginners to see the entirety of motif object and to draw sketch as they see it by showing a blurred motif object image. We conducted user studies and evaluated the effectiveness of the system.

2 Method and System

2.1 Proposed Method

We propose a novel method for beginners to experience how they should see the motif object. Two problems mentioned in previous chapter are caused by the fact that beginners can see all parts of motif object in detail. To avoid it, we propose an approach that artificially and temporarily deteriorates the beginners’ eyesight. Since they have good eyesight, they unintentionally focus on the details and ignore entirety. If that is the case, by deteriorating their eyesight, they become unable to unintentionally (and also intentionally) look at them. Our method disturbs beginners to see the details of the object when it is unnecessary and unpreferable.

In our proposed method, the motif objects are blurred when beginner focused on the detail, while those are always clearly shown in the conventional methods. Beginners can hardly draw a detail part and remember that they have to consider the balance of the whole motif object. Accordingly, they are led into experts’ procedure in which they start drawing from the whole so as to prevent them from losing entire balance. Also, since blurring might prevent them from understanding what they see, it becomes difficult for them to label conceptual meaning to each part of the motif object. It might help them concentrate on drawing sketch of what they see without unconscious interference of L-mode. To substantiate this idea, automatic detection of focus on detail and adjustment of the degree of blur are required. Since we aim at improving balance and accuracy of the sketch, we consider only contour line; other sophisticated expressions such as shadows and texture are out of the scope of this research.

2.2 Developed System - VujaDessin

Based on the proposed method described in previous section, a sketch learning support system VujaDessin (VJD) was developed. The name “VujaDessin” is a coined word that combines the word “Vuja DE” which means observing familiar object as if they see it for the first time, and the word “Dessin”.

Figure 1 shows user interface of the system. Image of motif object is displayed on the left area of screen (motif object display area). A user performs sketch drawing on the right area (sketch drawing area) while seeing the motif object on the left area.

Fig. 1.
figure 1

User interface of the system.

Full size image

We assumed that the user will bring his/her face closer to the screen to gaze at the motif object when they pay attention to the detail parts. The system makes the image of the motif object blur when his/her face gets closer to the screen. In the most intense blur condition, the user cannot see any detail parts but can identify only whole contour of the motif object.

Figure 2 shows the overview of the structure of the system. The system includes several key modules: a focus monitoring module and a blur generation module. The focus monitoring module determines how much the user pay attention to the detail parts according to a distance between the face and the screen. The distance is estimated by the face size captured by a web camera. Then, the blurred motif image generation module decides the degree of blur of the motif object according to the degree of user’s attention and focus on the detail parts, and generates a blurred image.

Fig. 2.
figure 2

Overview of the system

Full size image

A pen tablet and a drawing software allow the users to draw sketch while seeing the image of the motif object. We adopted XP-Pen Deco 03 for the pen tablet device and Microsoft Paint for the drawing software. In order to simplify experimental condition, advanced functions (e.g. a layer function and a zoom function) were inactivated, and only pen, eraser, and undo functions were permitted to use.

Figure 3 shows a relationship between the user’s face position, i.e. distance between eye position and screen, and degree of blur. Adjustment of the degree of blur was conducted in the preliminary calibration by the second author. In the calibration, we determined a standard distance between the user’s face and the screen as 70 cm: the system applied minute blur on motif object image at this position to notify users that the system is working. User can identify detail parts easily in this condition. As the distance decreases, the system applies stronger blur. We set the shortest distance as 40 cm where the second author put his face to see the detail parts of the motif during the calibration. At this position, the most intense blur is applied: the users cannot identify any detail parts. We applied gaussian blur algorithm and dynamically changed width and height values of kernel. Both modules were implemented by using OpenCV.

Fig. 3.
figure 3

Relationship of user’s face position and degree of blur

Full size image

3 User Study

In order to investigate how the blurring affects the user’s behavior, we conducted a user study with VujaDessin. The purpose of this user study is to evaluate the direct effect of the system that is how much the seeing behavior and outcomes (sketches) were improved by the system. Concretely, the system was evaluated according to following two viewpoints.

  1. (1)

    Can the system help users grasp the entirety of the object without excessively focusing on the detail in the drawing process?

  2. (2)

    Can the system help users draw a well-balanced sketch as a result?

In order to evaluate the first point, we recorded eye movement for seeing the motif object by using an eye tracking device (Tobii X120 Eye Tracker). Movement and retention of point-of-regard on the motif object image were quantitatively recorded and analyzed. Moreover, after the sketching was completed, an interview was conducted in order to ask how users drew sketches.

In order to evaluate the second point, the sketches were evaluated by a questionnaire. Subjects were asked to evaluate balance of the sketches by using five-point Likert scale. To evaluate comprehensive quality of the sketches, they were also asked to answer its accuracy.

In this user study, we employed two subjects and requested them to draw the sketches of a human model. Both subjects are master course students of the authors’ institute in their 20’s who do not have experience of receiving higher education of drawing. Subjects were asked to draw sketches in two conditions. In the first condition, they drew a sketch without using the system. In the second condition, they were asked to draw sketch with using the system. The subjects were allowed to draw their sketch in their own ways without any constraints or time limit. Figure 4 shows appearance of the setting of the user study. As shown in Fig. 5, We adopted the picture of the human body model as the motif object (the image was quoted from Posemaniacs.com [10]). Recorded data of both conditions were compared and analyzed to reveal effects of the system.

Fig. 4.
figure 4

Setting of user study

Full size image
Fig. 5.
figure 5

Motif object image

Full size image

4 Results and Analysis

Both subjects could complete their sketches in two conditions. Table 1 summarizes total spent time for completing the sketches. Table 2 shows outcomes (sketches) of each condition of both subjects. Figures 6 and 7 shows chronological shift of degree of blur in the drawing process in the second condition (with VJD).

Table 1. Total spent time to complete sketch of each condition
Full size table
Table 2. Motif object and outcomes of each condition
Full size table
Fig. 6.
figure 6

Degree of blur in drawing process (subject A)

Full size image
Fig. 7.
figure 7

Degree of blur in drawing process (subject B)

Full size image

Figures 6 and 7 show that degree of blur changed throughout the drawing processes of both subjects. Especially, subject A periodically got closed to the screen, which caused the strong blur. He reported he was curious about the blur and his seeing behavior was influenced. As for subject B, relatively strong blur was generated at the later phase of the drawing process while she reported she did not care about the blur.

While both subjects moved their face position in this user study, we also have to note that other two users did not get closed to the screen in preliminary tests that were conducted before this user study. They retained straight posture and the degree of blur hardly changed. The way of system usage possibly depends on user’s drawing style. Some users could focus on the detail parts without changing their face position. We can improve focus detection module to acquire point-of-regard retention from the eye tracking device directly. Another possible factor is the effect of the eye tracking device. The subjects could unconsciously keep their face position to maintain eye tacking accuracy. We thought wearable camera type eye tacking devices would be more suitable for the user study of this system.

4.1 System Effect on the Way of Seeing

We analyzed eye tracking data in order to explore the user’s seeing behavior in the drawing process. Figures 8 and 9 are heatmaps that express total fixation time on each part in their drawing process. Red means longer total fixation time.

Fig. 8.
figure 8

Duration of stay of point-of-regard on each part of motif object image (subject A), (1) fist condition and (2) second condition with VJD

Full size image
Fig. 9.
figure 9

Duration of stay of point-of-regard on each part of motif object image (subject B), (1) fist condition and (2) second condition with VJD

Full size image

Figure 8 shows that subject A spent longer time on the head part of the human model in the second condition with using the system. In addition, he also spent relatively longer time to both legs, foots, and toes parts that are the peripheral parts of the motif object in the second condition, comparing to the first condition. It suggests that expected effect was partly verified; subject A payed attention to whole of the motif object while bias to the head part was also observed. He reported that the blur made him become more conscious for outer contour of the human model image. He also reported he could be conscious of “thickness” of each parts without being aware of detailed inside lines.

Figure 9 shows, on the other hand, that it is difficult to conclude that the system provided expected effect to subject B. Subject B also saw the head part for longer time in the second condition, but did not pay enough attention to the lower body parts including both legs, foots and toes parts.

Table 3 summarized the results of quantitative analysis of seeing behaviors of both subjects. Velocity and total moving distance of point-of-regards, number and average duration of fixation were calculated. Threshold value of fixation was 60 ms. Subject A spent longer time for seeing at motif object (and also drawing area) in the second condition compared with the first condition. Total moving distance and number of fixations were increased, and it was implied subject A took exploratory seeing behavior. Since the average duration of one fixation was slightly declined, degree of focus on specific parts was not intensive in the second condition. On the other hand, subject B less saw the motif object display area in the second condition, explored smaller distance, and more concentrated during one fixation.

Table 3. Summary of analysis of seeing behavior and fixation
Full size table

Interview results suggested subject B felt that she is not good at drawing sketch. She reported that she did not notice that system worked and blur was generated, thus she saw the motif object in the same way. She would not carefully observe the motif object compared with the case of subject A. She spent approximately only five and three minutes to see the motif object display area and its percentage in the total duration of seeing screen was 24% and 27% in the first condition and the second condition. They were smaller time and ratio than those of subject A who spent approximately seven (39%) and eight (32%) minutes.

It was suggested that the proposed system would not be effective to very beginners because they might not spend enough time to see the motif object in the first place. We should support them to see the motif object itself rather than preventing the excessive focus on the object. For example, the blur could be applied to the sketch drawing area to prompt the users to see the motif object when they excessively concentrate on drawing sketch.

By the way, in this user study, we did not take user’s learning and mastering effect into account. Experiment in more strict condition is a future issue.

4.2 System Effect for Quality of Outcome

Table 4 shows results of answers of questionnaires about self-evaluation of the outcomes and comments.

Table 4. Evaluation for outcomes according to five-point Likert scale (1: poor ~ 5: excellent) and comments.
Full size table

It suggests the usage of the system did not contribute to subjective quality of outcomes. As for subject A, he talked the sketch in the second condition was poor compared with the first condition. Although he reported he paid more attention to whole contour than detailed parts of the motif object, the score for balance was declined. Drawing environment with VJD was unusual for subject A and it might provide some stress and anxiety in drawing process. He also reported that he felt difficulty when he drew the detail parts and it causes the decline of the score for accuracy. On the other hand, subject B gave low evaluation for all her outcomes regardless of the system.

Those results show that seeing behavior for grasping entirety in the process is not a sufficient condition and did not always directly improve the quality of outcomes. Many factors such as drawing skills also effects on quality of outcome. Subject’s comments gave us hints to consider approaches to improve the system. One possible approach is to improve the system to restrain unintended negative effect for other aspects (e.g. drawing behavior) in drawing process. We could let users understand how the system works and reduce their stress. Also, the degree of blur should be reduced when the users started to draw details after they completed rough contours. Another approach is to regard the system as an enlightenment material for appropriate seeing rather than a support system. Current system gives a message about the importance of grasping entirety of motif object to the users. We have to evaluate long-term learning effect of the system in future study.

5 Conclusion

Excessive focus on the specific detail parts of the motif object is a common problem of beginner’s sketch drawing and it might result in collapse of the balance of outcome. In this study, we proposed a sketch learning support system VujaDessin (VJD), which prompts the users to see the entirety of the motif object by blurring the motif object image when excessive focusing on the detail parts is detected. We conducted user study of the system targeting two beginners to evaluate the effectiveness; whether the system can help users grasp the entirety of the object and draw a well-balanced sketch. Although it did not bring positive effect to quality of outcome, we confirmed the blur changed users seeing behaviors in drawing process. Subject paid more attention to the peripheral parts of the motif object and spent longer time to explore the motif object display area. Several suggestions for improving the system were obtained. Another type of support such as support to pay attention to the motif object itself is required in order to help very beginners who strongly feel they are not good at drawing sketch. If we develop this system as an immediate support system in drawing process, the system should be improved to avoid negative effects e.g., user’s stress and difficulty of drawing detail parts in finalization phase. We also need to explore its long-term learning and enlightenment effect in the future study.