Development of Gesture Recognition-Based Educational Dance Game for Children with Intellectual Disability

Abstract

The purpose of this study is to develop an educational dance game for children. Successful experience on the stage improves the confidence of the participants. People with intellectual disabilities also have personal satisfaction and confidence as they finish the stage with dance and theater. However, during the process of explaining the movement, it is not easy to maintain a constant interest for the children with intellectual disabilities in the dance class. Children with intellectual disability are struggling with participating in dance class because of their low ability of body awareness. This makes children with the intellectual disability more inactive in dance class. Gesture recognition-based interface is an excellent educational tool to make students more active. This game contains 40 different dance pose images. The children match their head, hands, and legs to the images’ pose. The game was conducted at the intellectual disability child education center of the Lords of the Republic of Korea. The following results were obtained after conducting the dance lesson using a game. When children align their poses with the pose of the photo, they give auditory feedback as a means of praise and score the number of poses matched. In the dance lesson using these games, the children with intellectual disabilities were very interested in the appearance of their own images on TV. In addition, the children were more interested and actively participated in the class than the existing dance class. The use of Kinect in the dance lessons of children with intellectual disabilities is a good tool to increase students’ interest and excitement

1 Introduction

Successful experience on the stage improves the confidence of the participants. People with intellectual disabilities also have personal satisfaction and confidence as they finish successfully the stage with dance and theater. However, it is not easy to maintain constant interest and during the process of explaining the actions and attitudes for artistic activities to the people with intellectual disabilities in dance class.

Intellectual Disability Students’ ability to adjust their body movements is less than that of regular students. Not only choreography but also activity in everyday life, the degree of motion development is insufficient and below average. Therefore, in the instructional style in which teachers demonstrate and follow the dance teaching time, the children with the intellectual disability take more time to learn choreography than the general children. As the choreography takes more time, children with intellectual disabilities also become less distracted and more distracted.

In addition, teachers are well aware of these difficulties because they show very slow movements of dance lessons and behave in a way that they cannot concentrate on their intellectual disabilities. In particular, children with disabilities often fail to follow the dance movement when they learn to dance. To help with this difficulty, a game that can give feedback to immediate audiovisual stimulation can be effectively used. It is expected that the TV screen will provide feedback on the choreography movement and the identity of oneself, and if it poses correctly, it will lead to a positive attitude by giving praise through effect sound. The purpose of this study is to introduce an application that helps the mild intellectually disabled people to practice dancing choreographed using KINECT for the work “Les Miserables” in a dance education class.

2 Educational Dance Game Using KINECT

The KINECT used in this study uses only body movements as a controller without any other devices such as keyboard, mouse, and joystick. In this game, after taking a dance action with a camera, the still image is used to make a pose image, as shown in Table 2 (Green shape). In the game, it proceeds by moving the body along with the pose image that is stopped. The KINECT camera reflects the participant’s movements on the screen. This makes it easy to use the technique used by the Chroma key with KINECT. By linking the actions learned through this content, dance performances can be completed naturally as a stage performance.

3 Game Program Scheme

The works used for the dance performances were based on “Les Miserables”, which is widely known to the public. It is a famous musical work intended to induce the interest of people with intellectual disabilities. The inducement of interesting is necessary for active participation. In the Korean version of the musical “Les Miserables”, two rhythmic songs were selected for the background music, and the scenes from the movie were captured were used for the background (Table 1).

Table 1. Screen shots of dance education game

Table 2. Dance pose composition and sample pose image

The first screen was adapted from a poster of “Les Miserables”. It starts when the teacher clicks the screen. Every time a person with intellectual disabilities aligns his or her movements, the ‘action’ sound provides feedback. The dance motion images contains 40 different poses. Each time provides 3 min of game/dance play, and their score is shown after. The result show up the total number of actions taken correctly. The difficulty level does not change during the game. However, the game mode was applied differently according to the level of disability of children with intellectual disabilities before game start. If the level of intellectual disability is severe, the option is changed to a mode that provides more visual cues.

The choreography (40 poses) used in the game was based on the opinions of teachers in the Center for Children with Intellectual Disabilities and dance instructors. The athletic ability of children with intellectual disabilities is very large, at the level similar to that of the general public to the level where they can not recognize the positions of their arms and legs. Therefore, when composing choreography movements, easy movement and difficult movement were classified by level. Easy dance moves begin with a simple pose that moves only the upper body. Somewhat difficult movements consist of a pose with the upper and lower body moving together and a pose with the center with one foot. When the game starts, the user is presented in random order so that the order of poses cannot be determined. However, if a child with an intellectual disability does not act at all, he added the ability to move to the next action by the teacher’s veneer.

4 Demonstration Class at the Center of Children with Intellectual Disability

The dance education game designed in this study was used for 4 days as an education program at a center for children with intellectual disability in Yeongju-city, Gyeongbuk-province, South KOREA. The total hours of education program was 20 h for mildly handicapped people. At first, they were awkward following the images and dance movements on the screen, but they gradually got used to it. In conclusion, the Chroma key technique using KINECT was very useful for inducing the interest of the people with intellectual disability (Table 3).

Table 3. The place where the training is conducted by the participants.

Unlike general dance lessons, the characteristics of children with intellectual disabilities appearing in dance lessons using games are as follows. Children with intellectual disabilities have had fun with the their appearance on the TV screen using chroma key effect. After a student with a disability matches his/her body’s pose, he or she begins to perceive hearing sound feedback and score increases, and then the students begin to participate in the game harder to get high scores as well as motivation.

Particularly, the handicapped students were interested in concentrating on oneself in the pose matched while moving the body, concentrating on the dance movements. It can be seen that there are many differences from the passive figure of intellectual disability children participating in the traditional dance lesson. As an example, the dance movement was always difficult and burdensome. The accuracy of the dance movement was poor, and the students could see themselves around them, and they could not concentrate and appeared to have decreased self-confidence. As we progressed through the game, we became more focused on the fast-paced game and were able to work more happily because of immediate auditory feedback. In addition, I was able to know that I was confident when I showed my intention to show it to my friends around me whenever I set the motion for the students with disabilities.