Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Japan is currently entering a hyper-aging society. The Japanese Ministry of Health, Labour and Welfare estimates that the number of elderly people over the age of 65 will increase to 36.57 million, which is 30.3 % of the total population [2]. Due to the increase of the elderly, more and more people will suffer from dementia. The number of people with dementia grows to 7 million in 2025, which is one-fifth of all elderly. Thus, assistive methods and technologies for preventing, nursing and supporting people with dementia are strongly required.

The person-centered care [6] is an ideal principle for dementia care, where caregivers understand individual situations of people with dementia, and provide personalized support and care for them. Among various ways of the person-centered care, this paper especially focuses on the dementia counseling [8, 11]. Due to BPSD (behavioral and psychological symptoms of dementia), people with dementia often fall into psychologically unstable modes, including fear, sad and anger. The dementia counseling is a care that cures such unstable situations through receptive and attentive conversations. Preferably, any person with dementia should receive the counseling as often as possible. However, it is unreasonable to consult a professional doctor or therapist every day. Also, even for caregivers at home (e.g., family), the time and effort spent on the counseling are quite limited since they have many other supports to do.

We have been studying an assistive technology, called Virtual Care Giver (VCG) [10], that supports elderly people at home by using the virtual agent (VA) technology. The VA is an animated chat-bot software with speech recognition and synthesis technologies. Through a PC screen, a user can talk to the VA who behaves like a human being. Integrated with smart home and cloud services, the VCG provides information, communication and assistive services for elderly at home. The VCG can become a companion for a user regardless of time, and it never gets tired. Therefore, we consider that the VCG is a very promising technology for the daily dementia counseling at home.

However, we have found it difficult to apply the current version of VCG to the dementia counseling. To perform effective counseling, a counselor carefully chooses appropriate conversation topics for individual people with dementia. It should reflect the personal situation, including the life history, the current living, hobby and preference, and so on. However, the VCG currently relies on a static playscript for what VA should speak to a user. The playscript must be programmed by a service developer, and thus the contents of the conversation is almost fixed before the counseling. To adapt individual people with dementia, the developer has to write many different scripts reflecting their situations. This causes much development effort.

Our long-term goal is to develop a system that provides daily dementia counseling for people with dementia living at home. Especially in this paper, we propose an efficient method that can dynamically generate personalized dialogues for every person with dementia. More specifically, we develop the method with the following three approaches.

  • A1: Extracting life history with Center Method

  • A2: Expanding conversation with linked open data (LOD)

  • A3: Generating personalized dialogues using life history and LOD

As for A1, we propose a method where a system obtains the personal background of a user (i.e., person with dementia) based on his/her life history. The life history is personal information of a person with dementia about how he/she has been living so far. It includes birthplace, family, school, work, reminiscence, hobby and so on. In the proposed method, the system obtains user’s life history from the given care management sheets of the Center Method [9], or by asking questions so that the user fills data items of the sheets. In A2, we propose a method that can expand and enrich simple conversation, according to the response from the user. Specifically, when a user answers a word, the system tries to find other relevant words by exploiting the Linked Open Data (LOD) [1], and uses the new words for the next conversation. In A3, we propose a mechanism that dynamically generates personalized dialogues using the life history and the LOD. For this, we use dialogue templates, which specify common outlines of conversations. During run-time, the system chooses a dialogue template, and fills it with personal topics obtained from the life history and the LOD to build a personalized dialogue.

Based on A1, A2 and A3, we develop a prototype of the daily counseling system. The prototype system provides personalized dialogue based on birthplace of a user. First, the system asks user’s birthplace. Then, using LOD, the system finds a specialty associated with the birthplace and generates a new question with the specialty. Thus, we can see that the prototype system can provide personalized dialogues without pre-defined playscript. Thus, the proposed method can contribute significantly to the daily counseling system for people with dementia.

2 Preliminary

2.1 Home Demantia Care

With the increase of elderly people, the number of people with dementia grows accordingly. Many facilities of welfare and nursing care suffer from chronic shortage of workers. The number of job openings is increasing year by year (as of Dec. 2016) [5]. The number of nursing home is not sufficient against the number of applicants, who are over 524,000 elderly people. The Japanese government starts to encourage home care rather than building new facilities. It is more and more important to consider quality of life of elderly, and to support their independent life at home.

As symptoms of dementia vary greatly from one person to another, the person-centered care [6] is considered as an ideal principle of dementia care. In the principle, one must respect every patient as a human being, understand the patient from his/her perspective, and provide a tailor-made care for the patient.

2.2 Dementia Counseling

The dementia counseling is a non-drug treatment for people with dementia. A caregiver relieves patients in a psychologically unstable condition through receptive and attentive conversations. The caregiver should take appropriate topics and conversation attitudes carefully, based on the principle of person-centered care [8, 11].

The counseling has effects of removing fear, stimulating positive feeling via long-term memory, keeping good condition, giving vitality for life, and so on. In the communication, attitudes of accept, sympathize and listen are important. The dementia counseling is normally operated by a professional counselor and a speech therapist. However, more casual (but practical) counseling can be performed by a family caregiver.

Preferably, the dementia counseling should be performed on a daily basis. However, it is expensive to receive professional treatment every day. It is also difficult for a family caregiver to spare sufficient time and effort just for the counseling. Thus, in reality, it is quite challenging to achieve daily and on-demand dementia counseling.

2.3 Virtual Care Giver [10]

Our research group has been studying smart services that exploit the virtual agent (VA) technology to assist elderly people at home. In a project, we are developing a system, called Virtual Care Giver (VCG) [10], where the VA integrates smart home and cloud services to provide home care.

Fig. 1.
figure 1

Screen shot of virtual care giver

Full size image

Figure 1 shows a screen shot of VCG. The VA appears in the left. The VA is an animated human-like chatbot program, implemented with the speech recognition and synthesize technologies. A user can interact with the VA via voice. Connected with behaviors of the VA, the VCG can display supplementary texts, pictures and movies in a Web browser, as shown in the right side of the figure. Using VCG, we have implemented elderly care services, such as daily greeting, routine reminder, and favorite song movie. The VCG can be an accompany of a user regardless of time, and it never gets tired. Therefore, the VCG is a quite promising and realistic solution for the daily dementia counseling.

However, we have found it difficult to apply the VCG directly to the daily dementia care. Currently, every conversation of VCG relies on a playscript. The playscript is written as a program by a service developer, and it must be prepared before execution of the care scenario. During run-time, the VCG chooses a designated script and starts conversation. On the other hand, the dementia counseling requires person-centered topics and conversation attitudes. Therefore, to adapt individual people with dementia, the developer has to write the enormous number of playscripts to cover all possible situations.

2.4 The Center Method [9]

The Center Method [9] is known as a practical tool to support person-centered dementia care. The method provides a pack of sheets (forms), where every person with dementia (or caregiver) writes personal circumstances in the sheets. The sheets are shared by his/her surrounding people to consider person-centered treatments. The method aims to cover the following five categories:

  • (A) Basic Profile: describes the basic profile, degree of independence, conditions of disease to grasp the current situation of the person.

  • (B) Life: describes preferences and histories of the way of life and environment most important for the person.

  • (C) Mind and Body: objectively summarizes physical or mental problems that the person wants to be supported.

  • (D) Focus: identifies what the person can do and know based on clinical observation.

  • (E) Care Plan Introduction: creates a preferred care plan based on topics identified from A to D.

There are 16 sheets within the above five categories. One can start with even a single sheet. Filling personal information in the sheets reveals ideas and possibilities useful for the person-centered care.

2.5 Linked Open Data

Linked Open Data (LOD) is a technology of sharing open data, where the published data is linked on the Web. LOD is represented in RDF (Resource Description Framework), which describes every data as a machine-readable Web resource. RDF specifies every resource as a triple of subject, predicate, and object. Figure 2 shows a schematic representation of an RDF data model. In the figure, an oval represents a resource, a rectangle represents a literal (constant value), and an arrow represents a predicate. This example describes two facts: “the area of Tokyo is 2,188 km\(^2\)” and “The country where Tokyo exists is Japan”. Since every data is referred by URI, a resource can be linked with other relevant resources. For a given original concept, linked data across different domains allow machine processing to derive various associated concepts automatically. Famous LOD datasets include DBpedia, which is LOD version created from Wikipedia. Some LOD publish a SPARQL endpoint, which allows a client to query RDF in the SPARQL query language. If no SPARQL endpoint is available, a client downloads whole dataset as a file, or uses WebAPI to obtain data in designated format.

Fig. 2.
figure 2

Graph representing RDF

Full size image

3 Generating Personalized Dialogue for People with Dementia

3.1 Goal and Approach

The goal of this paper is to propose a method that generates person-centered dialogues essential for daily dementia counseling. When we let a system provide a dementia counseling, we have to consider carefully what topics should be appropriate. Due to the memory impairment, which is the core symptom of dementia, the person forgets recent events and information quite easily. Therefore, even if the system asks about timely news or recent topics, the person would not be able to follow the conversation. This would lead to the loss of confidence or physiological anxiety of the person.

Moreover, the system should behave attentively according to what the person says. For example, suppose that the system asks a person with dementia about his/her birthplace. Depending on the person, of course, the answer varies like Hiroshima, Okayama, Tokyo or so on. In our previous VCG (see Sect. 2.3), the developer had to prepare playscripts that cover all possible birthplaces. Moreover, to expand the conversation associated with the birthplace, further scripts are needed. So, it is quite expensive for the developer to write such a lot of playscripts in advance.

To cope with the challenge, we take the following three approaches.

  • A1: Extracting life history with Center Method

  • A2: Expanding conversation with linked open data (LOD)

  • A3: Generating personalized dialogues using life history and LOD

3.2 A1: Extracting Life History with Center Method

For an effective dementia counseling, it is essential to affect the long-term memory of a person with dementia. Therefore, we propose to choose topics of counseling based on life history of the person. The life history refers to information about how the person has been living so far, which includes birthplace, family, school, work, reminiscence, hobby, and so on. Compared to timely news, the life history is more robust information in the long-term memory. Also, the person can easily explain it, since it is his/her own history. Moreover, the life history represents unchangeable facts, which can be easily managed by the system.

To extract the personal life history, we extensively use four sheets from the Life category of the Center Method (see Sect. 2.4).  

(Sheet B-1) My Family: :

describes family of the person with dementia.

(Sheet B-2) My History: :

describes history and records of life of the person.

(Sheet B-3) My Style: :

describes preferred life style of the person.

(Sheet B-4) My Living: :

describes preferred living environment of the person.

 

Figure 3 shows an example of Sheet B-3. We can see the personal life style like the dietary habit, favorite music, what he is good/bad at, and so on.

Fig. 3.
figure 3

Example of sheet B-3 - my history

Full size image

Using the life history information derived from the above sheets, the system finds a clue to the person-centered conversation in the dementia care.

To extract the life history information, we consider two methods. The first method is that a user (person with dementia or caregiver) manually fills the Sheets B-1 to B-4 and the system operator registers the information to the system. This method would cause expensive effort for the user to fill many data items in the sheets.

The second method is that the virtual agent (VA) interactively asks the person with dementia to fill the sheet. The VA asks questions like “Where were you born?”, “What did you do for living?”, “What is your favorite music?”. The person answers each question via voice. Although this method takes time, the system can create opportunity of conversations, in addition to the acquisition of the life history.

3.3 A2: Expanding Conversation with Linked Open Data

“A system asks a question, then a person with dementia answers it. The system moves to the next question.” Such mechanical dialogues make the dementia counseling boring. For example, consider the following dialogue where the VA tries to extract the life history (as mentioned in A1).

VA: “Where were you born?”, Person: “I was born in Hiroshima.”

If the conversation just ends here, then the counseling would be quite poor. For more person-centered counseling, it is important to expand the conversation, based on the answer from the person with dementia. For instance, following the above conversation, if the system could produce one more question like:

VA: I know Hiroshima’s specialty is Okonomi-yaki. Do you like it?

then the counseling would be enriched significantly. However, it is not easy for the system to manage all possible prior knowledge necessary to expand the conversation.

To cope with this, the proposed method uses the linked open data (LOD, see Sect. 2.5) to implement on-demand expansion of the conversation. More specifically, when the system detects a characteristic word in user’s answer, the system tries to find “linked” words using LOD, and use the new words for the subsequent conversations. In the above example, suppose that the system recognizes the word “Hiroshima” in the answer. Then, the system looks up LOD to obtain information associated with Hiroshima. From “Hiroshima”, if “Okonomi-Yaki” is found as a linked word with predicate “specialty”, the system says “I know Hiroshima’s specialty is Okonomi-Yaki”.

The implementation of the method is as follows. First, if a SPARQL endpoint of the LOD is available, the system just queries to the endpoint. For example, a query of DBpedia Japanese that extracts neighboring prefectures of Hiroshima can be written as follows:

figure a

If the LOD is provided via WebAPI, the system executes the API and parses the obtained data. If the LOD is provided as a file, we download the file in advance and import the file to the system database. During run-time, the system looks up the database.

3.4 A3: Generating Personalized Dialogues Using Life History and LOD

In the proposed method, we define an outline of each counseling scenario by a dialogue template. A dialogue template is a template of conversation commonly used by all users in a counseling scenario. Each dialogue template has variables to be changed. During run-time, the system updates the variables based on the personal life history and LOD, which generates the person-centered dialogue.

To cover a wide variety of topics, the proposed system has a set of dialogue templates, corresponding to data items in the Life category of the Center Method. For example, there are birthplace template, hobby template, work template, school template, and so on. When a counseling starts, the system chooses an appropriate dialogue template based on the registered life history information. During the counseling, the system instantiates the template by filling actual values to the variables, using the available life history and LOD.

We illustlate an example of the dialogue template as follows.

figure b

This dialogue template specifies a conversation where a system first asks birthplace of the user and then expands the conversation for a specialty of the birthplace. #{info.name} represents a variable where user’s name is assigned. The function getAnswer() in the second line represents an operation that obtains a word from user’s answer. The result is assigned to a variable pref, representing a prefecture of birthplace. The last variable in the third line shows an operation that obtains LOD from pref linked with “specialty”.

When applying the above dialogue template to user “Seiji”, the following personalized dialogue is generated.

figure c

3.5 System Architecture

Figure 4 shows the overall architecture of the proposed counseling system. We call the system that implements the proposed methods A1, A2 and A3 Virtual Counselor. To provide a counseling service for many households of people with dementia, the virtual counselor is deployed within a cloud. In each house, a client PC with the virtual agent and the Web browser is deployed. The virtual counselor pushes counseling dialogues via WebAPI of the client PC, and pulls user’s response.

Fig. 4.
figure 4

System architecture of virtual counselor

Full size image

We explain how the virtual counselor service is provided. First, a person with dementia and caregiver manually fill the sheets of the Center Method to the extent that they can. The operator registers the information to the system using the feature A1. When a counseling service is executed, the virtual counselor picks up a dialogue template based on the available life history information, and starts a conversation. During the conversation, the virtual counselor dynamically fills variables of the template with feature A3, to generate personalized dialogue. For this, if the feature A3 requires unknown life history data, the virtual counselor asks a question with the VA using feature A1. As the user answers the question via voice or a button, the virtual counselor registers the answer in the database. When expanding the conversation, the virtual counselor queries LOD using feature A2, to retrieve necessary data. All components in the cloud and the client PCs are integrated with Web service, considering the principle of service-oriented architecture (SOA) [4]. The architecture facilitates further integration, in the future, with IoT and smart home.

4 Implementation of Prototype

Based on the proposed method, we have implemented a prototype system of virtual counselor. Technologies used in the implementation are as follows:

  • System Language: Java 1.8.0_25, Ruby

  • Web Server: Apach Tomcat 7.0.69

  • Web Service Framework: Jersey 1.19, Apache Axis2 1.6.3

  • Virtual Agent: MMDAgent version 1.4

  • LOD: DBpedia Japanese [3], LinkData [7]

This prototype implements the personalized dialogue for the user’s birthplace. The dialogue template specifies conversations, where the VA first asks a birthplace, and then talks about the specialty of the birthplace.

Figure 5 shows an example conversation generated by the prototype. First, the VA asks the user: “Where were you born, Seiji?” Suppose that the user Seiji answers “I was born in Fukuyama-city.” Now, the answered birthplace is not a prefecture, the system looks up DBpedia to find a prefecture where Fukuyama-city exists. The VA confirms that “Do you mean Fukuyama in Hiroshima prefecture?” Then, the user says “Yes.” As the prefecture is confirmed to “Hiroshima”, the system looks up LinkData to retrieve “Okonomi-Yaki” as a specialty of Hiroshima. Finally, the VA says that “I like Okomomi-Yaki of Hiroshima very much!”

Fig. 5.
figure 5

Personalized dialogue generated by prototype system

Full size image

5 Discussion

Towards the practical use, we discuss some issues learned through the prototype implementation. First, we have found that only DBpedia and LinkData cannot cover a wide range of topics and individual hobbies and preferences. We may need a method to discover necessary LOD dynamically during the counseling.

Second, the heterogeneous data access methods to LOD, such as SPARQL endpoint, WebAPI or file, make the system difficult to scale. In case that the variety of LOD increases, we need an extra service layer that abstracts the heterogeneous data accesses. With the service layer, the counseling system can acquire data in a uniform format (subject, predicate and object).

Third, in terms of generating effective counseling scenarios, it is important to cooperate with caregivers of people with dementia to know his/her favorite topics. Our current system assumes that an expert service developer creates dialogue templates as system program. In the future, however, we want to develop a method that allows caregivers to generate the templates easily without programming knowledge.

Finally, we have to consider the evaluation method. The evaluation should be conducted quantitatively; how often and how much the counseling system can cure physiologically unstable conditions of people with dementia. We need an empirical analysis of system logs and clinical data. It is also interesting that the system autonomously learns the data to find better counseling. The system collects answers of not only multiple-choice questions, but also free-form questions. For this, we need natural language processing to analyze user’s utterance.

6 Conclusions

To achieve the daily counseling system for home dementia care, we have proposed a new method that dynamically generates personalized dialogues for individual people with dementia. The proposed method obtains the life history in order to choose person-centered conversation. It also expands the conversation with associated topics using LOD. We have also implemented a prototype system to show practical feasibility of the proposed method. In our future work, we develop methods of triggering a counseling, dynamic discovery of LOD, template creation by non-experts. We also plan to conduct experimental evaluation with actual people with dementia.