Dialog systems have been studied separately as task-oriented dialog (TOD) to accomplish a specific task, such as booking a trip, and non-task-oriented dialog (CC: Chit-Chat) to entertain the user for the purpose of the dialog and the continuation of the dialog itself. However, when dialog systems are applied to the elderly, not only TOD but also robot-initiated dialog, which is classified as CC, is required. This is because, in order to encourage users to use the system over the long term, it is necessary to implement a system that suggests useful tasks for the user, in addition to tasks that the user selects on his/her own, such as QA for schedule confirmation and physical condition management. The purpose of this study is to add autonomy to TOD to activate users and encourage long-term use. Therefore, we propose to define the tasks of the dialog system as hidden tasks and to integrate dialog robot-initiated dialog into TOD. In particular, the system is expected to enable long-term use of user data and minimally invasive evaluation of the user's cognitive functions in monitoring the elderly.
Oriented Dialog (TOD) to accomplish specific tasks, such as travel reservations, and Non-Task-Oriented Dialog (CC: Chit-Chat) to entertain users for the purpose of the dialog or the continuation of the dialog itself. Therefore, as the conditions for constructing TOD, natural dialog and user consideration are required, but research on user consideration is not as advanced as CC on natural dialog. Therefore, this study proposes to integrate TOD and CC by using a technique of swapping user information storage and conversation control to expand the potential of TOD. Specifically, the proposed method makes it possible to integrate TOD and CC by defining tasks that operate separately from the original task, called hidden tasks. The hidden task is explained in section 3.
One application of this integrated method is in the area of elderly monitoring. When applying dialog systems to the elderly, not only TOD but also Robot-Initiated Dialog classified as CC are necessary. This is because, in order for users to use the system for a long time, it is necessary to implement a system that proposes useful tasks for users, such as checking schedules and managing their health, in addition to tasks that users choose themselves.
This paper describes the integrated method and its efective application.
In this study, we propose the system that can be applied to elderly monitoring, which is believed to have the effect of reducing the social isolation of the elderly and improving or delaying cognitive decline. To measure the efectiveness of this system, we utilize a user information management ontology to measure factors such as the user’s social relationships, memory, and sleep quality.
These factors can be recorded as usage history, allowing
for the system to capture dynamic changes. The
system primarily uses interfaces such as voice recognition,
speech synthesis, and display screens, with no variation
in format based on the user, ensuring fairness. However,
since the system includes a configuration to adapt
dialog content to the user, there is a risk of losing fairness
in determining the efects on the user based on usage
frequency.
such as BERT, has realized a system that uses machine diagnoses based on that information. To give an example,
learning to pre-training a large dialog corpus to infer if the user’s emotions are inferred from his/her facial
tasks from user input sentences and generate questions expressions and boredom (intent) is read, a hidden task
to gather information[
Recently, however, there have been studies on the in- topic and user information at that time. It is necessary to
tegration of the two systems, Kai et al.[
On the other hand, unlike Kai et al., we aim at the integration method that use CC in order to make also efective use of the free time when TOD is not being performed. This approach is also a solution to the problem that, in a monitoring system that requires all-day operation, TOD is defined as a conversation for medical checkups and schedule confirmation, but no conversation takes place during the time when the user is not engaged in such conversations unless the user asks.
collaboration, the hidden task used as a clue for TOD and CC collaboration, and its application to monitoring the elderly. The main components of the dialog robot used in the experiment are described in section 4.
3.1. How to structure TOD and CC sume that it will be used by elderly. In TOD for elderly, it Collaboration is required to conduct QA such as schedule confirmation and physical condition management. In addition, unlike Here, we explain how to compose the coordination of ordinary TOD systems, it is essential to consider the use TOD and CC under the assumption that the dialog system of the system in such a way that the system suggests the is always in operation. First, we consider the coordina- user some task that might be useful to the user, rather tion of TOD and CC, and adapt TOD-based composition than in such a way that the system waits for the user to to both of them. The proposed system segregates in- ask for a specific task the user chooses by himself/hertentions (boredom or tension) in chats and sets hidden self. Therefore, our method, integrating CC with TOD, is tasks such as "getting close" or "relaxing", and fills slots suitable for dialog robots for the elderly. with emotions inferred from tone of voice, facial expres- In this section, referring to the content of tasks, since sions, gestures, topics, and so on, through multimodal elderly people spend a lot of time in institutions and coordination. Then, according to the hidden task, the homes, where there are few topics of conversation, it is speaker’s response or the next topic is selected and the essential for the dialog system to spontaneously provide chatting proceeds. Specifically, the scenario proceeds by topics for chats. Furthermore, when an elderly person estimating the speaker’s intentions, setting tasks, analyz- shows a tendency toward dementia, it is necessary to ing responses, deciding slots, and then evaluating action ask questions to confirm the disease. Considering the policies. consistency of the dialog and the structure of the scenario that takes the elderly into consideration, there is a 3.2. Hidden Task requirement to ask questions in the form of chats rather than formalizing dementia screening.
Hidden Task assist the dialog robot in autonomous in- Therefore, this study envisions the use of User Informateraction, allowing it not only to integrate CC into TOD, tion Management Ontology (UIM Ontology) to manage but also to elicit information from the user and make user information and apply it to chats and the identification of dementia tendencies. For example, when there is a topic about a favorite food, the user and the food can be managed in the ontology, and the user’s favorite food can be guessed in the next dialog, or the memory of the previous dialog can be checked, thereby identifying the tendency toward dementia while entertaining the user as chats. A simple data flow diagram is shown below. The structure of UIM Ontology is also described in section 4.
The files contained in the KB are divided into three user’s needs while fulfilling its role as a TOD. In addition, main types: an intention dictionary, a slot dictionary, in order to make the dialog more user-friendly, we have and a meta-word dictionary. The intention dictionary de- added UIM Ontology in this study. scribes a combination of scenario names and keywords, and enables the user to invoke scenarios by inferring intentions from the keywords in the utterances. The slot dictionary is divided into json files according to abstraction level, and in order of abstraction level, the files are ( slot: pattern name ), ( pattern name: pattern ). Slots are extracted by pattern of the assigned pattern name. Finally, the meta-word dictionary describes combinations of commands and keywords, allowing users to freely manipulate the scenario using commands by uttering keywords during scenario activation.
Generator The Generator consists of a NLG module for generating the robot’s speech text and an image acquisition module for displaying images. The NLG module generates the speech text by assigning the dialog text defined in the scenario to the dialog state. The image display module is implemented in such a way that it retrieves and displays images by specifying the path of the location where the images are stored or URL .
DM to make it clear that it was added later to the main structure of the dialog system, but as shown in Figure 2, it plays the same role as Task Decision, so Hidden Task Decision is implemented within DM. The role of Hidden Task Decision is to determine the hidden task by estimating the user’s intention in his/her free time, as described in section 3. Here, a diagram of an example dialog is shown below.
The role of Hidden Task Decision is to determine the hidden task by estimating the user’s intention in his/her free time, as described in section 3. Here, a diagram of an example dialog is shown below. The blue frame is user-initiated and the green frame is robot-initiated. The flow of the dialog is as follows: First, the TOD has a high priority, so when the user asks a question, the robot responds to it with priority. In this example, the user greeted the robot, so the first task is to respond to the greeting. At the same time, since the task after the greeting is not set in stone, a hidden task works for the next topic. Here, the hidden task of "activating the user" is set to continue the dialog, since the user’s emotion recognition by his/her facial expression was read to be "smile". Then, in the middle of this dialog, the scenario is interrupted by the user’s question, "Wait a minute". In this way, the introduction of Hidden Task Decision enables the system to present topics according to the User Information Management Ontology (UIM Ontology) UIM Ontology only interacts with Hidden Task Decision as shown in Figure 4. Therefore, functions are defined in Hidden Task Decision to manipulate the ontology using owlready2 in order to mediate with the ontology. In this section, we describe how the ontology is created by the manipulation.
An example of the ontology is shown in Figure 5. We assume that the user’s name is Yoko. First, we describe the input example. When the user advances the topic of his/her favorite food, the user’s favorite food comes back as a slot value. Then, the slot value is entered as a subclass in the food class. Furthermore, we connect the user’s name and the food’s subclass with a property. In this way, information about the user can be accumulated each time the dialog is repeated. As an example of output, when the topic of lunch is first advanced, information about the food is given. The information about the food is queried to the ontology and relevant information is elicited. In this case, we know that it is the user’s preference, and this information can be applied to the dialog.
AIML for ChatBot have been proposed[
The scenario defines the domain, the control of the frame representation requested to the user, the actions of the system to ask the user for the frame representation, and the way to cooperate with external functions. The following is an example of scenario description by HDS-R.
Listing 1: hdsr_scenario.xml
Early Detection Methods for Dementia Medical tests for dementia, such as Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET), have been cited as necessary but time-consuming and expensive. There are also methods such as the Mini Mental State Examination (MMSE) and the Hasegawa Dementia Scale-Revised (HDS-R), but direct question-and-answer sessions with these professionals may be invasive to the patients. Therefore, patient-oriented testing is now required. As an example, an automated screening test based on the processing of patients’ conversational and communicative abilities is attracting attention.
It has been found that the language ability of patients with dementia is inferior to that of normal subjects, and the evaluation of language ability based on the interaction with the dialog system can be used to statistically predict the suspicion of dementia. Furthermore, prediction can be made based on acoustic features such as voice condition and time between utterances.
The scenario tag is the root tag, with the attribute Linkage between CC and HDS-R We describe the name representing the intent. The sequential tag selects structure of scenarios to accomplish the hidden task of dichild elements in sequence until all of them have been agnosis. As scenarios for diagnosis, we attempted to creexecuted. The sequential tag plays the role of action ate scenarios by referring to questions about proverbs[? selection. In addition to the sequential tag, the follow- ] and the HDS-R, which are considered to be a tool that ing action selection tags are implemented: conditional, can easily reveal the tendency to dementia. For questions which selects child elements according to conditions, ran- measuring cognitive abilities (arithmetic, sequencing, obdom, which selects child elements at random, and loop, ject recognition, image recognition, recitation, regression, which keeps selecting until the conditions are satisfied. and repetition), questions corresponding to the HDS-R[? These action selection tags can define action selection ] were used. For environmental register (name, age, place, tags, frame tags, etc. for child elements, and can be used date), we used UIM Ontology to automatically make corfor scenarios with complex state transitions. The frame rect and incorrect decisions. tag is used to fill the frame representation. The actions tag is a tag that summarizes the actions of the system and simultaneously executes the contents of the actions tag of the child elements. The request tag specifies a set of slots that the user is expected to fill. The slot tag defines the slots and can specify attributes such as default values or a list of slot values.
The Developed Dialog Robot So far, we have evaluated the functionality of the dialog robot.
As for the functionality of the dialog robot, the developed dialog robot is equipped with a pre-installed dialog system (Raspberry Pi 3, speakers, microphones, and other devices) in a familiarly shaped container designed for use by the elderly(figure 6). It also implements a specification that allows the user to switch between voice(figure 7) and text-based(figure 8) dialogs on the browser screen.
When we conducted a survey of developers and related parties for evaluation, many of them requested a dialog to confirm that metacommands were used and that speech recognition failed. In the future, it is necessary to improve these functional aspects and make the system more suitable for use by the elderly.
system for elderly. The system is normally working as TOD with multiple tasks: if the user asks to perform some task, the system responses as a normal TOD. In the proposed method, during free time, system will start CC by selecting non-serious topics, such as greeting, simple game, etc. During conversation he system collects the user’s responses and predictions are made based on the conversation data obtained by the topics. When the