=Paper=
{{Paper
|id=Vol-3124/paper15
|storemode=property
|title=ReSmart-15: An Information Gain based Questionnaire for Early Demetia Detection
|pdfUrl=https://ceur-ws.org/Vol-3124/paper15.pdf
|volume=Vol-3124
|authors=Hyeseong Park,Myung Won Raymond Jung,Ji-Hye Kim,Uran Oh
|dblpUrl=https://dblp.org/rec/conf/iui/ParkJKO22
}}
==ReSmart-15: An Information Gain based Questionnaire for Early Demetia Detection==
https://ceur-ws.org/Vol-3124/paper15.pdf
ReSmart-15 : An Information Gain based Questionnaire for
Early Dementia Detection
Hyeseong Park1 , Myung Won Raymond Jung1 , Ji-Hye Kim2 and Uran Oh3
1
AKA Cognitive Corp., Seoul, South Korea
2
Department of Family Medicine, Yonsei University College of Medicine, Seoul, South Korea
3
Department of Computer Science and Engineering, Ewha Womans University, Seoul, South Korea
Abstract
To build an effective questionnaire for detecting early dementia, we propose ReSmart-15 which is a dementia detection
questionnaire that includes daily behavior-based questions in five categories (i.e., attention (3Q), spatial ability (3Q), spa-
tiotemporal ability (3Q), memory (3Q), and thinking ability (3Q)). As for the evaluation, we first collected responses from two
different screening tests with 87 participants. Then we used a machine learning method called "information gain" ranking to
show the effectiveness of ReSmart-15 compared to another representative screening test. As a result, we found that the top 2
questions were from ReSmart-15, and 60 percent of ReSmart-15 questions were in the top 10.
Keywords
early dementia, questionnaire, information gain
1. Introduction selection through information gain to rank the impor-
tance of all features and then filter out the low-ranking
Existing screening tools for dementia have several lim- features. In information gain, each question in the ques-
itations. For example, Mini-Mental State Examination tionnaire was treated as a feature when it had different
(MMSE) [1], one of the widely used tests for measuring importance in the prediction of a dementia diagnosis. In
the clinical dementia rating scale (CDR) [2], is insensitive the same way, we computed the information gain to see
to detecting the early stages of dementia [3], especially if ReSmart-15 is ranked higher than NMD-12 [8]. The
for highly educated individuals [4, 5]. results show that most questions in ReSmart-15 were
Using a screening test with low specificity could lead to ranked higher than the NMD-12 questionnaire. This sug-
an incorrect diagnosis of dementia in elderly individuals. gests that ReSmart-15 was composed of influential ques-
Therefore, SED-11Q [6] aimed to investigate the state of tions filtered by information gain, which may increase
daily activities performed in various aspects by including the accuracy of the early diagnosis of dementia.
questions that dealt with social interactions and person-
ality. In this study, inspired by the SED-11Q, we propose
a questionnaire named ReSmart-15 by modifying the ex- 2. Method
isting MMSE questionnaire for better detection of early 2
dementia. The questionnaire consists of daily behavior- We collected audiences from SurveyMonkey to recruit
based questions in five categories (i.e., attention (3Q), 87 participants (53 female) excluding four who dropped
spatial ability (3Q), spatiotemporal ability (3Q), memory out. Their average age was 38.0 (ππ·=13.4, range=18-65).
(3Q), and thinking ability (3Q)), which are explained by To show the effectiveness of ReSmart-15 compared to an-
CogniFit1 which designed cognitive assessment through other existing screening questionnaire for early dementia
monitoring the patientβs cognitive rehabilitation process detection, we conducted a user study where participants
[7]. were asked to submit their responses to 27 different ques-
As for the evaluation, we collected responses from tions: 15 questions from ReSmart-15 and 12 questions
87 participants who were asked to answer 27 questions from NMD-12 [8]. It took 4 minutes and 25 seconds on
from two different screening tests for dementia, ReSmart- average to complete the task.
15 and NMD-12 [8]. NMD-12 uses automated feature
3. Evaluation
IUI β22 Workshops: Joint Proceedings of the ACM IUI 2022 Workshops,
March 22β25, 2022, Helsinki, Finland To evaluate the importance of the questions in ReSmart-
$ julie@akaintelligence.com (H. Park); rjung@akaintelligence.com
15 compared to NMD-12, where it uses the information
(M. W. R. Jung); BLES4YOU@yuhs.ac.kr (J. Kim);
uran.oh@ewha.ac.kr (correspondence) (U. Oh) gain (IG) ranking in machine learning [8]. Let πΈ(π·) be
Β© 2022 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
an entropy, and πΈπ (π·) be the amount of information to
CEUR
Workshop
Proceedings CEUR Workshop Proceedings (CEUR-WS.org)
http://ceur-ws.org
ISSN 1613-0073
1 2
https://www.cognifit.com/ https://www.surveymonkey.com/
Table 1
The Questions ranks through information gain
Rank IG Questionnaire Questions
1 0.28 Resmart-12 Do you often forget the points that you want to talk about?
2 0.27 Resmart-9 Are you having a hard time remembering where things are usually kept?
3 0.25 NMD-8 Is it difficult to learn how to use tools or equipment?
4 0.23 Resmart-3 Do you suspect others of hiding, or stealing items when they cannot find them?
5 0.22 NMD-2 Do you forget the correct year and month?
6 0.22 NMD-6 Are regular activities or ordinary hobbies less than before?
7 0.21 Resmart-8 Do you have trouble concentrating more than an hour?
8 0.20 NMD-11 Are working or professional skills getting worse?
9 0.17 Resmart-5 Do you often forget appointments?
10 0.16 Resmart-4 Do you have trouble handling money, such as when giving tips or calculating change?
11 0.15 Resmart-7 Do you become disoriented in unfamiliar places?
12 0.15 Resmart-15 Do you confuse names of family members or friends?
13 0.14 Resmart-14 Do you feel that learning a new stuff takes longer than before?
14 0.14 NMD-9 Is it difficult to get out (ride or drive to destination)?
15 0.14 NMD-7 Is it difficult to take medicine by yourself?
16 0.14 Resmart-11 Are you getting lost in familiar surroundings such as their own neighborhood?
17 0.14 NMD-3 Is it difficult to remember the time of appointment?
18 0.13 NMD-10 Is it getting worse to manage money?
19 0.13 Resmart-10 Do you repeat questions or statements or stories in the same day?
20 0.13 NMD-5 Is it difficult to learn how to use tools or equipment?
21 0.12 Resmart-1 Is cognitive function significantly worse than before?
22 0.12 NMD-4 Is it difficult to deal with complicated financing issues?
23 0.11 NMD-12 Do you often forget what youβve talked about recently?
24 0.11 NMD-1 Has cognitive impairment influence daily life, social networking and work?
25 0.11 Resmart-2 Do you misplace items more than once a month?
26 0.08 Resmart-13 Do you have to drink coffee to wake yourself up?
27 0.01 Resmart-6 Do you remain energetic in every day life?
make an exact classification based on the partition by 4. Findings and Future Work
questions π, where π· is the data sample in the training
set. Then, πΈ(π·) and πΈπ (π·) can be calculated as follows: The purpose of this experiment was to select influential
questions based on a machine learning technique called
βοΈπ information gain. This technique treats each question
πΈ(π·) = β ππ log2 (ππ ), as a feature and each of them has a different level of
π=1 importance in the prediction of a dementia diagnosis.
π
As shown in Table 1, our experiment found that the top
βοΈ |π·π |
πΈπ (π·) = β Γ πΈ(π·π ), 2 questions were from ReSmart-15, and 60 percent of
π=1
|π·| ReSmart-15 questions were in the top 10. This suggests
that ReSmart-15 was composed of influential questions
where π is the number of classes, ππ is the probability filtered by information gain, which may increase the
that an arbitrary tuple in π· belongs to class π, and π is accuracy of the early diagnosis of dementia. Furthermore,
the number of distinct values in π. In this study, we make information gain can be used to remove redundant or
total two classes where each class represents whether unnecessary features with low importance (i.e., ReSmart-
the user is in early dementia or not, which makes π = 2. 13 and ReSmart-6), and can simplify the procedure of
Also, each questionnaire in ReSmart-15 and NMD-12 can diagnosis.
only be answered by either yes or no, which makes π = 2. In the next experiment, to make our social robot as a
For simplicity, we diagnose early dementia when the health care device, we would like to examine whether
number of "yes" answers is 14 or more, i.e., more than the questions with the high impact selected by the ma-
half of the total 27 questions. IG of each questionnaire π chine learning the technique is likely to be more effective
can be calculated by the difference between πΈ(π·) and with a conversational voice-based interactive chatbot in-
πΈπ (π·). terface where it is known to have several benefits. It
has the potential to act as a doctor for people with de-
Jan
23 24 25 26
β’ Today, letβs improve memory!
β’ What is the date today? β’ (The user solves the problem) β’ Yes, you did well.
Figure 1: Questionnaire Interface.
mentia by providing deep learning chat and awareness [7] R. Jung, B. Son, H. Park, S. Kim, M. Wijaya, Res-
combination of information. It helps users [9] to drive mart: Brain training games for enhancing cognitive
conversations with users rather than simple Q&A ser- health, in: Advances in Computer Vision and Com-
vices, and to ease technical barriers and build intimacy. putational Biology, Springer, 2021, pp. 331β337.
We would like to experiment with a conversational voice- [8] P.-Y. Chiu, H. Tang, C.-Y. Wei, C. Zhang, G.-U. Hung,
based user interface called Musio 3 as shown in Figure 1. W. Zhou, Nmd-12: A new machine-learning de-
It is deployed with Muse, which has a natural language rived screening instrument to detect mild cogni-
processing (NLP) engine. Musio introduces himself at tive impairment and dementia, PloS one 14 (2019)
first and asks the questions (ReSmart-15) in a conver- e0213430.
sational format and we expect that Musio will relieve [9] C.-L. Liu, S.-R. Chang, A pilot tool of the virtual sce-
the usersβ burden of testing their cognitive abilities by nario initial dementia cognitive screening (vsidcs)
providing a user interface with a familiar voice [10]. with a cultural exhibition for improving the stan-
dard traditional test, in: Healthcare, volume 9, Mul-
tidisciplinary Digital Publishing Institute, 2021, p.
References 1160.
[10] S. Kim, J. Lee, G. Gweon, Comparing data from
[1] J. R. Cockrell, M. F. Folstein, Mini-mental state
chatbot and web surveys: Effects of platform and
examination, Principles and practice of geriatric
conversational style on survey response quality, in:
psychiatry (2002) 140β141.
Proceedings of the 2019 CHI conference on human
[2] J. C. Morris, The clinical dementia rating (CDR):
factors in computing systems, 2019, pp. 1β12.
Current version and, Young 41 (1991) 1588β1592.
[3] J. Lawrence, D. Davidoff, D. Katt-Lloyd, M. Auer-
bach, J. Hennen, A pilot program of improved meth-
ods for community-based screening for dementia,
The American Journal of Geriatric Psychiatry 9
(2001) 205β211.
[4] W. J. Lorentz, J. M. Scanlan, S. Borson, Brief screen-
ing tests for dementia, The Canadian Journal of
Psychiatry 47 (2002) 723β733.
[5] D. V. Espino, M. J. Lichtenstein, R. F. Palmer, H. P.
Hazuda, Evaluation of the mini-mental state ex-
aminationβs internal consistency in a community-
based sample of mexican-american and european-
american elders: results from the san antonio lon-
gitudinal study of aging, Journal of the American
Geriatrics Society 52 (2004) 822β827.
[6] Y. Maki, T. Yamaguchi, H. Yamaguchi, Symptoms of
early dementia-11 questionnaire (sed-11q): a brief
informant-operated screening for dementia, De-
mentia and geriatric cognitive disorders extra 3
(2013) 131β142.
3
https://akaintelligence.com/