=Paper=
{{Paper
|id=Vol-1982/paper1
|storemode=property
|title=Towards a Digitized Protocol based on Handwriting for the Assessment of Neurodegenerative Disorders
|pdfUrl=https://ceur-ws.org/Vol-1982/paper1.pdf
|volume=Vol-1982
|authors=Donato Impedovo,Giuseppe Pirlo,Maria Teresa Angelillo,Gennaro Vessio
|dblpUrl=https://dblp.org/rec/conf/aiia/ImpedovoPAV17
}}
==Towards a Digitized Protocol based on Handwriting for the Assessment of Neurodegenerative Disorders==
https://ceur-ws.org/Vol-1982/paper1.pdf
Towards a Digitized Protocol based on Handwriting
for the Assessment of Neurodegenerative Disorders
Donato Impedovo1, Giuseppe Pirlo1, Maria Teresa Angelillo2 and Gennaro Vessio1
1 Department of Informatics, University of Bari, Bari, Italy
2
I.R.C.C.S. Maugeri, Cassano delle Murge, Italy
{donato.impedovo, giuseppe.pirlo, gennaro.vessio}@uniba.it,
mtangelillo@gmail.com
Abstract. Kinematic features of the handwriting process are promising for dis-
criminating patients probably affected by neurodegenerative disorders from
healthy controls and potentially for identifying the stage of the illness. Many
tasks have been proposed and tested, however the scientific community still lacks
a complete protocol able to collect different traits useful to the research commu-
nity and at the same time to be of real use for doctors in their daily activities. This
paper collects all the different tasks already applied and proposes a protocol for
handwritten traits acquisition to be integrated into standardized cognitive and
functional tests already accepted, tested and used by the medical community
working on the assessment of neurodegenerative diseases.
Keywords: Neurodegenerative disorders, Handwriting, Acquisition protocol.
1 Introduction
Neurodegenerative disorders, such as Alzheimer’s disease (AD) [1] and Parkinson’s
disease (PD) [2], affect the structure and functions of brain regions resulting in a pro-
gressive cognitive, functional and behavioral decline. In this context, the assessment of
signs and manifestations of the disease is crucial for at least three main reasons:
1. The early diagnosis of the disease;
2. Monitoring and tracking of its progression;
3. The diagnostic differentiation of the disease with respect to similar disorders.
Recently, a growing research interest has arisen towards the possibility to automati-
cally discriminate between impaired subjects and healthy controls on the basis of kine-
matic features obtained through simple and easy-to-perform handwriting tasks, e.g. [3‒
5]. Handwriting, in fact, is a complex activity entailing cognitive, kinesthetic and per-
ceptual-motor components [6], whose changes seem to be a prominent biomarker for
the assessment of some neurodegenerative diseases.
However, although encouraging results have been obtained, there are several open
issues that demand further research [7]. Among the others, one problem is connected
D. Impedovo and G. Pirlo (Eds.), Workshop on Artificial Intelligence with Application in Health, Bari, Italy, November 14, 2017.
Copyright held by the authors.
2
to the possibility of collecting a big amount of samples involving a statistically signif-
icant number of patients. Current available datasets, e.g. [4], are typically composed by
very few subjects. Another problem deals with the acquisition tool and protocol, since
data are typically obtained from different devices and different handwriting tasks. The
number and periodicity of sessions should be also taken into account in order to under-
stand the evolution of the illness over time. The final aim is the development of a Com-
puter Aided Diagnosis (CAD) system, similarly to those already available in other
fields [8, 9].
To address these issues, the present paper aims to propose a suitable protocol for
handwriting data acquisition. It is important to note that, since a handwriting-based de-
cision support system is not expected to replace standard techniques, but to strengthen
them by providing precise and quantitative measurements, the proposal here presented
is designed to integrate standardized cognitive and functional tests (already accepted,
tested and used by the medical community). To this end, the protocol focuses on:
• Handwriting tasks for the extraction of kinematic features;
• Standard routine tests that are suitable to be carried out in a digitized way.
The rest of the paper is structured as follows. Section 2 concerns a literary review
that focuses on the most employed handwriting tasks for clinical assessment. Section 3
advances our proposal which takes into account both standard and handwriting tasks.
Section 4 concludes the paper and sketches future developments of our research.
2 Literary Review
Among the various methodological approaches, those related to Alzheimer’s disease
and Parkinson’s disease have been considered: being the two most widespread neuro-
degenerative disorders, they have been the most widely investigated by researchers.
As mentioned above, writing is a skill that involves a complex feedback system and
implicates the participation of several cognitive processes. For this reason, most of the
works in literature focus on tasks such as the Archimedes spiral, single letters or simple
words/sentences writing in order to investigate purely motor aspects of handwriting.
These tasks have been extensively used in studies based on static features of handwrit-
ing and significant differences between impaired subjects and healthy controls, based
on these features, have been revealed [10, 11].
Similar findings have also been reported in papers focusing on the kinematic features
of the handwriting process. The Archimedes spiral is probably the most popular hand-
writing exercise. Spiral, as well as circle drawing, in fact, have been frequently used
for the evaluation of the motor performance in various movement disorders, including
AD and PD, e.g. [12, 13, 4]. The works of Pullman [14], in particular, pioneered the
application of spiral analysis to the measurement of tremor.
Words containing one or more repetitions of the cursive letters “l” and “e” are the
second-most common exercises in handwriting assessment, e.g. [15‒18]. This because
such letters are easy to write in a recursive and continuous way; moreover, their use
minimizes the linguistic-comprehension processes.
3
In addition to these tasks, writing simple words and short sentences have also been
widely adopted, e.g. [19, 20]. Typically, the words/sentences used in these tasks are
chosen on the basis of their simple orthography and easy syntax.
In [21] and [22], the authors asked the participants of the studies to perform func-
tional writing tasks, such as copying the details of a bank check into the appropriate
places and copying an address. These tasks are particularly indicated for the analysis of
AD, as this disease is primarily characterized by cognitive deficits. Note that, in [22],
participants are asked to write their name as additional task.
Finally, in [3] the authors suggest considering also handwritten signature, as a sig-
nature conveys several information about the signer [23].
3 Protocol Proposal
The goal of this paper is to propose a digitized protocol for the early assessment of
neurodegenerative disorders, including handwriting tasks for a subsequent extraction
and analysis of kinematic features of the handwriting process. The protocol is made up
of three parts: an initial screening; a battery of a digitized version of standard cognitive
and functional tests; finally, some of the most relevant handwriting tasks already re-
ported in the literature. The first two parts mostly require the examiner to provide a data
entry; the last one requires the patient to perform the tasks. An overall model of the
protocol is depicted in Fig. 1.
It is worth noting that the proposal is conceived to be deployed on inexpensive and
commercially available digitizing tablets. Typically, these tablets are able to capture
not only temporal and spatial variables of handwriting, but also the pressure exerted
over the writing surface, the azimuth, i.e. the angle of the pen in the horizontal plane,
the altitude, i.e. the angle of the pen with respect to the vertical axis, and so on. More-
over, these devices can capture pen movement not only while the pen is on the writing
surface, but also when the pen is in close proximity of the surface, i.e. in-air.
In the following, the three parts are described.
3.1 Initial Screening
The first part consists in a simple screening routine allowing the examiner to collect
personal and clinical information of the patient. In addition to general questions, the
family history of the patient and the presence of risk factors, such as diabetes, hyper-
tension, obesity, etc., can be evaluated. Caregivers are then asked to answer questions
about the most common symptoms of dementia. The interview is mainly concerned
with memory deficit. If at least one question is answered positively, the protocol con-
tinues; otherwise, it ends. In the former case, the patient is asked to provide his/her
signature as first handwritten trait captured by the device. The signature is acquired
both in this initial stage and at the end of the second stage, as signature may vary de-
pending on the emotional state of the signer.
4
General information
Initial screening
Common no
symptoms of
dementia?
yes
Signature
Mini-Cog
Standard assessment
no
Suspected
dementia?
yes
IADL scale/
MMSE/
Additional tasks
Signature Handwriting tasks
Fig. 1. Overall model of the proposed protocol.
3.2 Standard Assessment
A standard assessment typically includes cognitive and functional tests. The tests suit-
able to be carried out in a digitized way included in our proposal are: Mini-Cog [24];
the Instrumental Activities of Daily Living (IADL) scale [25]; and Mini-Mental State
Examination (MMSE) [26]. These three tests were chosen since they are the most
widely used in clinical settings.
Mini-Cog is a short screening test, as it lasts no more than 3 minutes. The test is
composed of two parts: a 3-item recall test and a simple clock drawing test. The score
is calculated as follows: 1 point for each word recalled without cues, and 2 points for a
normal clock drawing. A total score of 3, 4 or 5 indicates lower likelihood of dementia.
One may proceed with the administration of the following tests only after obtaining a
low Mini-Cog score.
The IADL scale is an instrument to assess independent living skills. More precisely,
it provides information about 8 skills necessary for living in a community, for example
food preparation, housekeeping and laundering. Individuals are scored according to
their highest level of functioning in a specific category. A summary score, therefore,
ranges from 0 (low function, dependent) to 8 (high function, independent).
Then, MMSE may be administered. It consists in a 30-point questionnaire including
questions and problems in many areas: orientation to time and place; repetition of lists
of words; attention and calculation; registration recall; speaking back a phrase; basic
motor skills, such as drawing figures previously shown. Any score greater than or equal
5
to 24 points indicates a normal cognition. Below this, the score can indicate severe (≤9
points), moderate (10-18 points) or mild (19-23 points) cognitive impairment.
In conclusion of the second stage, additional well-known tasks, not included in
standard tests, may be administered, for example matrix and trail making tests. Then, a
second signature of the subject is acquired.
It is worth noting that all these tests already include handwriting tasks, and their
contribution and importance within the tests has been already clinically validated. In
this case, the use of a digital on-line (real-time) acquisition tool gives the possibility to
acquire kinematic features.
Furthermore, it is worth remarking that the inclusion in the proposed protocol of
standardized cognitive tests helps in specifying the stage of disease of the example ac-
quired, resulting in a multiclass classification problem.
3.3 Handwriting Tasks
The tasks described within the previous two paragraphs have three major advantages:
• They are useful for doctors to perform the assessment as they usually do;
• They can provide the so called “ground truth” to be used in the research activities;
• They provide a set of extra data to be combined (for analysis and classification aims)
to those related to handwriting.
Among the various tasks proposed by the literature, the following ones were ob-
served to be the most employed and discriminant:
• Archimedes spiral drawing;
• Copying the details of a check into the appropriate places;
• Repetitions of the cursive letters “l” and “e”;
• Writing a dictated word and sentence. Note that, in order to keep the dictation speed
constant, an audio trace should be preferred to the examiner’s voice. Moreover, note
that, according to [9], words or sentences without linguistic or spelling difficulties
should be chosen.
In addition, following (with the pen) a trajectory drawn on the screen may be of
interest to assess the hesitation of the subject in performing such a task.
Clearly, the handwriting tasks have to be administered not only to impaired patients
but also to healthy controls, in order to provide examples for both kinds of subjects.
A summary of the handwriting tasks included in all the three stages of the protocol
is reported in Table 1.
4 Conclusion
In this paper, an integrated protocol for the assessment of neurodegenerative diseases
has been proposed. It integrates handwriting tasks to a digitized version of standard
cognitive and functional tests.
6
Table 1. Handwriting tasks.
Stage Tasks
Initial screening Signature
Standard assessment Clock drawing test
Spontaneous writing
Copying geometric shapes
Matrix test
Trail making test
Signature
Handwriting Archimedes spiral
Copying the details of a check
Repetition of “l” and “e”
Word/sentence writing
Trajectory following
On one hand, the protocol may be useful, to the research community, to collect dif-
ferent handwritten traits. One main open issue, in fact, concerns the development of a
large benchmark database so that different pattern recognition tools can be effectively
evaluated and compared. At the same time, the protocol may be of real use for doctors
to support their daily activities. The inclusion of standardized tests is also important to
specify the stage of the illness of the patterns acquired: indeed, the main open problem
is to support the early diagnosis of the disease.
Future developments of our research should face some open issues. First, the proto-
col must be implemented and validated in a real-world scenario. The use of a digitizing
tablet may be unfamiliar to elderly people; therefore, a comparative study between this
acquisition modality and the usual one based on pen and paper should be carried out in
order to evaluate if any bias is introduced. Second, other characteristics of the partici-
pants, for example the medical treatment, should be taken into account.
Acknowledgment
This work was supported by the Italian Ministry of Education, University and Research
within the PRIN2015-HAND Project under Grant H96J16000820001.
References
1. Alzheimer’s Association: Alzheimer’s disease facts and figures. Alzheimer’s & dementia:
the journal of the Alzheimer’s Association, 11(3): 332 (2015).
2. Lang, A.E., Lozano, A.M.: Parkinson’s disease. New England Journal of Medicine, 339(15),
pp. 1044-1053 (1998).
3. Pirlo, G., Diaz, M., Ferrer, M.A., Impedovo, D., Occhionero, F., Zurlo, U.: Early diagnosis
of neurodegenerative diseases by handwritten signature analysis. In: International Confer-
ence on Image Analysis and Processing, pp. 290-297 (2015).
7
4. Drotár, P., Mekyska, J., Rektorová, I., Masarová, L., Smékal, Z., Faundez-Zanuy, M.: Eval-
uation of handwriting kinematics and pressure for differential diagnosis of Parkinson’s dis-
ease. Artificial intelligence in medicine, 67, pp. 39-46 (2016).
5. Pereira, C.R., Weber, S.A., Hook, C., Rosa, G.H., Papa, J.P.: Deep Learning-Aided Parkin-
son’s Disease Diagnosis from Handwritten Dynamics. In: 29th SIBGRAPI Conference on
Graphics, Patterns and Images, pp. 340-346 (2016).
6. Tseng, M.H., Cermak, S.A.: The influence of ergonomic factors and perceptual-motor abil-
ities on handwriting performance. American Journal of Occupational Therapy, 47(10), pp.
919-926 (1993).
7. De Stefano, C., Fontanella, F., Impedovo, D., Pirlo, G., Scotto di Freca, A.: Handwriting
analysis and e-Health: a brief overview. In: 18th International Graphonomics Society Con-
ference (2017).
8. Rizzi, M., D’Aloia, M., Guaragnella, C., Castagnolo, B.: Health care improvement: Com-
parative analysis of two CAD systems in mammographic screening. IEEE Transactions on
Systems, Man, and Cybernetics-Part A: Systems and Humans, 42(6), pp. 1385-1395 (2012).
9. Rizzi, M., D’Aloia, M., Cice, G.: Computer Aided Evaluation (CAE) of Morphologic
Changes in Pigmented Skin Lesions. In: International Conference on Image Analysis and
Processing, pp. 250-257 (2015).
10. Neils-Strunjas, J., Groves-Wright, K., Mashima, P., Harnish, S.: Dysgraphia in Alzheimer’s
disease: a review for clinical and research purposes. Journal of Speech, Language, and Hear-
ing research, 49(6), pp. 1313-1330 (2006).
11. Letanneux, A., Danna, J., Velay, J.L., Viallet, F., Pinto, S.: From micrographia to Parkin-
son’s disease dysgraphia. Movement Disorders, 29(12), pp. 1467-1475 (2014).
12. Eichhorn, T.E., Gasser, T., Mai, N., Marquardt, C., Arnold, G., Schwarz, J., Oertel, W.H.:
Computational analysis of open loop handwriting movements in Parkinson’s disease: a rapid
method to detect dopamimetic effects. Movement disorders, 11(3), pp. 289-297 (1996).
13. Schröter, A., Mergl, R., Bürger, K., Hampel, H., Möller, H.J., Hegerl, U.: Kinematic analysis
of handwriting movements in patients with Alzheimer’s disease, mild cognitive impairment,
depression and healthy subjects. Dementia and Geriatric Cognitive Disorders, 15(3), pp.
132-142 (2003).
14. Pullman, S.L.: Spiral analysis: a new technique for measuring tremor with a digitizing tab-
let. Movement Disorders, 13(S3), pp. 85-89 (1998).
15. Poluha, P., Teulings, H.L., Brookshire, R.: Handwriting and speech changes across the levo-
dopa cycle in Parkinson’s disease. Acta psychologica, 100(1), pp. 71-84 (1998).
16. Slavin, M.J., Phillips, J.G., Bradshaw, J.L., Hall, K.A., Presnell, I.: Consistency of hand-
writing movements in dementia of the Alzheimer’s type: A comparison with Huntington’s
and Parkinson’s diseases. Journal of the International Neuropsychological Society, 5(01),
pp. 20-25 (1999).
17. Teulings, H.L., Contreras-Vidal, J.L., Stelmach, G.E., Adler, C.H.: Adaptation of handwrit-
ing size under distorted visual feedback in patients with Parkinson’s disease and elderly and
young controls. J. of Neurology, Neurosurgery & Psychiatry, 72(3), pp. 315-324 (2002).
18. Tucha, O., Mecklinger, L., Thome, J., Reiter, A., Alders, G.L., Sartor, H., Naumann, M.,
Lange, K.W.: Kinematic analysis of dopaminergic effects on skilled handwriting move-
ments in Parkinson’s disease. Journal of neural transmission, 113(5), pp. 609-623 (2006).
19. Impedovo, D., Pirlo, G., Mangini, F.M., Barbuzzi, D., Rollo, A., Balestrucci, A., Impedovo,
D., Sarcinella, L., O’Reilly, C., Plamondon, R.: Writing generation model for health care
neuromuscular system investigation. In: International Meeting on Computational Intelli-
gence Methods for Bioinformatics and Biostatistics, pp. 137-148 (2013).
8
20. Kawa, J., Bednorz, A., Stępień, P., Derejczyk, J., Bugdol, M.: Spatial and Dynamical Hand-
writing Analysis in Mild Cognitive Impairment. Computers in Biology and Medicine
(2017).
21. Werner, P., Rosenblum, S., Bar-On, G., Heinik, J., Korczyn, A.: Handwriting process vari-
ables discriminating mild Alzheimer’s disease and mild cognitive impairment. The J. of Ger-
ontology Series B: Psychological Sciences and Social Sciences, 61(4), pp. 228-236 (2006).
22. Rosenblum, S., Samuel, M., Zlotnik, S., Erikh, I., Schlesinger, I.: Handwriting as an objec-
tive tool for Parkinson’s disease diagnosis. J. of neurology, 260(9), pp. 2357-2361 (2013).
23. Pirlo, G., Cuccovillo, V., Diaz-Cabrera, M., Impedovo, D., Mignone, P.: Multidomain Ver-
ification of Dynamic Signatures Using Local Stability Analysis. IEEE Transactions on Hu-
man-Machine Systems, 45(6), pp. 805-810 (2015).
24. Borson, S., Scanlan, J., Brush, M., Vitaliano, P., Dokmak, A.: The Mini-Cog: a cognitive
‘vital signs’ measure for dementia screening in multi-lingual elderly. International journal
of geriatric psychiatry, 15(11), pp. 1021-1027 (2000).
25. Lawton, M.P., Brody, E.M.: Assessment of older people: self-maintaining and instrumental
activities of daily living. Nursing Research, 19(3): 278 (1970).
26. Folstein, M.F., Robins, L.N., Helzer, J.E.: The mini-mental state examination. Archives of
general psychiatry, 40(7): 812 (1983).