A Brief Overview on Handwriting Analysis for Neurodegenerative Disease Diagnosys C. De Stefano1 , F. Fontanella1 , D. Impedovo2 , G. Pirlo2 and A. Scotto di Freca1 1 Dipartimento di Ingegneria Elettrica e dell’Informazione (DIEI) Università di Cassino e del Lazio meridionale Via G. Di Biasio, 43 02043 Cassino (FR) – Italy {destefano,fontanella,a.scotto}@unicas.it 2 Dipartimento di Informatica Università di Bari (Italy) {donato.impedovo,giuseppe.pirlo}@uniba.it Abstract. Degenerative nerve diseases affect many of your body’s activ- ities, such as balance, movement, talking, breathing, and heart function. These disease cannot be cured, nonetheless an early diagnosis can help to better manage the symptoms and the evolution of these diseases. Since handwriting involves several cognitive abilities, clinicians started to consider handwriting analysis as an effective tool for early diagnoses for this kind of diseases. Moreover, as they show different handwriting impairments as they evolve, handwriting analysis can be also used for monitoring them along the clinical course. This paper provides a brief overview on the use of handwriting anal- ysis for early diagnosis, monitoring and tracking of neurodegenerative diseases. In particular, we taken into account Alzheimer and Parkinson diseases. 1 Introduction Neurodegenerative diseases (NGD) affect the peripheral nervous system which includes muscles, the nerve-muscle junction, nerves in the limbs, and motor nerve cells in the spinal cord. Nerve cells send the messages that control these muscles in order to allow movements, including handwriting. Sick/died neurons cannot properly control muscles. These diseases are incurable, but early screening and identification can reduce the ”diagnostic odyssey”. On the other hand NMD often result in progressive cognitive, functional and behavioural changes. The current clinical diagnostic tools include imaging (e.g. magnetic resonance imaging, or MRI), blood tests, lumbar puncture (spinal tap). Handwriting results from a complex network composed by cognitive, kines- thetic, and perceptual-motor abilities [28]. Furthermore, visual and kinesthetic perception, motor planning, eye-hand coordination, visual-motor integration, dexterity, and manual skills are involved. Significant changes of the handwrit- ing performances are a prominent feature of Alzheimer Disease (AD) as well as D. Impedovo and G. Pirlo (Eds.), Workshop on Artificial Intelligence with Application in Health, Bari, Italy, November 14, 2017. Copyright held by the authors. Parkinson Disease (PD). Learning and performing handwriting requires the in- teractions of multiple brain areas, comprising cerebral cortex, basal ganglia and cerebellum [10]. In seeking to understand all the breadth and facets of motor learning, many researchers have used different approaches, such as neuroimag- ing and intracranial recordings, clinical treatments, and proposed neural schemes aimed at evaluating the viability of theories regarding the way these areas co- operate for motor learning/generation. One of the earliest neural schemes for motor control, which envisages the cooperation among cortex, cerebellum and basal ganglia, was proposed in 1974 [1]. On the other hand, the perspective of signal and image processing must be considered: there are certain aspects of the writing process that are more vulnerable than other and may present diagnostic signs [13]. Dysgraphia has been observed in patients presenting mild to moderate levels AD [14] and PD [16]. The idea of handwriting analysis within this field of application is also en- couraged by the fact that the Minnesota Handwriting Assessment (MHA) is used to identify students (6-8 years old) with difficulties related to autism. The test is also used to evaluate treatment effectiveness over time. It inspects the legibility, handwriting speed, legibility, form, alignment, size and spacing. The MHA is a standard in US, it requires 10 minutes to be performed and it costs less than 130$. This last aspect opens for a real possibility of having, in future, a similar NGD assessment. The paper is organized as follows: Section 2 describes the state of the art for early detection and monitoring of NGD by handwriting analysis; Section 3 illustrates the open issues that still need to be addressed; finally, Section 4 is devoted to the conclusions. 2 State of the Art As mentioned in the Introduction, handwriting involves several cognitive abil- ities. For this reason, handwriting analysis can be used as an effective tool for early diagnoses for NGD [14, 23]. Moreover, since this kind of diseases show dif- ferent handwriting impairments as they evolve, handwriting analysis can be also used for monitoring them along the clinical course [8, 25, 33]. Studies involving neural recordings have provided a large body of knowledge about the neural pro- cesses occurring in the brain areas related to motor learning. First studies on the brain areas governing handwriting observed that it implies the learning of motor sequences by two distinct neural systems, comprising cortex-basal ganglia and cortex-cerebellum loop circuits [12, 6]. A neural model of cortico-cerebellar in- teractions during attentive imitation and predictive learning of sequential hand- writing movements, suggests how cortical mechanisms interact with predictive cerebellar learning during movement imitation [5]. Recently a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation- based learning model of the cerebellum was proposed suggesting that basal gan- glia and cerebellar learning systems work in parallel and interact with each other. However, these works did not provide computational models to test the validity of the neural schemes or comprised only basal ganglia [6] or cerebellum [5], or were built with a simplified level of biological abstraction [2]. To develop effective and efficient systems for the early detection and monitoring of NGD by Hand- writing analysis, defining effective features plays a key role. For this reason, new methodologies of features extraction and classification have been proposed, tak- ing into account both image processing techniques and writing generation model techniques. In particular, it has been observed that the use of Sigma-Lognormal [15] and Delta-Log [18] models can be adopted to generate features representing the strokes [9]. These models have been developed from the kinematics theory of rapid human movements [19, 20] and in[9] the authors presented a system for handwriting analysis to investigate insurgence and monitoring of the Alzheimer’s disease. Significant handwriting difficulties were already reported by Alois Alzheimer when describing the first patient with Alzheimer’s Disease (AD) in 1907. He observed that the patient reduplicated the same syllable and forgot some others. The evolution of agraphic impairments in AD was described in [21] and included lexicosemantic disturbances at the beginning of the disease, with impairments becoming more and more phonological as the dementia becomes more severe. More recently, several studies analyzed the dynamic of the handwriting process in order to detect and monitor AD [31, 9, 24, 17, 32]. In [31] the authors performs kinematic measures of the handwriting process of persons with mild cognitive im- pairment (MCI) compared with those with mild Alzheimers disease and healthy controls; the aim was to assess the importance of measures for the differentiation of the groups and to assess the characteristics of the handwriting process across different functional tasks. Impedovo D et al. [9] use the Delta-Log and Sigma-Log models mentioned above to investigate on the handwriting generation processes and present a computational system to investigate insurgence and monitoring of AD. In [24] the authors analyze handwriting kinematic to quantify differences in fine hand motor function in patients with probable AD and mild cognitive impairment compared to depressed patients and healthy controls. The authors found that both patients with MCI and patients with probable AD exhibited loss of fine motor performance and the movements of AD patients were significantly less regular than those of healthy controls. Recently, also handwritten signa- tures have been investigated for early diagnosis of NGD [17]. Pirlo et al. used the sigma-lognormal model for the signature representation, then they analyzed the health condition of the signer in terms of Alzheimer disease. The proposed approach has shown to be cheap and effective. In the study presented in [32], the patients performed four types of handwriting movements on a digitizer. Move- ment time and smoothness were analyzed between the groups of patients take into account (probable AD, MCI and normal controls) and across the movement patterns. Kinematic profiles were also compared among the groups. AD and MCI patients demonstrated slower, less smooth, less coordinated, and less consistent handwriting movements than their healthy counterparts. Parkinson’s disease (PD) is a long-term degenerative disorder of the central nervous system that mainly affects the motor system. Even if, to date, clinical assessment remains the gold standard in the diagnosis of Parkinsons disease, many studies have been proposed that use handwriting for detecting and moni- toring PD, since abnormal handwriting is a well recognized manifestation of PD, with micrographia being characteristic [7]. Handwriting anomalies may appear years at the early stages of the disease and thus may be one of the first signs of impending PD. Previous research has shown that handwriting measures have the potential for identifying various stages of PD, effects of varied interventions [4] and the effect of medication [22]. Moreover, studies focusing on understanding the mechanism underlying micrographia found significant differences between the handwriting of PD patients and healthy subjects[27, 29]. More recently, fur- ther studies have been conducted to analyze the handwriting of patients affected by PD [23, 11, 26, 3]. In [23] the authors try to identify simple characteristics of handwriting which could accurately differentiate PD patients from healthy con- trols. Patients were asked to write their name and to copy an address on a paper affixed to a digitizer. Mean pressure and mean velocity was measured for the entire task and the spatial and temporal characteristics were measured for each stroke. The experimental results confirmed that these routine writing tasks can be used to differentiate PD patients from healthy controls. Letanneux et al. [11] identified several studies that investigated handwriting in PD, either with conventional pencil-and-paper measures or with graphic tablets, and re- ported their findings on key spatiotemporal and kinematic variables. They found that kinematic variables (velocity, fluency) differentiate better between control participants and PD patients, and between off- and on-treatment PD patients, than the traditional measure of static writing size. Moreover, since handwriting deficit for PD patients is not restricted to micrographia, they propose the term ”PD dysgraphia”, which encompasses all deficits characteristic of Parkinsonian handwriting. In [26] in order to assess whether standardized handwriting can provide quantitative measures to distinguish PD patients from healthy controls, the authors recorded pen tip trajectories during circle, spiral and line drawing and repeated character ’elelelel’ and sentence writing. The experimental results show that these tasks can provide objective measures for bradykinesia, tremor and micrographia to distinguish Parkinson patients from healthy controls. Fi- nally, Drotár et al. in [3], present a novel PD handwriting database consisting of handwriting samples from (PD) patients and healthy controls. Each sample contains kinematic and pressure data of height handwriting tasks. The tasks include drawing an Archimedean spiral, repetitively writing orthographically simple syllables and words, and writing of a sentence. To discriminate between PD patients and healthy subjects, the authors use three well known and widely used classifiers: K-nearest neighbors, ensemble AdaBoost classifier, and support vector machines (SVM), which was the best performing one. 3 Open Issues Although some research has been already carried out and some encouraging result has been observed, there are still many open issues that must be addressed. First of all there is the lack of a well designed dataset [30]. This involves many different aspects: – Cardinality of the set: in fact even considering papers dealing with PD, most of them make use of datasets composed by very few subjects. More recently some effort has been done in order to get an acceptable dimension (55 individual) [16]. – Acquisition tool and protocol: in many cases off-line acquisition has been performed due to the availability of handwritten document, however it must be considered that on-line acquisition is able to provide a wide set of useful dynamics. On the other hand, the choice of the acquisition tool also affect the amount of dynamics that can be taken into account (e.g. pen-based camera, pad, frontal video, etc.). – Cognitive model: as already mentioned, neurodegenerative diseases do not involve only functional and behavioural changes (can be encountered within the handwriting), but also result in progressive cognitive decay. The acqui- sition protocol should take into account, to some extent, also this aspect in order to be able to convey as much information as possible. – Number and periodicity of sessions: in order to be able to identify disease at different stages, a set of different users is needed. At the same time in order to have the possibility to understand the evolution of the disease over time, the same patient must be enrolled into the system in a periodic way, or when some specific event is occurred. The second issue is that to face the classification problem. Often standard Signal Processing and Pattern Recognition techniques are applied with very few cases of specialization to the field. The main problem is that the medical knowl- edge of the evolution of the disease cannot be ignored: an automatic system able to distinguish an healthy person and a late-stage sick one has a very re- duced usefulness in real word. From this perspective the challenge is to identify patients at different stages, tracking the evolution and to understand/identify signs of worsening. It must be underlined that today there is no cure for AD but it can only be somehow managed, so that an early diagnosis and follow up may have profound implications for carers and doctors. Research on handwriting and neuro-muscular diseases is not expected to replace standard techniques, but to strengthen them by allowing an earlier diagnosis. To this aim Pattern Recogni- tion approaches should be specifically studied and coupled with Cognitive and neuro-muscular generation models. 4 Conclusions In this paper we propose a brief overview of the handwriting analysis approaches for early diagnosis, monitoring and tracking of neurodegenerative diseases. In particular we taken into account Alzheimer and Parkinson diseases. Furthermore, we also discuss the still open issues in the field that must be addressed. Handwriting analysis is an effective tool for dealing with the diagnosis and monitoring of the above cited diseases, nonetheless some issues are open, and are mainly related to: (i) because of their cardinalities, most of the datasets currently available does not allow pattern recognition tools to be effective; (ii) since hand- writing kinematics has shown to be useful for discriminating between patients and healthy controls, new protocols for the on-line acquisition of handwriting should be defined; (iii) defining pattern recognition tools specifically devised for the automatic diagnosis and monitoring of neurodegenerative diseases. Acknowledgment This work is supported by the Italian Ministry of Education, University and Research (MIUR) within the PRIN2015-HAND project. References 1. Allen, G.I., Tsukahara, N.: Cerebrocerebellar communication systems. Physiologi- cal Reviews 54(4), 957–1006 (1974) 2. Dasgupta, S., Wrgtter, F., Manoonpong, P.: Neuromodulatory adaptive combi- nation of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control. Frontiers in Neural Circuits 8, 126 (2014) 3. Drotár, P., Mekyska, J., Rektorová, I., Masarová, L., Smékal, Z., Faúndez-Zanuy, M.: Evaluation of handwriting kinematics and pressure for differential diagnosis of parkinson’s disease. Artificial Intelligence in Medicine 67, 39–46 (2016) 4. Eichhorn, T.E., Gasser, T., Mai, N., Marquardt, C., Arnold, G., Schwarz, J., Oertel, W.H.: Computational analysis of open loop handwriting movements in parkinson’s disease: A rapid method to detect dopamimetic effects. Movement Disorders 11(3), 289–297 (1996) 5. Grossberg, S., Paine, R.: A neural model of cortico-cerebellar interactions during attentive imitation and predictive learning of sequential handwriting movements. Neural Netw. 13(8-9), 999–1046 (2000) 6. Hikosaka, O., Nakahara, H., Rand, M., Sakai, K., Lu, X., Nakamura, K., Miyachi, S., Doya, K.: Parallel neural networks for learning sequential procedures. Trends in Neurosciences 22(10), 464–471 (1999) 7. Horowski, R., Horowski, L., Vogel, S., Poewe, W., Kielhorn, F.: An essay on wilhelm von humboldt and the shaking palsy: first comprehensive description of parkinson’s disease by a patient. Neurology 45(3), 565–568 (1995) 8. Hughes, J.C., Graham, N., Patterson, K., Hodges, J.R.: Dysgraphia in mild de- mentia of alzheimer’s type. Neuropsychologia 35(4), 533 – 545 (1997) 9. Impedovo, D., Pirlo, G., Mangini, F.M., Barbuzzi, D., Rollo, A., Balestrucci, A., Impedovo, S., Sarcinella, L., O’Reilly, C., Plamondon, R.: Writing Generation Model for Health Care Neuromuscular System Investigation, pp. 137–148. Springer International Publishing (2014) 10. Kandel, E.R., Schwartz, J.H., Jessell, T.M.: Principles of Neural Science. McGraw- Hill Medical, 4th edn. (Jul 2000) 11. Letanneux, A., Danna, J., Velay, J.L., Viallet, F., Pinto, S.: From micrographia to parkinson’s disease dysgraphia. Movement Disorders 29(12), 1467–1475 (2014) 12. Nakahara, H., Doya, K., Hikosaka, O.: Parallel cortico-basal ganglia mechanisms for acquisition and execution of visuomotor sequences - a computational approach. J Cogn Neurosci. 13(5), 626–647 (2001) 13. Neils-Strunjas, J., Groves-Wright, K., Mashima, P., Harnish, S.: Dysgraphia in Alzheimer’s disease: a review for clinical and research purposes. J Speech Lang Hear Res 49(6), 1313–30 (2006) 14. Onofri, E., Mercuri, M., Salesi, M., Ricciardi, M., Archer, T.: Dysgraphia in re- lation to cognitive performance in patients with Alzheimer’s disease. Journal of Intellectual Disability - Diagnosis and Treatment 1, 113–124 (2013) 15. OReilly, C., Plamondon, R.: Development of a sigmalognormal representation for on-line signatures. Pattern Recognition 42(12), 3324 – 3337 (2009) 16. Pereira, C.R., Pereira, D.R., da Silva, F.A., Hook, C., Weber, S.A.T., Pereira, L.A.M., Papa, J.P.: A step towards the automated diagnosis of parkinson’s disease: Analyzing handwriting movements. In: 28th IEEE International Symposium on Computer-Based Medical Systems, CBMS 2015, Sao Carlos, Brazil, June 22-25, 2015. pp. 171–176 (2015) 17. Pirlo, G., Cabrera, M.D., Ferrer-Ballester, M.A., Impedovo, D., Occhionero, F., Zurlo, U.: Early diagnosis of neurodegenerative diseases by handwritten signature analysis. In: ICIAP Workshops. pp. 290–297 (2015) 18. Plamondon, R.: Handwriting generation: the delta-lognormal theory. In: Proc. Of the 4th International Workshop on Frontiers in Handwriting Recognition. pp. 1–10 (1994) 19. Plamondon, R.: A kinematic theory of rapid human movements. i. movement rep- resentation and generation. Biol. Cybern. 72(4), 295–307 (1995) 20. Plamondon, R.: A kinematic theory of rapid human movements. ii. movement time and control. Biol. Cybern. 72(4), 295307 (1995) 21. Platel, H., Lambert, J., Eustache, F., Cadet, B., Dary, M., Viader, F., Lechevalier, B.: Characteristics and evolution of writing impairment in alzheimer’s disease. Neuropsychologia 31(11), 1147–58 (1993) 22. Poluha, P., Teulings, H., Brookshire, R.: Handwriting and speech changes across the levodopa cycle in parkinsons disease. Acta Psychologica 100(1), 71 – 84 (1998) 23. Rosenblum, S., Samuel, M., Zlotnik, S., Erikh, I., Schlesinger, I.: Handwriting as an objective tool for parkinsons disease diagnosis. Journal of Neurology 260, 2357– 2361 (2013) 24. Schröter, A., Mergl, R., Bürger, K., Hampel, H., Möller, H.J., Hegerl, U.: Kine- matic analysis of handwriting movements in patients with alzheimer’s disease, mild cognitive impairment, depression and healthy subjects. Dementia and geriatric cog- nitive disorders 15(3), 132–42 (2003) 25. Small, J., Sandhu, N.: Episodic and semantic memory influences on picture naming in alzheimer’s disease. Brain Lang. 104(1), 1–9 (1997) 26. Smits, E.J., Tolonen, A.J., Cluitmans, L., van Gils, M., Conway, B.A., Zietsma, R.C., Leenders, K.L., Maurits, N.M.: Standardized handwriting to assess bradyki- nesia, micrographia and tremor in parkinson’s disease. PLOS One 9(5) (May 2014) 27. Teulings, H.L., Contreras-Vidal, J.L., Stelmach, G.E., Adler, C.H.: Adaptation of handwriting size under distorted visual feedback in patients with parkinson’s disease and elderly and young controls. Journal of Neurology, Neurosurgery & Psychiatry 72(3), 315–324 (2002) 28. Tseng, M.H., Cermak, S.A.: The influence of ergonomic factors and perceptual– motor abilities on handwriting performance. American Journal of Occupational Therapy 47(10), 919–926 (1993) 29. Tucha, O., Mecklinger, L., Thome, J., Reiter, A., Alders, G., Sartor, H., Naumann, M., Lange, K.: Kinematic analysis of dopaminergic effects on skilled handwriting movements in parkinson’s disease. Journal of Neural Transmission 113(5), 609–623 (2006) 30. Wan, J., Byrne, C.A., OGrady, M.J., OHare, G.M.P.: Managing wandering risk in people with dementia. IEEE Transactions on Human-Machine Systems 45(6), 819–823 (Dec 2015) 31. Werner, P., Rosenblum, S., Bar-On, G., Heinik, J., Korczyn, A.: Handwriting process variables discriminating mild alzheimer’s disease and mild cognitive im- pairment. Journal of Gerontology: PSYCHOLOGICAL SCIENCES 61(4), 228–36 (2006) 32. Yan, J.H., Rountree, S., Massman, P., Doody, R.S., Li, H.: Alzheimers disease and mild cognitive impairment deteriorate fine movement control. Journal of Psychi- atric Research 42(14), 1203–1212 (2008) 33. Yasuda, K., Nakamura, T., Beckman, B.: Brain processing of proper names. Apha- siology 14(11), 1067–1089 (2000)