=Paper=
{{Paper
|id=Vol-3309/paper4
|storemode=property
|title=Analysis technology of neurological movements considering cognitive feedback influences of cerebral cortex signals
|pdfUrl=https://ceur-ws.org/Vol-3309/paper4.pdf
|volume=Vol-3309
|authors=Mykhaylo Petryk,Mykhaylo Bachynskyi,Vitaly Brevus,Ivan Mudryk,Dmytro Mykhalyk
|dblpUrl=https://dblp.org/rec/conf/ittap/PetrykBBMM22
}}
==Analysis technology of neurological movements considering cognitive feedback influences of cerebral cortex signals==
https://ceur-ws.org/Vol-3309/paper4.pdf
Analysis technology of neurological movements considering
cognitive feedback influences of cerebral cortex signals
1
Mykhaylo Petryka, Mykhaylo Bachynskyia , Vitaly Brevusa , Ivan Mudryka , and Dmytro
Mykhalyka
a
Ternopil Ivan Puluj National Technical University, 56 Ruska str., Ternopil 46001, Ukraine
Abstract
A high-performance information technology for evaluating human neurological movements
has been developed based on a hybrid model of wave signal analysis, considering the cognitive
feedback effects of cerebral cortex neuro-nodes.
With the use of hybrid Fourier transforms, a high-speed analytical solution of the model in
vector form was implemented, which allows determining the elements of movements on each
segment of a complex spiral trajectory performed by the patient with an electronic pen on a
digital tablet, and identified the parameters of the studied neuro-systems with feedback.
Keywords
Neurological movements, cognitive neuro-feedback signals, abnormal tremor, numerical
diagnostics, hybrid Fourier transform, hardware and software
1. Introduction
New modeling methods are used to provide an approach to the design of digital diagnostic health
systems for patients with neurological diseases. The creation of new software and hardware solutions
for medicine and automated diagnostic systems to identify new phenomena of the body and human
health is an urgent task.
The latest information technologies and modeling methods in the design of a computer diagnostic
system improve the solution to the problem of treating critical diseases in the world, especially people
affected by neurological diseases, such as abnormal neurological movements (ANM) or tremors and
their extreme forms in the form of Alzheimer's, Parkinson's diseases [1]. ANM - unwanted oscillating
movements of certain part of the body (hands, organs of speech, eyeballs), arising as a result of
involuntary contraction of human muscles [2]. Movement regulation disorders signs of human
movements are an increase in their amplitude, a change in the frequency and form of oscillations. The
analysis of these ANM parameters is crucial for understanding the role of feedback dysfunction in
cerebral cortex (CC) neural nodes in cognitive control processes of human movements and early
detection of neuromotor disorders. The difficulty of identifying ANM lies in the imperfection of
existing diagnosis methods, their low accuracy, and the lack of mathematical and software tools for
identifying the neurofeedback influences of CC nodes on their behavior [2].
Studies of neuro-systems related to the analysis of the behavior of patients with tremor symptoms
(T-objects) were conducted by a number of researchers, such as Pullman S. L., Legrand A.-P., Vidailhet
M. (ESPCI Paris Tech, ICEM CNRS), Wang J.-S., Louis E., Haubenberger D., Kalowitz D. and others.
Here, the main attention was paid to the analysis of parameters of relatively normal conditions and
behavior of patients using classical methods of digital processing based on the Fourier transform [2-4].
However, such methods have already exhausted themselves to date and do not allow the analysis of
1
ITTAP’2022: 2nd International Workshop on Information Technologies: Theoretical and Applied Problems, November 22–24,
2022, Ternopil, Ukraine
EMAIL: mykhaylo_petryk@tu.edu.te.ua (A. 1); m.bachynskyi@gmail.com (A. 2); v_brevus@tntu.edu.ua (A. 3); i1mudryk@ukr.net (A. 4);
dmykhalyk@gmail.com (A. 5)
ORCID: 0000-0001-6612-7213 (A. 1); 0000-0003-4139-7633 (A. 2); 0000-0002-7055-9905 (A. 3); 0000-0002-4305-1911 (A. 4); 0000-0001-
9032-695X (A. 5).
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
abnormal states with complex, hard-to-predict behavior, which is inherent in real T-objects with a high
degree of tremor. Due to the imperfection of such methods, there is loss of 60-80% of important
information from the description of patients' real conditions, which de facto determines the low quality
level of such analysis.
The authors proposed a high-performance information technology for ANM research, built on the
basis of a hybrid model analysis of neuro-system's signals, which describes the state and behavior of
the 3D elements of ANM trajectories of the T-object, considering the cognitive neuro-feedback effects
of the identified CC nodes. Using the methods of hybrid Fourier transform, high-speed analytical
solutions of the model were built in the form of vector functions that determine the elements of the
trajectories on each ANM segment. On their basis, high-performance algorithms for the identification
of tremor-movements parameters are proposed for the component-wise evaluation of neuro-feedback
effects, which allow parallelization of calculations.
2. A brief description of the hardware used in the research
2.1.The method of collecting movement data using a graphic tablet
In addition to qualitative characterization, quantitative improvement of this test requires knowledge
of the position and pressure as a function of the time the patient uses the handle throughout the
experiment. This requires the use of a graphics tablet with specially adapted software. Basically, this
common tool is usually used for artistic drawing, which is provided by specialized software. The goal
here is not the same as above, we need to collect the X and Y coordinates as a function of time, the
frequency and accuracy of which depends on the manufacturer.
Figure 1: A visual representation of how to use a Wacom Bamboo graphics tablet
For these reasons, the WACOM Bamboo Fun Medium graphics tablet was chosen. Its active
area (corresponding to 217mm x 137mm) is compatible with the generally accepted patterns of Fahn-
Tolosa-Marin Tremor Rating Scale (FTRS). The reader has a pen input resolution of 2540 dpi, an
accuracy of 0.25 mm and a recognition speed of 133 points per second (according to the manufacturer's
specifications). In addition, this tablet allows you to display pen pressure while drawing on a plane and
measure movements at a distance of up to 16 mm above the surface of the pen, which allows you to
visualize the patient's movements in space.
2.2. NeuroCom computer electroencephalography system produced by KHAI-
MEDYKA
The NEUROKOM computer electroencephalograph, the fifth generation of developed computer
electroencephalography complexes [6], was chosen for the study of electroencephalography (EEG)
signals of the brain. The encephalograph in the complex is designed for registration, in-depth analysis
and interpretation of EEG and evoked potentials, conducting various analyzes for scientific research.
Figure 2: Visual representation of the data collection process from the NeuroCom
electroencephalography complex
Figure 3: Demonstration of using research hardware
The helmet with the installed hardware and software platform of the manufacturer provides 16-
channel selection of encephalograms and their transmission to a personal computer using the
appropriate protocol. Software conditioning of EEG signals and post-processing takes place on a PC.
Data is stored both in raw text and in a visualized representation at each point in time. Data is read from
each lead channel at 2ms intervals (500 Hz frequency).
3. A hybrid mathematical model of ANM-analysis considering the cognitive
neuro-feedback-influences of СС nodes
The hybrid model analysis of ANM built on the basis of the concept of propagation of a wave signal
determines the segment-by-segment description of the elements of ANM trajectories (hands of the
patient) considering the matrix of cognitive effects of groups of neuro-nodes of the CC on the movement
segments [5]. The implementation is based on the method of determining the position of the patient's
hand with an electronic pen, which reproduces the trajectory of the template (Archimedes' spiral) on
the screen of the interactive tablet [7, 8]. The deviation of the trajectory of the pen movement from the
template has a complex shape (Figure 4) and provides digitized information for determining the patient's
neurological condition. The pen movement trace is broken down into simpler elements for the purpose
of decomposing complex ANM-movements within the schematization of the model. The number of
divisions depending on the complexity of the ANM image can be chosen arbitrarily. The hybrid model
provides quantitative amplitude and frequency characteristics of ANM.
For considering the cognitive neurofeedback effects of the system, the obtained digital sets of
indicators of EEG signals are used, which synchronously with the movement of the electronic pen in
the patient's limb, come from a certain set of CC neuro-nodes. EEG signals in general determine the
dynamics of ANM for each 𝑗 -th segment of the trace, 𝑗 = 1, 𝑛1 + 1 where 𝑛1 is the number of points
of breakdown of the ANM track (Figure 4). In the model, the partition can be set automatically in an
arbitrary way, with any finite number of segments. Their lengths can be different depending on the level
of detail of the traffic sections.
Figure 4: Schematization and visualization of the influence of vector of connections components of
the of cognitive feedback-influences of EEG-signals of neuro-nodes 𝑆(𝑡) on individual elements of the
ANM–trace (𝑙𝑗−1 , 𝑙𝑗 ), 𝑗 = 1, 𝑛1 within the framework of the hybrid model of analysis
Model description. In order to present a mathematical solution in the form of implementing the
procedure for the functional identification of the amplitude components and the phase velocity of the
ANM wave propagation 𝑏𝑘2 , 𝑘 = 1, 𝑛1 + 1 as functions of time in the framework of the model
decomposition, taking into account the conditions that the traces (observation data in the form of digital
data of ANM-movements of the patient) of the solution for each 𝑘-th segment are known, 𝑘 =
1, 𝑛1 + 1, a system of initial-boundary value problems (micromodels) is obtained for successive
segments of ANM [6, 9]:
𝜕2 2
𝜕2 (1)
𝑢 (𝑡, 𝑧) = 𝑏 𝑢 + 𝑆𝑘∗ (𝑡, 𝑧),
𝜕𝑡 2 𝑘 𝑘
𝜕𝑧 2 𝑘
with initial conditions:
𝜕𝑢𝑘 (2)
𝑢𝑘 (𝑡, 𝑧)|𝑡=0 = 0, | = 0, 𝑘 = 1, 𝑛1 + 1,
𝜕𝑡 𝑡=0
Boundary conditions on each of the ANM segments along z:
𝑢𝑘−1 (𝑡, 𝑧)|𝑧=𝑙𝑘−1 = 𝑈𝐿𝑙 , 𝑢𝑘 (𝑡, 𝑧)|𝑧=𝑙𝑘 = 𝑈𝑙𝑘 , 𝑘 = 1, 𝑛1 + 1, (3)
𝑘−1
Selection of the residual functional. We assume that the components of the phase velocity of the
ANM wave propagation 𝑏, 𝑘 = 1, 𝑛1 + 1 of the boundary value problem (1) - (3) are unknown
functions of time. With known values of the position of the pen 𝑢𝑘 (𝑡, 𝑧) at observation points on ANM-
segments𝛾𝑘 ⊂ 𝛺𝑘 , 𝑘 = 1, 𝑛1 + 1
𝑢𝑘(𝑡, 𝑧)|𝛾𝑘 = 𝑈𝑙𝑘 (𝑡, 𝑧)| , (4)
𝛾𝑘
problem (1) - (4) can be considered for each point z for each 𝑘1 -th segment of the ANM-trajectory
and will consist in finding the functions 𝑏𝑘 ∈ 𝐷, where
𝐷 = {𝜈(𝑡, 𝑧): 𝜈|𝛺𝑘 ∈ 𝐶 (𝛺𝑘1 𝑇 ) , 𝜈 > 0, 𝑘 = 1, 𝑛1 + 1}.
1𝑇
Functional-incoherence𝛾𝑘1 ∈ 𝛺𝑘1 , according to [6, 10] will be written in the form
1 𝑇 (5)
𝐽𝑘 (𝑏𝑘 𝑘 ) = ∫ (‖𝑢𝑘 (𝑡, 𝑧, 𝑏𝑘 ) − 𝑈𝑘∗ ‖2 ) 𝑑𝑡,
2 0
The construction and mathematical substantiation of the solution of the hybrid model is included in
[9].
4. Modeling and identification of parameters of neurological movements
signals of a person under the influence of cognitive neuro-feedback effects
of cerebral cortex neuro-nodes
To configure the model of identification, fragments of the ANM-trajectory of the spiral type, which
was performed by the patient on a digital tablet (Figure 1) were used.
Figure 5: Spiral-type ANM trace created by the patient on an interactive tablet
Modeling and identification of the parameters of ANM movements was carried out within the
framework of the task of identifying parameters of cognitive neuro-feedback effects of EEG on ANM
trajectories developed using the ANM hybrid model considering the feedback effects of EEG-signals.
To set up the identification model, we used a fragment of the ANM-trace, made by the patient using an
electronic pen on an interactive digital tablet according to Figure 5.
The corrected fragment of the trace of this spiral example of the test pattern (Archimedes spiral) by
the patient with an electronic pen on a digital tablet in the number of discretized 2400 points - positions
of the track is presented in Figure 6. Here, the abscissa k is the number of positions of the deviation of
the pen from equilibrium during the passage of the spiral sample.
Figure 6: Straightened fragment of the electronic trace of the spiral circuit with an electronic pen on
the tablet. Number of points 𝑛 = 2422
The analysis used a test set of EEG signals of the Fp1- tap sensors of NeuroCom (according to the
CC node classification map), which was measured synchronously with the movement of the electronic
pen when the patient traced the test pattern (Figure 7).
Figure 7: Test set of EEG-signals of the Fp1- tap sensors of the CC of the patient when tracing the test
pattern
Setting up the model ANM track and their step-by-step and segment-by-segment identification
(amplitude and frequency parameters for each segment taking into account the integrity of the system)
to a specific sample of the trace performed by the patient (observation curve or experimental curve)
was performed according to the feedback scheme and the analytical solution of the hybrid ANM model
tracks (Figure 8).
The degree of tremor in this part of the recording ETG (essential tremors graphic), which is based
on the matrix of interactions of brain signals (EEG) 𝑆.
Опис АНТР – моделі: Пряма задача
Математичний опис елементів траєкторії АНТР на множині I n1 у:
lk ln +1
u1 ( t , z ) 11 ( t − , z, ) ... 1,n1 +1 ( t − , z, ) S ( , )
l0 1
ln
...
... ... ...
...
u j (t, z ) = ( ) d d
t lk
j ,n+1( t − , z, ) Si ,
ln +1
0 l ( t − , z, ) ...
j1
ln
...
0
... ... ... ...
u n+1 ( t , z )
lk
( t − , z, ) ... n1 +11,n1 +1 ( t − , z , ) n
ln +1
(
S , )
n +1,1
l0 ln
ETG EEG
feedback
Figure 8: Model of the ANM track with feedback
The results of simulation of ANM of the limb with an electronic pen of the patient, their verification
Михайло ПЕТРИК (ТНТУ) кафедра програмної інженерії 121 – Software Engineering 8/365
according to the data of the neuro-experiment according to Figure 4, which were accompanied by the
synchronous cognitive neuro-feedback effect of the Fp1 neuro-node (a set of EEG signals according to
Figure 7) and the use of a hybrid model of ANM analysis (based on the hybrid Fourier transform) are
presented in Figure 9-Figure 13. At first, we took a relatively small number of points in an effort to
reproduce the profile of the observation curve (the profile of the ANM-track performed by the patient
taking into account the picture of the EEG feedback curve (Figure 7).
Figure 9: Comparative analysis of the ANM-model track (red solid line) and the real patient trace (blue
square markers) for the first 60 points (the degree of model verification is more than sufficient)
As can be seen from Figure 9, the accuracy of matching the model track and the patient's real trace
is very high (up to 1.5-2%) for 60 observation points. Amplitude and frequency characteristics due to
the hybrid spectral function built by us, built systematically for all segments of the breakdown (taking
into account their connectivity, not each separately), made it possible to obtain an almost complete
coincidence of the model track with the real patient trace. Then we gradually increased the number of
points on the track. For the number of points 200, the results turned out to be practically the same
(Figure 10).
Figure 10: Comparative analysis of the ANM model track (red solid line) and the real patient trace
(blue dot markers) for the first 200 points (a deeper level of model verification on real data)
In the future, we again gradually increased the number of points to 600, 1200, 2400 and 4 and studied
the behavior of the model curves, evaluating their possible deviations from the patient's experimental
ANM-traces.
Figure 11: Analysis of the ANM model track (red solid line) and the patient's real trace (blue dot
markers) for the first 600 points (model verification by track curvature elements)
In Figure 11, for a segment with 600 points, we observe a slight deviation in the saddle zone around
the 60th point of the track, about 3-5%. However, this problem can be technically solved by making the
track segmentation in this area smaller. By the way, the model itself allows for arbitrary partitioning
with arbitrary sizes of each segment and making them as small as necessary.
Figure 12: Analysis of the ANM model track (red solid line) - and the patient's real trace (blue dot
markers) for the first 1200 points (broader verification of the model by elements, which includes
several sinusoidal inhomogeneous sections of the wave motion)
Figure 13: Analysis of the ANM model track (red solid line) - and the patient's real trace (blue dot
markers) for the first 2400 points (model verification in a wide range of the track with complex sections
of wave motion)
A positive point is that on all graphs we see a complete reproduction of the frequency characteristics
of the track (the periodicity of the model curves almost completely corresponds to the periodicity of the
curves performed by the patient). As the number of examined points in individual saddle or ridge points
increases, it decreases. But this can be corrected by choosing a smaller division in these zones.
As can be seen from the presented graphs, the developed model reproduces the patient's behavior at
a high level, displaying an ANM track that practically coincides with the one drawn by him on the
tablet. The most important thing is that the model includes the possibility of displaying the mechanisms
of its feedback effects of CC signal in the form of a matrix of EEG signals that determine the behavior
of these movements. Further research may include a change in this behavior, apparently for the better,
depending on the change in the magnitude of these EEG feedback effects after certain therapeutic
procedures and expand the limits of application.
5. Conclusions
Information technology for evaluating the neurological movements of a person is proposed, the one
is based on a hybrid model of wave signal analysis, considering the reverse cognitive effects of cerebral
cortex neuro-nodes. Using hybrid Fourier transform, a high-speed analytical solution of the model in
vector form was obtained and implemented. It determines the elements of trajectories on each segment
of a complex spiral-drawing performed by the patient with an electronic pen on a digital tablet with the
identification of the parameters of the researched feedback systems.
The proposed hybrid model provides deep decomposition of the system without affecting its
integrity and connections, which is not possible for by classical methods of signal processing, that lead
to the loss of 60-80% of information about the real condition of the object. Weighting coefficients which
characterize the influence of the digital recordings sets of the cognitive influence signals produced by
neuro-nodes of the patient's cerebral cortex are pre-specified by machine learning methods during of
the patient-executed test examples of drawing spiral trajectories on a Wacom digital tablet.
Compared to the paper method of drawing a spiral, the following indicators were obtained:
• reduction of test duration by 2.5-3 times (about 1 minute instead of 2.5-3 minutes of manual input);
• efficiency and accuracy of the assessment - a significant increase in productivity due to the high
speed of data analysis, the quality of the process of obtaining input data of the 3D drawing of the
spiral, the unambiguous interpretation of the results.
This approach makes it possible to more qualitatively describe the complex mechanisms of human
neurological conditions under the influence of cognitive connections of the nervous system, which
determine abnormal behavior deviations caused by various man-made and other factors providing a
high degree of data completeness, clarifying diagnoses for treatment.
6. References
[1] A.N. Khimich, M.R. Petryk, D.N. Mykhalyk, I.V. Boyko, A.V. Popov, V.A. Sydoruk, Methods
for mathematical modeling and identification of complex processes and systems based on
visoproductive computing (neuro- and nanoporous cyber-physical systems with feedback,
models with sparse structure data, parallel computing). Monograph, Kiev: National Academy
of Sciences of Ukraine. Glushkov Institute of Cybernetics. 2019. - 176 p. ISBN: 978-966-02-
9188-1.
[2] R. Bhidayasiri, Z. Mari, Digital phenotyping in Parkinson's disease: Empowering neurologists
for measurement-based care. Parkinsonism Relat Disord. 2020 Nov; 80:35-40. DOI:
10.1016/j.parkreldis.2020.08.038.
[3] D. Haubenberger, D. Kalowitz, F.B. Nahab, C. Toro, D. Ippolito, D.A. Luckenbaugh, L.
Wittevrongel, M. Hallett, Validation of Digital Spiral Analysis as Outcome Parameter for
Clinical Trials in Essential Tremor. Movement Disorders 26 (11), 2073-2080, (2011)
[4] V. Rajaraman, D. Jack, S.V. Adamovich, W. Hening, J. Sage, H. Poizner, A novel quantitative
method for 3D measurement of Parkinsonian tremor. Clinical neurophysiology, 11(2), 187-369
(2000)
[5] H. Xie, Z. Wang, Mean frequency derived via Huang-Hilbert transform with application to
fatigue EMG signal analysis. Comput Meth Progr Biomed 2006; 82:114–20
[6] Electroencephalography complex NEUROKOM, NEUROLAB. Instructions for medical
application AINC.941311.001 I1 U 33.1-02066769-001-2002
[7] A.P. Legrand, I. Rivals, A. Richard, E. Apartis, E. Roze, M. Vidailhet, S. Meunier, E. Hainque,
New insight in spiral drawing analysis methods – Application to action tremor quantification.
J Clinical Neurophysiology, 128 (10), 1823–1834. (2017)
[8] J.-S. Wang, F.-C Chuang. An Accelerometer-Based Digital Pen with a Trajectory Recognition
Algorithm for Handwritten Digit and Gesture Recognition. IEEE Transactions on Industrial
Electronics, 59(7), 2998-3007 (2012)
[9] E.D. Louis, A. Gillman, S. Böschung, C.W. Hess, Q. Yu, & S.L. Pullman, High width
variability during spiral drawing: Further evidence of cerebellar dysfunction in essential tremor.
Cerebellum, 11, 872-879 (2012).
[10] I. Mudryk, M. Petryk, Hybrid artificial intelligence systems for complex neural network
analysis of abnormal neurological movements with multiple cognitive signal nodes. 2020 IEEE
Third International Conference on Data Stream Mining & Processing (DSMP) : Conference,
Lviv, 21-25 August 2020. P. 108–111. DOI: 10.1109/DSMP47368.2020.9204341
[11] P. Viviani, P.R. Burkhard, S.C. Chiuvé, C.C. dell’Acqua, P. Vindras, Velocity control in
Parkinson’s disease: a quantitative analysis of isochrony in scribbling movements. Exp Brain
2009; 194:259–83.
[12] A. Salarian, H. Russmann, C. Wider, P.R. Burkhard, F.J. Vingerhoets, and K. Aminian,
“Quantification of tremor and bradykinesia in Parkinson's disease using a novel ambulatory
monitoring system,” Biomedical Engineering, IEEE Transactions on, 54. Jg., Nr. 2, pp. 313-
322, 2007.
[13] G. LO, A.R. Suresh, L. Stocco, S. González-Valenzuela, and V. C. Leung, “A wireless sensor
system for motion analysis of Parkinson's disease patients,” (PERCOM Workshops), 2011
IEEE International Conference on. IEEE, pp. 372-375, 2011
[14] I. Mudryk, M. Petryk, High-performance modeling, identification and analysis of
heterogeneous abnormal neurological movement’s parameters based on cognitive neuro
feedback-influences. Innovative Solutions in Modern Science. 2021. New York. TK Meganom
LLC. V1(45). P. 235-249 doi: 10.26886/2414-634X.1(45)2021.16.