<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>median filter application to deal with large windows of missing data in eye-gaze measurements</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Arnaud Gucciardi</string-name>
          <email>arnaud.gucciardi@toelt.ai</email>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff6">6</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Monica Crotti</string-name>
          <email>monica.crotti@kuleuven.be</email>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nofar Ben Itzhak</string-name>
          <email>nofar.benitzhak@kuleuven.be</email>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Lisa Mailleux</string-name>
          <email>lisa.mailleux@kuleuven.be</email>
          <xref ref-type="aff" rid="aff4">4</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Els Ortibus</string-name>
          <email>els.ortibus@uzleuven.be</email>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Umberto Michelucci</string-name>
          <email>umberto.michelucci@toelt.ai</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff6">6</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Vida Groznik</string-name>
          <email>vida@neus-diagnostics.com</email>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff5">5</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Aleksander Sadikov</string-name>
          <email>aleksander.sadikov@fri.uni-lj.si</email>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff5">5</xref>
        </contrib>
        <contrib contrib-type="editor">
          <string-name>Eye-Gaze, Software, Median Filter, Unilateral Cerebral Palsy, Kinarm</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Computer Science Department, Lucerne University of Applied Sciences and Arts</institution>
          ,
          <addr-line>6002 Lucerne</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Faculty of Computer and Information Science, University of Ljubljana</institution>
          ,
          <addr-line>Ljubljana</addr-line>
          ,
          <country country="SI">Slovenia</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Faculty of Mathematics, Natural Sciences and Information Technologies, University of Primorska</institution>
          ,
          <addr-line>Koper</addr-line>
          ,
          <country country="SI">Slovenia</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Katholieke Universiteit Leuven, Department of Development and Regeneration</institution>
          ,
          <addr-line>Leuven</addr-line>
          ,
          <country country="BE">Belgium</country>
        </aff>
        <aff id="aff4">
          <label>4</label>
          <institution>Katholieke Universiteit Leuven, Department of Rehabilitation Sciences</institution>
          ,
          <addr-line>Leuven</addr-line>
          ,
          <country country="BE">Belgium</country>
        </aff>
        <aff id="aff5">
          <label>5</label>
          <institution>NEUS Diagnostics d.o.o.</institution>
          ,
          <addr-line>Ljubljana</addr-line>
          ,
          <country country="SI">Slovenia</country>
        </aff>
        <aff id="aff6">
          <label>6</label>
          <institution>TOELT llc, Machine Learning Research and Development</institution>
          ,
          <addr-line>Birchlenstr. 25, 8600 Duebendorf</addr-line>
          ,
          <country country="CH">Switzerland</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Eye-hand coordination is a challenging skill to measure objectively, especially in children with motor disabilities such as Cerebral Palsy (CP). The recent development of robotic technology provides noninvasive tools for the simultaneous acquisition of eye and hand movement data. One such technology is the remote eye-tracking and virtual-reality systems namely the Kinarm Gaze-TrackerTM installed in the Kinarm ExoskeletonTM. Unfortunately, no standard software interface exists to extract the data contained in the Kinarm proprietary files for an eficient further analysis in common programming languages such as Python. Additionally, in the standard Kinarm reports only hand movements parameters are available, while eye movements are only stored as raw data files. These limitations lead to dificulties in performing a complete analysis of eye-hand coordination in research settings. Additional problems can arise in the case of missing data (due to loss of tracking). The software described in this paper allows the extraction of the hands and eye-gaze time series for eficient further analysis directly from the raw data. Furthermore, a study of the distribution of missing data is presented. Finally, this paper describes a revised median filter application to deal with large windows of missing data.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>with large</p>
    </sec>
    <sec id="sec-2">
      <title>1. Introduction</title>
      <p>
        https://github.com/toelt-llc/KiPy (A. Gucciardi)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
progressive disturbances that occurred in the developing foetal or immature brain. The motor
disorders of CP are often accompanied by disturbances of sensation, perception, cognition,
communication, and behavior, by epilepsy, and by secondary musculoskeletal problems” [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
The development of movement capacity in addition to muscle tone and posture [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ] is afected
by brain injury in the prenatal, perinatal, and postnatal phase up to the age of two years [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. CP
is a neurological disorder recognized as the leading cause of childhood motor disability and its
appearance is estimated from 1 to nearly 4 per 1,000 live births [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Children with CP develop a
wide range of conditions that may afect their functional abilities [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. The clinical variability
of children with CP can be explained by the heterogeneity of the underlying brain injury [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ],
which also afects the nonmotor pathways of the developing brain. Among these, the visual
network is often afected in children with CP [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. This leads to impairments in visual function
[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] which is a prerequisite for typical eye-hand coordination [8] since it is crucial for planning
and performing movements [9, 10]. Therefore, children with CP also sufer from dificulty in
grasping objects [11].
      </p>
      <p>
        Accurate reaching develops in children between 5 to 13 months of age [12] and is fine-tuned
over a longer period of several years (often more than 8) [ 12, 13]. In this process, eye-hand
coordination plays a fundamental role. Despite a large amount of research in this area, several
aspects of the development of eye-hand coordination remain unsolved in children with CP.
In addition to motor problems, 60 to 75% of children with CP also have visual deficits [
        <xref ref-type="bibr" rid="ref7">14, 7</xref>
        ].
Eye-tracker systems, which allow quantification of looking behaviour, nowadays are considered
a valid tool for investigating visuomotor coordination in CP children [15, 16]. Furthermore, their
implementation with robotic technology can provide an in-depth quantification of eye-hand
movement impairments in the pediatric neurological population. Previous studies [15, 16, 17]
attempted to quantify eye-hand coordination in children with CP using diferent methodologies.
Results showed that children with CP have increased visual attention towards the impaired
limb during object grasping and reaching [18, 17] and impaired anticipatory visual control in
eye-hand coordination when compared to typically developing children [15, 19].
      </p>
      <p>One novel application is the use of the Kinarm Exoskeleton [20] which allows an in-depth
quantification of bimanual motor control during symmetrical and asymmetrical tasks and
the simultaneously recording of eye movements via the Kinarm Gaze-Tracker [21]. With
this technology, both motor and gaze measures can be seamlessly integrated for efective
experimental control and data analysis. To our knowledge, no previous work fully evaluated
eyehand coordination in children with CP with the use of such a technology, although investigating
this relationship would provide a better understanding of the complex function of the visual
motor system. In addition, such results would provide useful information for clinicians and
researchers to be applied in diagnosis and possible rehabilitation settings.</p>
      <p>The first step for such an analysis is the extraction of information, such as gaze and hands
parameters over the time course of a movement, from the Kinarm saved files. This is not a trivial
task as the data is stored in proprietary file formats, making the desired analysis very dificult.
This work addresses this problem and describes a possible solution. The contributions of this
paper are fourfold. First, it describes a software framework able to extract hands and eye-gaze
coordinates with time from the Kinarm Exoskeleton files as time series (a time series is, in its
most common occurrence, a sequence of points taken at successive equally spaced points in
time). Secondly, this work describes and discusses a variation of the median filter for data with
large windows of missing values. Thirdly, this paper analyses missing data windows in terms
of distributions of width and frequency of the gaps in the data in two separate cases. Finally,
it demonstrates the median filter variation described in this paper applied to two diferent
examples with very diferent distributions of missing data windows.</p>
      <p>The paper is organised as follows. In Section 2, the Kinarm Exoskeleton and the eye-gaze
module are briefly described. The data and the median filter are also discussed and defined
respectively. In Section 2.4, the diferent tasks possible with the Kinarm exoskeleton are
described. In Section 2.6, the monitored parameters are listed. In Section 3, the Software is
described. In Section 4, the results are presented and in Section 5, the conclusions are discussed.</p>
    </sec>
    <sec id="sec-3">
      <title>2. Methodology and Data</title>
      <sec id="sec-3-1">
        <title>2.1. The Kinarm Exoskeleton</title>
        <p>Data collection was carried out with the Kinarm Exoskeleton Lab (BKIN Technologies, Kingston,
ON, Canada) [20] combined with an integrated EyeLink 1000 Plus eye tracking system (SR
Research, Ottawa, ON, Canada) [21]. The Kinarm Exoskeleton can be seen in Figure 1. The
Kinarm Exoskeleton Lab (BKIN Technologies, Kingston, ON, Canada) allows movement of the
arm in the horizontal plane such as flexion and extension of the shoulder and elbow joints
[20]. The hands are free to interact with objects in the environment surrounding the subject.
Patterns of joint motion are recorded and the system computes muscular torques, allowing the
study of upper limb movement and coordination. The use of Kinarm Lab’s operating system
and its control software, Dexterit-E TM [20], allows data collection in a user-friendly way. At
the end of each experimental task, reports can be extracted from the Dexterit-ETM software in a
Comma-Separated Values format (CSV) where a division in LEFT and RIGHT-hand parameters
is available.
2.2. Eye-Gaze tracking system and parameters
Eye tracking and gaze estimation systems are well-established techniques used to study eye
movements and position, both in clinical and research settings [22]. In eye tracking systems, the
eye position is calculated through diferent sequential steps (detection of the eyes, interpretation
of eye positions, and frame-to-frame tracking) with the help of the pupil or the iris centre [23].
Gaze estimation, that is, the process made to estimate and track the 3D line of sight, is calculated
from the analysis of eye movements through a device called gaze tracker [23]. A gaze tracker
simultaneously records the location of the eye position and its motion to determine the direction
of the gaze [24]. The EyeLink 1000 Plus system (SR Research, Ottawa, ON, Canada) integrated
into the Kinarm Exoskeleton Lab (BKIN Technologies, Kingston, ON, Canada) allows recording
binocular eye movements at up to 2000 frames per second. Camera images are processed using
a real-time operating system from which gaze data is recorded. More information on the eye
gaze estimation system can be found in the work of A. Kar et al. [25]. Eye tracking systems
allow the quantification of diferent types of eye movements such as fixations, saccades, and
smooth pursuit.</p>
        <p>This paper will specifically focus on the data used to estimate fixations and saccades [ 26].
A visual fixation is the maintenance of gaze in a single location or area [ 27]. Fixations phases
are defined as moments where the eyes are stationary between movements while the visual
input occurs. A saccade is a quick and simultaneous movement of the eyes between phases of
ifxation in the same direction [ 27]. Saccades are mainly used for orienting the gaze towards an
object of interest. They can be triggered voluntarily or involuntarily, with both eyes moving in
the same direction.</p>
        <p>Fixations and saccades can be quantified in terms of diferent parameters which can be used
for further analysis (i.e., eye-hand coordination). The mathematical algorithms to compute
them are not discussed in the present paper. For further information, the interested reader can
refer to the following papers [28, 25].</p>
        <p>The data output from Kinarm software includes gaze position (  and  positions), gaze
direction, pupil position ( ,  and  positions) and area, time stamps, and events such as start
and end of fixations and saccades. The Kinarm software automatically saves these features in
stored files, making them available to researchers. If necessary, averages and other statistical
analyses of the available metrics are then possible. Additionally, by adding the formerly listed
features, the parameters mentioned below can be calculated.</p>
        <p>Fixations and saccade parameters include:
• Fixation Duration - total duration of a fixation in seconds.
• Fixation Area - position where the fixation is recorded in meters.
• Saccadic Peak velocity - the highest velocity recorded during the saccade in metres per
second.
• Saccadic amplitude - the horizontal displacement during eye movement in meters.
• Saccade Duration - total duration of a saccade in seconds.
• Gaze latency - time taken from the appearance of a target to the beginning of a saccade
in response to that target in seconds.
• Gaze Accuracy - the average distance between the target and the participant’s eye position
in meters.
2.3. Data
In the present paper, data from two subjects, namely A and B, are discussed. Both participants
have been diagnosed with unilateral CP (mean age: 11y4m). Test subjects are chosen only if they
have minimal ability to actively grasp and hold an object and suficient cooperation to perform
the assessments. None of the participants received botulinum toxin injections six months before
testing or had a history of arm surgery two years prior to the assessment. Each experimental
session lasted about one hour. After the experimental session, the data were anonymised and
extracted from Dexterit-ETM software in a Comma Separated Values (CSV) format where a
division in the left and right upper limbs and the left and right eye gaze parameters is available.</p>
        <p>The available records for this study consist of two separate file groups: group A is defined
as the group that contains the first cohort of experiments, and group B is the second group.
Both groups contain eight files for the three diferent tasks studied: Ball On Bar task (2 files),
Object Hit task (2 files), and Visually Guided Reaching task (4 files). The files of the second group
contain incomplete and damaged data: missing values are detected for the Gaze X and Gaze Y
positions. In addition, the remaining values translate into a diferent gaze behaviour.</p>
        <p>A comparative view of the gaze behavior in the three tasks can be seen in Figure 4. Due to the
individual diferences in the task protocol and the numerous rows that make up the experiment
dataframes, the CSV files are of various sizes. In Table 1, the average total lines and the size of
the CSV files related to subjects A and B are reported.</p>
        <p>Size diferences are directly correlated with the duration of recorded trials ( number of
attempts) on tasks (i.e., exercise type).
2.4. Experiment tasks and datasets diferences
In this paper, the focus is on three custom experimental tasks (i.e., Kinarm standard test-KST),
namely, the Ball On Bar, Object Hit, and Visually Guided Reaching task. Each task is standardised
and performed with the Kinarm Exoskeleton, allowing the assessment of upper limbs’ motor
control and the simultaneous acquisition of eye-movements data [29]. A description of the KST
taken from the Kinarm manual is provided below.</p>
        <p>Ball On Bar The Ball on Bar task assesses the ability of subjects to perform a motor activity
that requires coordination of the two arms. [30] A virtual bar is presented between the
subject’s hands, and a virtual ball is placed on the bar. The objective of the task is to move
the virtual ball on the bar to successively presented targets as quickly and accurately as
possible.</p>
        <p>Object Hit The Object Hit task [31] assesses rapid motor skills throughout the workspace. It
is developed to assess the ability of a subject to select and engage in motor actions with
both hands over a range of speeds and a large workspace. Good performance requires the
ability to generate a goal-directed motor action on a moving target, bimanual planning to
select which arm to use to hit each object, and spatial awareness across the workspace.
Visually Guided Reaching The purpose of the Visually Guided Reaching task is to quantify
voluntary control directed toward the goal [32]. This task assesses visuomotor response
time and arm motor coordination. During this task, a central target is presented, and the
subject must move a cursor (white circle) representing hand position to this target.</p>
        <p>For each task, the Kinarm software can automatically compute a standard report (SR), as
well as a CSV file [ 33]. CSV files contain a metadata header with calibration and experimental
set-up information such as the Kinarm experiment instructions, the accessories used and the
calibration values. The file header also includes the definition of all recorded channels that
measure features with their unit of measurement. For each task, a diferent number of trials
are presented, namely an attempt to accomplish the exercise. Note that the number of trials
varies depending on the task exercise in both groups (A and B). For the Object Hit task, there
is a single trial in each file. A total of 1 to 3 trials are presented for the Ball On Bar, and 1 to
24 for the Visually Guided Reaching task. In the trial information, the diferent time series are
provided in terms of a dataframe where each row contains the data measured at a given time in
milliseconds. Rows are separated by 1 ms intervals. The individual trials also include a trial
header with several lines regarding metadata that needs to be removed in pre-processing.</p>
      </sec>
      <sec id="sec-3-2">
        <title>2.5. Files processing</title>
        <p>The experiment files are presented as CSV files containing dataframes. In this case, the columns
are the measured kinematics features, and the rows are the individual values of each frame
saved. The extraction process allows the reading of the dataframes contained as raw content.
Furthermore, the goal is to obtain the experiment data as time series. In the files, each trial
is represented as a separate dataframe. The pre-processing algorithm allows us to extract
these dataframes by splicing the CSV files and removing the general metadata header and the
individual trial headers. This is done for each of the four possible tasks. The methods involved
can be used for new experiment files to automate the analysis.</p>
      </sec>
      <sec id="sec-3-3">
        <title>2.6. Monitored parameters</title>
        <p>The features of interest used for visualisation and analysis are listed in Table 2. Note: the total
real time of each trial is equal to sample count multiplied by sample duration. It can also be
retrieved by looking at frame time in the last row of the selected trial. All tasks from both file
groups are saved in a single trial (that is, in one dataframe). Some files contain more than one
trial: in the first group of files, the two Ball-On-Bar tasks include, respectively, 3 and 2 trials,
and the two Visually Guided tasks include 3 (for the child practise set) and 24 (for the complete
exercise) trials. And in the second group, two out of four Visually Guided tasks contain 9 trials.
2.7. Analysis of Missing Data in the Eye-Gaze Measurements
As mentioned in Section 2.3, some of the experiment data lines contained in the CSV files of the
second group of files appear incomplete. The features presented previously require continuous
data to study the detailed actions of gaze and hands. In practise, the missing values detected
in the files make a complete analysis and visualisations impossible; since both gaze and hands
position data in multiple time windows are missing. Indeed, events like fixations and saccades
cannot be entirely detected and understood when positions are only partially saved during a
given time window. As a first step, the number of extended regions of continuous missing data,
or gaps, are counted and their respective lengths measured. Depending on the type of trial,
the complete length of recorded experimental data changes, but it is possible to estimate the
percentage of missing data for each trial to quickly quantify the impact on the final analysis.
And this is possible for each experiment regardless of the length. This result is represented in
Table 3.</p>
      </sec>
      <sec id="sec-3-4">
        <title>2.8. Median Filter</title>
        <p>To preserve the recorded data saved in the incomplete files and to reduce noise, a median filtering
technique is applied. The method is compatible with missing data. Due to the noise reduction
obtained, the local trend can be preserved by replacing incorrect data. The median filter
eliminates extreme or empty values without having to do a mean averaging of the neighbour
values, which would heavily impact the correct values. Although typically used for image
pre-processing [34], the algorithm can be applied to one-dimensional signals [35] as is the case
here. When used on one-dimensional input, the process is simplified : the neighbourhood
includes values before and after the index.</p>
        <p>Definition 1. Given an array of  values ( 1, ...,   ), the median filter (MF) of size  (in this
paper  is taken to be odd for simplicity of notation) is a mapping ℝ → ℝ . By defining
 = 2 + 1 , the output of the MF will be an array with elements    given by</p>
        <p>= median({  − , ...,   + })
The median filter algorithm replaces each individual value   (starting from  =  ) of the original
array by the median of the following and previous  values. The window or filter size of total
size  defines the amount of neighbour values considered to compute the new filtered signal   .
Since there are no values preceding the first and last elements of the signal, the first and last values
are repeated until enough values are reached to fill the window.
0.4
2.1
2.7
1.9
1.8
0.8
(1)</p>
        <p>Figure 2 shows an example of the median filter applied to a hypothetical data set with some
quantity (  ) measured at specific time points   . In Step 1, a time window is selected (in Figure 2
the selected time window includes the time points  2 to  6). In Step 2, the values are sorted, and
the median value is selected (Step 3). The time window then moves along the entire data set.</p>
        <p>The median filtering is applied to the experimental data in Figure 5, with two filter sizes
presented. Fixations and saccades rely on specific windows of gaze position. It is important to
note, as a downside, the risk of losing short movement detections as the filter size selected gets
larger. Movements shorter than the filter window might be lost. This is a risk in the time series
containing gaze positions if the filter is longer than some of the actual events happening within
the task. The window size choice is an important parameter further developed in the article.
2.9. The Median Filter for Large Windows of Missing Data (MFLWMD)
A statistical analysis of the features of saccades and fixations is made dificult, if not impossible,
when large windows of missing data are present. The most appropriate solution is to split the
time series each time a gap larger than a specific size is encountered. As a possible solution
to the presence of large windows of missing data, the following median filter application is
presented. Given a certain measurement of a generic quantity   (for example, the  coordinate
of the eye-gaze) at various   time points (for  = 1, ...,  ). A median filter of size  can be applied
by sliding a window of size  on the measured data. Let us also suppose that  ∈ ℕ windows
of missing data, in which gaps of size   are present at various positions along the array   . If
a missing data window is encountered, there are two possible scenarios. Let us indicate with
 ∈ ℕ an integer that can be called threshold.
1.   ≥ : the array   is split at that point and every calculation of the statistical estimators is
stopped. The two parts are considered for all purposes as separate files.
2.   &lt;  : the median filter can be applied simply by removing the missing data and
considering only the available values.</p>
        <p>The best choice of  is, of course, related to the median filter window size.  should be larger
than  to make this proposed variation of the median filter meaningful. From the available
missing values files and experiments, and since the filters span across windows of size  , the
authors propose that a good choice is  ≳ 2 to 3 . If the threshold is less than  , complete
missing value gaps are never considered since the filter size will cover all possible cases. On the
other hand, if the threshold is greater than 3 the starting and end points are considered too far
apart for the filter to suficiently fill the gap considered.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>3. Software Functionalities</title>
      <p>The KinarmPython extraction library presented in this article, called KiPy, can read CSV files
generated from the Kinarm Dexterit-ETM software [ 33] and support their analysis. The software
provides the user with the ability to parse all accessible kinematics logs. Concerning the research
questions previously mentioned in Section 2.3, the software extracts relevant information and
can create visualisations which can also be easily changed by the user. The tool currently uses
command-line scripts and Python functions; the goal is to keep it simple, configurable, and
performant. The application is written in Python 3.8 and requires the additional Pandas, NumPy,
Pickle, and Matplotlib libraries. It is open source and is available on GitHub [36]. Once the files
are read, all metadata are excluded, and the remaining data are read as a Pandas DataFrame. The
Pandas DataFrame is an eficient data structure to store structured data and provides powerful
functions to filter and search for specific rows of columns.</p>
      <p>From the extracted dataframes, the algorithm accesses the visual data events. For each trial,
the duration and count of all the events are given. This information can be further used to
analyse the frequency of the events over a single trial, a complete experiment, or a group of
experiments (e.g., the events statistics for a given type of experimental task).
4. Results
a.1
b.1
20
40
60
80
100
120
140
0
5
10
15
25
30
35
40
10
20
30
40
50
60
0
10
20
30
40
50
60
70
a.2
b.2
0
10
20
30
40
50
60
0
10
20
30
40
50
60
70
a.3
b.3
)(m0.6
Ye 0.3
z
aG 0.0 0
)(m0.2
eX 0.0
z
aG−0.2
)(m1.0
Ye 0.5
z
aG 0.0 0
)(m0.5
eX 0.0
z
aG−0.5</p>
      <sec id="sec-4-1">
        <title>4.1. Eye-gaze and Hand Position Data</title>
        <p>The central idea behind the software tool is to give the user the ability to read the CSV files and
visualise and analyse the Kinarm experiments. The software makes it possible to first, given
any task input, visualise the movements of the gaze and hands. The visualisation can be done
with both static plots and animations. Animations are short videos that the user can produce at
a chosen speed. Examples of visualisations can be seen in Figure 4 for gaze movements and in
Figure 6 for hands movements over the duration of the task in the three diferent tasks. The
ifgures display the diferences between the two groups of files. One can identify gaps of missing
values, saved as NaNs in files, and numerous peaks, hence the name of noisy flickering data.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Median Filter Application</title>
        <p>Examples of the distribution of NaN gaps in gaze time series are presented in Tables 3 and 4.
The examples selected are described with a precise description of the impact of the missing
values. The diference is comparably significant over all the tasks and trials. In group A of the
ifles, only 17 of 65 total trials contain missing values, and the average proportion of missing
values within the 17 afected trials is 2.8%. However, 29 of the 29 trials in group B have missing
values, which represent 18.8% of the total trial values on average. Due to the high variation
rates and missing values in group B, the median filter technique was developed and applied to
gaze X and Y time series of this group. An example of a median filter application is shown in
Figure 5. In each of the three panels, missing data points are not replaced by the median filter.</p>
        <p>0.8
)
(m0.6
e
c
r
u
so 0.4
Y
e
z
aG 0.2
0.0
0.4
)
(m0.2
e
c
r
ou 0.0
s
X
e
za−0.2
G
−0.4
)
m
(
0
4
1
d
e
r
e
t
lif
Y
e
z
a
G
)
m
(
0
4
1
d
e
r
e
t
lif
X
e
z
a
G
)
m
(
0
0
5
2
d
e
r
e
t
ilf
Y
e
z
a
G
)
m
(
0
0
5
2
d
e
r
e
t
lif
X
e
z
a
G
0
10
40
50
0
10
40
50
0
10
40
50
20 30
Trial time (s)</p>
        <p>20 30
Trial time (s)</p>
        <p>20 30
Trial time (s)</p>
        <p>Two diferent filter sizes are applied: 140 and 500, respectively, on the centre and right panels,
with the filter centred on the replaced value. On the gaze Y, the range of values went from
0.0-0.83, to 0.17-0.61 in the 140-filtered time series, and to 0.22-0.46 in the 500-filtered time series.
A similar result is obtained for the Gaze X values. Although some flickering outliers remain
in the 140-filtered time series, they are all smoothed in the 500-filtered time series. In this
example, the smoothing appears to be efective. In practice, a 500-sized filter may be damaging
for some tasks since it represents half a second of gaze movements; ruling out the gaze events
with shorter durations. The filter of 500ms is used here as a display example of a high value.
0.4
0.8
1.0
1.2
0
1
2
3
4</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions</title>
      <p>Eye movement tracking combined with bimanual motor movement recording allows a
complete experimental analysis during symmetrical and asymmetric tasks. From the combined
measurements, precise parameters can be extracted and analysed. In this paper data extracted
from measurements obtained with the experimental setup described in Section 2.1 and 2.2, is
used to highlight the possible applications of the KiPy software analysis library. For each task,
it is possible to extract the recorded parameters, including the gaze and hands positions with
the timestamp, thanks to various automated functions. Additionally, statistics of ranges of
movements and speeds can be individually calculated. If files with flickering and missing values
are present, the software can smooth the flickering signals, detect the size of missing value gaps,
and efectively count the number of consecutive data sections. The visualisations and statistics
on incomplete experimental data are therefore possible, allowing the analysis of all datasets,
including those that present a high percentage of missing data. In the available data, half of the
ifles displayed such problems.</p>
      <p>The ability to obtain as much information as possible from the experiments is crucial. The
variation of the median filter described in this paper can be applied to all kinds of datasets. To
derive an optimal strategy for the choice of the parameter  , more experimental and diferent
data will be needed. This analysis is planned for a future publication.</p>
      <p>To the best knowledge of the authors, no previous work has looked at the automatic detection
and filtering of missing data in Kinarm report files. With access to more statistical samples, the
KiPy software can be confidently used to extract the desired features and make the analysis
much more robust. The authors plan to use and apply the KiPy software to the analysis of a
larger dataset to study the eye-hand coordination in a group of children with unilateral cerebral
palsy.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgments</title>
      <p>This work was supported by the project: “PARENT” funded by the European Union’s Horizon
2020 Project MSCA-ITN-2020 – Innovative Training Networks Grant No. 956394.
[8] S. Chokron, G. N. Dutton, Impact of cerebral visual impairments on motor skills:
implications for developmental coordination disorders, Frontiers in psychology 7 (2016)
1471.
[9] F. R. Sarlegna, R. L. Sainburg, The roles of vision and proprioception in the planning of
reaching movements, Progress in motor control (2009) 317–335.
[10] S. A. Mutalib, M. Mace, H. T. Ong, E. Burdet, Influence of visual-coupling on bimanual
coordination in unilateral spastic cerebral palsy, in: 2019 IEEE 16th International Conference
on Rehabilitation Robotics (ICORR), IEEE, 2019, pp. 1013–1018.
[11] O. Martinie, C. Mercier, A. M. Gordon, M. T. Robert, Upper limb motor planning in
individuals with cerebral palsy aged between 3 and 21 years old: A systematic review,
Brain sciences 11 (2021) 920.
[12] C. von Hofsten, L. Rönnqvist, Preparation for grasping an object: a developmental study.,</p>
      <p>Journal of experimental psychology: Human perception and performance 14 (1988) 610.
[13] M. Favilla, Reaching movements in children: accuracy and reaction time development,</p>
      <p>Experimental Brain Research 169 (2006) 122–125.
[14] A. Schenk-Rootlieb, O. Van Nieuwenhuizen, P. Van Waes, Y. Van der Graaf, Cerebral
visual impairment in cerebral palsy: relation to structural abnormalities of the cerebrum,
Neuropediatrics 25 (1994) 68–72.
[15] S. M. Surkar, R. M. Hofman, B. Davies, R. Harbourne, M. J. Kurz, Impaired anticipatory
vision and visuomotor coordination afects action planning and execution in children with
hemiplegic cerebral palsy, Research in Developmental Disabilities 80 (2018) 64–73.
[16] S. Saavedra, A. Joshi, M. Woollacott, P. van Donkelaar, Eye hand coordination in children
with cerebral palsy, Experimental brain research 192 (2009) 155–165.
[17] J. Verrel, H. Bekkering, B. Steenbergen, Eye–hand coordination during manual object
transport with the afected and less afected hand in adolescents with hemiparetic cerebral
palsy, Experimental brain research 187 (2008) 107–116.
[18] B. Steenbergen, W. Hulstijn, A. De Vries, M. Berger, Bimanual movement coordination in
spastic hemiparesis, Experimental Brain Research 110 (1996) 91–98.
[19] S. James, J. Ziviani, R. S. Ware, R. N. Boyd, Relationships between activities of daily living,
upper limb function, and visual perception in children and adolescents with unilateral
cerebral palsy, Developmental Medicine &amp; Child Neurology 57 (2015) 852–857.
[20] Kinarm, Kinarm exoskeleton product webpage, 2022. URL: https://kinarm.com/
kinarm-products/kinarm-exoskeleton-lab/, last accessed on 2022-07-03.
[21] Kinarm, Kinarm exoskeleton integrated gaze-tracking datasheet, 2022. URL: https://kinarm.</p>
      <p>com/download/kinarm-gaze-tracker/, last accessed on 2022-07-03.
[22] F. R. Danion, J. R. Flanagan, Diferent gaze strategies during eye versus hand tracking of a
moving target, Scientific reports 8 (2018) 1–9.
[23] M. Q. Khan, S. Lee, Gaze and eye tracking: Techniques and applications in adas, Sensors
19 (2019) 5540.
[24] R. A. Naqvi, M. Arsalan, G. Batchuluun, H. S. Yoon, K. R. Park, Deep learning-based gaze
detection system for automobile drivers using a nir camera sensor, Sensors 18 (2018) 456.
[25] A. Kar, P. Corcoran, A review and analysis of eye-gaze estimation systems, algorithms and
performance evaluation methods in consumer platforms, IEEE Access 5 (2017) 16495–16519.
[26] D. D. Salvucci, J. H. Goldberg, Identifying fixations and saccades in eye-tracking protocols,
in: Proceedings of the 2000 symposium on Eye tracking research &amp; applications, 2000, pp.
71–78.
[27] B. Cassin, S. Solomon, M. L. Rubin, Dictionary of eye terminology, Triad Publishing</p>
      <p>Company Gainesville, 1990.
[28] S. Tangnimitchok, A. Barreto, F. R. Ortega, N. D. Rishe, et al., Finding an eficient threshold
for fixation detection in eye gaze tracking, in: International Conference on
HumanComputer Interaction, Springer, 2016, pp. 93–103.
[29] Kinarm, Kinarm standard tests summary, 2022. URL: https://kinarm.com/kinarm-products/
kinarm-standard-tests/, last accessed on 2022-07-03.
[30] C. Lowrey, C. Jackson, S. Bagg, S. Dukelow, S. Scott, A novel robotic task for assessing
impairments in bimanual coordination post-stroke, International Journal of Physical
Medicine and Rehabilitation S3:002 (2014). doi:10.4172/2329- 9096.S3- 002.
[31] K. Tyryshkin, A. M. Coderre, J. I. Glasgow, T. M. Herter, S. D. Bagg, S. P. Dukelow, S. H.</p>
      <p>Scott, A robotic object hitting task to quantify sensorimotor impairments in participants
with stroke, Journal of neuroengineering and rehabilitation 11 (2014) 1–12.
[32] A. M. Coderre, A. Abou Zeid, S. P. Dukelow, M. J. Demmer, K. D. Moore, M. J. Demers,
H. Bretzke, T. M. Herter, J. I. Glasgow, K. E. Norman, et al., Assessment of upper-limb
sensorimotor function of subacute stroke patients using visually guided reaching,
Neurorehabilitation and neural repair 24 (2010) 528–541.
[33] Kinarm, Dexterit-e., 2022. URL: https://kinarm.com/kinarm-products/dexterit-e/, last
accessed on 2022-07-13.
[34] G. Gupta, Algorithm for image processing using improved median filter and comparison
of mean, median and improved median filter, International Journal of Soft Computing and
Engineering (IJSCE) 1 (2011) 304–311.
[35] B. Justusson, Median filtering: Statistical properties, Two-Dimensional Digital Signal</p>
      <p>Prcessing II (1981) 161–196.
[36] TOELT, Github repository, 2022. URL: https://github.com/toelt-llc/KiPy, last accessed on
2022-08-20.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>P.</given-names>
            <surname>Rosenbaum</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Paneth</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Leviton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Goldstein</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Bax</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Damiano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Dan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Jacobsson</surname>
          </string-name>
          , et al.,
          <source>A report: the definition and classification of cerebral palsy april</source>
          <year>2006</year>
          ,
          <source>Dev Med Child Neurol Suppl</source>
          <volume>109</volume>
          (
          <year>2007</year>
          )
          <fpage>8</fpage>
          -
          <lpage>14</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>D. R.</given-names>
            <surname>Patel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Neelakantan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Pandher</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Merrick</surname>
          </string-name>
          ,
          <article-title>Cerebral palsy in children: a clinical overview, Translational pediatrics 9 (2020) S125</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>M.</given-names>
            <surname>Sadowska</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Sarecka-Hujar</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Kopyta</surname>
          </string-name>
          ,
          <article-title>Cerebral palsy: Current opinions on definition, epidemiology, risk factors, classification and treatment options</article-title>
          ,
          <source>Neuropsychiatric disease and treatment 16</source>
          (
          <year>2020</year>
          )
          <fpage>1505</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Kinarm</surname>
          </string-name>
          ,
          <article-title>National institute for health and care excellence (uk). cerebral palsy in under 25s: assessment and management, 2022</article-title>
          . URL: https://www.ncbi.nlm.nih.gov/books/, available online.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>L.</given-names>
            <surname>Mailleux</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Simon-Martinez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Klingels</surname>
          </string-name>
          , E. Ortibus,
          <string-name>
            <given-names>H.</given-names>
            <surname>Feys</surname>
          </string-name>
          ,
          <article-title>Brain lesion characteristics in relation to upper limb function in children with unilateral cerebral palsy</article-title>
          , in: Factors Afecting Neurodevelopment, Elsevier,
          <year>2021</year>
          , pp.
          <fpage>411</fpage>
          -
          <lpage>420</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A.</given-names>
            <surname>Guzzetta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Fazzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Mercuri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Bertuccelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Canapicchi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. Van</given-names>
            <surname>Hof-van Duin</surname>
          </string-name>
          , G. Cioni,
          <article-title>Visual function in children with hemiplegia in the first years of life</article-title>
          ,
          <source>Developmental Medicine and Child Neurology</source>
          <volume>43</volume>
          (
          <year>2001</year>
          )
          <fpage>321</fpage>
          -
          <lpage>329</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>E.</given-names>
            <surname>Fazzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. G.</given-names>
            <surname>Signorini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. La</given-names>
            <surname>Piana</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bertone</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Misefari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Galli</surname>
          </string-name>
          , U. Balottin,
          <string-name>
            <given-names>P. E.</given-names>
            <surname>Bianchi</surname>
          </string-name>
          ,
          <article-title>Neuro-ophthalmological disorders in cerebral palsy: ophthalmological, oculomotor, and visual aspects</article-title>
          ,
          <source>Developmental Medicine &amp; Child Neurology</source>
          <volume>54</volume>
          (
          <year>2012</year>
          )
          <fpage>730</fpage>
          -
          <lpage>736</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>