=Paper=
{{Paper
|id=Vol-2343/paper7
|storemode=property
|title=Artificial Neural Network for Sit-to-Stand classification based on Inertial Measurement Units Data
|pdfUrl=https://ceur-ws.org/Vol-2343/paper7.pdf
|volume=Vol-2343
|authors=Sandy Rihana,Patrick Hockayem,Ramy Salloum
|dblpUrl=https://dblp.org/rec/conf/bdcsintell/RihanaHS18
}}
==Artificial Neural Network for Sit-to-Stand classification based on Inertial Measurement Units Data==
https://ceur-ws.org/Vol-2343/paper7.pdf
Artificial Neural Network for Sit-to-Stand
classification based on Inertial Measurement Units
Data
Sandy Rihana
Biomedical Engineering Department Patrick Hockayem Ramy Salloum
Holy Spirit University Of Kaslik, Biomedical Engineering Department Biomedical Engineering Department
USEK Holy Spirit University Of Kaslik, Holy Spirit University Of Kaslik,
Jounieh, Lebanon USEK USEK
sandyrihana@usek.edu.lb Jounieh, Lebanon Jounieh, Lebanon
Abstract— According to some statistics published by method can be done by taking several images during the
Centers for Disease Control and Prevention (CDC), 1 in 50 experience, processing, feature extraction and classification.
people approximately around the world is incapable of [1].
achieving some of daily’s life activities by his own due to
paralysis. Paralysis is the partial or total inability of the human
body to perform some movements caused by stroke, spinal As its name indicates, floor force sensors technique uses a set
cord injury, multiple sclerosis, birth defect, etc. Today, number of force platforms which enable the user to extract and
of paralysis “victims” is increasing dramatically making over 6 calculate pressure, force or pressure and force based data
million people paralyzed around the world, with some cases when the subject perform gait or STS tasks on the sensors.
were the physical therapy becomes unable to heal. This technique [2], like the previous one, needs special
equiped high end laboratories. Wearable sensors is a new
Consequently, Technology has constantly been a major
technique that provides great results and same accuracy as
player in a large number of physical therapy applications, and
offers many advantages for paralyzed people that the physical image processing and floor force system, and overcomes
therapy is not able to provide. Recently, exoskeleton patient these methods by the fact that it is less complex, since it
motion aiding technology was introduced in order to supply doesn’t require special environment and can be performed
disabled people and regain mobility. in-lab, less time consuming and less costly. [3] During a STS
or gait study, the participant has a number of sensors
The main goal of this project is to study the Sit-to-Stand and positioned on his body, usually segments and joints, in order
Stand-to-Sit transfer in 10 young healthy subjects. IMU to extract desired data. [3]
(Inertial Measurement Unit) wearable sensors, more
specifically MPU6050 sensors, are used in the performed
experiences in order to extract desired raw data such as the Tay et al. study was founded on Newton’s.motion.law.
acceleration, angular rate and inclination of lower limb Since fair forces applied on the human body are pro rata to
different segments including metatarsal, shank and thigh the acceleration of the latter, Tay et al introduced a method
segments and the inclination of the trunk. Thus, ankle, knee by integrating two accelerometer sensors, positioned on the
and hip joints angles were derived. left and right ankles of a subject to stalk walking, and an
Furthermore, extracted features are studied, analyzed and accelerometer near the cervical vertebra in order to supervise
used to establish epochs and recognize phases of the Sit-to- different body positions during the task. [4]
Stand gesture based on a number of previous Sit-to-Stand
literature of art. This is accomplished using Artificial Neural
Network, using different architectures and choosing the best Likewise, they were capable of evaluating some
one, which resulted in four main phases: flexion phase, transfer interesting parameters like peak acceleration and time during
phase, extension phase and stabilization phase. a sitting and standing tasks.
Finally, and using the proper Neural Network with the
higher accuracy (92.3% accuracy using the 30 layers Inspired from Android Phone Sphere application
architecture), a Sit-to-Stand algorithm is proposed and allowing to estimate the direction of rotating the smartphone,
modeled. Kai-Yu Tong & Granat studied gait and STS movements by
placing 1-axis gyroscopes on the shank and thigh of each
Keywords— Artificial Neural Network, Sit-To-Stand subject in order to extract every segment velocity rate. They
algorithm, Gait analysis, Classification, IMU sensors conclude the efficiency of this method in order to derive
other parameters such as inclination, orientation and number
I. INTRODUCTION
of movements. [5]
Gait and Sit To Stand (STS) analysis are studies based on
different techniques using various devices and sensors able Our experience and study were basically based on Tadano et
to capture some human motions and extract the al. [6]. They proposed a three dimension gait analysis using
corresponding parameters and data for further applications. wearable, combination of 3-axis accelerometer and 3-axis
Usually, three common techniques are used: Image gyroscope, sensors positioned on lower limb segments.
Processing, floor Force Sensors and Wearable Sensors. Acceleration and angular velocity data were recorded, in real
time, during walking task.
Image processing technique is performed in a special
laboratory equipped with a set of cameras, either analog or The direction and orientation, respectively to the
digital ones. Extracting gait and STS parameters using this gravitational force, were extracted from the accelerometer
21
sensors in order to compute original positions, while various II. MATERIALS AND METHODS
displacements and angular velocities were recorded using the
A. Inertial Moment Unit Sensors
gyroscope sensors.
Motion Processing Unit 6050 (MPU-6050), a part of
Tadano et al. conclude that IMU based method showed IMU-6000 family, consists of a micro-electromechanical
accurate and validate results, comparatively to other common system, providing a 3 Degrees of Freedom of
techniques, of quantitative data and parameters like segments acceleration and 3 Degrees of Freedom of angular
acceleration, inclination and angular displacement. velocities values, using respectively a 3-axis
accelerometer and a 3-axis gyroscope. It provides 6
Our suggested methodology in this paper will be described Degrees of Freedom as a final result. This low priced
as follow : first, 50 STS tasks will be experimented on 10 sensor consumes less power and has a great performance
subjects. The subjects have been asked to perform 5 trials, comparatively to other IMU sensors. MPU-6050 sensors
while 7 IMU sensors are placed on lower limb segments and are compatible with Arduino and Raspberry pi boards
the trunk. During the task, acceleration, angular velocity and and softwares using I2C and Serial Peripheral Interface
inclination of lower limb segments and inclination of the (SPI) protocols.
trunk are recorded, using Arduino software, displayed on B. Experimentations
graphs, using MATLAB software, and interpreted. Joints 10 healthy men participated in this experience, all these
angles will be derived and calculated. All these data are then
participants are between 20 and 25 years (Mean= 23 years,
classified using Artificial Neural Netowork, ANN, using
SD= 1.84), heights between 169 cm and 182 cm (Mean= 175
various architectures. Afterwards, Receiver Operating
Characteristic, ROC and confusion matrixes results of the cm, SD= 4.7) and weights between 85 Kg and 99 Kg (Mean=
most ideal ANN architecture are analyzed. Finally, all results 91.8 Kg, SD =3.89).
will be used in order to suggest a STS conceptual model
algorithm. All participants are free of any orthopedic or arthritic
disorder, and thus were all-able to perform the sit-to-stand
independently without any human support or device
assistance (e.g. knee support).
Participants were requested to sit on an armless chair with no
active role of their arms in the sit-to-stand task. In order to
maintain the vertical position of the trunk, each subject uses
the back support of the chair, and his knees flexed to
approximately 90o with a space of 20 cm between his feet
while placed on the ground.
Every participant was asked to perform 5 sit-to-stand-to-
sit trials for 13 to 14 seconds after verbal signals.
TABLE I. EXPERIENCE PROCESS
Phase Recording time
Sitting position 3 seconds
Transition phase (sit-to- 2 seconds
stand)
Standing position 3 seconds
Transition phase (stand-to- 2 seconds
sit)
Sitting position 3 seconds
Figure 1- Flowchart Methodology Instrumentation consists of 7 IMU sensors (MPU 6050),
Arduino software and armless back supported seats.
Sensors were positioned on 2/3 of each segment length
(metatarsal, shank and thigh) of the left and the right parts,
and 1 IMU is placed on the T3 thoracic vertebrae of the trunk
illustrated in Fig.2 and Fig.3.
In this study, segments acceleration values, acquired
from the accelerometer and segments angular velocity
22
values, acquired from the gyroscope, were collected
simultaneously, displayed on the serial monitor of Arduino
software and saved in a .txt format file.
Hip, knee and ankle angles were calculated from the
acceleration values. The right placement of the IMU sensors
is crucial to features extraction, because each sensor will
indicate to a certain list of variables all obtaining to one
parameter.
Each sensor will be coded to one parameter as illustrated in
table II.
The angles calculation is shown in the following formulas:
𝑨𝒙
𝒑 = 𝒂𝒓𝒄𝒕𝒂𝒏 ( )
𝑨𝒚 𝟐 + 𝑨𝒛 𝟐 Figure 3- Trunk IMU sensor Placement
𝑨𝒚 C. Proposed protocol
𝜱 = 𝒂𝒓𝒄𝒕𝒂𝒏 ( )
𝑨𝒙 𝟐 + 𝑨𝒛 𝟐 In the first place, the subject is in a sitting position with feet
𝑨𝒙 𝟐 + 𝑨𝒚 𝟐 positioned on the ground.
Ɵ = 𝒂𝒓𝒄𝒕𝒂𝒏 ( ) • Trunk, maintained in a nearly vertical position, and
𝑨𝒛
thigh, maintained in a nearly horizontal position
relatively to the ground, forms a hip angle of 95 o.
• Shank, maintained in a nearly vertical position
TABLE II. SENSORS PLACEMENT relatively to the ground, forms with the thigh a
knee angle of 85 o.
• Metatarsal, maintained in a nearly horizontal position
relatively to the ground, forms with
Sensor Place the shank an ankle angle in a range of 85 o.
S1 Trunk
The first step, flexion phase, is demarcated by the
movement of the subject’s upper body part, especially the
S2 Right thigh trunk, while lower body segments are immobile. The main
base support in this phase is the seat or the chair.
S3 Left thigh
So it starts from the sitting position and end when
S4 Right shank the trunk is flexed before lifting off from the seat or initial
position.
S5 Left shank • Knee and ankle joints angles remain the same
S6 Right metatarsal
while the trunk is flexed in a forward position resulting in a
decrease of the hip joint angle of 35o to achieve a flexion
S7 left metatarsal position of 60 o.
• Thigh, shank and metatarsal segments accelerations
remain the same.
• Thigh, shank and metatarsal segments angular
velocity remain the same
The second step, extension phase is differentiated from the
first one from by several mechanical points. It is completed
when maximum forward flexed posture is reached. A
“transfer” of the base support occurs: the subject passes
from the seat support to the feet support. The lower body
movement accompanies the upper body movement.
This step begins when the subject starts to lift off from
the seat and ends before starting the extension motion of
different joints.
The third step, extension phase, also mechanically
differs from the first and the second phase. We can simply
say that the extension phase is the translation of the body in
a vertical direction.
The Stabilization phase represents the final stage of a STS
transfer.
Figure 2- IMU sensors placement Hip and knee angles of 170o allow us to conclude that
metatarsal and shank segment are nearly orthogonal, while
shank, thigh and trunk are approximately aligned in a
23
vertical direction nearly perpendicular to the horizontal body parts and the trunk to study the Sit-to-Stand task, it is
direction of the ground. fundamental to use some anatomical terms related to
The different STS phases are illustrated in Fig.4. different joints movement:
-Flexion: causes two parts or segments to get closer,
decreasing the separating angle between them. It is usually
used to describe a flexion movement of the hip and knee
joints.
-Extension: causes two parts or segments to separate,
increasing the separating angle between them. It is usually
used to describe the extension movement of the hip and
knee joints.
-Dorsiflexion: is the flexion of the ankle joint
-Plantar flexion: is the extension of the ankle joint.
Figure 4- STS Phases algorithm
Sit-to-Stand phases will be named respectively class 1, class Fig. 5 represents an ankle joint, formed by the metatarsal
2, class 3 and class 4. and the shank segments, with M and S the angles of
D. Features Extraction respectively metatarsal and shank segments, while M’ is the
internal alternate angle of the S relatively to the ground. M
All extracted features were recorded using Arduino
is measured from the IMU sensor of the metatarsal segment,
software. Here is a table that displays each feature number
and thus M’ is concluded from this sensor. S is measured
and name.
from the IMU sensor of the trunk segment. Ankle angle can
be seen on the figure as the sum of M’ and S angles.
TABLE III. FEATURES EXTRACTION
Feature Number Feature Name
F.E.1 Trunk angles
F.E.2 Thigh Angles
F.E.3 Thigh Angular Velocity
F.E.4 Thigh Acceleration
F.E.5 Shank Angles
F.E.6 Shank Angular Velocity
F.E.7 Shank Acceleration
Figure 5- Ankle joint extraction
F.E.8 Metatarsal Angles
Same studies have been done for the knee joint, the hip
F.E.9 Metatarsal Angular Velocity angle. We only limited the calculation on the X-plan, since
flexion and extension are only performed in the lateral plan.
F.E.10 Metatarsal Acceleration
E. Processing workflow
Each experience is made for 13 seconds approximately; Extracted features have been fed into an artificial neural
each sensor can sense a value at 0.2 seconds. Thus for each network. Fig.6 illustrated the general workflow.
feature status above we will have, a certain number of
values that will be noted N, where N = 65 values. However,
the total amount of values obtained at each experience that
will be noted X, is equal to the N x Number of features = 10
x 65 = 650. To add, each experience is repeated 5 times, to
get the most accurate values possible. Total Amount of
Values (for each subject) , knowing that the experiment has
been repeated 5 times is equal to 3250 values. For the 10
subjects, the total Amount of Values (for all subject) is
equal to 32500 values.
In general, when two bones are re-joined, a joint is
developed. The joint movement is completed by the
presence of muscles. Since we are interested in the lower
Figure 6- General workflow
24
After extracting features from MATLAB workspace, 32 480
data were introduced as input vectors to the ANN. In this
statistical analysis, 10 samples were tested. The set of
variables could be resumed as follow : X-axis metatarsal
angles, X-axis shank angles, X-axis thigh angles, X-axis
trunk angles, X-axis metatarsal angular velocity, X-axis
shank angular velocity, X-axis thigh angular velocity, X-axis
metatarsal acceleration, X-axis shank acceleration, X-axis
thigh acceleration. Several neural network architectures were
employed in order to find the most appropriate one using 5,
10, 15, 20, 25, 30 and 35 neurons per hidden layer. Figure 8- Trunk Angles Graph
Phase 1: window between 0 x 0.2 sec and 10 x 0.2 sec. It
displays a constant curve of 87 degrees, which allows us to
conclude that the subject is at rest. In fact, these values are
related to a perpendicular posture of the trunk respectively to
the horizontal (ground) with values approximately close to
90 degrees.
Phase 2: window between 10 x 0.2 sec and 15 x 0.2
sec. It displays a rapid decrease of the trunk inclination curve
of 40 degrees, going from 87 degrees to 47 degrees. In
includes a forward movement of the trunk since the angle
formed with the horizontal is decreasing and the trunk is
getting closer to the thigh.
Phase 3: window between 15 x 0.2 sec and 20 x 0.2
sec. The curve is still in a descending direction, but relatively
slower than the previous one. Inclination value passes from
47 degrees to 30 degrees where it attempts its minimum
value. The trunk still performs a flexion motion.
Phase 4: window between 20 x 0.2 sec and 25 x
0.2sec. A sudden increase of the curve occurs from 30
degrees to 97 degrees concluding that the trunk is moving in
the opposite direction of phases 2 and 3: it is getting further
from the thigh, and angle between the trunk and ground is
getting bigger. The trunk can be said to perform an extension
motion.
Phase 5: window between 25 x 0.2 sec and 40 x 0.2 sec.
the curve maintains a constant value of 97 degrees, which
means that the trunk is in a vertical posture relatively to the
Figure 7- ANN Algorithm ground.
Fig. 7 illustrates the adopted ANN algorithm. The study has been elaborated for the different features.
The features were fed into an artificial neuronal network.
III. RESULTS Different ANN architecture accuracy results are presented in
Fig. 8 illustrates different trunk inclinations during the table IV. The best results have been obtained for the 30
task. As seen, 5 different steps can be recognized easily. hidden layers ANN architecture.
TABLE IV- DIFFERENT ANN ARCHITECTURE AD THEIR RESULTING
ACCURACY Referred to the resulted test confusion matrix, trained inputs
ANN architecture (hidden layers) Accuracy result (%) provides a total precision of 92.3% and total error of 7.7%,
showed in the blue cell of the matrix. Precision and accuracy
5 64.1 values corresponding to each output-target class, represented
10 76.0
in green cells in the matrix, were as follow:
15 88.5 -Class-target 1: 95.0% and 93.3%.
20 89.9
-Class-target 2: 90.4% and 94.7%.
-Class-target 3: 95.8% and 88.7%.
25 90.6 -Class-target 4: 89.4 and 92.3%.
Results are illustrated in Fig.9.
30 92.8
35 92.6
25
However, the 4 curves fits perfectly the description
stated above. With area under curves of the 4 classes ranging
from, 0.887 to 0.993, this is by far the best classification the
MLP can reach. Having a maximum ration of true positive
rate over the false positive rate reaching a 0.993 AUC.
IV. CONCLUSION
We presented a full study of the STS motion based on
experiences made on 10 healthy subjects. After
understanding different factors that affect the movement and
biomechanics behind, we used 6 IMU sensors placed on the
metatarsal, shank and thigh segments of the lower limb parts,
and 1 IMU sensor on the trunk to extract and record STS
motion parameters such as the acceleration, angular velocity,
inclination and joint angles. In order to classify this data, we
used the Artificial Neural Network Toolbox in MATLAB.
By trying different architectures, we were able to each an
accuracy of 92.3% using the 30 layers architecture. Finally,
we proposed a conceptual algorithm model of the STS
motion based on analyzed results and visual observation of
different steps during the task. STS analysis has been always
an interesting study in different biomedical, biomechanics
and physical therapy. Improving our suggested method can
Figure 9- Confusion matrix using 30 hidden layers be used to study STS performance in elderly and obese
people or patients representing a Parkinson disease, impaired
postural control, etc. Higher accuracy, using ANN based
classification can be reached by increasing number of trials
per subject. Furthermore, our extracted features can be used
as a reference input to design a Patient Motion Aid
exoskeleton to assist disabled people in achieving a STS task
by their own.
V. REFERENCES
[1] Courtney, J. B. (2001). Application of Digital Image Processing to
Marker-free Analysis of Human Gait.
[2] Whittle, M. W. (May 12, 2014). Gait analysis: An introduction.
[3] P., B. (2003). Wearable sensors/systems and their impact on
biomedical engineering.
[4] Fathima S.M.H.S.S., B. R. (March 21, 2012). Human Gait
Recognition Based on Motion Analysis Including Ankle to Foot
Angle Measurement.
[5] Raymond Kai-Yu Tong, M. G. (n.d.). practical gait analysis system
Figure 10- ROC of STS classes using 30 hidden layers using gyroscopes in Medical Engineering & Physics.
[6] R. Takeda, S. T. (2009). Gait posture estimation using wearable
acceleration and gyro sensors. Journal of biomechanics.
As already stated the best ROC wished, is when the
curves starts so close to the y-axis, to reach the closer it can .
get, to the left-upper corner and then to go parallel with x-
axis to reach the closest to the right-upper angle.
26