=Paper=
{{Paper
|id=None
|storemode=property
|title=Automatic Patient Pose Estimation Using Pressure Sensing Mattresses
|pdfUrl=https://ceur-ws.org/Vol-715/bvm2011_84.pdf
|volume=Vol-715
}}
==Automatic Patient Pose Estimation Using Pressure Sensing Mattresses==
https://ceur-ws.org/Vol-715/bvm2011_84.pdf
Automatic Patient Pose Estimation Using
Pressure Sensing Mattresses
Robert Grimm1,2 , Johann Sukkau 3 , Joachim Hornegger1 , Günther Greiner2
1
Pattern Recognition Lab, University of Erlangen-Nuremberg
2
Chair for Computer Graphics, University of Erlangen-Nuremberg
3
Siemens AG, Healthcare MR, Erlangen, Germany
robert.grimm@informatik.uni-erlangen.de
Abstract. We present a system to automatically estimate the body
pose of a reclined patient, based on measurement data from a pressure
sensing mattress. It can be used to replace or reduce manual input in
clinical imaging procedures and thus improve the workflow. The pro-
posed method consists of two stages. First, the body posture is classified
into prone, supine, and left and light lateral orientation by a k-nearest-
neighbor classifier. In the second algorithmic stage, a modified optimiza-
tion scheme based on Powell’s direction set method fits a model of the
human body to the observed pressure distribution. Thus, the position of
important body landmarks is estimated. For our database of 143 mea-
surements from 16 subjects, a mean classification rate of 96.0 % was
achieved for the posture, and an average localization error of 6.95 cm for
the body parts.
1 Introduction
Detailed knowledge about a patient’s position on an examination table is very
valuable information in many clinical workflows. For example, prior to a Mag-
netic Resonance Imaging (MRI) examination, the radiologist has to define the
center of the examination region on the patient by means of a laser crosshair.
Moreover, the fundamental alignment of the patient has to be entered into the
system, e.g. that he is in left lateral posture with the feet towards the scanner
bore. In this example, the desired localization information are the orientation
(feet first) and the posture (left lateral). Knowledge of the pose (position and
orientation of body and limbs) supersedes the laser crosshair, as it implicitly
defines the location of a given body region on patient table. A similar scenario
is the positioning of C-arm Computed Tomography (CT) scanners [1]. In this
paper, we show how these tedious steps of manual input can be made redundant
by an automatic pose estimation system.
Approaches to markerless human pose estimation or patient localization
based on optical sensors [2, 3] are, in general, not sufficiently robust since they
tolerate only limited visual occlusion of the subject. Instead, our system relies
on an array of pressure sensors that is placed between the mattress and the
patient.
410 Grimm et al.
Harada et al. [4] compared measurement data of a pressure sensing mattress
to a database of 180 templates that were pre-computed by estimating the pres-
sure distribution of a simple human body model. The dimensions of the model
have to be adjusted manually to match those of the actual subject. The limited
number of training data accounts for merely three to five different configurations
of the four degrees of freedom in their model. With today’s computing power,
larger databases are feasible, but an exhaustive database for a detailed body
model would still be too big. In later work, Harada et al. [5] described a motion
tracking system based on the physical forces exerted on the mattress surface.
However, this approach requires manual initialization of the pose of the body
model. Seo, Oh, and Lee [6] also utilized an array of pressure sensors to discrim-
inate between supine, left, right, and sitting posture of a human reclining on a
bed. Without specifying the size of the tested data set, the authors quote an
accuracy of 93.6 % for classification with a radial basis function neural network.
Unlike previous approaches, our system is fully automatic. It can reliably
classify the fundamental posture with an accuracy of 96.0 %, even in the chal-
lenging case where the knees are not bent in lateral positions. Since the sensors
are placed underneath the patient, our method is not impaired if the patient is
covered by a blanket.
The detailed body pose is analyzed using a custom synthetic body model
with 15 degrees of freedom that allows arbitrary joint angles.
2 Materials and Methods
Arrays of pressure sensors are available from several vendors and in a variety of
sizes. A popular use case is ergonomic optimization, e.g. for prostheses, seats,
or mattresses. Typically, they are used for long-term monitoring, but, as they
can be read out at a rate of 10 Hz or more, also real-time measurements are
possible. At each node with index (i, j) of the array, a sensor measures the
applied physical force F(i, j) ∈ R+0 . The pressure sensors can be calibrated to
quantify the force in metric units.
On the algorithmic side, our system is composed of two stages that first
classify the posture and then estimate the articulated body pose.
2.1 Posture Classification
Most pressure is applied to the mattress by the shoulders and hips. By local-
izing these regions of maximal pressure relative to the whole body, the patient
orientation (head first/feet first) is determined. To further distinguish between
prone, supine, and left and right lateral postures, a k-nearest-neighbor (kNN)
classification compares the measured pressure distribution to a set of labeled
training data. The measurement is assigned the same class as the majority of
the k most similar training data belong to. The similarity between two measure-
ments is computed as the sum-of-squares distance when both data are aligned
such that their center of mass coincides.
Automatic Patient Pose Estimation 411
2.2 Model-Based Pose Estimation
In the second stage, we use a 3D human body model composed from elliptic
cylinders and ellipsoids to generate a synthetic pressure distribution. In the
pose estimation process, the configuration of this model is iteratively adjusted
to maximize the similarity between the generated pressure distribution and the
measurement data. The model is augmented by additional geometry to heuris-
tically simulate points of particularly high forces on the surface, as shown in
Figure 1(a). For example, in supine posture, the weight of the legs rests mostly
on the heels. By contrast, the knees are usually clearly visible in prone posture.
For each posture, the 15 degrees of freedom in the model configuration are
described by a vector θ = (t, ϕ, s, δ) ∈ R15 . It defines the global translation
t ∈ R2 and a global rotation ϕ in the plane parallel to the sensor mattress,
the scale factor s determining the body height of the model, and a vector of
Euler angles δ ∈ R11 that indicate the joint angles in the plane parallel to the
mattress surface. A synthetic pressure distribution F̂(θ, i, j) is generated by
sampling a depth map of the model at coordinate (i, j) using OpenGL. In an
iterative process, the model configuration vector θ̂ is estimated that minimizes
the sum-of-squares difference between the generated and the observed data:
∑( )2
θ̂ = arg min F̂(θ, i, j) − F(i, j) (1)
θ
i,j
The algorithm is based on Powell’s direction set method [7] that performs con-
secutive optimizations along linear dimensions and requires no gradient compu-
tation. Our implementation performs several cycles; in each cycle, a linear search
is conducted individually for each degree of freedom. The model configurations
are generated according to a fixed, heuristically determined schedule.
2.3 Experiments
To evaluate the accuracy of our algorithms, a total of 143 different pressure dis-
tributions was acquired from 13 male and 3 female volunteers. For every subject,
at least two measurements were conducted in each of the four fundamental pos-
tures. In our experiments, we used the XSensor X3 PX100:26.64.01 mattress.
Its array of 64 × 26 capacitive sensors provides a spatial resolution of 3.175 cm.
The sensor sheet is flexible, as thin as 1 mm, and sized 203.2 cm × 81.28 cm. It
was placed on top of a camping mat and a soft blanket.
The two algorithmic stages of posture classification and pose estimation were
analyzed independently. For kNN classification, cross-validation was performed,
using, in turns, the data of four volunteers for training and testing on the rest.
For pose estimation, the data were upsampled to a resolution of 256 × 104
pixels. Five cycles of the optimization procedure, corresponding to approxi-
mately 500 tested configurations, were performed for every dataset. From the
estimated model pose, the metric location of important body landmarks was
computed and compared to the corresponding, manually labeled gold standard
coordinates.
412 Grimm et al.
Table 1. Confusion matrix of kNN clas- Left Right Prone Supine
sification results. Rows indicate the true
label. Left 74 1 0 9
Right 1 87 0 2
Prone 0 0 85 2
Supine 2 0 0 166
Table 2. Average Eu- Crest Neck Shoulder Hip Knee
clidean distance between
estimated landmark posi-
L R L R L R
tions and gold standard, Left 11.0 6.2 7.4 7.2 5.2 6.6 8.3 −
in cm. L and R denote the Right 7.6 7.7 6.1 5.8 8.0 5.2 − 8.4
left and right joints.
Prone 14.9 7.0 12.1 9.8 7.2 6.1 5.1 3.1
Supine 2.9 4.2 6.5 6.8 4.7 4.0 6.5 6.9
3 Results
The heuristic algorithm to determine the orientation of a patient succeeded in all
143 cases. For the kNN classifier (k=5), the cumulative confusion matrix from
the four cross-validation runs is shown in Table 1. The average classification
rate, ie. the percentage of correct classifications, is 96.0 %.
Figure 1(b) shows measurement data and, as an overlay, the skeleton of the
estimated model pose. The average Euclidean distances between the estimated
position of selected landmarks and the corresponding gold standard coordinates
are given in Table 2. The mean error over all landmarks in is 6.95 cm. Since
in lateral positions the opposite side of the body is not captured, no errors were
computed for the affected knees. The computation time for pose estimation in
one dataset is about 5 s with a single-threaded implementation on a 2.26 GHz
Intel Core2 Duo CPU and an ATI Mobility RadeonHD 3400 GPU.
(a) (b)
Fig. 1. (a) Models for prone and supine posture. (b) Input data (left, right, prone,
supine), overlaid with model skeleton after five pose estimation cycles.
Automatic Patient Pose Estimation 413
4 Discussion
With the sensor mattress embedded e.g. into the patient table of a Magnetic
Resonance Imaging system, the presented system automatically provides local-
ization information about the patient. This reduces manual interactions that
are required before every examination today.
The qualitative and quantitative results confirm the robustness of the pro-
posed methods to compute the orientation and posture of a patient on a bed
as well as the location of individual body parts. The reason for the lower ac-
curacy for head and shoulders in prone position is that some volunteers placed
their head on the arms, which leads to an unspecific pressure distribution in that
area. By contrast, in some small regions a lot of force is concentrated, e.g. at
the knees in prone posture and the head in supine posture. A mismatch of the
corresponding model geometry in such areas is heavily penalized by the objective
function, leading to higher accuracy.
The presented approach is limited to the localization of extremities that touch
the mattress. Incorporation of an optical camera could provide complementary
information from a viewpoint above the patient. Since currently only a single
data acquisition is used for pose estimation, another aspect subject to future
work is the extension to motion tracking over a period of time.
Acknowledgement. The authors gratefully acknowledge support for this work
provided by Siemens AG, Healthcare MR, Erlangen, Germany and Siemens Min-
dit Magnetic Resonance Ltd., Shenzhen, P.R. China.
References
1. Schaller C, Rohkohl C, Penne J, et al. Inverse C-arm positioning for interventional
procedures using real-time body part detection. Lect Notes Computer Sci. 2009; p.
549–56.
2. Moeslund TB, Hilton A, Krüger V. A survey of advances in vision-based human
motion capture and analysis. Comp Vis Imag Under. 2006;104(2-3):90–126.
3. Lee MW, Nevatia R. Human pose tracking in monocular sequence using multilevel
structured models. IEEE Trans Pat Anal Mach Intell. 2009;31(1):27–38.
4. Harada T, Mori T, Nishida Y, et al. Body parts positions and posture estimation
system based on pressure distribution image. In: Proc IEEE Int Conf Robot Autom.
vol. 2; 1999. p. 968–75.
5. Harada T, Sato T, Mori T. Human motion tracking system based on skeleton
and surface integration model using pressure sensors distribution bed. In: Proc
Workshop on Human Motion; 2000. p. 99–106.
6. Seo KH, Oh C, Lee JJ. Intelligent bed robot system: pose estimation using sensor
distribution mattress. In: Proc IEEE Int Conf Robot Biomim; 2004. p. 828–32.
7. Powell MJD. An efficient method for finding the minimum of a function of several
variables without calculating derivatives. Comput J. 1964;7:155–62.