=Paper=
{{Paper
|id=Vol-260/paper-18
|storemode=property
|title=Hand Gesture Recognition for Table-Top Interaction System
|pdfUrl=https://ceur-ws.org/Vol-260/paper18.pdf
|volume=Vol-260
|dblpUrl=https://dblp.org/rec/conf/isuvr/ParkL07
}}
==Hand Gesture Recognition for Table-Top Interaction System==
https://ceur-ws.org/Vol-260/paper18.pdf
International Symposium on Ubiquitous VR 2007 1
Hand Gesture Recognition for Table-Top
Interaction System
Sejong University, Game Interface Research Center
Hyoung Il Park and Jong Weon Lee
Abstract—Hand interface is well known for its intuition and
convenience. User can use their real hands to interact with the
system without any assistant device. In this paper, we propose a
method which uses vision-based hand interface for table-top
interaction system. To do this, first, we find skin region in
camera image. But the result might include false-regions which
are likely to skin-color, so we use region-based segmentation.
To use PCA, a method of hand gesture recognition, we have to
split detected region into hand region and arm region, then
eliminate hand yaw angle component. We demonstrated the (a) (b)
usefulness and possibilities of our method by developing 'omok'
game and testing in table-top interaction system. Figure 1. (a) The original image, (b) detected skin region
B. Detecting Hand Region from Skin Region
Index Terms—hand recognition, skin detection, table-top
interaction system, principal components analysis. After having detected the skin region, we have to separate
hand region and arm region for hand gesture recognition. We
I. INTRODUCTION base on the thickness variable of skin region. The point
where the thickness changes most is the wrist point. To
T able-top interaction system using hand interface has
been an active research topic in recent years. Its main identify the most thickness variable, we do the following
steps:
advantage is to allow users to play game or work like the way
they do in real life. Without any required devices for input 1) Finding the orientation vector, centroid point and
interface, the system becomes very convenient. To obtain this rectangle boundary of the skin region (see Fig. 2 (a)).
interface, a method of hand gesture recognition which is 2) Determining the two intersection points of the
based on detected hand region is necessary. In relation to this orientation vector with the rectangle boundary using
work, K. Oka, Y. Sato, H. Koike use fixed-size search positive orientation vector and negative orientation
window to separate arm from whole image depending on the vector (see Fig. 2 (b)).
distance from the camera to a user's hand [1]. R. Lockton, A 3) Determining the middle points of two lines connecting
and W. Fitzgibbon use a wristed band for detecting wrist the centroid and each intersection point. By our
position [2]. J. New, E. Hasanbelliu and M. Aguilar use a experimental research, the wrist position definitely lies
method that depends on hand size [3]. We propose a hand between these two middle points (see Fig. 2 (c)). This
gesture recognition method which is independent on hand can help us to save the detection time.
size or the distance between camera and hand. Our approach 4) Determining interval points in the two lines connecting
is to base on the thickness variable of skin region. the centroid with the two middle points (see Fig. 2 (d)).
5) Basing on the obtained interval points, calculating the
thickness of skin region by calculating the distance
II. HAND REGION DETECTION IN IMAGE
between two intersection points of the cross vector and
A. Skin region Detection the boundary of the skin region (see Fig. 2 (e)).
6) We could find out the most thickness variables by
To extract a skin region from the whole image which
finding the maximum difference between two
comes from camera, we use YCrCb color-space.
consecutive thicknesses.
Transformation simplicity and explicit separation of
After finding out the most thickness variable, we could draw
luminance and chrominance components of YCrCb make it
the wrist line and identify which is the hand region and
attractive to skin modeling [4]. While detecting a skin
which is the arm region by comparing the thicknesses on the
region, the result could contain some false-regions which are
two sides spitted by the wrist line. We know that hand region
likely to skin region. We use labeling method to group pixels
has bigger thicknesses than the arm region. So we could
into regions. Then we assume the largest area is the skin
eliminate the arm region which is unnecessary for hand
region (see Fig. 1).
gesture recognition (see Fig. 2(f)).
International Symposium on Ubiquitous VR 2007 2
new images.
IV. EXPERIMENT AND FUTURE WORK
A. Overview of ‘Omok’ game
The table-top interaction system consists of a display
screen on its top and a camera above which points
perpendicularly to the table’s screen (see Fig. 4 (a)). The
(a) (b)
application we developed to demonstrate the usefulness and
possibilities of our proposed method is 'Omok' game. The
game rule is that the first player who aligns 5 balls is the
winner. Players just need to point their index finger to the
expected position to make a playing step and use another
gesture for undo action (see Fig. 4 (b)).
(c) (d)
(a) (b)
Figure 4. (a)Table-top interaction system, (b) The developed application.
(e) (f)
Figure 2. (a) detected centroid point, rectangle boundary, (b) B. Future work
intersection points, (c) Middle points, (d) Interval points, (e) calculated The proposed method still requires much improvement.
thicknesses, (f) wrist line
The first problem is detecting skin region. If the background
C. Yaw Component Elimination of image includes skin-like regions which are bigger than
After obtaining two end points of the wrist line, we can hand region, then skin detection will fail. And if users wear a
define a wrist vector as a vector which goes from the left end long sleeve which covers the wrist point detection will fail
point to the right end point. Then we calculate the angle too. We will overcome these problems by improving our
between the basic vector (the unit vector of the Ox axis) and method.
the wrist vector. Using this angle, we could rotate the hand
region to compare with the trained images to identify which ACKNOWLEDGMENT
trained image is the most similar to the user hand for PCA This work was sponsored and funded by Korea Game
method. Development & Promotion Institute as a Korean government
project (The Ministry of Culture and Tourism).
REFERENCES
[1] K. Oka, Y. Sato, H. Koike, "Real-Time FingerTip Tracking and Gesture
Recognition" In Proceedings of the Automatic Face and Gesture
Recongition, pp.429-434, 2002.
[2] R. Lockton, A. W. Fitzgibbon. Real-time gesture recognition using
deterministic boosting, Preceedings of British Machine Vision
Conference, 2002.
[3] J. New, E. Hasanbelliu, and M. Aguilar, “Facilitating user interaction
(a) (b) with complex systems via hand gesture recognition”.
Figure 3. (a) The image have a yaw angle, (b) Eliminated yaw angle image ACMSE'03.Knowledge Systems Laboratory, Jacksonville State.
[4] V. Vezhnevets, V. Sazonov, A. Andreeva, “A survey on pixel-based skin
III. HAND GESTURE RECOGNITION USING PCA color detection techniques”, GRAPHICON03, pp. 85-92., 2003
[5] F.Torre and M. J. Black, "Robust principal component analysis for
PCA (Principal Components Analysis) method has been computer vision", Int. Conf. on Computer Vision, volume I, pp. 362?349,
proven useful for solving problems such as face and object 2001.
recognition, tracking, detection and background modeling
[5]. Therefore we use this method. It is separated into two
parts: an off-line and an on-line part. The off-line part is
performed in order to find the transformation matrix and
generate a classifier, all based on a set of training images.
The on-line part uses the transformation matrix and the
classifier computed off-line to transform and classify any