-Hand interface is well known for its intuition and convenience. User can use their real hands to interact with the system without any assistant device. In this paper, we propose a method which uses vision-based hand interface for table-top interaction system. To do this, first, we find skin region in camera image. But the result might include false-regions which are likely to skin-color, so we use region-based segmentation. To use PCA, a method of hand gesture recognition, we have to split detected region into hand region and arm region, then eliminate hand yaw angle component. We demonstrated the usefulness and possibilities of our method by developing 'omok' game and testing in table-top interaction system.
Index Terms—hand recognition, skin detection, table-top interaction system, principal components analysis.
Tabbeelen-taonp aicnttievreacrteisoenarscyhstteompicuisninrgecheannt dyeianrtse.rfIatscemhaains
advantage is to allow users to play game or work like the way
they do in real life. Without any required devices for input
interface, the system becomes very convenient. To obtain this
interface, a method of hand gesture recognition which is
based on detected hand region is necessary. In relation to this
work, K. Oka, Y. Sato, H. Koike use fixed-size search
window to separate arm from whole image depending on the
distance from the camera to a user's hand [
To extract a skin region from the whole image which
comes from camera, we use YCrCb color-space.
Transformation simplicity and explicit separation of
luminance and chrominance components of YCrCb make it
attractive to skin modeling [
After having detected the skin region, we have to separate hand region and arm region for hand gesture recognition. We base on the thickness variable of skin region. The point where the thickness changes most is the wrist point. To identify the most thickness variable, we do the following steps: 1) Finding the orientation vector, centroid point and rectangle boundary of the skin region (see Fig. 2 (a)). 2) Determining the two intersection points of the orientation vector with the rectangle boundary using positive orientation vector and negative orientation vector (see Fig. 2 (b)). 3) Determining the middle points of two lines connecting the centroid and each intersection point. By our experimental research, the wrist position definitely lies between these two middle points (see Fig. 2 (c)). This can help us to save the detection time. 4) Determining interval points in the two lines connecting the centroid with the two middle points (see Fig. 2 (d)). 5) Basing on the obtained interval points, calculating the thickness of skin region by calculating the distance between two intersection points of the cross vector and the boundary of the skin region (see Fig. 2 (e)). 6) We could find out the most thickness variables by finding the maximum difference between two consecutive thicknesses.
After finding out the most thickness variable, we could draw the wrist line and identify which is the hand region and which is the arm region by comparing the thicknesses on the two sides spitted by the wrist line. We know that hand region has bigger thicknesses than the arm region. So we could eliminate the arm region which is unnecessary for hand gesture recognition (see Fig. 2(f)).
After obtaining two end points of the wrist line, we can define a wrist vector as a vector which goes from the left end point to the right end point. Then we calculate the angle between the basic vector (the unit vector of the Ox axis) and the wrist vector. Using this angle, we could rotate the hand region to compare with the trained images to identify which trained image is the most similar to the user hand for PCA method.
(a) (b) Figure 3. (a) The image have a yaw angle, (b) Eliminated yaw angle image
III. HAND GESTURE RECOGNITION USING PCA
PCA (Principal Components Analysis) method has been
proven useful for solving problems such as face and object
recognition, tracking, detection and background modeling
[
The table-top interaction system consists of a display screen on its top and a camera above which points perpendicularly to the table’s screen (see Fig. 4 (a)). The application we developed to demonstrate the usefulness and possibilities of our proposed method is 'Omok' game. The game rule is that the first player who aligns 5 balls is the winner. Players just need to point their index finger to the expected position to make a playing step and use another gesture for undo action (see Fig. 4 (b)).
The proposed method still requires much improvement. The first problem is detecting skin region. If the background of image includes skin-like regions which are bigger than hand region, then skin detection will fail. And if users wear a long sleeve which covers the wrist point detection will fail too. We will overcome these problems by improving our method.