Semantic Human 3D Shapes Annotation for Animation F. Dellas, L. Moccozet, N. Magnenat-Thalmann, G. Patanè, M. Mortara, M. Spagnuolo, and B. Falcidieno associated to this default shape and is used to animate the Abstract— The problem of identifying particular points or areas Virtual Human and adjust the 3D default shape to animated on 3D meshes is closely related to several outcomes in computer postures [1, 2]. Extracting an accurate skeleton is a tedious graphics: when animating virtual characters the animator must task. We propose to use multiscale shape analysis tools in first identify which part of the 3D envelop can be assigned to which corresponding part of the animated skeleton, to obtain a order to automate the process. visually coherent animated shape. Every shape has to be segmented to be usable after while. For instance, the CAESAR body database has been built using 3D scans plus a set of landmarks to identify body measurements. Unfortunately this is not the case when acquiring scanned data in general, and particularly for human scanned bodies. We demonstrate that it is possible to get rid of noisy and complex data so as to extract from any human body closed mesh its skeleton of animation. Assuming the joints are located where the shape has the more variations and based on a multi-scale analysis, we are able to deduce main joints positions. We also build a control skeleton and label automatically all detected joints relying on a priori knowledge on human anatomy, independently from body postures. We demonstrate our approach with several examples. Index Terms—Semantic Annotation, 3D Shape Analysis, Computer Animation, Virtual Human. I. INTRODUCTION A skeleton for animation is a hierarchical structure composed of joints and segments linking them. A joint stands for a possible degree of freedom of the Figure 1: Detected areas on a scanned body at all scales; in the last picture, the centroids corresponding to these regions are located by the circles and they creature’s shape. Each joint is considered as a local coordinate globally correspond to the anatomical joints. system and its position in the world coordinate system depends on its parent joint position. The root of the hierarchy II. BODY SHAPE ANALYSIS is the basin center for a human. This structure is necessary to compute efficiently the shape deformation during animation, The Tailor algorithm [3] detects different kind of features or to evaluate body measurements and meta-data. For a (sharp protrusions, wells, dips, and branching parts) at human, we can consider a joint for each anatomical joint and different scales on a given shape (c.f. figure 1). Vertices are the segments are the bones linking the different joints. In a labeled with different colors according to the feature they are standard animation pipeline, we consider around 20 degrees of associated to. We performed tests on a set of bodies with freedom (i.e., joints). Virtual Humans are traditionally different corpulence. For each labeled area of vertices, its produced from a single 3D shape representing the body shape centroid is defined as the mean of triangles barycenters. We in a default posture. An articulated animation skeleton is computed them across the different scales and filtered these features points according to their position invariance. We did experiments on the average centroids displacement across Manuscript received October 9, 2001. This work was supported by the scales and noticed it stays approximately located at a same AIM@SHAPE European Network of Excellence (www.aimatshape.net) F. Dellas, L. Moccozet, and N. M-Thalmann are with MIRAlab at the position considering a set of bodies, regardless of fatness, size University of Geneva, Battelle, Route de Drize, 7, 1227 Carouge, Switzerland and morphology, which confirm our hypothesis: the anatomic (phone: +41 22 379 76 19; email: moccozet@miralab.unige.ch). joints of a human on a scanned mesh correspond to features G. Patanè, M. Mortara, M. Spagnuolo, and B. Falcidieno are with IMATI CNR, Via de Marini, 6, 16149 Genova, Italy. that can be detected by Tailor algorithm. Some of the detected points do not correspond to real joints: we can filter them by taking into account only the blends and concave features. The with Tailor onto tubular and non-tubular segments to extract Plumber algorithm [4], which builds on the vertex potential location of joints. These results are combined with a classification provided by Tailor, identifies tubular features priori knowledge on human anatomy in order to select the best together with their axes (cf figure 2). candidates. The first segmentation provides arms and legs, which are tubular. Each part contains only 2 joints: in the case of arms, we have bends and wrists. In the case of legs, we have ankles and knees. Tailor centroids are then filtered in order to extract for each limb the 2 best candidates corresponding to the joint location. For the trunk, Tailor allows extracting the spine curve. This curve is used to estimate the height of the shoulders and hip joints. Extracted landmarks are further projected onto the axes of the Plumber tubes in order to estimate the joints locations. Figure 2: Tubular features with their axes. IV. RESULTS III. SKELETON EXTRACTION In order to correct possible incoherencies due to bad scan Our strategy relies on the observation that the shape data, we use the symmetry property: since the human skeleton variations match areas corresponding to skeletal joints. is symmetric, the lengths of symmetric bones segments should Intuitively, we expect that the areas where we detect small be the same. We also use anatomic proportions, according to variations lay in the same location with bigger variations. statistical data: proportions between long bones of the Since the skin is thinner where we have natural articulations, skeleton are very stable with a negligible variation. We it produces small concavities and variations at small scales. present in figure 5 three different scanned bodies with There are also less fat tissues and no muscles around joints, so different morphotypes. it induces variations at bigger scales among body parts[5]. The experiments on various morphotypes demonstrate that the resulting method is able to handle noisy 3D input data for variable 3D body shapes and postures and to extract with a reasonable accuracy the main 16 joints and the 5 tip joints corresponding to a simple H-Anim [6] hierarchy (c.f. figure 3- e). (a) (b) Figure 4: Skeleton extraction for different morphotypes. REFERENCES [1] N. Magnenat-Thalmann and D. Thalmann. Handbook of Virtual Humans, ed. Wiley, 2004. [2] J. P. Lewis, Matt Cordner, and Nickson Fong, Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation, In Proc. SIGGRAPH 200, pp. 165 - 172, 2000. [3] L. Moccozet, F. Dellas, N. Magnenat-Thalmann, S. Biasotti, M. Mortara, B. Falcidieno, P. Min and R. Veltkamp, Animatable Human Body Model Reconstruction from 3D Scan Data using Templates, In CapTech Workshop on Modelling and Motion Capture Techniques for Virtual Environments, Zermatt, Switzerland. December 2004. (c) (d) (e) [4] M. Mortara, G. Patane, M. Spagnuolo, B. Falcidieno, and J. Rossignac, Figure 3: (a) Landmarks extracted from the 2 steps shape analysis; (b) tubular Blowing bubbles for multi-scale analysis and decomposition of triangle limbs axes and spine curve; (c) estimated hip and shoulder joints location from meshes, In Algorithmica (2003), Vol. 38, pp. 227–248. (a) and (b) and (d) final joints estimation corresponding to H-ANIM joints in [5] M. Mortara, G. Patane, M. Spagnuolo, B. Falcidieno, and J. Rossignac, (e). Plumber: A method for a multi-scale decomposition of 3d shapes into tubular primitives and bodies, ACM Symposium on Solid Modeling Our proposed approach is based on two main steps: 1) an 2004, ACM Press, 2004. [6] Henry gray, anatomy of the human body 1918. initial segmentation is applied on 3D scanned data with http://www.bartleby.com/107/. Plumber to extract tubular (limbs) and non-tubular parts of the [7] H-anim specification. http://www.h-anim.org/. body (trunk); 2) a multi-scale shape analysis is further applied