The ability to grasp objects of di erent size and shape is one of the most important skills of a humanoid robot. Human grasping integrates a lot of di erent senses. In particular, the tactile sensing is very important for a stable grasping motion. When we lift a box without knowing what is inside, we do it carefully using our tactile and proprioceptive senses to estimate the weight and thus, the force necessary to hold and to lift this box. In this paper we present an adaptive controlling mechanism which enables a robot to grasp objects of di erent weights. Thereby, we only use the proprioceptive sensors like positions and electric current at the joints and force sensors at the end-e ectors providing the robot with tactile feedback. We implemented and tested our approach on a humanoid robot.
One of the features that made humans a very successful living being is their ability to grasp and manipulate objects. Such feature granted humans the possibility to modify and adapt the surrounding environment making it more suitable to their own needs. For such reason, grasping and manipulating objects can be seen as a strategic goal for robotics. Restraining objects is a not trivial task due to each object's geometrical and physical peculiarities.
In particular, to grasp objects of di erent weights requires di erent force to be
applied for holding as well as for lifting. It has been shown in case of humans that
the force is adjusted anticipatory for both, the grasping as well as the lifting of
the object. At this junction anticipatory means a pre-evaluation based on certain
assumptions, e.g., on experience or visual analysis of the object. However, a
wrongly estimated force can quickly be adjusted while grasping before the ngers
are slipping on the surface (cf. [
In this paper we present an implementation of an adaptive bimanual grasping
motion on a humanoid robot Nao [
These discussed methods adapt in a re ex-like manner to the respective
situation without planning more than one step in advance as well as without extensive
models or knowledge about the environment and about object to be grasped.
These local cognition methods may be embedded in existing grasping methods
and as a result, make them more robust to noise and environmental changes.
By the way we implicitly explore properties of the object to be grasped, like the
weight. This task belongs to the eld of haptics more than to robotics according
to [
The robot's grasping capabilities highly depend on the hands' mechanical
structure, its sensors and, of course, the available computational power.
Moreover, a robot has to be able to perform stable and exible motions in order
to act in a dynamic environment, moving the whole body whenever necessary.
This is especially important if an object has to be grasped with both hands.
The presented dynamic control is integrated in a complete grasping behavior as
described in [
A detailed survey about the modeling of the grasping movement is
demonstrated in [
The remainder of the paper is structured as follows. At rst we brie y outline the hardware of the robot used. Thereby, we make a particular accent on its sensing capabilities and its kinematic constraints. In the third section we present the general design of the grasping algorithm and the dynamic control. In the fourth part we show some experimental results benchmarking the control e ectiveness and we suggest some ideas where to address the further developments in the last part. 2
Nao robot is a humanoid robot produced by the French company Aldebaran Robotics and is currently used in RoboCup competitions within the Standard
Platform League [
Nao robot has a very articulate body, it is 58cm of size and weighs about 4.8kg including the battery. Each arm has four degrees of freedom describing a workspace quite similar to the human arm's one. The joints are actuated by DC motors and the platform is equipped with a low power and low consumption Geode LX 800 processor with just 500 MHz. Since the CPU processing power is quite limited compared to the robot's physical structure, it is challenging to implement very complex algorithms for motion controlling. Therefore simplicity in design is the preferred approach. Figure 1 shows the robot Nao, while standing and while grasping a ball.
Each of the joints is controlled by a PID controller. The API provides two
values for controlling each joint: target angle which should be reached and the
maximal electric current which is used to drive the joint. The last one is also
called sti ness of the joint, since it de nes how hard the joint will try to reach
the requested position. Further details can be found in [
The robot Nao is equipped with four force sensors on each foot, a gyroscope, an
accelerometer, two ultrasounds in the chest and two VGA cameras (operating
on a single bus) in the head. Each joint is equipped with sensors measuring the
actual angular position and the electric current consumed by the motor. The
camera images can be received up to 30 times per second, while all other sensor
data, like joint's positions and electric current, can be read every 10 ms. The
motion system requires a control signal at the same frame rate, i.e., every 10 ms,
to ensure the correct execution of the planned movements. The gyroscope and
the accelerometer can be used together with the feet's force sensors for inferring
robot body's posture, while ultrasounds can be used for inferring the position
of obstacles in the front of the robot. The camera provides information about
the surrounding environment that is very e ective for navigation and object
recognition. In particular, visual sensing can be used to control the high level
parts of the grasping motion, like aligning the hands around the ball, which do
not require very high reactivity. In [
A robot's grasping ability is highly dependent on its kinematic constraints. These are, among other things, determined by the reachable space of the robot's hands. The reachable space is usually de ned as the set of points that can be reached by its end e ector, e.g., the hand, with respect to a reference frame of the robot. In general, this space is de ned by some basic physical constraints that a humanoid robot has to satisfy during the motion, including the kinematic constraint (e.g., the limits of joint angles; and the collision constraint) and the balance constraint. We represent the reachable space by a three dimensional grid, thereby we consider basically the positions of the end e ector but not its rotation. Figure 2 illustrates the reachability grid for a hand of the Nao robot.
Nao's arms are equipped with four joints, two for the shoulder and two for the elbow as shown in the Figure 1 (right). Both links can be controlled along the roll, while the second joint controls the pitch and the yaw for the shoulder and the elbow respectively. The end e ectors can operate in relatively large workspaces that are partially overlapping each other. However, such freedom of movement is a further node of complexity for determining a successful grasping pose, because the same point in the space can be reached in many ways that di ers only on the end e ector orientation. To reduce complexity, we xed one joint, thereby the degrees of freedom of each arm reduced by one. We decided to x the yaw-joint of the elbow, because its xation limits the robot at least. The right grid in the Figure 2 was generated with a xed yaw-joint in the elbow. Considering the di erence between the full and the restricted grid, shown in the center of the Figure 2, it can be seen that only some positions behind the robot are lost by this restriction.
An interesting challenge to overcome on the Nao robot is dealing with the hand's structure itself. In the used version the Nao robot typically owns passive hand e ectors, which are equipped with three ngers. The passive ngers of the robot result in very irregular surfaces making the control of the object more di cult. Therefore we replace the hands by at gripping adapters, as described in 2.2, with expanded material to have friction characteristics similar to human skin. Additionally, force sensors were built into the adapters, which provide better control of the exerted force. 3
In this section we illustrate some simple controllers used for implementing a grasping motion. At rst we show the system's infrastructure and an outline of the control mechanism, on the second part we discuss three possible regulators and nally we provide some experimental data to better analyze the motion control. 3.1
The whole grasping motion can be divided in two parts: approaching the object and restraining it. The rst part of the motion is needed to bring the item in the hands' reachability space and can be further decomposed in the tasks of recognizing the object, reaching it and crouching. Once the target is reachable the core of the grasp motion is executed. At rst the hands are aligned to the object and then they are moved in order to squeeze the target. This part of the motion assumes as reference system the robot's chest as visualized in the Figure 3 (right).
The end e ectors are driven by inverse kinematic, so the same point can be reached with di erent arm con gurations. This feature makes the motion more general but introduces an extra degree of complexity due to the freedom μC pL
z x pR μC pL y of rotation of the arm. In fact, hand's orientation is coupled with elbow's orientation making impossible having a direct in uence on hand's rotation without modifying the end e ector position, as already pointed out in Section 2.3.
As a consequence, an item may be approached using di erent arm con gurations but not all of them o er a convenient grasping surface. For this reason, the elbow's rotation is xed for the duration of the entire grabbing motion, forcing the hands to touch an object always on the side with the at gripping adapters. In this way the Nao's arms can be seen as a big two ngered gripper.
Thus, we formulate the geometry of the grasping task as follows: in the rst step align the hands around the point C 2 R3, representing the center of the object, with a certain distance 2 R+. In the second step close the hands reducing the distance between the hands and the point C . Thus, the actual target positions for the hands can be calculated as points with the distance left and right from C , i.e., pL = C + e2 and pR = C e2 for the right and left hand respectively, whereas e2 = (0; 1; 0)T . The point C is controlled by vision and is used for choosing a useful spot where to grasp the object, since the hands will be placed according to it. The distance is driven by joint sensor feedback and a mapping of the end e ector force applied to the target: the smaller the bigger the force intensity. Figure 3 visualizes the geometrical con guration of the grasping. In this section we discuss an integrated controlling mechanism adapting dynamically to the weight of the grasped object. This allows for grasping and lifting objects having di erent weights. The main task here is to estimate the right force which is needed to grasp the object. The trivial solution is, of course, to take the maximum force available. However, this strategy is obviously very ine cient and may destroy fragile objects, e.g., a paper cup. Thus, our general strategy for the grasping is: as soft as possible, but as strong as necessary.
The problem to determine the force necessary for grasping is stated in [
The whole controller consists mainly of two parts: controlling of the sti ness and controlling of the distance between the hands. Both parts are designed in a way allowing for them to be considered independently.
Sti ness The sti ness is controlled for each joint separately by a P-controller. Thereby, the sti ness is locally determined to be proportional to the di erence between the requested and the measured angle of the joint. Formally, the sti ness at a joint is determined by = j ^ j C where ^ is the measured angle and the requested one. The constant C can be determined experimentally.
I.e., the sti ness of a joint is reduced to a minimum in the case if the requested angle position can be reached. However, if the joint is prevented from reaching the requested position by some external force, the sti ness is increased and the joint is working with more force against the external obstacle. Thus, each of the joints is reacting locally according to an external force.
Distance between Hands In order to control the distance between the robots hands we use a threshold controller based on the force F measured at the hands of the robot, i.e., the distance between the hands is successively reduced by a until the force F exceeds a certain threshold M . By this we ensure that a certain minimal force M is applied to the object during the grasping.
(t) = (t (t 1) 1) for F (t otherwise 1) < M Thereby, for each object with a di erent weight we need another appropriate force in order to lift it, i.e., in particular we need for each object a di erent threshold for the controller. This threshold can be estimated by the means of the controlling electric current of the joints (cf. Section 2.2). The more load is on a joint, the higher is the corresponding electric current. In particular, the pitch joints of the shoulders appeared to be the best suitable for this estimation. This is because they are the only arm joints in our grasping geometry which apply vertical force to the object. Our experiments have shown that for the used test objects a cubic dependency between the threshold M and the electric current is su cient, i.e., for the measured current IL and IR at the left and right shoulder respectively we can write
M = max (IL; IR)3 C with an experimentally determined constant C. Intuitively, this rule means that for a bigger weight of the object a larger grasping force is applied, i.e., the heavier an object, the stronger the robot is grasping.
It should be remarked that the relation between the vertical lifting force which is produced by the shoulder joints and the tangential force which is necessary to hold the object between the hands strongly depends of the friction between the hand palms and the object. 4
To study the behavior of the integrated controller III-B an isolated scenario was set up. The general robot's task is to grasp and lift an object placed in front of itself. For this challenge we use objects of di erent weights and consistencies. We chose an empty co ee cup and a full cola plastic bottle as representative objects. The cup weighs about 30g by the robot and can be crushed easily, whereas the bottle is very sturdy and weighs approximately 500g.
The robot behavior in the experiment consists of three phases: 1. sit down and stretch the arms (positioned left and right of the object); 2. clasp the hands around the object; 3. stand up with the grasped object and lifting it;
During the second and third phase the intrinsic grasp control mechanism is active. The series of photographs shown in the Figure 4 illustrates the progress of the experiment in which the robot lifts a bottle. Figure 5 visualizes some sensor values, which were recorded during the experiment with the cup respectively with the bottle. The upper graph shows the development of the force measured on the hands of the robot, the center graph illustrates the development of the controlling electric current measured at the shoulder pitch joints. For symmetric reasons, in both cases the maximum is calculated between the left and the right sensor. The vertical dashed lines separate the di erent phases of the grasping motion.
At the end of the second phase you are able to spot the rst contact with the object in the image above, exactly in this moment the force increases for the rst time. The contact with the cup occurs earlier, because it has a slightly larger radius than the bottle. In this example it can be clearly noticed how the movement is adapted to di erent forms. In the third phase, the robot tries to stand up while grasping the object. Concurrently the robot tries to lift the object slightly with its arms. Therefore the shoulder pitch joints are actuated (stretched) and we are able to measure the increasing controlling electric current. If the object cannot be lifted the control current would increase steadily, which can be observed in the middle graph of Figure 5.
The increase of the current leads to the reduction of the distance between the hands and increases the force on the object. This strengthens the connection between the object and hands. If the object is lifted the shoulder current will not continue to increase and the force on the object remains on the current state of the power in the shoulders.
This behavior can be observed very well in the top two graphs. In the case of the cup (thin line) the current in the shoulders increases slightly till the cup is lifted, whereas in the case of the bottle (thick line) the current increases more signi cantly, with the result that the force rises to about 8N and the friction between the hand and the bottle is large enough to lift it.
In this way just as much force is exerted as needed to generate the necessary friction for lifting, thereby a behavior is originated that allows a lifting of light and heavy objects with a minimum e ort. That means the robot does not spend more force than necessary. Another aspect is that in this way light and fragile items, such as a cup, are not deformed or even broken. 5
Grasping is still a hard task for a robot. As the main result of this paper an algorithm was presented which enables a robot to grasp and control objects with di erent weights. In the experimental setup the robot was able to lift a fragile cup and a comparably heavy bottle. The most remarkable aspect is the actuation of the arms while grasping the objects. By measuring the controlling 0.2 1 1 1 2 2 2 3 3 3
Time in s Time in s 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8
Time in s Fig. 5. Sensory data of the robot Nao recorded while lifting two di erent objects: a light paper cup (thin line) and a full plastic bottle of coke 0,5L (thick line). At the top the progress of the force measured at the hands of the robot can be seen, thereby the maximum of both hands is plotted. The middle plot illustrates the electric current measured at the pitch joints of the robots shoulders, again the maximum of both shoulders is visualized. At least, the bottom graph shows the measured roll-angle of the left shoulder (both shoulders are moved symmetrically, so it is enough to plot only one angle). The vertical dashed lines separate the di erent phases of the grasping motion. electric current at the shoulder joints, an estimation of the tangential force which is applied to the arms is allowed. If the arms are not actuated the electric current does not behave proportional regarding the object's weight due to the friction in the gears.
Our future research will focus on more general rules for the adaptation and for a more precise estimation of the object's weight. In particular, measurements of other body joints, e.g., the electric current of the knee, could also be incorporated in order to exploit redundancy and to archive a better estimation of the force.
Additionally, we are working on a compensation of the weight of the object by balancing the body. In order to do this the force resistive sensors in the feet of the robot could be used. Thinking ahead, we hope to enable the robot to estimate the actual weight of an object with respect to its own by exactly that kind of balancing. Last not least, it is intended to incorporate this knowledge in its own kinematic model so that as a consequence, a robot is able to walk with an object, e.g., a bottle, still compensating its weight and inertia.