Experimenting with real robots is limited by the available ressources: Complex hardware is costly, and it needs time and experience for setup and maintenance. Simulated robots can be used as alternative. Our RoboNewbie project is a basic framework for experimenting with simulated robots. It serves as an inspiration for beginners, and it provides room for many challenging experiments. The RoboNewbie agents run in the simulation environment of SimSpark RCSS, the o cial RoboCup 3D simulator, where the simulated robots are models of the humanoid Robot NAO of the French Company Aldebaran. Di erent example agents provide easily understandable interfaces to simulated sensors and e ectors of the robot as well as simple control structures. The framework has been successfully used at di erent courses where the participants needed only few hours to understand the usage of the framework and to develop own agents for di erent tasks.
Understanding grows with active commitment: to "do" something, to master it,
provides a deeper understanding. Experiencing with own experiments is of course
an important prerequisite for studies in Robotics and Arti cial Intelligence as
well. But experimenting with real robots is di cult not only because of expensive
hardware. Maintaining the robots and set ups for experiments are very time
consuming even for experienced people. Experiments at home as needed for
elearning require a deep technical understanding by the students, i.e. experiences
that they are just going to learn. So it is not surprising that simple hardware is
still broadly used in robot experiments, hardware which is far behind the recent
technical developments, not to talk about e.g. complex humanoid robots. The
collection of papers in [
Simulated robots in simulated environments can be used as an alternative
for complex hardware. The RoboCup community has more than 15 years of
experiences with real and simulated robots in the eld of soccer playing robots
[
Our RoboNewbie Project is a basic framework based on JAVA for the development of simulated humanoid robots. It provides easy understandable interfaces to simulated sensors and e ectors of the robot as well as a simple control structure. It runs in the environment of the SimSpark RCSS, thus it can but need not be used for soccer playing robots. Users can develop their own motions, e.g. for dancing, gymnastics or kicking a ball.
The RoboNewbie Project implements some kind of "minimalistic approach" with respect to Robotics. Users are able to start without special knowledge about robots. They can learn by their own experiences about the basic concepts of perception, motion, control, synchronization, and integration. All related program code in RoboNewbie is understandable from simple principles without further knowledge. That concerns the structure of the code as well as the underlying computational methods. As soon as users learn more about Robotics, they will be able to extend the programs accordingly, e.g. concerning complex motions or world modelling.
Following some hints of the reviewers, we would also like to emphasize the potential of the framework for the research on foundations. e.g., on computational models as well as on di erent problems in cognitive science. It can be useful in verifying models and in gathering large data sets for experiments in data mining.
The paper is organized as follows: After an overview about the concept and the downloadable resources of the RoboNewbie project, we give a short overview about SimSpark RCSS, and we describe the communication between the RoboNewbie agents and SimSpark RCSS. The main part of the paper in Section 5 discusses the details of the RoboNewbie framework, and the paper ends with results of practical evaluations and our conclusions. 2
The main goal of the RoboNewbie Project is to provide an uncomplicated
starting point to the programming of complex robots with minimal requirements and
pre-knowledge. The users are only supposed to have some programming
background (Java) and some technical/mathematical understanding. More knowledge
about robotics can be provided in parallel to the exercises with RoboNewbie,
e.g. in introductory tutorials (as we already did) or by e-Learning material. At
its present stage, RoboNewbie is not prepared as a complete course material on
university level like e.g. the course "Autonomous Multiagent Systems" at the
University of Texas [
The objectives behind RoboNewbie is the realization of the following requirements: { Holistic view on robots: For beginners and especially for pupils in schools, it is more appealing to see a robot behave like a human than to test and calibrate the behavior of a sensor. Of course, when dealing with more complex tasks, users will experience the need to have better knowledge about the usage of sensors and actuators, and then they may draw their own conclusions. { Motivating scenario: Application elds from daily life with known properties and rules are well suited. Robots which imitate human skills are especially motivating. { Scalable tasks: Unexperienced users should have no di culties to perform rst steps with own experiments and later move to more complex tasks with unlimited challenges. { Low requirements: The usability would be restricted if peoples need to have education on Robotics or if they are supposed to have deep knowledge in hardware and software. To be usable at schools, basic programming skills and interests in mathematics and natural sciences should be su cient. { Low costs: The costs of a learning system include money and e orts for purchase, set up, and maintenance, respectively. They should be as low as possible to permit a broad usage.
The users of the RoboNewbie project can nd all materials on the web page
of Berlin United { Nao Team Humboldt [
SimSpark RCSS is developed and used by the RoboCup community in the 3D
simulation league. SimSpark is a generic physical multi agent simulator system
for agents in three-dimensional environments. It uses the Open Dynamics Engine
(ODE [
The Simulator SimSpark RCSS consists of two programs (server for simulation and monitor for visualization and interaction) and con guration les. It models a soccer eld with the player bodies (adapted from the robot hardware of Nao) and the ball. It also controls the rules of the soccer game, i.e. it controls the game according to the decisions of a referee.
SimSpark RCSS can be used as open source software. This was also an
important criteria for its usage. It can be downloaded from [
The SimSpark RCSS project itself is constantly evolving according to the progress in the RoboCup initiative. The version (compiled in June 2012) on the RoboNewbie web pages serves for stable usage and avoids potential incompatibility problems by new RoboCup versions.
SimSpark RCSS is documented in a Wiki [
To provide an easy access, the down loads of the RoboNewbie project contain an introduction to SimSpark RCSS which refers to the provided version (as described above). It gives the user an overview about { Simulation using SimSpark RCSS: The SoccerServer and the Monitor. { The Nao-Model used by SimSpark RCSS. { Communication between Agents and SimSpark RCSS (with explanations of the message formats as background information). { Synchronization between SimSpark RCSS and the Agents. { Monitor and User Interface.
{ Running a Game.
Actually, our description of SimSpark RCSS provides also some "background" information which is not needed for beginners, e.g. details about the message formats. Since RoboNewbie permits an easy and direct access to the items of messages like sensor values and motor commands, the syntax of messages must not be known by users. Nevertheless, we have included the information for deeper understanding of RoboNewbie in case of interest.
SimSpark RCSS implements the soccer environment including the bodies of the Nao robots. It models all physical interactions between players, ball and environment. The agents implement the control of the players.
The interface between the physical environment and the control of real robots is constituted by sensors and actuators: Robots perceive the world by sensory data (e.g. by vision, accelerometer, force sensors etc.), and in uence the world by their actuators (motors, voice etc.).
In simulation, the sensory data are calculated by the simulator according to the situation in the simulated world (e.g. observable objects) and sent via message exchange to the agent. Then, like a real robot, the agent can update its belief about the situation and decide for actions it wants to perform. A real robot would then activate its actuators (e.g. motors at the joints) to perform the intended actions. In simulation, the agent communicates with SimSpark RCSS again by messages which transmit the actuator commands. Both are synchronized by a communication cycle of 20 milliseconds.
In SimSpark RCSS, the message transfer is optimized for minimizing the server load: All sensory data are packed in one server message to be sent at the beginning of a communication cycle. Vice versa, the agent can send all action commands by a single agent message before the end of a cycle. The message formats follow a special syntactic scheme based on symbolic expressions (Sexpressions). As a consequence of collecting data into one message, the preparation of the data in an agent needs more e orts than in a real robot. It is a special feature of the RoboNewbie agent that this preparation is hidden from the user: The agent provides special getter- and setter-methods which allow the access to the sensor (perceptor) data and the setting of actuator (e ector) commands in a similar way as in a real robot.
The interaction between the server and the agent works as follows: 1. At the beginning of a cycle at a time t, the server sends individual server messages with sensations to the agents. 2. During this cycle, the agents can decide for new actions depending on their beliefs about the situation. 3. Before the end of this cycle, the agents should send their agent messages to the server for desired actions. 4. The server collects the agents messages and calculates the resulting new situation (poses and locations of the players, ball movement etc.) according to the laws of physics and the rules of the game. This is done during the following cycle at time t+1. (Note that the server message sent at the beginning of this cycle regards the situation calculated in the previous cycle at time t). 5. At the beginning of the subsequent cycle, at time t+2, the sensor data in the server message is based on the e ects of the actions at time t+1 which were chosen by the agent according the information from time t.
A special feature of SimSpark RCSS is the use of so-called perceptors instead of sensors. The perceptor data can be regarded as already pre-processed sensor data. For example, the image data from the camera are not presented by a pixel matrix. Instead, the vision perceptor sends a collection of observable objects with egocentric coordinates relatively to the camera of the observing agent. In a similar way, actions commands of the agent are encoded as so-called e ector values and sent to the server which translates them to motor control commands. The calculation of perceptor values and the interpretation of e ector values are part of the simulator, too. On the agent side, a server message has to be parsed for the contained perceptor values, and the action commands have to be collected to the agent message. Both constitute a signi cant burden for a beginner while it provides only few insights to robotics. The RoboNewbie users need not to care about that, because the RoboNewbie agent does all this work in the background.
Besides some e ectors related to initial connection with SimSpark RCSS, there are Hinge Joint E ectors for each of the 22 hinge joints and a Say Perceptor (as of a loudspeaker with limited capacity). The following perceptors are available in SimSpark RCSS (for details see the Wiki or our SimSpark description): { Vision Perceptor (as of a camera in the center of the head). { Hinge Joint Perceptors at each of the 22 hinge joints. { Accelerometer in the centre of the torso. { GyroRate Perceptor in the centre of the torso. { Force Resistance Perceptor at each foot. { Hear Perceptor (as of a directed microphone with limited capacity). { Game State Perceptor (reporting the actual game state of the soccer match). 5
The RoboNewbie framework o ers a comfortable interface for agents interacting with SimSpark RCSS. It includes sample agents which illustrate basic concepts and methods of Robotics and Arti cial Intelligence. Users can start exercises with these agents and learn how to use RoboNewbie and what the programming of robots is like. They can make their own experiences with di erent topics and algorithm by modi cations and extensions.
It was a main goal of the project, to provide easily understandable concepts, methods and programs. There are no complicated structures, and all code is documented in detail. As a consequence, some more demanding concepts were replaced by simplier approaches (e.g. keyframe motions instead of inverse kinematics, approximated coordinates of observed objects etc.). Nevertheless, the clear structure of the project supports extensions for more challenging solutions if wanted. 5.1
The framework includes interface functionalities on two levels. The lower one
corresponds to the hardware-near functionalities of robots, while the higher one
is concerned with more abstract control functionalities. Especially for the lower
level, parts of the code of the team magmaO enburg [
The hardware-near layer encapsulates the network protocol for interaction with SimSpark RCSS and it allows access to the simulated hardware entities corresponding to sensors and motors. The access is implemented by getter functions for perceptor values of di erent perceptors which can be used similar to sensor signal queries of real robots. Related setter functions for e ector values can be used for the control of actuators.
Especially the low level interface functionalities for SimSpark RCSS are a hurdle for beginners and need time consuming work even for experienced users. They concern tasks like network connection, synchronisation with the server, parsing of nested server messages, syntactical analysis of S-expressions, synthesis of agent messages with a lot of technical non-robotics details. The users of RoboNewbie need not to care about all this details, the framework o ers ergonomic methods for the interaction with the simulated environment in an easy understandable way similar to the methods used by the operating systems of real robots. Users can learn to use these methods after a short training time (cf. the evaluation in Section 6).
The synchronization protocol was already described in Section 4. The user needs not to care about the communication, except the delays by the protocol and the duration of the cycles given by 20 msec. It is necessary to fetch a server message at each cycle and to send the agent message before the end of the cycle. The related control structures are already implemented in the examples and explained by the tutorial. Hence, if the calculations during one cycle do not exceed the cycle time, there will be no problem. The needed time depends of course on the used computer, the example agents run without problems even on less powerful machines.
The rst example "Agent BasicStructure" in the tutorial let the users start with an agent which already implements all low level communication. The agent simply rises an arm by setting related e ector values. The user can experiment with other values and other e ectors just to understand the basic structures. 5.2
The available perceptors were already listet in Section 4. All perceptor values can be queried by related getter methods using the perceptor names instead of the acronyms of the server messages. This allows a comfortable access to the perceptor data which corresponds to the access of sensor values by a related operating system of a real robot.
RoboNewbie has already implemented the necessary conversion from the
nested server messages to the perceptor values. For that, the server message
are parsed for the constituents of a tree like structure (again, thanks to the
code of the team magmaO enburg [
The programs "Agent TestPerceptorInput" and "Agent TestLocalFieldView" illustrate the usage of the related getter methods and the perceptor values. The examples serve also as an illustration to the usage of the logger functions described in Subsection 5.5. As an exercise of the tutorial, the user can implement an agent, which lifts the robots arm, when it senses another robot and moves the arm down, when it does not sense any robot. Which arm is lifted should depend on the side where the other robot is seen.
Special e orts are needed for the vision perceptor. It provides coordinates of all objects in the vision range of the camera. SimSpark RCCS in its common version does not communicate image data. Instead, the communicated information can be understood as the result of basic image interpretation, it contains coordinates of the goal posts, the lines, the ball, and the body parts of robots.
The vision perceptor provides values by egocentric coordinates relatively to the camera in the centre of the head. Further calculations are necessary to get the coordinates of objects relatively to the body of the robot. Accurate calculations would need the inspection of the cinematic chain. The necessary data are available by the hinge joint perceptors. More calculations including self localization are necessary for the transformation into alocentric coordinates. RoboNewbie does not provide related programs following the intended "minimalistic" approach, because they would not be understandable by beginners without preknowledge about Robotics. Instead, the implementation of related methods can serve as exercises during courses in Robotics.
As a simple substitute, we have decided to provide only approximations for the conversion from camera coordinates to robot coordinates. They are documented in the sources and easily to understand. Users can make experiments according to the accuracy and draw own conclusions on cinematic relations.
Visual information is provided by SimSpark RCSS only at each third cycle, and the robot would have to act blindly in between when there are no vision data available. Hence, the vision information should be stored for the following cycles. Moreover, the vision perceptor is limited by the camera view range of 120 degrees horizontally and vertically. The robot has to move its head to observe more objects in the world. Again it is useful to store objects seen before in other directions. In general, such updating and memorizing of observations is maintained as belief of the robot in a so called world model. Updates may regard corrections according to robot motion, guesses for movements of invisible objects and integration of information communicated by other robots.
Again, a fully elaborated world model is far behind the scope of beginners. Hence, RoboNewbie provides a very simple version, where just the observed objects are stored in a simple form. The coordinates of those objects are referenced with respect to the robots coordinates. Turnings of the head are already regarded by RoboNewbie, but only by the approximate calculations as described above. Other movements of the robot like turning or walking are not regarded. Time stamps indicate the last time of observing an object. The example "Agent TestLocalFieldView" illustrating the perception features of RoboNewbie is provided for the users.
All intentional motions are performed by controlling the hinge joints by sending e ector values (speed of motors) to SimSpark RCSS. Then the physical simulation engine calculates the e ects of the commands regarding physical laws and updates the simulated world accordingly.
Simple motions like turning the head or rising the arms can be easily programmed by the users as in the already mentioned examples. The motions can be controlled using the feedback of hinge joint perceptors. i.e. by sensor-actor coupling, where the delay of observing an action has to be regarded as described in Section 4. There is much room for own experiments of users.
More complicated motions like walking need coordinated movements of different joints, users may learn about these problems after some trials. We have decided to provide keyframe motions in RoboNewbie because they are easily to understand and to design. The interpolation mechanism for keyframe motions in RoboNewbie realizes a linear interpolation - users may implement other interpolation methods like splines if they want. Keyframes are stored as text les which can be edited by any text processing system. Therewith, users could even design and change motions while using the programs as a blackbox.
RoboNewbie comes with a set of prede ned keyframe motions for walking, turning, stand up and others. Users can change these motions (by changing the related text les). New motions need an integration into the program "keyframeMotion", details are explained in the tutorial and the source code documentation.
According to simplicity, there are no concepts implemented for interruption of motions: Each motion is performed completely until its end, and there are no cyclic motions, e.g. for walking. Instead, continuous walking can be performed by subsequent calls of a two-step-walk.
The design of keyframe motions is supported by a graphical Motion Editor. It can be downloaded from the RoboNewbie Web page as well. It shows the postures of the robot for selected keyframes. Then the keyframes can be edited in two ways. In the graphical representation the posture can be kneaded into the desired posture with the mouse. Alternatively, each joint angle can be set to speci ed values which are immediately presented by the graphics. Transitions between keyframes can be de ned with speci c transition times resulting in a keyframe sequence as usual.
The program "agentKeyframeDeveloper" helps in designing keyframes. A robot performs the motion of the actually edited keyframe le. After each change, the new motion is performed immediately. If the robot falls down during such a motion, it stands up by itself. Another helpful program can be used to mirror keyframes from one side to the other.
The example "agentSimpleWalkToBall" illustrates the motion concepts. As an exercise of the tutorial, the users can change that program to implement obstacle avoidance (walk around the ball without touching it). They can use motions for walk, stop and turn. Additionally, the agent must be able to recognize the ball and to decide for the appropriate motion according to the ball position. Another exercise is the design of a new motion for kicking the ball. Users can furthermore do their own experiments e.g. with dancing robots.
In general, keyframe motions are useful for special motions like standing up, but they are not so well suited e.g. for walking. Walking is still a challenging problem in Robotics. The users of RoboNewbie will get some understanding about the task. Moreover, the framework is well suited as a basis for other implementations and for Machine Learning by more educated users. But according to our "minimalistic" approach, related implementations are not provided. 5.4
The basic control cycle follows the classical deliberation approach, often denoted as the "sense{think{act{cycle", or by related similar names. This corresponds closely to the cycle given by SimSpark RCSS: At rst, sensations are provided to the agent, then the agent decides for appropriate plans and then it sends the related action commands back to the server.
Critical remarks may come from the community of Embodied Robotics/AI, e.g. concerning the centralistic and symbolic computations in the classical approach. To realize concepts of Embodied Robotics/AI one needs to put more emphasis on local sensor actor coupling, distributed control, embodiment, situatedness, emergent behaviour etc. The real robot Nao as well as its simulated counterpart with the central control (i.e. our agent) are not primarily designed for such purposes. It is possible to design sensor actor couplings and other behavioural concepts in the RoboNewbie framework, too. One might even split the agent into di erent "parallel" acting parts (implemented e.g. by threads) to simulate distributed controls, but some synchronization is unavoidable by the server cycles of SimSpark RCSS.
At the same time, thinking in terms of the "sense{think{act{cycle" is quite natural for beginners because it re ects some causal dependencies. It provides an intuitive and easily maintainable structure in the design of robots. Therefore, the control cycle in RoboNewbie adopts the related terms for structuring the run-methods of the agents by cyclic calls of methods sense, think and act. The think-method is sometimes omitted in case of simpler ("reactive") agents.
The sense method is reponsible for receiving and processing the perceptor data by the related RoboNewbie methods. The act methods calls the transfer of the agent message with the e ector commands. What is left is the further analysis of the perceptor data (e.g. a more elaborated world model) and the decision for plans and actions to be performed by the robot now and possibly in the future. By the given structure of RoboNewbie, all this can be included in the think method. The think method can of course be split into more dedicated deliberation methods which may be organized hierarchically if needed. Again, all this is left to exercises during related courses. RoboNewbie provides just a simple example for illustration, the Agent SimpleSoccer.
The Agent SimpleSoccer is able to perform a very simple soccer play: As long as it is behind the ball and sees the opponent goal, it walks forward while pushing the ball with its feets. If the condition is not ful lled, it turns around until it sees the ball, walks to the ball, turns around the ball until it sees the opponents goal, and then it starts walking towards the goal again. The decisions are made by a simple decision tree whenever the previous motion is completed (note that motions can not be interrupted as described above).
It is obvious, that the play of Agent SimpleSoccer can be improved in many ways. This is just what we want: The users can collect many ideas for improvements. Improvements may concern better usage of perception (e.g. by a ball model guiding the search), improved motions (like faster walk), new motions (like kick or dribble), better control (like path planning). It is also possible to have more players on the soccer eld such that players can cooperate (e.g. by positioning and passing). This gives room for simple contests during a course. 5.5
Runtime debugging of programs may be di cult because it a ects
synchronization with the server. Even simple debug messages printed on System.out may
need too much time such that the agent cannot respond in time. It is possible
to use the so-called sync mode which lets SimSpark RCSS wait until all agents
have sent their messages (see [
We have tested the RoboNewbie framework at di erent places. It was used at introductory Robotics courses of about 30 hours during 5-8 days at Ohrid, Warsaw, Novi Sad, and Rijeka, respectively. 20 hours were planned for lectures, 10 hours for introduction and rst usages of RoboNewbie. Additional 10-20 hours were used for further experiments by homework.
RoboNewbie served for illustrating experiments and for exercises in connection with the theoretical instructions. The participants of the courses learned to use RoboNewbie during short time and they programmed an improved soccer player at the end. The work with RoboNewbie was helpful to understand the theory, and the nal evaluation of the courses by the participants resulted in high marks. Especially the competitions with the improved soccer agent at the end of the courses were motivating.
This was also the case with the participants of a Robotics course at our university, where the students had more time (two months) for their studies and exercises. Students used the time to implement more sophisticated methods and to try out changes of the framework itself (e.g. other interpolation methods for keyframes). But it also turned out, that e orts for more sophisticated controls are limited by the available skills, especially for motions.
In contrast to other experiments in Robotics, the RoboNewbie framework can be used without special hardware. It simply needs a computer for simulation of the robot soccer scenario, which is more complex than experiments by many hardware equipments. It is easy to understand and to use after a short introduction. No special knowledge (except basic programming in Java) is required to start with own experiments, and while the users acquire more knowledge, they can work on more challenging tasks.
The "minimalistic approach" is useful especially for short courses and for introductions to longer courses. Later on, the disposability of non-minimalistic more sophisticated methods could be useful for higher level integrative tasks. It is impossible to let students implement all desirable algorithms in the limited time of a course. Joint activities of robots, for example, depend heavily on the available bodily skills and on the capabilities for interaction and coordination.
The practical evaluations have con rmed our expectations on the RoboNewbie project. Beginners in Robotics were able to use the framework after short introductions. They were able to program own methods in parallel to the theoretical concepts and methods provided by classes. Participants have attested the usefulness of own experiences (which again corresponds to our expectations).
Next plans concern the usage of the RoboNewbie framework in Secondary Schools, and the integration into an e-Learning course on Robotics.
We are thankful to the whole RoboCup community, especially to the developers of SimSpark RCSS, to the team magmaO enburg and to our team NaoTeam Humboldt, and especially to Yuan Xu.