FinE-R 2015
Along the history, there are many important discoveries that resulted from long trials and error processes (e.g. the electric light bulb from Edison) or from analysing 'failed' results (e.g. the Michelson-Morley experiment). In each case, the key contributor for the final success was the willingness to learn from previous mistakes and to share the gained experience with the research community.
The path to progress in the field of robotics is not free of failures and caveats. These failures provide valuable lessons and insights on future approaches by analysing errors and finding methods to avoid them. As such, the robotics community could benefit from the experience of those who had faced and overcome similar failures before.
The objective of this workshop is to provide a forum for researchers to share their personal experiences on their "failure to success" stories, to present what they have learnt, what others should avoid while experimenting in similar context, providing tips for better research practices and for creating more successful robots that meet people's expectations.
Topics of interest (but not limited to)
When failure is not an option: creating an outstanding robot from HW to SW
Matching the expectations and needs of industries and consumers with the current technology
INVI TED KEYNOTE SPE AKER S
Ryad Chellali Nanjing Robotics Institute- CEECS Nanjing Tech University, Nanjing, China Lessons Learned in Human Movements and Behavior Analysis Human Robots Interactions (HRI) is a hybrid filed mixing many domains including engineering, physics, social sciences, neurosciences, etc. Accordingly, research in HRI combines exact models and experimental approaches towards developing interaction frameworks and friendly robots. The inherent heterogeneity leads in many cases to ill-posed problems with oversimplification on one side (the engineering side) and forced (thus inexact) models on the other side. Indeed and for the "R" part in HRI, measurements and procedures are known to be exact and objective. Models in these fields are enough known allowing quantitative accurate observations that can be measured repeatedly supporting the original models. In psychology and social sciences the situation is different: the object of studies, namely humans, is much less known. Scientists in these areas are lacking in terms of accurate models compared physicists and engineers. Indeed, humans can be seen as high dimension multivariate systems, with complex dynamics, preventing from having complete explanatory models. This leads to qualitative approaches, where only isolated aspects (and most of the time related indirectly to the object of investigations) are considered. This situation is even worst when experiments are performed in real life conditions. Indeed, to obtain realistic observations, experiments in real world are needed; unfortunately, the control of experimental conditions is almost impossible out of laboratories, leading to higher difficulty and complexity in analysis and understanding.
Through our previous works, we found out that one should consider carefully modelling human behaviour. In analysing human gestures for instance, our system used to work well in lab conditions but failed completely in real situations. We imposed a model that cannot handle the variability both intra-personal and inter-personal. Likewise, in studying relationships between body movements and human physiology, we found out a new phenomenon that was not considered in our original model. Our conclusion (which is obvious a posteriori) is that model driven approaches are useless in human behaviour analysis. Instead, data driven techniques (unsupervised, latent modelling, etc.) seem to be more effective. Indeed, in both previous cases we achieved better results by removing modelling constraints and by using statistical tools extracting weakly hypothesized regularity.
Ryad Chellali is a distinguished professor at Nanjing Tech University, Nanjing China since 2015. From 2005 to 2015, he was senior research scientist at the Italian Institute of Technology. He created and leaded the Human Robots Mediated Interactions Group (2006-2011) and then joined the Department of Pattern Analysis and Computer Vision (PAVIS, 2011-2015). From 1995 to 2006, he was with Ecole des Mines de Nantes/CNRS (France), heading the automatic control chair. From 1993 to 1995 he was assistant professor at University of Paris. He served as Junior Researcher in 1992 at the French Institute of Transports (INRETS). He obtained his Ph.D. in Robotics from University of Paris in 1993 and his Dr. Sc from University of Nantes (France) in 2005. His main research interests include robotics, human-robot interactions, human behavior analysis (social signal processing and affective computing). Telepresence, virtual and augmented realities are also keywords of his activity.
Ryad Chellali co-authored more than 100 papers. In 2000 and 2005 the French Government awarded him for the creation of innovative technologies companies.
Laurence DeVillers LIMSI-CNRS, Sorbonne University, France laurence.devillers@limsi.fr The communication accommodation of the machine to deal with errors in Human
Talk during social interactions naturally involves the exchange of propositional content but also and perhaps more importantly the expression of interpersonal relationships, as well as displays of emotion, affect, interest, etc. Such social interaction requires that the robot has the ability to detect, interpret the social language and represent some complex human social behaviour. Cognitive decisions will be used for reasoning on the strategy of the dialog and deciding social behaviours (humour, compassion, white lies, etc.) taking into account the user profile and contextual information. The research challenges also include the evaluation of such systems and the various metrics that could be used like the measure of social engagement with the user. Engagement in dialog with a machine is not only linked to the error rates. We argue that the communication accommodation theory is a promising paradigm to globally consider the errors in the convergence or divergence dimensions.
Laurence DeVillers is a Professor of Affective Computing at Paris-Sorbonne University and she leads a team of research on "Affective and Social Dimensions of Spoken Interactions" at the CNRS. Her current research addresses the problem of sensing and understanding human non-verbal interactive language and intentions. Her background is on machine learning, speech recognition, spoken dialog system and evaluation. She participates in BPI ROMEO2 project, which has the main goal of building a social humanoid robot for elderly people. She leads the European CHIST-ERA project JOKER: JOKe and Empathy of a Robot. She is member of the working group on the ethics of the research in robotics (CERNA). She is also heading the "Humanmachine co-evolution" research group at the Numerical Society Institute (France). She has (co-) authored more than 140 publications. She is a member of AAAC (board), IEEE, ACL, ISCA, WACAI and AFCP. She is also involved in the Eurobotics Topic Groups: "Natural Interaction with Social Robots" and "Socially intelligent robots". (Video-demo1).
The Computer Sciences Laboratory for Mechanics and Engineering Sciences (LIMSI) is one of France's largest research laboratories of the CNRS working on language technologies. The team on "Affective and Social Dimensions of Spoken Interactions" (Head: L. Devillers) is working on affective computing and robotics applications2.
Page IV
SERC Robotics Program, A*STAR Institute for Infocomm Research, Singapore
SERC Robotics Program, A*STAR Institute for Infocomm Research, Singapore
TUM CREATE, Singapore Ad vi s o r y C o m m i t t e e
P r o g r a m
C o m m i t t e e ( i n a l p h a b e t i c a l o r d e r )
Aamir Ahmad, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
OR G AN I Z ATI O NS LIST OF ACCEPTED CONTRIBU TIONS
Luis Fernando D'Haro, Andreea Niculescu, Aravindkumar Vijayalingam, Marco Antonio Gutierrez Giraldo, Suraj Nair, and Rafael Banchs. “The path to success: Failures in rEal Robots (FinE-R)” Anders Billesø Beck, Anders Due Schwartz, Andreas R. Fugl, Marnti Naumann, and Björn Kahl. “Skill-based Excepotin Handling and Error Recovery for Collaborative Industrial Robots” Richard Wang, Manuela Veloso, and Srinivasan Seshan. “Using Autonomous Robots to Diagnose Wireless Connectivity” Hong Tuan Teo, and John-John Cabibihan. “Toward Soft, Robust Robots for Children with Autism Spectrum Disorder” Thommen Karimpanal George, Mohammadreza Chamanbaz, Abhishek Gupta, Wen Lizheng, Timothy Jeruzalski, and Erik Wilhelm. “Adapting Low -Cost Plaotfrms for Robotics Research” Felipe Cid Burgos, Pedro Núñez Trujillo and Luis J. Manso. “Improvements and considerations related to human -robot interaction in the design of a new version of the robotic head Muecas” Joel Stephen Short, Aun Neow Poo, Chow Yin Lai, Pey Yuen Tao, and Marcelo H Ang Jr. “Lessons from the Design and Testing of a Novel Spring Powered Passive Robot Joint” Zheng Ma, Aun-Neow Poo, Marcelo Ang, Geok-Soon Hong and Feng Huo. “Design, Simulation and Implementation of a 3 -PUU Parallel Mechanism for a Macro/mini Manipulator” Roger Bostelman, Tsai Hong, and Elena Messina. “Intelligence Level Performance Standards Research for Autonomous Vehicles” Fernando Fernández, Moisés Marntíez, Ismael García -Varea, Jesús Martínez -Gómez, Jose Pérez-Lorenzo, Raquel Viciana, Pablo Bustos, Luis Manso, Luis Calderita, Marco Guétirrez, Pedro Núñez, Antonio Bandera, Adrián Romero-Garcés, Juan Bandera and Rebeca Marfi l. “Gualzru's path to the Advertisement World” Page 1 5 11 15 20 27 36 42 48 55 SCHEDULE
Session 8:45 – 9:00 9:00 – 10:00 10:00 – 10:30 10:30 – 10:50 10:50 – 11:10 11:10 – 11:30 11:30 – 11:50 11:50 – 12:10 12:10 – 14:10
Lunch time 14:10 – 15:10 15:30 – 16:00
Cofee Break 16:00 – 16:20 Title Opening Ceremony Keynote: Lessons Learned in Human Movements and Behavior Analysis Ryad Chellali (Nanjing Roboctis Insttiute - CEECS, Nanjing Tech University, Nanjing, China) Cofee Break “Intelligence Level Performance Standards Research for Autonomous Vehicles” Roger Bostelman, Tsai Hong, and Elena Messina. “Lessons from the Design and Tesntig of a Novel Spring Powered Passive Robot Joint” - Joel Stephen Short, Aun Neow Poo, Chow Yin Lai, Pey Yuen Tao, and Marcelo H Ang Jr. “Improvements and consideraotins related to human -robot interacotin in the design of a new version of the robocti head Muecas” - Felipe Cid Burgos, Pedro Núñez Trujillo and Luis J. Manso. “Adapntig Low -Cost Platforms for Roboctis Research” - Thommen Karimpanal George, Mohammadreza Chamanbaz, Abhishek Gupta, Wen Lizheng, Timothy Jeruzalski, and Erik Wilhelm. “Using Autonomous Robots to Diagnose Wireless Connecvtiity” - Richard Wang, Manuela Veloso, and Srinivasan Seshan Keynote: The communicaotin accommodaotin of the machine to deal with errors in Human-Robot Spoken Interacotin
Laurence DeVillers (LIMSI-CNRS, Sorbonne University, France) 15:10 – 15:30 “Toward Soft, Robust Robots for Children with Austim Spectrum Disorder” Tuan Teo, and John-John Cabibihan. - Hong “Gualzru's path to the Adverstiement World” - Fernando Fernández, Moisés Marntíez, Ismael García -Varea, Jesús Marntíez -Gómez, Jose Pérez-Lorenzo, Raquel Viciana, Pablo Bustos, Luis Manso, Luis Calderita, Marco Guétirrez, Pedro Núñez,
Antonio Bandera, Adrián Romero-Garcés, Juan Bandera and Rebeca Marfil. 16:20 – 16:40 “Design, Simulation and Implementaotin of a 3 -PUU Parallel Mechanism for a 16:40 – 17:00
Macro/mini Manipulator” - Zheng Ma, Aun-Neow Poo, Marcelo Ang, Geok-Soon Hong and Feng Huo. “Skill-based Excepotin Handling and Error Recovery for Collaboravtie Industrial Robots”- Anders Billesø Beck, Anders Due Schwartz, Andreas R. Fugl, Marnti
Naumann, and Björn Kahl.
The path to success: Failures in rEal Robots (FinE-R)
Abstract— This paper presents our motivation for organizing the FinE-R workshop at IROS 2015, as well as a summary of all accepted papers. The main workshop goal is to provide an open exchange forum to the robotic community where participants can share their personal “failure to success” stories. We believe that such exchanges are of tremendous importance for the community as they provide a rich source of knowledge on how to avoid future mistakes with possible high impact. On the other hand, the papers accepted in the workshop give a good overview of different types of errors encountered in the robotic fields.
Through deep analysis and clear description of failures, the authors of these papers contribute to a learning process by extracting positive experiences and conclusions from negative results leading ultimately to success.
Keywords: Workshop goals, summary of accepted papers, failure analysis.
I. INTRODUCTION
Along the history there have been many important
discoveries that resulted from long trials and error processes,
like the ones done for the creation of the electric light bulb by
Tomas A. Edison [
1 Luis Fernando D’Haro, Andreea I. Niculescu and Rafael Banchs work at the Human Language Techonology Group in the Institute for Infocomm Research (I2R - A*STAR). 1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore 138632. (emails: {luisdhe, aandrea-n, rembanchs}@i2r.a-star.edu.sg).
2 Aravindkumar Vijayalingam and Suraj Nair work at TUM-Create, 1 Create Way, 8th Floor, Singapore 138602. (emails: {aravind.v, suraj.nair}@tum-create.edu.sg)
3 Marco Antonio Gutierrez is PhD student at the Robotics Laboratory (Robolab), Computer and Communication Technology Dept in the University of Extremadura, Spain. Polytechnic School, University of Extremadura Avda. de la Universidad s/n 10003 Cáceres-Spain. (email: marcog@unex.es). The author conducted this work as part of his A*STAR Research Attachment Programme (ARAP) at the Human Language Technology Department of Institute for Infocomm Research, Singapore.
As many other sciences, the path to progress in the field of robotics is not free of failures and caveats. These failures provide valuable lessons and insights on future approaches by analyzing errors and finding methods to avoid them. As such, the robotics community could benefit from the experience of those who had faced and overcome similar failures before.
The objective of this workshop is then to provide an international forum for researchers in robotics and its related fields, where they can share their personal experiences on their "failure to success" stories, to present what they have learnt, what others should avoid while experimenting in similar context, and providing tips for better research practices and for creating more successful robots that meet people's expectations.
II. MOTIVATION FOR THE WORKSHOP
Nowadays, in the scientific community only successful
theories and positive results have a chance of being regarded
as true, and then published in prestigious publications,
discarding odd and unexpected findings. However, the
success of these theories does not warrant that they are truth
neither prove their adequacy to realism. Unfortunately, the
current scientific publishing system privileges “successful”
results as it is expected that their research findings will be in
alignment with well-established literature or with expected
outcomes. However, as pointed by [
However, ignoring the huge amount of information that
negative results can provide (which, according to [
Page 1 unknowledgeable about the subject or incapable of tailoring more robust research hypotheses. To make things worse, some scientists will not report negative results just to avoid their papers to be rejected by the peer-reviewers, who could give priority to other studies with “successful” results or that follow a more popular theory or approach.
Fortunately, the scientific community is becoming aware that negative results are not meaningless and that there is a potential value in sharing also negative results and discussing the lessons learnt after analyzing the failures, as well as in explaining what were the keys to avoid problems and achieve successful results. Some examples of this tendency can be seem in the New Negatives in Plant journal4 that according to their scope is “an open access, peer reviewed, online journal that publishes hypothesis-driven, scientifically sound studies describing unexpected, controversial, dissenting, and/or null (negative) results in basic plant sciences. The journal also consider studies that validate controversial results or results that cannot reproduce previously published data”, or in the new approach supported by the Wealth Health Organization (WHO)5 that has a new policy of publishing, in their peer reviewed journal, results of clinical trials that include also negative findings.
Following such examples and taking into account that the scientist community working in the robotics field can also benefit of following a similar approach, we decided to propose FinE-R (Failure in Real Robots), a workshop in the context of IROS6 (IEEE/RSJ International Conference on Intelligent Robots and Systems) conference. For this, we decided not only to focus on presenting the negative results obtained while working on real robots, but also on how the researchers were able to extract meaningful lessons from their failures and what kind of solutions they proposed to finally overcome their problems. Then, we made the FinE-R’s call for papers targeting at the following topics: Analysis of failures when participating in robotic
challenges. Design of robust human-computer interfaces for robots. Description of problems and solutions faced when failure is not an option, therefore there is the need of creating an outstanding robot from hardware to software. Description of benchmarking and tools for testing and
creating robust robots. Description of techniques to avoid common but frequently seen errors when deploying robots for industrial or general public environments. Description of advanced techniques for failure recovery
and troubleshooting. Matching the expectations and needs of industries and
consumers with the current technology. 4 http://www.journals.elsevier.com/new-negatives-in-plantscience 5 http://www.who.int/ictrp/results/reporting/en/ 6 http://www.iros2015.org Description of alternatives to techniques and algorithms
that are prone to fail. Presentation of keys for successful research projects and
proposals on robotics. Analysis of failed results and projects when using smart algorithms, well-established techniques or brilliant designs.
These proposed topics not only were in line with the idea of learning from failures, that is central to our workshop, but also allowed to differentiate FinE-R from other workshops that are mainly centered on specific and vertical topics or areas of research. With FinE-R we aim at providing a space for sharing practices and experiences of robot design and construction across multiple disciplines, therefore making the workshop more interesting and open to a wider audience.
Finally, it is worth mentioning that it was gratifying for us to read comments from reviewers of the Workshop proposal about the appropriateness and timelines of an initiative such as FinE-R. Some examples of these are:
“This is a very interesting proposal as learning from failure in real-world applications is an important and essential capability for robots. This is not a topic not well addressed so far. It is very good to see a group of people discussing this”
“This workshop will provide such a unique opportunity that we can learn from not only our own failure but also others. We surely need such a workshop. Topics cover wide ranges. Speakers are from well-known organizations. Suggest leaving more time for discussions.”
III. SUMMARY OF CONTRIBUTIONS
In this section we summarize the accepted contributions to the first edition of FinE-R. All submissions went through a single blind review process. In average, all papers received three reviews.
A. Skill-based Exception Handling and Error Recovery for Collaborative Industrial Robots
Written by Billesø et al [
Autonomous Robots to
Diagnose
Wireless B. Using Connectivity
This paper, written by Wang et al [
Page 2 interest for most robotic laboratories when dealing with wireless connectivity problems. The authors claim that using this method they were able to improve the diagnosis of wireless connectivity issues as compared to manual methods.
C. Soft, Robust Robots for Children with Autism Spectrum Disorder
In this paper, Hong Tuan and Cabibihan [
D. Adapting Low-Cost Platforms for Robotics Research
In this paper [
E. Improvements and considerations related to humanrobot interaction in the design of a new version of the robotic head Muecas
In this paper [
F. Lessons from the Design and Testing of a Novel Spring Powered Passive Robot Joint
This article [
One problem engineers are often dealing with is the short time schedule they have to make certain assumptions and estimations. This can often lead to troubles in the assembly and testing phase. As such, the spring joint prototype designed by the authors went twice through a cycle of assembly, testing, and redesign before the arriving at the final stage. During this process, the authors mention that they identified three problems and reported five learned lessons from their design experience.
G. Design, Simulation and Implementation of a 3-PUU Parallel Mechanism for a Macro/mini Manipulator
In [
The authors further demonstrate how they learn from the initial attempt failures and device a new parallelogram based configuration for the universal joint mechanism in order to reduce backlash.
H. Intelligence Level Performance Standards Research for Autonomous Vehicles
In this paper [
Firstly, standards for vehicle navigation in order to measure uncertainties in navigation performance are detailed as currently this information isn't provided by the manufacturers.
Secondly, standards to determine uncertainties in vehicle docking by measuring relative displacement from each of the points are described. Thirdly, standards for obstacle detection and avoidance are presented to study the reaction of AGV in different situations such as when a human is detected and interaction with machines that are operated manually. And finally, standards for 6DOF optical measurement of dynamic systems are discussed as these systems are needed for performing ground truth measurements of AGV performance.
Experiments carried out for vehicle navigation, vehicle docking and optical measurement systems standards are also presented.
I. Gualzru's path to the Advertisement World
Presented by Fernández et al [
IV. CONCLUSION AND FUTURE WORK
With this first edition of the FinE-R (Failure in Real
Robots) workshop we pretend to open a door for researchers
Page 3 to address the analysis and discussion of failures and methodologies when creating or designing robots. The workshop allows for sharing research experiences with scientists facing similar situations and problems. In this paper we also have provided a summary of the accepted contributions, in which the authors were asked to describe their path to success roadmap and to provide clear explanations of what they learnt while deploying their robotic projects that could be of interest for other researchers working in the same area.
Taking into account the quality of the accepted papers, the good response from the reviewers, program committee, and scientific community, as well as the importance that brings doing a deep analysis not only on the successful results but also on the path followed to reach them, as future work, we plan to continue organizing FinE-R in the context of IROS conferences. Our desire is that by keeping open this forum, the expertise of worldwide researchers gained along several years of working on robotic projects can be share with the scientific community. By doing so, not only better research projects can be conducted, but specially common or subtle failures can be avoided. In addition, we plan to open a special session or discussion panel where people participating on shared tasks or competitions like the DARPA Robotics Challenge7, can explaining their experiences and problems encountered.
ACKNOWLEDGMENT
We want to thank to all the members of the program
committee who helped during the review process and during
the organization of the workshop. The full list of contributors
and program committee members can be found at the
workshop website8.
[
Martínez-Gómez, Jose Pérez-Lorenzo, Raquel Viciana, Pablo Bustos, Luis Manso, Luis Calderita, Marco Gutiérrez, Pedro Núñez, Antonio Bandera, Adrián Romero-Garcés, Juan Bandera and Rebeca Marfil. “Gualzru's path to the Advertisement World,” Proceedings FinE-R Workshop, pp. 55-65. IROS 2015 - Hamburg, Germany. October 2, 2015..
Page 4 Skill-based Exception Handling and Error Recovery
for Collaborative Industrial Robots
A. B. Beck, A. D. Schwartz, A. R. Fugl, M. Naumann, B. Kahl
Abstract— Moving robots from their carefully designed and encapsulated work cells into the open, less structured human workspace for collaboration with workers requires robust error detection and recovery strategies. Foreseeing all possible uncertainties and unexpected events and to program in recovery actions at setup time is unfeasible. Online learning of nominal execution behaviour and automatic detection of anomalies using an Extended Markov Model, combined with interactively trained Bayesian networks for mapping anomalies to error causes and recovery actions, enables automatic recovery from previously experienced errors. A three-layered user-friendly model of errors—causes— responses and a simple GUI allows non-expert user to define new recovery activities and error causes when not yet handled anomalies occur.
I. MOTIVATION
Today’s robot systems for industrial applications rely on a structured environment to avoid errors. Parts, fixtures, tools and stations have defined positions and the workspace is encapsulated to avoid intruders that could possibly endanger this defined environment. Expected exceptions from the nominal case that were either foreseen during the planning of the robot system, or occurred during the setup phase of the system are coped with by integrating additional sensors, adapting tool-, fixture and part geometries and adding additional branches to the robot program to cope with these deviations. Furthermore, as many robotic systems are complicated, any exceptions and breakdowns occurring after system setup often require external technicians or engineers to diagnose and solve problems.
Such strictly controlled and carefully designed work
cells are only economically feasible if the designed robot
system will run unobstructed for a long time. Small and
midsized enterprises (SMEs) are often characterized by a much
more agile production style and consequently rely on human
workspaces. Moving robots out of their strictly controlled
and carefully designed spaces into human workspaces,
which are by nature unstructured environments with a high
degree of uncertainty, requires significantly enhanced
robustness towards unforeseen events and geometric or
other uncertainties. (The additional need for safety measures
to protect the human co-worker from injuries is out of scope
of this work, see e.g. [22], [
* The research leading to these results has been funded by the European Union’s seventh framework program (FP7/2007-2013) under grant agreements #608604 (LIAA: Lean Intelligent Assembly Automation) and #287787 (SMErobotics: The European Robotics Initiative for Strengthening the Competitiveness of SMEs in Manufacturing by integrating aspects of cognitive systems).
A. B. Beck and A. R. Fugl are with the Danish Technology Institute. Email: anbb@dti.dk and arf@dti.dk recovery approach that allows non-robot expert users to operate a robotic system embedded in a human-centric workspace. We briefly introduce our execution model, detail the Extended Markov Chain based Situation Awareness, which forms the base for Exception Handling, and the Error Recovery module employing a Bayesian network and betabinomial inference algorithm. The prosed system has been implemented in a pick & place and in an assembly work cell, which are finally presented.
II. RELATED WORK
Research in exception handling is related to the area of error
or fault recovery [17]. Error recovery has been defined as
“the process by which the system returns to a state where
production can restart after an abnormal and disruptive
condition has occurred” [23]. For a robot coworker to
effectively handle an exception, whether through informing
the human worker or resolving the problem by itself, the
types of faults that typically occur in the manufacturing
robotic assembly cases needs to be understood. Fault
taxonomies have been presented in other related fields,
including mobile robots [
III. SKILL-BASED EXECUTION MODEL
At the base of the system is a Skill Execution Engine,
which allows a more goal-oriented task description than
strict motion based programming or planning. Without
going into details of the skill-model [
Robot tasks are constructed by chaining skills and control
flow instructions, forming a state machine [
N. Naumann is with the Fraunhofer Institute for Production Systems and Automation. E-mail: Martin.Naumann@ipa.fraunhofer.de
B. Kahl is with the Gesellschaft für Produktionssysteme GmbH Stuttgart. E-mail: bjoern.kahl@gps-stuttgart.de
1 Apache Commons SCXML executor, http://commons.apache.org/proper/commons-scxml/.
Page 5 of the skill executor that allow the implementation of error recovery functionality at higher layers are:
The skill executor knows and publishes the current state of the system at any time. This allows an error recovery module to relate errors on the one hand to specific skill models and on the other hand to specific application steps and therefore to draw conclusions like “this is an error that is very typical for a pick operation” or “this is an error that occurred already in the past at this specific execution step of the application”.
The skill executor has an interface for an error recovery module to stop and later continue the execution of the skill based application program thereby allowing worker interaction to recover from errors detected by an error recovery module.
The skill formalism used by the skill executor is built on the concept of reusable hierarchical skills that are easy to enhance or adapt. It is therefore easily possible to include additional mechanisms into an existing skill model to cope with errors that could be detected by the system but just have not been considered yet.
The Situation Assessment (SA) constantly monitors the overall situation (robot task execution) using data published by the skills executor as well as by additional sensors dedicated for situation assessment. Deviations flagged by the SA are further examined by the Exception Handling (EH), which devises a possible cause and corrective measure, potentially involving user interaction. The whole system of skill executer, situation assessment and exception handling is collectively referred to as “Exception Handling Framework” or “EHF”.
IV. SITUATION ASSESSMENT
The role of Situation Assessment (SA) is to learn and
monitor the (correct) skill execution and detect non-nominal
conditions. Deviations from the learned, nominal behaviour
are interpreted as Anomalies, which are passed on to the
Exception Handler (section V). Our implementation of SA
is based on prior work by [
To learn how to perform a skill correctly, SA captures the essence of the skill by learning the timing and sequence of events that make up the skill. Our approach is to generate one parameterized model that includes parameters in the space and time domain. SA learns the sequence of events within a skill execution by learning a set of parameters with a temporal component, recording the transition from one instantiation of the parameters to the next: p( ) = ( 1, 2, … , )
(1) where , a Situation Model, denotes a set of parameters (a state), denotes a discrete step in time, and , a Situation, denotes the complete distribution of all the states within a skill. Each state of is parameterized: = [ 1, 2, … , ]
(2) where are data components such as sensor values or robot’s internal state values.
A. Situation Model
Situation Assessment uses a Situation Model as a template description to fuse together the different data points for learning a Situation. The components of the Situation Model ( in (2)) are real number data, which can come from any source and have any meaning. In our experience, combining space and time is critical to the success of learning a skill. For instance, learning a skill using a 6D F/T sensor, the Situation Model could be defined as in (3).
= [
, x , y , , , , ] (3)
The component of a is a data point that uniquely identifies the current primitive being executed in the skill. In this case, the unique primitive ID provides the understanding of time while the understanding of space is provided by the F/T data. By using the primitive ID we can learn a skill time invariantly. This means that SA will only learn the sequence of the events and is invariant towards the duration of the execution of specific primitives. We have found this feature particularly useful when the duration of the primitives or skills is stochastic. Should it be necessary to catch anomalies in relation to when events occur (e.g. too early or late), the primitive ID in (3) can be substituted with a time data point. Throughout our research, we have successfully applied SA to monitoring digital inputs, such as the state of one or more grippers. Through the rest of the paper, we will use the following Situation Model for implementation and testing: = [ , en,
] (4) where and are binary outputs of reed switches of the gripper: is when the gripper is fully open and is when the gripper is fully closed. Our assumption is that the gripper is grasping an object when both readings are , indicating the gripper is neither fully open nor closed.
B. Data Processing and Clustering
The Situation Model serves as a template describing which sensors SA should fuse together into one single state.
In general, all data points in the Situation Model have to be real numbers. This allows the computation of one single metric for each in (1). We have so far used the Jaccard similarity coefficient as a method for clustering similar states. Through experimentation, we have found the algorithm to be useful despite its simplicity.
C. Dynamic Learning in Situation Assessment
SA can autonomously learn a skill without the user
having to manually specify the states of a skill. We have
implemented a spatiotemporal model that allows for online
dynamic learning of states over time. For this purpose, we
are currently using the Extensible Markov Model (EMM) as
it is useful for online learning of sequences of states [
Page 6
D. Anomaly detection
SA has two modes of operation: learning and detection during execution. In the learning mode, SA monitors the data points specified in the Situation Model and builds the Situation for the skill that is being learned. During execution of the same skill, SA loads the saved Situation and applies the same clustering process as during learning. However, should the clustering of the data result in a new state in (1), then SA will interpret that as an anomalous state has occurred and issue an Anomaly warning. Processing and handling the Anomaly is the task of the Exception Handler (EH) module.
V. EXCEPTION HANDLER
The task of EH is to receive an Anomaly from SA and provide a suggested solution that is most likely to solve the problem. For each robotic system, EH maintains a hierarchical four-layered Bayesian network with all exceptions and solutions relevant to that cell. The hierarchical structure allows EH to reason about the most suitable solution to a problem. EH provides the suggested solution to the user along with all other possible solutions.
The user is free to select the suggested solution, any other solution or to create a new solution. The selection is stored in EH as a sample of user solution preference. Such samples are used in priming the network for inference with future anomalies. With the feedback of user samples, a closed preference-learning loop is formed to provide suggestions for solutions to future anomalies. In this section, we provide a detailed description of EH and begin with the role of the Exception Scenario ES in EH.
A. Exception Scenario
The Exception Scenario (ES) is designed as a fourlayered model consisting of Anomaly, Error, Fault and Response, inspired by work in [18]. The hierarchy is a fourlayered binary Bayesian network that facilitates inferring the most likely Response (solution) to an Anomaly (a deviation), an example is shown in Figure 2. At the lowest level of the network, Anomaly nodes model anomalies detected by SA. Each Anomaly node corresponds to a data component (di of (2)) in the Situation Model. Above Anomaly, the Error node models which kind of error the Anomaly is and if the Anomaly should even be considered
2 Figure 2 shows a screen capture of GeNIe, a Bayesian modelling environment developed by the Decision Systems Laboratory of the University of Pittsburgh. Available at http://genie.sis.pitt.edu an error. The Fault node models the root cause of the Anomaly and the Response node models the solution to the Fault. This model resembles the diagnosis model used by physicians when examining a patient: Based on symptoms (here: the detected error) an illness is inferred (here the fault) and a therapy decided (here the response). The intermediate step of a fault is necessary, since one and the same observed error (symptom) can have multiple causes.
For example an unexpected gripper state can be due to a failed grasping operation, a missing object at the pickup position or a defective gripper itself.
B. Bayesian Network
Figure. 2. Inference of cause and solution to Gripper Open anomaly.
Both error nodes are dependent on both anomaly nodes, allowing the Bayesian network to further strengthen the belief about the cause of an anomaly. Simulated in GeNIe1.
Figure 2 is an example of a Bayesian network with three Exception Scenarios for the two Anomaly nodes of the sensors and in (4). Both Error nodes are dependent on both Anomaly nodes, allowing the Bayesian network to further strengthen the belief about the cause of an Anomaly (simulated in GeNIe2). The first two scenarios with nodes 1 = {1,3,5,8} and 2 = {1,3,6,9}, offer two Responses to the Gripper Open Anomaly while the third scenario 3 = {2,4,7,10}, offers a single Response to a Gripper Closed Anomaly. The numbers in curly braces indicate the node number in Figure 2. In this example we are modelling two faults {5,6} and Responses {8, 9} for the Gripper Open Anomaly. If a gripper is unexpectedly open (Gripper Open = true, Gripper Closed = false), we could interpret that as either a pneumatics failure (e.g. loss of air pressure) that can be solved by checking and replacing the air supply {5,8}, or an actuator failure (e.g. broken gripper) that can be solved by repairing the gripper {6,9}. In the reverse case of a closed gripper, we could interpret the failure as there was no object to grip and the solution is simply to replace the missing object. In Figure 2, the Faults {5,6} are modelled as belonging to the same Error, Gripper Operations Error {3}. This allows the network to learn user selections for a specific Fault, Response pair over other pairs belonging to the same Error node. The network is thereby able to encode knowledge specific to individual user environments.
Page 7
The process of inferring a Response to an Anomaly in the Bayesian network, is the inference process of the EH.
This process is an implementation of Bayes’ theorem: ∝ ∙ ℎ
(5) We have implemented Bayes’ theorem in three steps:
Calculate prior probabilities Introduce evidence to network
Infer posterior probabilities
In the following, we describe each of these steps.
D. Inference
A prerequisite for performing inference is the calculation of prior probabilities. As described in section IV, a feature of the EHF is to learn the user-preferred solution of a given anomaly. When the user selects a specific Exception Scenario (i.e. an Error, a Cause and a Solution) to solve a problem, it is fed back to the database as a sample of the user selection, thus learning the preference of selecting this Exception Scenario for a specific Anomaly. The sample data is used to calculate the prior probability for each node of the network. We treat calculating the node’s prior probability as an inference process that adds another layer of Bayesian inference as described in (5). We introduce the sample data from user selection of Exception Scenarios as the evidence to infer each node’s posterior probability. Each node of the Bayesian network is a binary random variable modelling an event that either occurs or not. For instance, if the user selects the ES {1,3,5,8} in Fig. 2, then the user is confirming that the specific ES solved the problem (e.g. that a Gripper Open Anomaly did happen, it was caused by missing air pressure and the solution was to resupply the air). At the same time and equally important, the user is also confirming that alternative events {4,6} to ES {1,3,5,8} did not occur.
Thus, with every selection of an ES, EH registers the confirmed nodes on all levels of the ES, as well as the rejected nodes. The process of selecting any node in the Bayesian network over time, can be viewed as a Bernoulli process following a binomial distribution as in (6).
~Binom( , ) (6) where is the number of times a specific node has been selected. is the number of samples drawn in the sequence.
If this process is sampled sufficiently, a distribution
reflecting the user selection can be inferred from the sample
set. However, in many cases it is not possible to provide a
sample set of sufficient size and inference will be subject to
uncertainty. To model this uncertainty, we model the user
selection for each node as a hyper-parameter , thereby
modelling the user selection as a random variable itself and
creating a hierarchical Bayesian model for calculating the
prior probability [
( | , ) = Be( , )
(7)
In (7), the user selection is modelled as the hyperparameter , drawing samples from the Beta distribution. and is respectively the number of samples confirming and rejecting the selection of the node. The Beta distribution is a conjugate distribution to the Binomial distribution, thereby offering analytical tractability of the Bayesian inference process. The conjugate property ensures that when updating the prior Beta distribution (7) with new evidence following the Binomial distribution, the resulting posterior distribution is also a Beta distribution (8).
( | ∗, ∗) =Be( ∗, ∗) (8) ∗ and ∗ is respectively the new number of selections and rejections for the specific node. Thus, obtaining the posterior distribution in (8) becomes simply a matter of adding new confirmations to the existing, and then calculating the mean ( ) and variance ( ) (9,10).
In Figure 3, examples of Beta distributions for different values of and are shown. Distribution 1: e( = 1, = 1) is a uniform distribution offering an uninformative prior with a mean, = 0.5 and a high variance (uncertainty) due to the low sample size. In this case, the posterior will largely be determined by the data. Distribution 4: e( = 30, = 5) has = 0.86 and a smaller variance, thus providing a comparably less uncertain estimate of the user selection preference, . The sequence of graphs 1-4 in Figure 3, can be seen as an example of a continuous learning cycle, starting with no knowledge of user selection (a uniform distribution with no samples) towards more informative distributions 2-4 as the sample size increases. When a new node is created with no samples available ( = = 1), the Beta distribution is uniform. However, to avoid the uninformative uniform distribution we propose to query the user to provide a subjective estimate of the selection (the mean) of this node along with a confidence level (the variance). Using the equations for the mean (9) and variance (10), suitable values for and can then be calculated.
E. Introducing evidence
The Bayesian network described in section V.B and Fig. 2 receives evidence in the form of Anomaly information gathered by SA. In the example shown in Figure 2, SA has detected that the gripper was unexpectedly fully open (thus providing evidence that Gripper Open = true, Gripper Closed = false. The evidence is in practice introduced to the network by clamping the two nodes to their respective values.
Page 8 F. Posterior probabilities
After introducing evidence, posterior probabilities for all
nodes are calculated. We have used the SMILE reasoning
engine [
For each Response, the tree is descended towards the root Anomaly nodes, thus mapping out each possible path towards the root. The resulting list will have the most probable ES listed first with all other less likely alternative ES following in descending probability.
VI. USER INTERFACE FOR ERROR RECOVERY
While section V discussed the inner working of the actual mapping process, we focus on a more user-centric view in this section.
Whenever an anomaly is detected, the error layer classifies it into an error cause. If no cause is found the user is inquired and given the option to assign an existing cause, dismiss the anomaly as not indicating an error or to create a new cause (including a resolution, if known). Figure 4 shows the dialog box after successfully mapping an anomaly to an error and further to a recovery action. The user can accept
Figure 4 The system identified an error including a recovery action.
In case of misclassification the user can add a new exception or dismiss the anomaly as not indication an error (button “Continue
Learning”). this solution or add a new solution. Figure 5 shows the corresponding dialog box for adding a new triplet of error, error cause and recovery action. The dialog boxes shown in Fig. 4 and 5 are designed for use at system runtime and therefore as simplistic as possible. A more elaborated interface for managing the entire network of anomalies, errors, causes and recovery actions is also provided and targeted at specifically trained users that setup a new robot application.
VII. EXPERIMENTAL EVALUATION
Within the scope of SMErobotics, this framework has been intensively evaluated using various experiments. A detailed example is the failure to grasp as described in the following section. We have tested the system’s ability to learn the preference of selecting a solution by manually introducing the Gripper Open (GO) Anomaly, shown in Fig. 6, during the execution of a skill. In this test, we have tested the system’s ability to learn the user preference of selecting the Repair Actuator (RA) Response over the Replace Pneumatics (RP) Response. For the purpose of the test, the system had initially no knowledge of user selections (samples), except for five samples confirming the choice of the RP Response as the user preferred solution to the GO
Figure 5 Adding a new error cause or fault to the system.
Anomaly. Hereafter, we introduced the GO Anomaly repeatedly, selecting the RA Response as the solution each time. This process was repeated until EH started to suggest the RA Response, thus demonstrating EHF’s ability to learn the user preference of selecting the RA Response over the RP.
Test results are shown in Fig. 6. Initially, EH has five samples confirming the selection of the RP Response for the
GO Anomaly. Thus, when the GO Anomaly is introduced, EH suggests RP as the most suitable Response to the GO Anomaly with probability ~ 0.553. However, the user ignores the EH suggested RP Response and instead selects RA. Thus, when the GO Anomaly is introduced again, EH now has six samples (five for RA and one for RP), computing the most likely Response to be RP with probability ~ 0.545, and so on. At RA sample 5, EH computes the probability for each Response being identical (~ 0.526). Again, the GO Anomaly is introduced and this time EH suggests the RA Response with posterior probability ~ 0.535. Thus, with five samples confirming RP, it took six samples of RA for EH to suggest RA.
VIII. CONCLUSION AND FUTURE WORK
Through the test results in section VII, we showed that EH is able to learn the user preference of selecting a solution, even when it had learned a different preference earlier. As the user selects a specific solution to an Anomaly, the solution becomes more probable for future selection. This is normally helpful, but can be problematic if the user wishes the system to select a different solution, since learning a new preference can take several iterations, as the test results showed. This is especially true when the sample count for the prior solution is high. A possible future solution could be
Page 9 B. Kinematic Analysis of 3-DOF Translational Motion
With knowledge of the 3-DOF translational mobility, the
kinematic model of the parallel mechanism can be derived [
The top view of the base and top platform is shown in Fig. 3, where Ai and Bi are the locations where the prismatic joints and the universal joints are mounted to the base and the top platform respectively. Coordinate Frame O and Frame O’ are respectively attached at the centre of the base and the top platform. The distance from the center of the platforms to Ai and Bi are R and r respectively. Let the displacement of the ith prismatic joint attached at Ai be zi. All the universal joints are passive.
Since the parallel mechanism are constrained to have only translational motions, the transformation matrix for rotation from frame O’ to frame O is an identity matrix. Let the position vector of Frame O’ in Frame O be A3
According to the mechanism structure shown in Fig. 1 and the geometric conditions shown in Fig. 3, the inverse and forward kinematics of the parallel mechanism can be obtained.
By assuming the top platform has only translational motion with respect to the base platform, position vector Bi in frame O’ is [Bi ]O' (r cos i
r sin i 1 30 , 2 150 , 3 90 0)T ,
Therefore the position vector Bi in frame O is and the position vector Pi in frame O is [Bi ]O (r cos i x r sin i y
z)T [Pi ]O (R cosi
Rsini
zi )T For all three limbs, if the distance between the two universal joints, Bi to Pi is L. The constraint equation can then be written as L
[Bi Pi ]O After substituting Bi and Pi into (7), we have (x xi )2 ( y yi )2 (z zi )2 L2 , xi (R r) cos i , yi (R r) sin i
The inverse kinematics thus can be obtained as Page 44 zi L2 ( x xi )2 ( y yi )2 z
In the same way, the forward kinematics can be obtained by applying the same constraint equation.
III. FAILURES IN SIMULATION AND IMPLEMENTATION
With the kinematic model obtained, the parameters R, r and L were chosen to meet the workspace criteria. Solid models were then established for motion and stress analysis, the former to confirm the translational motions of the top platform within the specified workspace and the latter for sizing the components for strength and stability.
During simulation, some unexpected results were observed when the top platform moved away from being parallel to the base platform. Unacceptable motion performance was also obtained with the first prototype developed using off-the-shelf universal joints. These will be discussed in the following
sections.
B2 B1
A. Extra DOF observed in Simulation
Solid models of the parallel mechanism were created using the software SolidWorks®. Motion studies were done simulating motion at the three prismatic joints. This caused the three lower universal joints, P1, P2, and P3 in Fig. 1, to move vertically. Various combinations of linear motions for the three prismatic joints were used to study the movement of the top platform relative to the base platform, as well as to verify the size of workspace of the parallel mechanism.
The top platform was expected to remain parallel to the base platform at all times since the design of the mechanism constrained it to have only 3-DOF translational motion.
However, it was noted that for some motion combinations of the prismatic joints, the top platform does not always remain parallel to the base platform but moved into a non-parallel mode of motion after remaining parallel for some time. Fig. 4 shows an example of how the roll-pitch-yaw angles of Frame O’ with respect to Frame O change with time for one such instance. From the figure, it can be seen that the top platform moves with only translational motion for about 11s after which it has rotational motions.
To explain the unexpected rotational motion, the assumption of pure translational motion was reviewed. A typical drawing of a universal joint is shown in Fig.5.
Ux a hole to accommodate the external shaft and a dowel pin is used to hold the shaft to the joint as shown in the figure.
UzP
Platform side
Uy
Link side
UzL
Consider one of the three universal joints attached to the
top platform as shown in Fig. 5. With the other end, Pi, of the
link fixed, there will be no rotation about the axis UzL, The
universal joint can only rotate about the Ux and Uy axes,
enabled by the cross component in the joint. With only two
degrees-of-freedom, there will not be any rotation about the
axis UzP, and thus no rotation of the platform [
Since there are three universal joints attached to the top platform, therefore no rotation of the platform is allowed about three axes. When these three axes are linearly independent in 3 , the top platform will lose all the rotational motion and its 3-DOF motions will be purely translational. Based on this analysis, the rotational motion of the top platform during simulation as shown in Fig. 4 is thus unexpected.
This rotational motion observed in simulation is suspected
to be caused by the loss of independence among the three axes
UzPi. When two or more axes become linearly dependent, the
parallel mechanism will be in a singular position. Unlike the
singularities in serial-link robots, instead of losing degrees of
mobility, a parallel mechanism gains extra degrees of freedom
at a singular position [
In Fig. 4, it is likely that the parallel mechanism reached a singular position at about 11s, gained an extra degree of rotational mobility and the top platform became non-parallel to the base platform. Thereafter, the motion of the mechanism was no longer constrained to be purely translational.
Referring to the Chebychev-Grübler–Kutzbach criterion, the mechanism should have three degrees-of-freedom when it is not in a singular position. It is likely that the motion of the mechanism after passing through the singular position is a combination of three degrees of motion with both rotation and translation. Further investigation will be needed explain and to understand this unexpected simulation result.
B. Backlash in Implementation
The universal joints used in the construction of the first prototype were off-the-shelf good quality joints the schematic of which is shown in Fig. 6. Each side of the universal joint has
Figure 7 shows the first prototype of the mini manipulator mechanism using these universal joints. Three linear actuators, labeled with 0, 1 and 2, are used for the prismatic joints. Each link connecting the prismatic joint to the top platform is made up of a circular shaft with a universal joint at each end. The universal joint at one end of each link is fixed to a linear actuator and the other end to the top platform.
When the three linear actuators are fixed in any position, i.e. not moving, the top platform should also remain in a fixed position parallel to the base platform. However, it was found that with the actuators fixed in their positions, the horizontal slack of the top platform was 4 to 5mm, which is unacceptably large, together with unacceptably large angular rotations.
Investigations showed that these unacceptably large motions, or “backlash”, are due to the clearances used in the manufacture of the mechanical components used. While pure translation motion of the top platform was observed in simulation for which perfect dimensions of the various components are used in computation, such perfectly formed parts are not available in practice, thereby resulting in the unacceptable results. A close examination of the first prototype showed that the exhibited backlash phenomenon is due almost entirely to clearances in the off-the-shelf universal joints used.
The universal joint, also known as a Hooke's joint, is a joint or coupling which is commonly used to transmit rotary motion from one rigid shaft to another rigid shaft when the axes of the two shafts are at a small angle to each other. The rotary motion transmitted is usually in one direction only. Because there is no change in direction of the transmitted rotary motion, the small clearances designed into them for ease of manufacture does not cause any backlash problem.
The universal joints used in the TPM mechanism in the work here serve a different purpose. They serve as joints providing two degrees of freedom (rotary motion) constraining
Page 45 the motion of the parallel mechanism as required from the structure shown in Fig. 1. Referring to Fig. 5, the universal joints used should rotate only about axes Ux and Uy to cause the top platform of the TPM mechanism to move. There should be no rotation about axis UzL or UzP. However, when one side of the U joint, say the link side, is fixed and not allow to rotate about its axis UzL, it is observed that the other side has freedom to rotate, about axis UzP, to some significant degree. This is due to manufacturing clearances designed into the joints, in particular at the four ends of the cross component in the joint.
The resulting free-play or backlash is accentuated due to the short lengths of the two rods forming the cross component in the joint. The off-the-shelf U joints thus did not have sufficient stiffness along the Uz axes and are not suitable for the TPM mechanism.
Another significant cause of the free-play or backlash problem in the motion of the TPM mechanism is due to clearance applied during the fabrication of the mechanism. As mentioned earlier and with reference to Fig. 6, dowel pins were used to connect the external shaft to each end of the U joints.
Ideally, the two holes in the U joint and the one in the shaft to accommodate the dowel pin should all be of exactly the same diameter, corresponding to the diameter of the dowel pin, with their centers perfectly aligned. However, as the holes were drilled at different times, if they were to be made of the same diameter with very little clearance, the centers of the holes need to be perfectly aligned in order for the dowel pin to be inserted. Alignment of the holes, when drilled separately, is not easily done. As such, the fabricator introduce some clearance and made the hole in the shaft larger (Fig. 8) than that of the holes in the U joint, which is of the same diameter as the dowel pin. While this allowed for the insertion of the dowel pin even if there is some slight misalignment of the holes during manufacture, it caused significant rotational free-play or backlash between shaft and the universal joint. Here again, the rotational backlash is accentuated by the small diameter of the shaft, and thus the length of the hole in it.
are used for their typical functions of transmitting rotary motion between two shafts.
The first prototype failed to meet the requirements for its intended application and a review of the design, and where it failed, was carried out to come up with the second prototype.
IV. LEARNING FROM THE FAILURES
In the process of developing and building the first prototype, two valuable lessons were learned. One is the unexpected results during simulation studies and the other is the poor performance in the fabricated mechanism due to manufacturing clearances and backlash in the off-the-shelf universal joints used.
It is noted that that the top platform of the mechanism does not remain parallel to the base platform under all circumstances. Rather, when starting from a parallel position, the top platform may move into a mode, or region of its workspace, where it gains rotational motions after passing through a singular position. This problem occurred during simulation when it is put all possible motions within its total workspace. In practice, this problem can easily be overcome by constraining the motions of the three actuators such that its workspace clearly does not contain any singular positions.
The first prototype has unacceptably poor accuracy in its motion and positioning. The top platform has some degrees of mobility, of about 5mm due to backlash when the actuators are fixed in their positions. This mobility is not acceptable as the mini manipulator is required to have high stiffness and precision. It is clear that this problem is caused by the manufacturing clearances in the off-the-shelf universal joints used. To overcome this problem, while still using lower-cost off-the-shelf components, other type of joints which has the same motion properties as universal joints but do not suffer from the same backlash problem was investigated as replacements.
The mechanical structure to replace the link with its pair of universal joints is shown in Fig. 9. It is composed of four ball joints connected in a way to form a parallelogram.
The unsatisfactory motion of the first prototype of the mechanism is largely due to the clearances in the off-the-shelf universal joints and the limited machining accuracy of the fabricated parts. Information on clearances for off-the-shelf universal joints are not readily available from manufacturers as such information may not have been important when they
According to the property of an ideal parallelogram, the opposite sides of the parallelogram will always be parallel.
Therefore, the side AB will always be parallel to the side CD in Fig. 9. Since the side CD is mounted parallel and fixed to the base platform, the side AB will also always be parallel to the base platform. As there are three limbs in the TPM mechanism,
Page 46 there are three parallelogram with three sides AB attached to the top platform.
These three parallelogram limbs are attached to the top platform such that the three sides AB all lie in a plane and the top platform is parallel to this plane. Since all the three sides AB are parallel to the base platform, the plane formed by them will be parallel to the base platform. Therefore, the top platform will also always be parallel to the base platform. With the top platform constrained to be parallel to the base platform, and the base platform is fixed and immobile, the motion of the top platform will be constrained to be translational only.
If there is free play or backlash in the ball joints at A, B, C, or D in Fig. 9, then the parallelogram formed will not be an ideal parallelogram. In this case, the sides AB may become non-parallel to the side CD. The amount of non-parallelism depends on the amount of free play in the ball joints and the length of the sides AB and CD, the longer the sides are, the smaller the degree of non-parallelism.
For the typical applications they are intended for, good quality ball joints have almost no free play or backlash. The length of the sides AB and CD of the parallelogram are also much longer than the length of the cross component in the universal joints. As such, the use of ball joints with a parallelogram structure for the three limbs of the TPM mechanism effectively eliminated the free play and backlash problem. The resulting second prototype is rigid and has high precision in positioning. With the actuator fixed in their positions, there is no measurable backlash in the top platform.
The backlash found in the first prototype had been effectively eliminated and this second prototype will be suitable as the mini in a macro-mini manipulator to be used for finishing and deburring applications for which both position and force/position control are required. Unlike a serial-link robot, the parallel structure of this robotic device gives it the high rigidity and thus the capability of exerting large forces on the workpiece in force-controlled polishing applications
V. CONCLUSIONS
A parallel mechanism, based on the structure of the Delta robot, was designed and implemented to serve as a mini manipulator, acting as an end-effector, in a macro-mini manipulator configuration for polishing and deburring applications.
Kinematic models of the mechanism were first obtained and applied to fulfil the given criteria. Solid models were created to simulate and analyze the resulting motions and workspace of the mechanism which was design. Unexpected and unacceptable motions of the top platform in the mechanism were observed during the simulation experiments.
The kinematic models failed to explain the motion since the assumption of pure translational motion of the top platform did not hold. It is likely that the non-parallel motions of the top platform in the mechanism was due to it passing through a singular position at which it gained extra degrees of freedom.
With the motion of the actuators in the mechanism constrained such that no singular positions lie within the workspace, the problem of non-parallel motions can be resolved. Further research will be done to determine the exact cause of the rotational motions of the 3-PUU parallel mechanism during simulation.
Unacceptable free play and backlash was exhibited by the first prototype. This was not evident in the simulation experiments which are based on perfectly manufactured components. Investigations showed that this problem was due to inaccuracies in the dimensions of the components used. The main cause was the free play in the off-the-shelf universal joints used for the first prototype. To overcome this problem the universal joints were replaced by off-the-shelf ball joints forming a parallelogram structure for the three limbs of the mechanism. The kinematic model of the mechanism remains the same but the free play problem was effectively eliminated and the second prototype exhibits high stiffness and positioning accuracy.
Lessons were learned from unexpected outcomes and failures during the simulation experiments and in implementation. Properly designed simulation experiments may produce results not predicted by theoretical studies as these studies are normally based on certain simplifications and assumptions, which cannot be completely replicated in simulation experiments.
Furthermore, straightforward simulation experiments which are based on perfect physical properties of the component parts may not show up possible inadequacies in the design. These inadequacies may show up only in the prototypes built due to unavoidable imperfections in the physical components making up the whole system.
ACKNOWLEDGMENT
The authors acknowledge the support from the Collaborative Research Project under the SIMTech-NUS Joint Laboratory (Industrial Robotics). This work was also supported in part by the Science and Engineering Research Council (SERC) A*STAR Industrial Robotics Program Grant 12251 00008.
Page 47 Intelligence Level Performance Standards Research for
Autonomous Vehicles
Roger B. Bostelman, Tsai H. Hong, and Elena Messina
Abstract— United States and European safety standards have evolved to protect workers near Automatic Guided Vehicles (AGV’s). However, performance standards for AGV’s and mobile robots have only recently begun development. Lessons can be learned from research and standards efforts for mobile robots applied to emergency response and military applications. Research challenges, tests and evaluations, and programs to develop higher intelligence levels for vehicles can also used to guide industrial AGV developments towards more adaptable and intelligent systems. These other efforts also provide useful standards development criteria for AGV performance test methods. Current standards areas being considered for AGVs are for docking, navigation, obstacle avoidance, and the ground truth systems that measure performance. This paper provides a look to the future with standards developments in both the performance of vehicles and the dynamic perception systems that measure intelligent vehicle performance.
Automatic Guided Vehicles (AGV’s) have typically been
used for industrial material handling since the 1950’s. Since
then, U.S. [
Recently AGV and mobile robot performance standards developments have begun to limit measurement method differences. Initial developments began with a review of other research and standards efforts for mobile robots as applied to
R. V. Bostelman is with the National Institute of Standards and
Technology, Gaithersburg, MD 20899, USA and with the IEM, Le2i,
Université de Bourgogne, BP 47870, 21078 Dijon, France (phone:
301-9753426; fax: 301-990-9688; e-mail: roger.bostelman@nist.gov).
emergency response and military applications [
Performance test methods for docking, navigation, (see
Figure 1) [
This paper will specifically discuss measurement of: vehicle navigation (e.g., commanded vs. actual AGV pathfollowing deviation), vehicle docking (e.g., AGV stop point positioning vs. known facility points), and obstacle detection and avoidance of standard test pieces (e.g., comparison of realtime AGV path-planning and new path following vs. commanded path) towards smart manufacturing applications, such as assembly and unstructured environment navigation.
Additionally, this paper will discuss a new ASTM Committee
on 3D Imaging Systems E57.02 [
II. PERFORMANCE STANDARDS THRUSTS
AGV navigation, docking, and obstacle detection and avoidance tests were conducted in support of future performance standard test methods and are described in this section. In some instances, typical industry practices were evaluated as well as the improved AGV performance tests.
T. H. Hong, is with the National Institute of Standards and Technology, Gaithersburg, MD 20899, USA (phone: 301-975-3444; fax: 301-990-9688; e-mail: tsai.hong@nist.gov).
E. Messina is with the National Institute of Standards and Technology, Gaithersburg, MD 20899, USA (phone: 301-975-3510; fax: 301-990-9688; e-mail: elena.messina@nist.gov).
Page 48 A. Vehicle Navigation The most basic functions of mobile robots and AGV’s are navigation to and docking with equipment in the workspace.
However, the description of how well the vehicle navigates
(i.e., commanded vs. actual AGV path-following deviation)
has certain ambiguities. For example, navigation implies that
the vehicle measures its current position, plans a route to
another location, and moves from the current location to
planned location upon command. Most vehicle
manufacturers don’t provide specifications for how uncertain
the navigation performance is (i.e., the error bounds on
position or velocity), other than perhaps radius of vehicle
turns, maximum velocity, and maximum acceleration. The
vehicle velocity sets limits on the allowable turn radius for
particular vehicles. Some controllers [
Industrial vehicles may eventually become uncalibrated through regular use. An uncalibrated vehicle does not follow a commanded path or stop/dock at a commanded point with minimal relative uncertainty (standard deviation of measured vs. ground truth) as does a calibrated vehicle. To correct this, vehicle manufacturers have calibration procedures for their vehicles, although these procedures can be tedious, timeconsuming, and may not be appropriate for all vehicles. For example, calibration of Ackerman steered vs. ‘crab’ steered (sometimes called quad) vehicles have different calibration procedures. It is not always clear what will happen when a vehicle is uncalibrated nor when the vehicle becomes uncalibrated. The effects of calibration on vehicle control and uncertainty are typically not specified either. There is also typically no specification describing how far from the commanded path a vehicle navigates. This may be important to users who have tight tolerance AGV paths (e.g., paths between infrastructure) that must be followed. A test can be developed to uncover the effects of uncalibrated vs. calibrated vehicle navigation performance when commanded to move along a path, as shown as a dashed line in the example in Figure 1. Should objects be near the vehicle path, such as walls or obstacles, depicted in Figure 1 as bordering lines along the path, the vehicle may stop, slow, or worse, collide with the boundary object. A user would then be required to provide additional, perhaps unnecessary space for one manufacturers’ vehicle and not for another. How the vehicle handles (slow, stop, etc.) the event is also ambiguous. For example, some, but not all vehicles are equipped with obstacle detection based on non-contacting sensors that provide detection beyond the physical vehicle footprint.
To address AGV navigation uncertainty, with an eye
towards a potential test method for all automatic industrial
vehicles, tests were executed, both with an AGV prior to and
after being calibrated. The uncalibrated AGV test is similar to
typical industry methods since not all AGVs can be frequently
calibrated. An uncalibrated AGV was moved along a straight
line path between two commanded points in an open area and
spaced approximately 5 m apart [
Pt 1
Pt 2
Page 49
Another test setup was tried, with an eye towards a relatively less expensive test method that will allow all AGV systems to be measured, ideally, with an independent measurement method that doesn’t use AGV controller tracking, yet captures the full AGV configuration (i.e., including safety sensing). The AGV was commanded to drive back and forth between temporary barriers, along a straight line defined by commanded points spaced approximately 10 m apart. The goal of the experiment was to measure the AGV deviation from the commanded path. A critical AGV navigation performance area is also deviation from the commanded path after turns so a 90° turn was added to the end of the straight path beyond the barriers to measure the vehicle navigation uncertainty when moving from/to a straight path to/from a turn. Figure 3 shows the test setup and Figure 4 shows (a) a B56.5 test piece being used to define the safety laser stop field edges, (b) the barriers and lines to which barriers are moved between trials, and (c) the AGV emergency-stopped upon detection of the barriers. The safety laser, stop field edges were marked on the floor, as a ground truth, zero-tolerance spacing that the vehicle can navigate, when the vehicle was at position 1 and again at position 3, shown in Figure 3, for both left and right vehicle sides. The barrier position lines were measured from the edge line using a ruler and marked at 2 cm increments from the edge up to 10 cm away from the edge line. Smaller spacing between lines (e.g., 1 cm) could also be used for finer uncertainty measurement. For each test trial, the barriers were moved towards the AGV to the next line beginning at 10 cm for trial 1, 8 cm for trial 2, and so forth until the navigating vehicle detected a barrier, and emergency-stopped the AGV, thus completing the test run.
A series of eight trials were completed with nearly all trials including three or more runs each to demonstrate the navigation test method concept. Ten or more runs are ideal for statistical analysis. The optical measurement system mentioned earlier was used as an experimental ground truth (GT) to measure the barrier and vehicle position during experiments to further understand the test method and vehicle performance. The barriers and AGV were marked with spherical reflectors (visible in Figure 4 (a, b, and c) detectable from the GT system. Figure 5 presents GT data plotted for navigation tests showing ground truth data of: (a) test 8 vehicle path and emergency stopped vehicle (red circle) when a wall was detected, (b) test 1 path, and (c) test 1 path data from (b) zoomed in to show data points of three runs.
blue barrierposition lines a b
c
Experimental results from the barriers demonstrated a path uncertainty of between 6 cm and 8 cm maximum when the vehicle detected the boundaries at nearly the center of the straight line path and when moving at either 0.25 m/s or 0.50 m/s. The navigation test method using barriers is simple and cost-effective for manufacturers and users to employ, as compared to the higher accuracy, but more expensive ground truth visual tracking system used for test method development.
A simple straight line with one turn was tested. However, more complex test configurations, such as shown in Figure 1, could be set up using B56.5 test pieces instead of larger, physical barriers as were used in this research.
Page 50 when a wall was detected, (b) test 1 path, and (c) test 1 path data from (b) zoomed in to show (red, green and blue) data points from three runs.
A working document that addresses quantifying vehicle navigation uncertainty is being developed as an initial step towards a performance standard for ASTM F45.02 subcommittee on Docking and Navigation. Based on consensus of the task group developing this standard, as was tested at NIST, the simple path-bounding test method using temporary reconfigurable barriers made from readilyavailable, off-the-shelf materials is being proposed.
B. Vehicle Docking
Vehicle docking is another common application of mobile robots and AGVs. Unit load (tray, pallet, or cabinet carrying), tugger (cart pulling), and fork/clamp (pallet or box load/unloading) are typical industrial style vehicles that require different docking uncertainties. For example, a unit load vehicle that places/retrieves platters during wafer manufacturing would no doubt require less uncertainty than a fork style vehicle that places/retrieves pallets. As robotics advances, current and potential users are requesting mobile manipulators to perform tasks such as unloading trucks.
Eventually, it is expected that mobile manipulators will be
used for smart manufacturing assembly applications [
Similar to navigation, there are no performance measurement test methods that define how manufacturers and users characterize their vehicle’s docking capabilities. Figure 6 (a) shows an example method for docking for any style vehicle. A vehicle approaches and makes contact with ‘a’ and/or ‘b’ docking points dependent upon the vehicle type.
Relative displacement from each of the points would be measured to determine vehicle docking uncertainty. A forktype AGV is shown docked with a test apparatus in Figure 6 (b). The fork tips are marked with yellow points. Figure 6. (a) Example docking test method using various AGVs (e.g., 1 and 2 for AGV unit load tray table docking, 3 for fork and tugger AGV docking).
“a” and “b” are fixed points in space (e.g., contact or non-contact sensor locations in space). Approach vectors and sensor point spacing and locations are variable. (b) Fork-type AGV docking with a docking apparatus.
Two experiments were simultaneously performed: AGV
docking relative to known facility locations and GT system use
for measuring AGV docking. Two different GT measurement
systems were used to measure AGV performance: a laser
tracking GT with an uncertainty of approximately 10 µm [
An experiment using an uncalibrated AGV that was
programmed to stop at various points yielded an uncertainty
range of approximately 1 mm to 50 mm. Figure 7 (a) shows
the vehicle paths and Figure 7 (b) shows average errors for five
runs at stop or dock points. The vehicle position was measured
using a laser tracking GT system which provided
highprecision measurement of AGV stop points. [
A 6 DoF optical tracking GT system was used instead to measure AGV docking. Docking was measured again after the AGV was calibrated using the manufacturer’s procedures. The AGV approached similar dock locations and after AGV calibration, provided consistent 5 mm uncertainty. Standards development for optical tracking systems is also underway and is discussed in section 2 D, 6 DOF Optical Measurement of Dynamic Systems.
(a) Docking points
(b) Figure7. (a) Commanded paths and stop points and (b) stop point errors of a
single AGV point for each location in (a) averaged over 5 runs.
Additional AGV equipment docking experiments were
also performed using a mobile manipulator and a
reconfigurable mobile manipulator artifact (RMMA)
developed at NIST (see Figure 8). [
Page 51 AGV provided relatively low repeatability uncertainty (e.g., +/-5 mm) although more than 10 mm from the commanded docking points. This manipulator could reach the commanded points on the RMMA even with 10 mm uncertainty in AGV position. The mobile manipulator corrected for the position uncertainty after being taught the actual RMMA locations. At the RMMA, the manipulator, wielding a laser retroreflector, was commanded to move in a spiral pattern to detect 6 mm diameter reflectors. The reflectors provide non-contact alignment detection of the tool point position and orientation.
The experiment provided results demonstrating that this relatively inexpensive ground truth measurement method was sufficient for measuring docking accuracy. As the reflector based measurement system is inexpensive compared to the optical tracking-based GT, it may prove ideal for use as a precision vehicle/mobile manipulator docking test method that both manufacturers and users can replicate.
Manipulator RMMA
AGV
C. Obstacle Detection and Avoidance
Obstacle detection and avoidance (ODA) research is well
documented in the literature for mobile robots. However,
there are few citations for AGVs perhaps due to the relatively
closed nature of commercially available AGV controllers and
because ODA is not often implemented on AGVs deployed in
large manufacturing facilities. In [
The navigation performance measurement experiment discussed previously in section II A. Vehicle Navigation can be similarly applied for obstacle detection and avoidance. In fact, the ASTM F45.02 subcommittee navigation and docking task groups have discussed the potentially overlapping nature of the two vehicle capabilities. The ASTM F45.03 Obstacle Detection and Protection subcommittee is currently in the process of considering standards in this area. Questions have been raised regarding standards development as follows:
How well does the AGV react to situations? For example: Obstacles appearing in the path Potential obstacles headed towards the path Unstructured (i.e., changing obstacle locations) areas not on the original planned path or that rapidly change How far off the commanded navigation path can an AGV be, and at what speeds, before it violates the path and causes a stop? For example, due to environmental factors such as: Offset-pitched/rolled AGV can’t see guidance
markers, such as reflectors, magnets, wire, etc. Guidance or boundary-marking tape is worn or
broken Terrain causes “bouncing” or moving laser or other
navigation sensors How well does the vehicle react when a human is detected and how should the human be represented? For example: By test pieces, mannequins, humans With what coverings? (i.e., what clothes should be
worn?)
How to interact with manual equipment (e.g., forklifts,
machines)
How to standardize communication of vehicle
intelligence for obstacle detection and avoidance? For
example:
Contextual autonomy levels [
Experiments to support ODA performance test method development will be performed based on forthcoming guidance from the ASTM F45 subcommittee. However, a prototype safety test method that has been developed to evaluate a vehicle’s response to obstacles in its path and within its stop zone, as noted in the Introduction, can be considered a first step towards full ODA standard test methods. ASTM F45 is meant to dovetail with safety standards such as
ANSI/ITSDF B56.5. Therefore, providing an initial test
Page 52
method for detection of obstacles is ideal as a starting point for
F45.03. The ‘Grid-Video’ detection method [
D. 6 DOF Optical Measurement of Dynamic Systems
ASTM’s draft Standard for the Performance of Optical
Tracking Systems that Measure Static and Dynamic Six
Degrees of Freedom (6DOF) Pose (see Figure 10) is the next
step beyond the static case covered by ASTM E2919-14 [
Optical tracking is being used for robot and autonomous
vehicle GT measurement, as discussed in this paper. Optical
tracking measurement systems [
Ground truth cameras
AGV
In the initial test procedure, measurements with uncertainties were computed using an artifact – namely a metrology bar as shown in Figure 9 (a). Current optical tracking systems utilize a three-marker metrology bar with all markers in a line which does not provide 6 DOF system performance measurement. A metrology bar made of carbon fiber with length 620 mm and with five reflective markers attached on each end was used as the 6 DOF artifact. A carbon fiber bar is used since it limits the effects of thermal expansion. The metrology bar markers on each end form a constant relative 6 DOF pose between the two ends. A shorter bar length should be used for smaller space measurements to maximize metrology bar measurements.
movement during dynamic
Most optical tracking systems have at least a 30 Hz data collection rate. Therefore, a minimum of 5 min of data needs to be collected. The workspace is uniformly divided by the artifact length. The artifact is moved using at least the minimum and maximum motion capture velocity specified for the system.
The static test procedure for measuring the performance of the optical tracking system is to divide the test space into a grid and place the artifact at intersections of the grid and at various orientations. The dynamic test procedure also divides the test space into a grid where the metrology bar is moved in a raster scan pattern forward-to-back and left-to-right throughout the space.
The metrology bar maintains a constant separation and orientation of the two marker clusters along all the paths and can be rigidly attached to and moved using a wheeled frame as illustrated in Figure 9 (b) that is pushed/pulled by a human, a mobile robot, or other mover to closely follow the path.
The metrology bar is moved at the maximum specified
velocity of the optical tracking. Pose error measurement and
reporting methods are also described in the ASTM WK49831
[
III. CONCLUSION
The AGV standards development process has been limited for many years to considering only safety standards. Starting in late 2014, ASTM F45 Driverless Automatic Guided Industrial Vehicles performance standards are being developed to include navigation, docking, terminology and several other key areas for AGV’s, mobile robots, and mobile manipulators. As discussed in this paper, standard test methods for measuring vehicle performance are being developed so that manufacturers and users of these systems can easily replicate the measurements in their own facilities and at minimal cost and effort. More AGV and mobile robot systems, instead of just the one AGV used in these experiments, would ideally validate the generic test method proposed.
A comparison of GT measurement systems was also made to support the test method development. It was determined that for dynamic AGV measurement, an optical tracking system provided a suitable ground truth measurement. At the same time, a standard for these dynamic measurement
Page 53 systems is also being developed. The standard will allow vehicle and robot performance standards developers to use the systems as ground truth with known measurement uncertainty. Optical tracking systems users and manufacturers can replicate the same test methods with similar tracking systems and use the results to compare their performance at dynamic tracking tasks.
ACKNOWLEDGMENT
The authors would like to thank the ASTM F45.02 subcommittee navigation task group and Omar Y. AboulEnein, Salisbury University student, for their recent input to navigation test method development and experimentation.
Also, we thank Sebti Foufou, Qatar University, Doha, Qatar, for his guidance on the mobile manipulator docking performance measurement research.
Page 54
Gualzru’s path to the Advertisement World F. Ferna´ndez, M. Mart´ınez Planning and Learning Group
University Carlos III Madrid Email: fffernandg@inf.uc3m.es
I. Garc´ıa-Varea, J. Mart´ınez-G o´mez
SIMD
University Castilla-La Mancha Email: fismael.garcia,jesus.martinezg@uclm.es
J.M. Pe´rez-Lorenzo, R. Viciana
M2P
University of Jae´n Email: fjmperez, rvicianag@ujaen.es
Abstract—This paper describes the genesis of Gualzru, a robot commissioned by a large Spanish technological company to provide advertisement services in open public spaces. Gualzru has to stand by at an interactive panel observing the people passing by and, at some point, select a promising candidate and approach her to initiate a conversation. After a small verbal interaction, the robot is supposed to convince the passerby to walk back to the panel, leaving the rest of the selling task to an interactive software embedded in it. The whole design and building process took less than three years of team composed of five groups at different geographical locations. We describe here the lessons learned during this period of time, from different points of view including the hardware, software, architectural decisions and team collaboration issues.
Gualzru is a social robot built as an advertisement tool for a consortium of technological and digital media companies within the ADAPTA1 project. The core of this project is an interactive panel able to provide personalized advertisement according to the preferences of the user. To achieve this goal, the consortium includes advertising companies, media asset management, software developers, technological consultants and software infrastructure providers, coordinated by the Software Labs group from the Spanish Indra company. The idea of using a robot as a more personal way of bringing people’s attention was suggested in order to endow the panel with the ability to recognize the emotional state of the user and to classify her according to the estimated age and gender.
After agreeing on creating a new social robot, it was decided that Gualzru, would team up with the interactive panel and boost the advertisement potential of the platform. The project started in May 2012 and this paper describes our experiences, successes and failures, during the three-year process.
From the early analysis of the problem we knew that the project needed the expertise from different research groups.
So, once a solid group of complementary researchers was agreed, we accepted to join the project’s consortium. Our 1See Acknowledgments section at the end.
first ’robotic’ consortium was composed by the Universities of Ma´laga, Extremadura and Carlos III of Madrid. From 2010, the first two groups were working together on the definition of a software framework for robotics, which could be used for the project. The University of Extremadura would also build the platform and would be the responsible of endowing the robot with the abilities for autonomous navigation and facial emotion detection. The University of Ma´laga would address the rest of vision-based problems (e.g. use facial descriptors to estimate the gender or age) and help with the navigation modules. Finally, the University of Carlos III of Madrid would be in charge of the high-level planning and learning modules.
Everyone agreed in using this project as a test for the initial proposal from the University of Extremadura: to organize the whole software architecture around a centralized internal model of the outer world. Such representation is accessed by all software components to keep them informed about the current world state. They can also update it as the result of processing the data from the sensors. The ADAPTA project will provide a controlled but realistic scenario for testing the idea.
One major requirement of the proposal that the robot initially lacked was the ability to dialogue with people. To solve this problem the SIMD group from the University of Castilla-La Mancha, summing a large expertise on automatic speech recognition and natural language processing, joined the Consortium. Furthermore, the human-robot interaction ability was strengthen with the incorporation of the researchers from the University of Jae´n. Six researchers from all groups were contracted during different periods of time to work on the project, however a larger group of researchers was always involved on the project.
A. Team coordination and sharing of resources
The coordination of this large group was supported by the use of collaborative tools. However, we soon understood that the only way to make a steady progress in the development of a large and complex project like ADAPTA, was by sharing a common code base and by scheduling periodic hackathons
Page 55
Fig. 1. Gualzru the robot in which the members of all the teams could seat together for a week and fight a specific, common battle. This strategy naturally led to the division of the project in well defined milestones, consisting on system features to be integrated and tested during the one-week period. Coordination was therefore subtended on:
A unique robotic prototype, available from month zero in a robotics simulator.
A common programming framework, RoboComp [17], used in several previous projects. All the software developed for the project had to qualify as a RoboComp component, meeting the established quality standards, and had to be uploaded to a common git repository.
A common cognitive architecture, RoboCog, available for all researchers and where individual modules could be inserted and tested minimizing the knowledge required about the rest of the architecture The organization of several intensive working weeks hackathons- coinciding with the project milestones. These meetings were intense and dedicated to integrate and debug specific target functionalities.
To maintain the global view of the project and of the specific requirements, all the members should meet for each milestone and have always access to an open document storing this information. The document was edited online by all researchers and also served as a battlefield to discuss technical issues. We did not always coincide about how to do things but we agreed that the digital arena was the right place to fight.
III. GUALZRU
Gualzru, a phonetic transcription of the English phrasal verb ”walk through” pronounced by a native speaker of Extremadura, is a 1.60m. tall robot with an external cover built of resin and fiber glass, and a differential base with two powered wheels and two casters. It includes gel lead batteries that provide an autonomy of three hours and all the necessary power electronics, recharging and power supplies for the sensors and processors. The complete fabrication of the robot was custom made by the groups of the consortium.
Table I shows the complex handcrafting process of Gualzru’s external cover. This step was one of the most exasperating and time-consuming in the overall development of the project. It was a relatively new process for us with many steps that were out of our direct control. Going in Table I from top to botton and from left to right, we can rapidly summarize the manufacturing steps: 1) Gualzru’s initial 3D design. To come up with a nice robot image we set up a public design contest among all Spanish universities and people and companies in the design business. One person from Ca´diz, Spain, was selected among more than 30 proposals with a poll among a selected resolution committee. 2) The 3D drawings were sent to a company specialized in manufacturing expanded polystyrene molds using industrial CNCs. We learned that the choice of prices and qualities here are apparently important, since the final quality of the surface of the cover and the number of hours spent by the sculptor in fixing the small imperfections generated in the machining process were closely related. It is important to assure the final quality level in this early stage. 3) An external coating over the mold is necessary to facilitate the unmolding process. The mold is split in two halves. 4) A thick silicone layer is manually applied on the mold with additives to avoid sagging. This layer is called negative. 5) On top of the silicone a resin with fiber glass layer is
applied to create a rigid external cover called mother. 6) Both layers are unmolded. 7) The silicone mold after being separated. 8) A positive mold is finally built by applying resin and fiber glass inside the negative. After drying, the cover is unmolded from the silicone and both parts are glued together. A final polish work is done to obtain a nice texture. 9) A solid and reliable differential base is built as the
mechanical core of the robot. 10) The cover is fit on the base. Additional holes and slits have to be carved to allow for laser, camera, fastening, etc. 11) Sensors are incorporated to the robot. The tactile screen is placed after a final coating is ordered to a car painting workshop. 12) Gualzru at the University of Ma´laga in a public event
with the University’s Provost.
As a summary of the experience it is evident that the process is slow, expensive in working hours and almost impossible to rectify if a new idea comes by. The whole process took us many more months than expected and we had to use a replacement Nomad 200 robot while the robot was being built.
In summary, it is a valid solution to the cover problem but with the arrival of 3D printing technology, all chances are that future robot covers will be divided in pieces small enough to be printed in a modern 3D printer, and then assembled together. There are also new small companies starting to offer these kind of services.
Page 56
TABLE I A SERIES OF SNAPSHOTS OF THE BUILDING PROCESS OF GUALZRU. SEE TEXT FOR DETAILS ON EACH STEP AND THE CONCLUSIONS OBTAINED AFTER IT
WAS FINISHED.
IV. ROBOCOMP
As commented in section II-A, one of the few things that were already clear when the project started, was the need of a common code base.
A big part of the group had been already working in previous projects together and sometimes with other partners.
From these works we learned that one of the main causes that prevented the formation of a cohesive, long lasting group with a common goal was the fact that each one was coding their own programs on different frameworks or without one at all. There are many robotics labs around, still unable to organize and create a coherent code base that grows from the accumulated work of dozens of researchers. After some tough negotiations involving the different frameworks that the groups were using or planning to use, we agreed to use RoboComp.
We believe there are several reasons that, in the hindsight, justify this decision:
We keep the control of the core and thus, we decide when to change and when to hold. It looks like a contradiction but when some complex open source software is very soon used by thousands of people, its evolution freezes or slows down almost immediately. The reason is that the core decisions made at the very beginning cannot be easily changed without generating compatibility problems and versions nightmares. As an example you can look at the widely expanded Microsoft’s operating system (Windows) and the relatively slow addition of new features with each release (mainly nothing on the core changes).
This does not mean that good software cannot be used by many people, but that complex software that deals with new, changing, not very well defined sort of things, takes its time to settle down.
RoboComp’s component model has been evolving since its beginning and has the necessary complexity for our needs. Not more.
The current communications middleware, Ice by ZeroC
[
New middlewares could appear in the future with some
game-braking features. In that case, if you control the
framework you can define a reduced set of
communication primitives, like the ones proposed by Schlegel in his
PhD thesis [
A code generator is mandatory so the generic part of the components is always the same, compiles without errors and keeps the required quality levels. Code re-generation might be trickier but there are several techniques. RoboComp splits the working part of the component in two using inheritance. The inherited part is always generated and the part that inherits is generated only the first time.
A lesson learned here is that it is easier if all tools and technology in the framework use the same development language and environment. Better if it is the one that most of the users are familiar with. Otherwise the no common specific tool becomes a bottle-neck that might delay and affect other parts of the framework.
Our initial code generator was created with the Eclipse ecosystem using the existing tools it provides for DSL designs. This tool turned out not to be easily adjustable by developers (since they mostly develop C++ and Python) and a heavy environment that would not exactly match the team needs. Therefore, we ended up rewriting a lighter code generator in Python using pyparse and COG so
Page 57 everybody could collaborate in the natural evolution of the tool. Now, RoboComp’s code generator generates also Python components, that are becoming more a more popular due to their simplicity.
We have developed all the tools we needed, although there are always tools that we would like to have but we have not had time to code them. RCIS deserves a special mention, RoboComp’s simulator, that it has been there almost since the beginning of the framework. By the time RoboComp started the only existing open source simulator was Gazebo and it was in its early versions. If you are building a robotics framework and have already decided on the communications middleware, the chances are that you want a simulator that speaks the same language as the components of the framework. Only doing so, the simulator would behave like a component or several components with all the advantages that come with that.
Therefore we wrote RCIS using Open Scene Graph [20] and an initial scene specification language that we named InnerModel. Later on, we discovered with great joy that having our own simulator would immediately provide us with an emulator. That is, a simulator that could be run inside the architecture computing in super real-time future courses of action and predictions. That is now part of our new architecture CORTEX, which is still in development.
V. ROBOCOG
Initially, we addressed the ADAPTA project from a very specific point-of-view. That is, giving the use case, we translated it to a finite state machine and assigned tasks to software components or groups of them that we call agents. The idea of using a finite state machine to manage the whole use case was soon unbearable. The number of states and transitions grew with every bit of reality added to scenario.
Even modern hierarchical and concurrent formalizations of
state-machines [
We thus decided to take the hard way to a fully fledged
symbolic planning system, in charge of the automatic
generation of those huge state machines. The Planning and
Learning Group at the University Carlos III of Madrid had
a very long trajectory in these disciplines and was the perfect
match to provide the needed technology. The use case was
translated into a PDDL domain specification [18] and several
planning algorithms were tested for that domain. A separated
interface was clearly defined between high and low level
domains. High-level being the domain of logic attributes and
predicates, and low-level the domain of behavior agents that
receive parametrized calls to act and provide metric values
for relevant variables of the world state. The interface layer
translates between high and low level, so both worlds are kept
communicated. Of course, it is also the main cause of the so
called, symbol grounding problem [
This now familiar scheme was synthesized by Erann Gat as
the three-layered architectures [
To us, it looks like that the good engineering practice of
decoupling the problem in parts of infinite impedance, took away
a crucial element, context. It is sometimes argued that context
is somehow coded in the interactions between agents [
To start, we already had a representation of the robot and its close environment in the form of a scene-graph, called InnerModel. This simple DSL served to initialize our RoboComp’s 3D simulator, RCIS. Therefore, RoboComp’s InnerModel was the perfect starting point to develop the idea of a shared representation of the robot, the environment and the people in it. The initial scene graph specification language was gradually extended to include more types of objects. Also a C++ class was written to hold in memory the graph and allow an easy and safe access to all the handy functionalities that this structure provided, such as coordinate transformations, measuring, insertion, modification and removing of nodes, perspective changing, frustum reachability, etc.
A basic scene-graph is essentially a kinematic tree with
some add-ons. We had to incorporate all the symbolic
information needed by the deliberative elements of the
architecture. The requirements were that the perception-action
related agents could update a fixed set of symbolic attributes
and predicates, and that the selected representation could be
efficiently translated to PDDL, so a specialized planning,
executing and monitoring framework like PELEA [
Page 58 The current solution we are working on is the embedding of the kinematic tree inside the symbolic graph, and the code necessary to efficiently extract and insert the tree in a format that can be used by the many components that were written before the integration. It is hard to evaluate if the other choice would have permitted us to finish the robot on time, saving the posterior integration step. Software developing time is really hard to estimate, specially when robots are in the loop.
The new graph was named AGM, for Active
Grammarbased Model [
AGM and InnerModel where independent structures and their coordination was managed by ad-hoc procedures, but AGM was finished on time and will maintain a complementary, symbolic representation of the robot and the objects in the scene. For example, a human in front of the robot being detected and represented as a skeleton inside InnerModel, could now be tagged happy, focused or woman in AGM. With AGM we had the missing part of the architecture and all the groups could start to meet in hackatons and reach, one by one, the urgent remaining milestones of the project. In summary, the main purpose of this dual representation was to provide both, a local description that could be updated and used by the different agents for their computations, and a shared context that is propagated among them to carry information that otherwise would remain hidden.
As a result of the graphs occupying their places with the
agents, the overall idea of RoboCog started to change and
we started to move from the three-tier original model to a
non-hierarchical disposition in which all agents gather around
these shared graphs and interact among them by reading,
writing and propagating the changes. The abstraction axis is
hidden inside the agents and defines what parts of the graph
are accessible by its internal components. This is discussed
in recent works by the group [
When a mission is assigned to the robot it is internally re-coded as a desired state in an AGM graph, which could include the whole world or just the symbols needed to satisfy the mission. The Executive module is the one in charge of achieving it. The steps needed to transform the current AGM graph are provided to the Executive by the Task-based Planning and Monitoring module PELEA as a sequence of tasks that are injected back in AGM. At this point, AGM holds the current belief about the world and the current desire about how the world should be. Agents scan the graph and find tasks that can be performed by them. Inside each agent there may be some limited capability of planning or sequencing sub tasks, e.g., maintain the interest of the person through dialog, monitor the correct execution of a gesture, recognize her facial emotions, etc. The most basic components are in charge of sensor motor loops and normally execute their commands without interaction with other components.
VI. THE USE CASE
We now describe the use case that constituted the main goal of the project. It was defined in the ADAPTA’s kick-off meeting. The first version of this use case is depicted in Figure 3. It can be textually described as,
Gualzru is waiting in the Waiting area. It is now ready to start one of its tedious working days. The Waiting area is at the middle of an uncluttered corridor in a large shopping center. People usually enter this side of the mall from the left side of the corridor, crossing in front of the Panel area before entering the shops. People going out the mall also cross in front of the Panel area, but walking toward the left part of the corridor. The objective of Gualzru is to offer products and services to all these people.
In fact, its aim is to drive potential consumers to an advertising panel, in which these products and services will be displayed. As there are products for everybody, it can choose any person in the corridor.
When it chooses a target, it moves from the Waiting area following an intersecting trajectory with the person’s heading direction. This displacement is very short (2-3 meters maximum) and allows Gualzru to wait for the person in a static pose, facing her at comfortable social distance (1,5-2 meters minimum).
Therefore, Gualzru can say ’hello’ to the person without scaring her even if she is not very used to interact with a moving robot.
If the person engages with him in this first contact, Gualzru will classify her into a group -using gender and age parameters- and will choose a product topic to offer. Product topics provide Gualzru an specific theme of conversation before inviting her to walk back to the Waiting area. During this short conversation, Gualzru will be always ready to say goodbye to the user if she shows the intention of leaving the conversation or if the presented product topic is not interesting to her. On the other hand, Gualzru must also check its batteries level to say goodbye and move to the Charging area if this level is under a minimum value. The Charging area is close to the Waiting area. In fact, when Gualzru arrives to the Waiting area, he will home itself to the Charging area. If the person agrees on going with Gualzru to the Waiting area, both move there and the robot says goodbye to her. Then, it returns to the Waiting area and waits for some time, which is the expected time that the person is going to be at the panel, before starting the process to select a
Page 59
Fig. 2. An overview of the RoboCog architecture (from [16]) new target. As before, if batteries level are under a certain value, Gualzru moves to the Charging area for a reload.
Fig. 3. The ADAPTA use case
VII. AN ANNOTATED DIARY
As was mentioned before, the coordination of the project was based on periodic hackathons. We think this decision was a real success. We have already noticed the difficulty to integrate complex software and reach milestones without a real motivation from the people working in different labs. Many times, the global objective or the potential implications of the work are not correctly perceived. Other, personal relations get in the way. Hackatons have turned out to be an effective way to code, debug, test, share, make progress and build a team spirit.
A. May 2012. Kick-off
The project initiated with a kick-off meeting at Ma´laga where the overall strategy was discussed and the periodicity of the meetings was set.
B. December 2012. The ”WORST” workshop at Ca´ceres
The main objective of the first hackathon was to explain and establish RoboComp as the common code base. All groups on the consortium had certain degree of knowledge about RoboComp, but it was considered mandatory to organize a workshop where simple examples could be programmed by all researchers under the supervision of experts from the Universities of Extremadura and Ma´laga. Fifteen people from all research groups and some more and some from Indra Software Labs met at Ca´ceres.
C. May 2013. First public demonstration at Ma´laga
For the first public demonstration of the whole project we had a simple prototype of Gualzru (Figure 4). Two autonomous behaviors were tested: the reactive navigation and a face detector. The AGM graph and the kinematic tree were able to internalize the perceived information. The seed of the architecture was planted. Obviously, not everything worked properly.
The algorithms underneath both behaviors were changed in the final version, for example. But this fact was rather usual during the project. Other issues were more time consuming as expected. During 2013 we tried to replace the laser by
Page 60
Fig. 4. The initial internal skeleton of Gualzru
an array of RGBD sensors, arranged in a configuration that
provided a wide field of view and theoretically gave good 3D
coverage. Researchers from the Universities of Extremadura
and Ma´laga were involved on achieving this goal. The sensor
had limitations to perceive at short distances and we had to
connect them to embedded computers like the Raspberry Pi of
the time to liberate the USB ports in the main computers of
the robot, Intel’s NUC. We could not make the Asus’ Xtion to
run reliably with the available Raspbian. A few months after
the end of the project, we succeeded with another board, the
ODroid C1 [
D. November 2013. First evaluation of the architecture at Albacete
One of the major goals achieved after the meeting at Ma´laga was the development of a complete architecture able to work with a simulated robot in a virtual environment. The so-called ’empty boxes’ architecture took this name from the fact that it included a complete version of the architecture RoboCog, although some of the components only had the public interface -IDL file- and the structure inside. Nevertheless, it included the two inner graphs -the symbolic and the geometric-, the conversational module, reactive navigation, person detector and high-level planning, executing and monitoring. To play with it we did not need the physical robot, but a computer with a RGBD sensor, speakers and a joystick. The simulated robot operated as an autonomous agent and we were able to move a virtual person in the simulator using the joystick. When robot and person were at interaction distance, the robot tried to convince the person to accompany it to the advertisement panel using his conversational skills. The speakers and the microphones on the RGBD sensor were used to support this Fig. 5. Playing with the ’empty-boxes’ architecture. The dashboard shows different panels: the one on the right shows the graph models that encodes (a) the goal to achieve -target model plan, and (b) the symbolic view of the outer world. It also includes the current action of the plan (’approachperson’ in this figure). The panel on the left shows a visualization of the kinematic tree -upand of the virtual world -down. We did not endowed the virtual robot with virtual sensors. The person is automatically detected if she is in front of the robot. interaction stage. The RGBD sensor was used to detect the face of a real person during the conversation. It was an intensive integration task.
One of the major successes of this architecture was the development of the triangle, high-level decision maker - executive - symbolic graph model. We were now able to translate to PDDL the information stored and updated in the symbolic graph (AGM) to the PELEA framework at the deliberative level. Furthermore, the Executive module was able to publish the graph to all software components on the architecture when a change was introduced. These components were arranged on networks, connected to the Executive through one distinguished component, the so-called agents. These agents were the responsible of maintaining the coherence of the information stored in the inner model, since they update the graph-model and, simultaneously, the geometric information of the kinematic tree. This second route was not supervised by the Executive. For the first time, the new definition of agent, included formally in the RoboComp component model and code generator, allowed all participants to share the graphs using the same interface. Our shared global representation on the state was now real and working.
We were able to launch more than fifteen software components. From this point of view, we were able to modify or add new components over a full-integrated architecture. Each successive meeting would imply a refinement of the previous proposal. While waiting for the robot Gualzru, see Section III for reasons explaining the long wait, we set up an old Nomad 200 robot with the RoboCog architecture and organized a new meeting at Albacete. See Figure 6.
At Albacete we evaluated for the first time the robot’s behavior through questionnaires filled by the people interacting with the robot. The questionnaire is designed as a Likert scale, although it uses six levels, from 0 to 5, to remove the
Page 61
Fig. 6. The old Nomad performing through the use case at Albacete
neutral option -middle point. It is similar to that employed
by Joosse et al. [
Its main difference is that it has been created not from the point of view of the person observing the behavior of the user against the presence of the robot, but from the point of view of the same user that interacts with the robot. In this sense, we can consider that it collects influences of questionnaires of the Almere original model for man-machine interaction. In particular, the questionnaire includes a collection of questions arranged in four blocks: navigation, conversation, interaction and general sensations. The user fills the questionnaire giving a value for each response between 5 (completely agree) and 0 (completely disagree). These questions are listed in Table II. From that point on we would use that tool to evaluate the evolution of the project. Although at Albacete we were able to run the software in a real robot, we only could finish 12 use cases with different users. The number of questionnaires were also reduced to take hard decisions based on it, however the results were promising.
E. May 2014. Second public demonstration at M a´laga
The Nomad 200 was moved from Albacete to Ma´laga and we continued testing specific problems related to the navigation module, speech generation and recognition, and person classification based on age and gender. The communication among components was forbidden and all information was transmitted through the inner representation. Things still did not work as we needed and the causes were not clear. Then, the robot Gualzru arrived and all efforts were translated to getting it ready.
For the second public demonstration Gualzru was already running RoboCog. Probably, this was not a mistake, but we did this integration without time to test the whole system.
Also, we put the emphasis on collecting a larger collection of questionnaires. And to worsen things even more, the meeting was not set as the other previous hackathons: the goal did not focus on solving technical problems that were really there, but on showing a prototype that, at the end of the day, we should have known that it would not do the job. And the results were not good. During the demonstration the robot was able to interact with a person but it showed its limitations: the odometry alone was not able to correctly solve the localization problem, speech recognition had problems to work on crowed environments, the person classification module blocked the full use case until a good image of a face was taken, and so on.
Another significant lesson was learned and never forgotten: you can not say that your robot will succeed in one trial until you have tested it for at least hundreds of times. There is a saying that can be applied here: ”let’s rehearsal so hard that the show looks like a rest”.
F. June 2014. Hackathon within the Workshop on Physical Agents
In June 2014, all groups had talks within the Workshop on Physical Agents (WAF2014) to be held in Leo´ n (Spain).
We asked the local organizers to facilitate us a working space to set up another hackathon during the week before. Vicente Matella´n, the conference director gave us a cordial welcome and provided an excellent place for testing.
Before the hackathon, we discussed and organized the problems to solve there and when we arrived to Leo´ n everybody knew what to do and joined in groups for a long week.
The result was a real success: the dialog module was largely improved and tested, the localization problem was solved using AprilTags landmarks, and so on. The use case was repeated and repeated, and for the first time we detected real bugs and problems to deal with. After several days of intense work,
Page 62 the robot. This situation is more common than expected due to the interest the robot produces. Additional issues such as different accents, voice volumes, etc. add more difficulties to the scenario. Despite these limited conversational skills, Gualzru achieved its main objective, to capture the attention of people. Most of them enjoyed the experiment and also would recommend the experience to friends or would like to repeat it. Comparing these results with the ones collected in the first experiments, revealed that successive updates in the robot have made it more robust and its conversational abilities, while still constrained, have been significantly improved.
H. Last stage: refining the HRI Gualzru was able to do its job relentlessly until the battery was off! The robot spoke with all of us in the Lab and accompanied us to the panel. We had the impression that all ours problems were solved. We were happy for the moment...
But we were not going to enjoy the success for a long time. After discovering that the learning module would make the robot avoid people that always answered: ’No, I do not want to go with you to the panel’, our host asked us to move Gualzru to the large hall where the conference was about to start and to have it welcoming the assistants. It looked like a good scenario for our use case. We accepted.
The moment we moved to the hall new problems appeared.
The robot was unable to talk to people because nobody, not even humans, could hear what the other was saying. The space was wide open and we could not find a good place for the AprilTags landmarks. Light conditions were changing and the algorithms in charge of the RGBD camera did not always run correctly. For our younger researchers the experience was really hard, as they passed from the complete success to a glaring failure in a short time. Nevertheless, we were now in the final scenario. A spacious environment where the robot must interact with a specific person while other people are speaking and moving around. The failure had an aftertaste of an approaching victory. We still were able to close some use cases in this challenging scenario and a new time for improvements had started.
The conversational abilities represented a severe drawback.
Despite our efforts, only 50 % of the people that interacted with the robot in these real scenarios thought that it was able to maintain a coherent conversation. This was not enough for a robust, useful robot. But if you cannot solve a problem, perhaps is a good option to totally change the way to solve it. The speech recognition issue is hard to solve in noisy and crowded environments, where even humans find difficulties in understanding each other. Therefore, our idea was to look for alternative methods to allow people communicate with the robot. Speech recognition was reinforced with the incorporation of a tactile screen installed on the chest of the robot. The verbalized phrases were now displayed on this screen and it was possible for the person to answer the robot by touching G. December 2014. Large evaluation test at Ma´laga it. This way, Gualzru retained its conversational abilities but
After a new demonstration at Ingenia (Ma´laga), in an the new interfaces increased its robustness and reliability.
environment very similar to the hall at Leo´n where we could Following this modification, a new set of questionnaires were
capture new questionnaires, we returned to the Lab. The array collected on the same scenario at the University of Ma´laga.
of microphones of the Kinect sensor was intensively tested These questionnaires showed us that the mean values related
and, finally, we decided to change it for a shotgun microphone. to questions 2.1 and 2.2 (Table II) improved from 3.57 to 4.27
As it is described with more detail in [
On December, 2014, the current version of Gualzru was % of the started use cases (on December 2014, this rate was 81 tested in a real working scenario. The system was deployed %). Furthermore, the unfinished use cases were always caused in the hall of the Escuela de Ingenier´ıas at the University of by the large amount of people in the place that would prevent Ma´laga. The area where the robot was operating was about 70 Gualzru from reaching the panel. square meters. Fixed obstacles included a column and some tables, but most of the area was free for the robot to move. VIII. CONCLUSIONS The hall was populated by students and the trials lasted two In this paper we have presented the long process of creation half-days. The robot worked without human intervention and of Gualzru, the salesman robot built for the ADAPTA research engaged with people passing nearby. These people had no a project. Looking at the starting requirements, we can firstly priori knowledge about the robot, nor its functionality. We conclude that the final version of the robot conforms with collected a large set of questionnaires. The results are shown the goals and expectations that we and the companies in on Table II. the consortium initially had. But it is also true that even
This data showed that the conversational system remained more rewarding than Ursus has been the whole process of as the weak point of the robot. Some people did not correctly collaboration and the knowledge distilled during these years. understood the robot due to the environmental noise and the It is not that common that basic research is taken close voice of the robot was perceived as not particularly pleasant. to the production line while all the intermediate steps are But the most important issue was related to the understanding registered and analyzed as a means to improve both, the capabilities of Gualzru. Even when using the shotgun micro- forthcoming research and the methods and ways to generate phone these capabilities were strongly limited. The system is reliable technology, given a limited amount of resources. For too sensitive to environmental noise and echos and it gets us, it has been a productive experience, both personally and also confused when there are several people speaking around professionally, and the capacity of the group to approach
Page 63 new technological challenges has increased notably. We have learned something useful in every step of the project, from the handcraft manufacturing materials and steps, to the way humans are starting to look at the (social) robots.
From the point of view of the technology that has been created and used in the project, we reaffirm the initial idea of the need for a common code base that brings together the work of all researchers. We still need some adjustments in the protocols and some refinements in the technology, and even more conviction by some doubters, but at the end of the day we might well be in the right track. The cost of maintaining and improving a framework like RoboComp is compensated by the flexibility of adapting it to your needs. Making good choices in this field, where Robotics meets Software Engineering, is not easy at all but once the software reaches a certain point of maturity, the leverage is undeniable. In the near future, we believe that these frameworks will play a crucial role in the evolution of intelligent robots. A role much more important that it is given today. It is needed the confluence of interested people from Software Engineering to gradually introduce new advances in DSLs, meta-models, model-based design and communication middlewares. From the recent evolution of social robot software, it looks like to us that the near future will bring larger and finer-grained networks of components, hundreds within the next years, that will demand more efficient software communications, self-diagnosis and repair, and sophisticated monitoring and deployment systems. Maybe classic, coarsegrained architectures will meet fine-grained ones at a point where interaction dynamics play a relevant role.
The cognitive robotics architecture, RoboCog, is a much more experimental and uncertain piece of handcraft. We started with a standard three-tier schema and managed to integrate symbolic planning with a fair amount of perception and action. To get there we re-introduced the idea of a shared representation among modules playing the role of an explicit context. It was implemented as two graphs, one geometric and one symbolic, and it proved enough for the required advertisement scenarios. Also, the introduction from the beginning of symbolic planning and learning technology in the project has proven a huge success. The initial idea of a using a flat PDDL description of the domain with a standard planner has evolved now into HRI specialized schemes, where hierarchical planners take care of quotidian, repetitive tasks and flat ones of the fine details and contingencies that might occur [19]. But, each solution takes to the next problem and before the end of the project, we were already working on integrating both graphs, reordering the classical hierarchies into more versatile organizations, infiltrate lifelong learning into all crevices of the system or use domain specific symbolic planning in classical low-level modules like navigation or object recognition. This issue, dealing with the overall organization of robotic intelligence, is undoubtedly the hardest one but projects like this motivate, and ultimately enforce, the search for new theoretical perspectives.
Other crucial part of the global Gualzru experience has been the use of evaluation metrics. User questionnaires turned out to be very important to improve the people’s attitude towards the robot, as well as to reveal the most urgent weaknesses in preliminary stages. It is a valuable lesson to be kept that periodic tests and surveys are an important part of HRI research, although they are often seen by roboticits as a dull, questionable use of the scarce human resources available.
Another important source of feedback are the robotic contests like RoboCup@Home2 or RoCKin3, that put all teams in the track of a common goal, and where real performances are evaluated in front of expert judges.
The ADAPTA project officially finished on May 2015 with a final public demonstration in Ma´laga. There Gualzru was able to interact with many people and successfully closed several difficult use cases. All partners in the consortium were satisfied and the robot will be serving from now on at the headquarters of Indra in Madrid. The research groups are now involved in more collaborative projects that hopefully will fund the construction of new social robots. We hope that the next generation will be capable of providing a better service to humans.
ACKNOWLEDGMENTS
This paper has been partially supported by the Spanish Ministerio de Econom´ıa y Competitividad TIN2012-38079 and FEDER funds, and by the Innterconecta Programme 2011 project ITC-20111030 ADAPTA.
REFERENCES
[
robotics,” Advanced Robotic, vol. 3, no. 1, pp. 049–054, 2006.
[
Mobile Robots, D. Kortenkamp, R. P. Bonnaso, and R. Murphy, Eds.
AAAI Press, 1998.
[
vol. 47, no. September 1987, 1991.
[
Olivares, and E. Quintero, “Pelea: a domain-independent architecture for
planning, execution and learning,” Proceedings of ICAPS’12 Scheduling
and Planning Applications woRKshop (SPARK), pp. 38–45, 2012.
[
spaces,” Ph.D. dissertation, 2013.
[
1990
[
P. Bustos, A. Bandera. The Cognitive Architecture of a Robotic Salesman,
in review.
[
The revised set of measures to assess users’ attitudinal and behavioral responses to a social robot, International Journal of Social Robotics 5(3): 379-388. 2013 2http://www.robocup.org 3http://rockinrobotchallenge.eu/
Page 64