-System developers spend a significant part of their time debugging systems (i.e., locating and fixing the cause of failures observed through verification and validation (V&V)). While V&V techniques are commonly used in model-driven engineering, locating and fixing the cause of a failure in a modelled system is most often still a manual task without toolsupport. Although debugging techniques are well-established for programming languages, only a few debugging techniques and tools for models have been proposed. Debugging models faces various challenges: handling a wide variety of models and modelling languages; adapting debugging techniques initially proposed for programming languages; tailoring debugging approaches for the domain expert using the abstractions of the considered language. The aim of the first edition of the MDEbug workshop was to bring together researchers wanting to contribute to the emerging field of debugging in model-driven engineering by discussing new ideas and compiling a research agenda. This paper summarizes the workshop's discussion session and distils a list of challenges that should be addressed in future research.
Model-Driven Engineering (MDE) aims at coping with the
complexity of systems by separating concerns through the use
of models, each representing a particular aspect of a system.
In the past years, significant effort has been directed towards
providing early verification and validation (V&V) techniques
to determine whether or not a set of models fulfils a set of
properties (e.g., [
To illustrate this activity, Figure 1 presents an overview of a typical system design process. First, a set of properties that the system has to satisfy is defined. Then, a system is designed as a collection of models that must satisfy these properties. To check that the properties are satisfied by the models, a wide range of verification and validation (V&V) techniques are available, such as theorem proving, symbolic execution, model checking, real-time simulation, and testing. Depending on the approach, it might be found that a property is satisfied (“pass”), not satisfied1 (“failure”), or that the result is inconclusive (“unknown”)—for example when the chosen technique cannot prove the property in a reasonable time frame. At that point, however, the cause of the failure (also called the defect) must still be identified (i.e., which parts of the models are causing the observed failure). A failure may also be observed if the properties were wrongly specified, in which case the properties themselves have to be fixed. Once they are identified, the cause of the failures must be fixed by changing either the models or the properties. Locating and fixing the cause of a failure can be accomplished manually given a good understanding 1Note that it is also possible to consider potential failures when the considered V&V technique may give false positives (e.g., static code analysis) ⊨ natural language, temporal logic, test cases, etc.
properties testing, model checking theorem proving, etc.
models conforming to DSLs, UML, SysML, etc.
slicing,
Fault interactive debugging, Localization cmoamnmuaelnsteinagrccho,de, automated search, etc. failure cause
Debugging
model element(s),
line number,
event sequence,
etc.
of the models. A wide range of debugging techniques can be
used to assist developers in finding the cause of the problem.
For example, interactive debugging techniques [
While debugging techniques and tools are very common for programming languages, very few debugging tools and techniques are available when it comes to models. Hence, modellers often have to resort to ad-hoc methods, such as inspecting and debugging the code generated from models. Although this allows the reuse of established and wellresearched program debugging techniques, it is not ideal since the developer has to switch contexts and must understand both the mapping to and the semantics of the underlying implementation language. Dedicated debugging support for modelling languages has potential to reduce or eliminate the need for this kind of context switching, and is an essential part of allowing a developer to remain in the modelling paradigm throughout the full development process.
In this context and scope, the goals of the MDEbug workshop were to: bring together interested researchers to optimize the research effort and establish collaborations; provide a forum for researchers to share new experiences, ideas and early results on the topic of debugging in model-driven engineering; define the scope of debugging within model-driven engineering; identify gaps in the current body of research.
The full-day workshop took place on September 17, 2017 as part of the satellite events of the ACM/IEEE 20th International Conference on Model Driven Engineering Languages and Systems (MODELS 2017) conference in Austin, Texas. The workshop started with a keynote presented by Andrei Chis, from feenk on the topic of Moldable Debugging. We received six submissions, and five submissions were accepted after a review process in which each paper was reviewed by at least three members of the program committee. One of the accepted submission was a full research paper, while two were tool demonstration papers, and two were position papers— including one by the keynote speaker that summarized his keynote presentation. The complete list of papers can be found below, in the order of their presentation at the workshop: 1) “Moldable Debugging (Position paper)” by Andrei Chis, and Tudor Girba 2) “Transformations Debugging Transformations” by Maris
Jukss, Clark Verbrugge and Hans Vangheluwe 3) “Towards Debugging the Matching of Henshin Model Transformations Rules (Position paper)” by Matthias Tichy, Luis Beaucamp and Stefan Kögel 4) “Domain-Level Debugging for Compiled DSLs with the GEMOC Studio (Tool demonstration)” by Erwan Bousse, Tanja Mayerhofer and Manuel Wimmer 5) “Debugging Non-Determinism: a Petrinets Modelling, Analysis, and Debugging Tool (Tool demonstration)” by Simon Van Mierlo and Hans Vangheluwe
The afternoon session of the workshop was reserved for an interactive discussion on the wide topic of debugging in model-driven engineering. We summarize these discussions in Section V.
All information can be found on the workshop website: https://msdl.uantwerpen.be/conferences/MDEbug. This includes the slides of all presentations given at the workshop.
The program committee of MDEbug 2017 was composed of 27 researchers and experts in the domains of modeling, debugging, and model execution, coming from 12 different countries. We sincerely thank the program committee members and external reviewers for their time in reviewing and discussing the submitted papers.
Debugging is not limited to MDE. Debugging embraces various activities, techniques, and application domains in software engineering prior to MDE, such as debugging objectoriented programs. Yet, to the best of our knowledge no recent state of the art has been published on debugging in software engineering. The interested reader has to explore the literature of various techniques applied in a debugging context, such a program slicing.
A direction of research can focus on designing a comprehensive survey on debugging in software engineering. The outcome(s) of this work would permit to clearly identify the challenges shared or specific to MDE debugging.
Vasco Amaral, Universidade Nova de Lisboa, Portugal B. Clarifying debugging terminology, classifying approaches Reda Bendraou, Université Paris Ouest Nanterre la Due to a rather rich history in the field of computer science, Défense, France the words bug, debugging, and debugger have been used in a Francis Bordeleau, CMind, Canada large number of contexts with diverse meanings. For instance, Benoit Combemale, IRISA, Université de Rennes 1, some might call “bug” the wrong behaviour of a program, France while some others consider that the “bug” is the piece of code Jonathan Corley, University of West Georgia, USA responsible for the misbehaviour. Likewise, while “debugging” Andrea d’Ambrogio, University of Rome Tor Vergata, literally means “removing a bug”, in some contexts it means Italy “finding the bug”, while in some others it means “executing Julien Deantoni, University of Nice-Sophia Antipolis, the model/program in an interactive debugger”. Lastly, the France word “debugger” is used on a daily basis to designate an Davide Di Ruscio, University of L’Aquila, Italy interactive debugger (e.g., gdb, or the Eclipse Java debugger), Holger Giese, Hasso-Plattner-Institut, Germany while more generally it can mean “a tool that can be used Martin Gogolla, University of Bremen, Germany for debugging”. Overall, to avoid confusion, it appears that Jeff Gray, University of Alabama, USA the community must either agree on a precise terminology, or Robert Heinrich, Karlsruher Institute of Technology, Ger- at least acknowledge that these words may convey different many meanings depending on the context.
Sebastian Herzig, Caltech/Jet Propulsion Laboratory, In addition to vocabulary issues, there is some confusion USA on the different kinds of debugging approaches that exist, and Levi Lucio, Fortiss, Germany what qualifies as debugging, understanding, or neither. For Tanja Mayerhofer, TU Wien, Austria instance, using an interactive debugger to explore the different Tim Molderez, Vrije Universiteit Brussel, Belgium states of a behavioural model step by step can be useful both Patrizio Pelliccione, Chalmers University of Technology for debugging (i.e., finding and fixing a bug), or simply to and University of Gothenburg, Sweden better understand the model. Likewise, a model slicer can be Arend Rensink, Universiteit Twente, The Netherlands used to debug a model but is commonly not called a debugger, Bran Selic, Malina Software Corporation, Canada nor is model slicing classified as a debugging technique since Eugene Syriani, University of Montreal, Canada it can be used for other purposes as well. It would be of great Jérémie Tatibouët, CEA, France benefit for the community to reach a classification of possible Massimo Tisi, Institut Mines-Telecom Atlantique, France debugging techniques and to understand the similarities or Javier Troya, University of Seville, Spain differences between them (e.g., interactive or not, automated or Antonio Vallecillo, Universidad de Málaga, Spain manual, static or dynamic, language-specific or generic). The Tijs van der Storm, Centrum Wiskunde & Informatica consensus at the workshop seemed to be that any technique (CWI), The Netherlands used during the “debugging phase” (denoted by the dashed orange line in Figure 1) can be called a debugging technique.
V. SUMMARY OF THE DISCUSSIONS Among many others, this includes print statements, interactive debugging, and model slicing. It is not so much the technique, but rather the intent of the use of the technique that defines it as a debugging technique.
This section summarizes the content of the discussions that took place during the workshop, in the form of a list of topics.
Each topic covers an aspect of debugging in model-driven engineering that has open research challenges. Therefore, these challenges can be used as a basis for future research, or for organizing discussions during a future workshop.
Semantics for modelling languages can be defined in a
variety of ways [
In a general sense, the mapping between source and target languages is not always one-to-one. One open problem is how to handle such situations. Should the target language concepts be available to the modeller? Should additional domain concepts be defined? Or should one consider that there is a bug in the translational semantics themselves?
While some approaches reuse a target language
interactive debugger to provide interactive debugging for modelling
languages with translational semantics, instrumentation offers
another way of debugging models. For instance, it is possible
to instrument the model with specific “traps” that interrupt
the normal flow of execution to provide source language
interactive debugging operations [
Interactive debugging is usually performed forwards: from
an initial state, one explores the execution trace according
to the semantics of the considered modelling language. It is
possible to extend this approach to allow the developer to
step backwards in an execution trace, leading to so-called
“omniscient debugging” [
To extend even further the possibilities of omniscient debugging, an interesting research direction is to study the potential for starting from a faulty model state to then compute the possible past execution paths that can lead up to that state, thereby debugging entirely backwards. This is related to program or model slicing, where the debugger produces a set of statements that can affect the values of a run-time state.
Debuggers may assume that the model source (a text document, diagram, etc.) is available. This might not always be true. In certain cases the source of the model might be hidden and provided as a compiled “black-box”. If so, the control an interactive debugger can exert over such a black box is limited: in program interactive debugging, these components are often ignored (or “stepped over”) and assumed to be correct. Failures that only appear when composing or integrating black-box components can be difficult to debug: the failure might actually be caused by an interaction in one of the components that is impossible to detect without observation of the state and control of the execution. If it is impossible to provide the full source (e.g., in case of intellectual property protection), a greybox approach might provide an interface to the component that allows a debugger to interact with it. This requires finding a balance between exposing enough detail and protecting the full details of the model.
Debuggers must be developed for developers and modellers, which are often domain experts with no experience with the technologies used to implement the modelling environments that they use. The usability of a debugger is therefore a key issue and may cover many aspects. In the case of an interactive debugger, such aspects may include the following:
The representation of the current execution state has to fit the mental model of the developer. For debugging visual languages, this representation can be close to the visual abstractions provided by the language. In the presence of additional run-time information, however, additional abstractions may have to be provided.
Similarly, not only the debugging state but also definitions of breakpoints need to align with the language. Here, domain-specific condition languages on states and state sequences need to be provided which enable the domain expert to easily specify conditions on the state according to the domain-specific needs.
The debugging operations provided have to fit the needs of the developers. They have to fit the developer’s mental model of the language’s semantics: a logical “step” does not necessarily correspond to a debugger or simulator “step”.
Suitable representations for the execution traces and their states have to be found. Especially for concurrent, non-deterministic formalisms and/or acausal languages, these might differ significantly from the usual sequential representation.
Furthermore, while the usual concepts of interactive
debugging (state, steps, breakpoints, etc.) are obvious to software
engineers and can be easily used also for models (e.g., [
While there is already existing and encouraging work on
domain-specific debugging [
Debugging is often defined as finding the cause of some observed faulty behaviour of a system, and changing the system to avoid this behaviour in the future. This suggests that debugging only makes sense for behavioural models, and H. Reuse of debuggers among languages therefore for modelling languages with execution semantics
Many formalisms have common aspects they share: syntax, implemented by a code generator or an interpreter.
semantics, visualization. In language engineering, efforts are Yet, a wide range of models focus only on structural
made to reuse different parts of language definitions [
A declarative language generally provides concepts that do not explicitly show or define in which order a conforming model will be executed. A prime example of this is a declarative model transformation: it defines the relation between input elements and output elements, but does not define how the transformation is executed. The engine responsible for executing the transformation is defined operationally, however. To debug such declarative languages interactively, one can ask whether we have to expose those operational semantics to the developer (in the form of steps, for example), and if so, to what extent. Since the language is declarative, the developer does not necessarily know its operational semantics, and exposing them might require a mental leap. Alternatively, debugging operations that are far away from the operational semantics but closer to the mental model of the developer might be more appropriate (e.g., by showing all maximal parts of a declarative model transformation which are still applicable to understand why the full declarative model transformation is not applicable). This would lead to debugging functionality that considers the constituents of the declarative model (e.g., debugging the formulas being part of the declarative model).
While many systems are deployed as software, synthesis to hardware is also possible. When a failure occurs in a synthesized hardware part, it can be necessary to debug this part to identify the cause of the failure, using physical probes to read the state and control of the system. Many tools and techniques exist for hardware monitoring, information,
While debugging commonly focuses on finding problems inside models, another possible cause for failures are problems in the semantics of the considered language (e.g., errors in the interpreter or in the code generator). Faulty semantics can lead to invalid or inconsistent debugging results. This can have different causes, such as:
mented, which means that the semantics itself must be debugged.
The semantics is incorrectly specified, which means that the set of properties that specify the language must be clarified.
The semantics is both correctly specified and implemented, but is not well understood by the modeller. This is a subtler problem, and one where debugging techniques can be useful for “model understanding”, or in this case “semantics understanding”, possibly by exposing certain details of the semantics in the debugging process.
Debugging aims at searching the cause of some observed failures within models or properties. In other words, the goal is to identify “guilty” model elements that cause trouble. However, it is equally important in the debugging process to identify the “innocent” model elements that definitively do not contribute to the fault. For instance, when using generalpurpose programming languages, it is common to comment portions of code to see whether the failure is still present without these portions. In a more advanced fashion, unit testing aims at testing independently the different parts of a system— for instance using partial and trusted implementations called stubs—in order to mark as many units as possible “innocent”, and to eventually isolate the guilty unit responsible for a failure.
While verification and validation is a necessary step to identify the defects in models, debugging is a crucial activity when it comes to dealing with such defects to improve the quality of models. The International Workshop on Debugging in Model-Driven Engineering (MDEbug) aims at providing an event for the community to share ideas and results in this research area. This first edition was well attended throughout the day, and the afternoon discussions were both lively and constructive, leading to a wide range of potential research topics. In addition, we observed that debugging was a quite recurrent topic throughout the presentations of the MODELS 2017 conference, which strengthens our belief that it is an important research direction for the success of MDE.
We thank everyone who took part in the success of the workshop, including the program committee members, the authors, our keynote speaker, and everyone who attended the workshop or took part in the discussions.