Recent research shows that runtime models can be used to build dynamic systems coping with changing requirements and execution environments. As software systems are getting bigger and more complex, locating malfunctioning software parts is no trivial task because of the vast amount of possible error sources and produced logging information. This information has to be traced back to faulty components, which often leads to editing of code scattered over di erent software artefacts. With such a fragmented view, challenges arise in nding out the root cause of the unwanted software behaviour. In this paper we propose the usage of runtime models in combination with model-driven techniques and interactive visualizations to ease tracing between log le entries and corresponding software artefacts. The contribution of this paper is a repository-based approach to augment root cause analysis with interactive tracing views while maximizing reusability of models created during the software development process.
Recent research proposes models used at runtime to realize adaptable and
manageable systems. Runtime models provide software with additional capabilities to
re ect on its own structure and adapt itself according to changing requirements
and execution environments. This is di erent to traditional software engineering
where models are artefacts created during the software development phase,
models have no connection to the resulting executable software, and the software is
usually modi ed by re-deployment of changed software components. In addition
to this extra deployment phase, such changes are often performed on artefacts
residing on a low abstraction level. To overcome these limitations, the running
system can use a model-based self-representation which is causally connected to
it [
The amount of runtime data produced by many applications tends to be huge
and is recorded in di erent forms. Relevant information has to be extracted and
put into context with corresponding artefacts produced in the development phase
to reveal the root cause of unexpected software behaviour. The purpose of such
an analysis is to nd out the root e ect within a chain of undesirable e ects
occurred during software execution [
In this paper we propose a novel approach to combine visualization techniques with models used at runtime (and usually created in the design phase) to their mutual bene t during root cause analysis: Model artefacts are utilized as additional information during root cause analysis while visualization techniques allow to focus on particular sections of logging output of the running application. We accomplish this by using model-driven techniques in combination with interactive visualizations to enable tracing between log le entries and corresponding software artefacts. A customized code generator produces annotated code which handles logging without the need of human intervention. Furthermore, model artefacts are stored in a repository to be accessible by a log le tracer, although our approach is not limited to such data access. That is, model data is also accessible to external applications to open the system for future extensions. At runtime, recorded logging data is fed back to interactive analysis views and associated model elements to support root cause analysis on a higher abstraction level, thus closing the loop between the development and runtime phases.
This paper is organized as follows: In the next section we give an overview of our approach to combine reusable models with interactive visualizations to gain aforementioned bene ts. In Section 3 we describe the model-driven part of our work which prepares runtime access of model information. Furthermore, we introduce interactive visualizations to analyze runtime information and present a prototype demonstrating our idea by using a model repository to trace log le entries to corresponding software artefacts. In Section 4 we give an exemplary scenario for the usage of our approach. In Section 5 we discuss our results and lessons learned. In Section 6 we take a look at related work in terms of model feedback and interactive visualization. We conclude in Section 7. 2
Our approach proposes semi-automated, interactive techniques to provide feedback from runtime information back to the relevant software artefacts. More precisely, the tool proposed in our approach listens to log le changes produced by the running system, extracts the information and puts it into context with corresponding model artefacts and their elements. As a result, models created in the development phase are utilized at runtime and act as a representation of system parts a ected by speci c log messages. We use a model-driven approach to generate the logging and tracing environment for the software where the amount of produced runtime information can be adjusted by the developer at design time (see Section 3.1). Furthermore, we use interactive visualizations to reveal the root cause of unexpected software behaviour by making dependencies between software parts explicit and providing improved navigation capabilities to model artefacts used at runtime (see Section 3.2). A summary of the development steps is depicted in Figure 1.
⑥ Iterative root cause search (repeat until root cause candidate is found) odels ①Createm e d o c e rt a e n e G ② ③Implem entlogic
④Compile/ Ex⑤ecuAtnealyze log data Design time
generation
refinement
log creation
The rst step of our approach is concerned with modelling of the system. In our approach, the modelling step can concern any model artefact that is compliant with a speci c meta-model, although the current version of our prototype focuses on UML models only (see Section 4). These created model artefacts are utilized at runtime where runtime information is fed back to the models to nd locations of errors and unexpected software behaviour. The next step deals with the development or adaptation of a code generator to produce model and logger code for the system. Furthermore, the code generator is responsible for storing models and their related elements in a repository which is accessible at runtime. This step requires no additional interaction of the developer and enables the resulting software to query its own structure and produce log le entries with a reference to the model that has created the software artefact producing the log entry. This part of the development process leaves room for future extensions to augment the logger, e.g. to generate reports of software failures.
During code generation model-related code as well as logger code is created which emits log le entries in a format speci ed by the code generator. In the phase of code re nement, the actual software product is developed and the generated model code is integrated or extended by the custom implementation. Within custom code, log le entries can be emitted in the correct format by making calls to the previously generated logger code.
The combination of custom and generated code is then compiled to an executable application which acts as system under observation for subsequent analysis. Implemented logger calls obtain model information from the repository and produce log le entries traceable to the related model elements. The last step of the development is concerned with the analysis of the produced log les and the interactive visualization of the recorded data to enable e cient tracing between log le entries and corresponding model artefacts. In this phase, root cause analysis is an iterative action by narrowing down problems step-by-step with the help of multiple interactive visualizations. These visualizations highlight a ected model elements and clarify their interrelationships.
As a summary, the key idea of realizing root cause analysis with interactive visualizations is a loop which is responsible for the feedback of runtime data to model artefacts created in design phase. We divide the steps into model-driven application development and feedback of runtime information with the help of interactive visualizations. Section 3.1 and 3.2 describe the details for these parts. 3
Based on the overview depicted in Figure 1, we give a more detailed view on the development process to provide insight into our model-driven development and interactive visualization parts. 3.1
We use model-driven techniques to prepare the resulting software product for runtime access of model-related logging data. Interactive visualizations are then used to process this data and provide root cause analysis capabilities. The rst step is the creation of model artefacts to act as a base for code generation and information feedback. These model artefacts are then fed into a code generator which is responsible for two tasks: On the one hand, it has to store the models and its associated elements in a repository to be accessible at runtime. On the other hand, it must generate code out of the input models which is then re ned by the developer. The generated code depends on the customized code generator and usually covers model-relevant code and additional components to enable logging. The generated logger code is responsible for writing log les in a format to be processable by a log le tracer or other external applications.
Within the repository, the model and all of its traceable model elements (e.g., packages, classes and methods) are stored and get a unique ID. In principle, our approach is independent of algorithms which generate such unique IDs { for our prototype, we used the hash code of the full quali ed name of a model element (see Section 4). The assigned IDs are used by the code generator to create annotations for corresponding elements in the generated code. These annotations can be accessed by the logger component at runtime to produce traceable log les entries. Such entries usually consist of the related IDs and a generic message, but can be adapted by modifying the code generator itself.
In the next step, custom code is implemented which extends or calls the generated model code and realizes the actual business logic. Logging is controlled by annotating elements of interest or by calling the generated logger directly. Extending generated code is usually achieved by inheritance relationships between custom and generated classes in combination with overriding of generated operations. Depending on the code generator, a logger can evaluate such inheritance hierarchies at runtime through re ection to locate IDs of annotated elements and produce log entries containing references to associated operations and classes. This is an important step to close the loop between produced runtime information and the static structure modelled at design time since it enables interactive analysis views to provide navigation between log les and design artefacts.
For the dynamic aspects of the system, the logger component alone is not adequate since the logger class can only trace the caller of the logger itself, but not calls between di erent components (e.g., a statement in an annotated method calls another annotated method). We achieve tracing of such calls through aspectoriented programming techniques: The code generator generates pointcuts (hooks to method invocations) for methods to be traced and advices (the actual hooking code) which call the logger with information about caller and callee.
After compilation and execution, calls to the generated logging code will produce log le entries in a format which can be interpreted and fed back to model elements which have been created in the rst step. For feedback of runtime information, a tracer component has to read produced log les and query associated model elements from the repository. To obtain live data from the application, the tracer must listen regularly to changes in these log les. In addition, it is also possible that the application itself queries the repository to obtain additional information about its own structure. The repository can also be queried from arbitrary clients which makes the architecture extensible for external tools. 3.2
After log entries have been read, interactive analysis views have to be populated so the user can select relevant data from the collection of runtime information. With visual support available, the amount of irrelevant messages can be minimized and a ected components and their dependencies can quickly be found.
Van Ham [
(a) Call Matrix [
Beside the call matrix, we implemented a matrix view where a simple list of all recorded log entries is set in relation to all corresponding model elements (see Figure 2c). Within this matrix view, sortable columns enable grouping of log A B C
A
B
C Entry 1 Entry 2 Entry 3 Entry 4 Entry 5 X Entry 6
A B C D E F ten ten ten ten ten ten lem lem lem lem lem lem e e e e e e l l l l l l eod eod eod eod od od
e e M M M M M M
X X
X X
X X
X entries for particular model elements. Such grouping improves root cause analysis because hot spots with a high error density can easily be detected. Furthermore, if a faulty component has been detected with the call matrix approach, similar errors for the same component can quickly be revealed. Since the matrix grows rapidly with the number of log entries and model elements, this technique is only recommended for small projects. For a great number of data, one should fall back to Table Lens or a combination of Table Lens and log entry matrix.
To provide more insight into the structure and dependencies of a system,
Linking and Brushing combines multiple visualizations by introducing
synchronization mechanisms between them [
We implemented an Eclipse-based1 prototype of our idea by using Papyrus as modelling toolkit, Acceleo as code generator and AspectJ for aspect-oriented capturing of dynamic calls. For log les, we included the le name, line number, a generic message and related IDs (packages, classes and methods) for all log le entries. The model repository is a relational database which stores the model le of UML models and holds references to each model element by their assigned unique identi er. In our prototype, we use the hash code of the full quali ed name of the respective model element as unique identi er, although our approach is not limited to this technique as long as there is a unique identi er available. We implemented four di erent visualizations for more e cient root cause analysis which utilize the repository to trace selections back to the UML models: { List view, a live view which lists all log entries in chronological order. { Matrix view, a sortable view which relates log entries with model elements (see Figure 2c). Model element grouping enables hot spot identi cation. { Outline view, a list of expandable log entries to view related model elements.
Enables quick navigation from log entries to model elements. { Call Matrix view, a view to trace dynamic calls between components (see Figure 2a). Reveals dependency chains of faulty components. 1 http://projects.eclipse.org/list-of-projects
Tracer
Load file
Once the logging data has been read, the matrix view and call matrix view are populated with the data (other views are neglected in this scenario). Such views are not available in traditional logging approaches and contribute to a more e cient root cause analysis: Assuming an anomaly has occurred, the developer is able to take a rst look into the matrix view to get an overview of all traceable model elements which were involved in log entries. Irrelevant model elements are not rendered as columns at all, providing a compact overview which is not available when analyzing log les by hand. For our example, the columns of the matrix view consist of three classes (Program, List, Element ) and two methods (initList, print ). The matrix view allows to group logging entries for suspicious model elements. If this grouping results in a large list of entries for an element, it is very likely that the error occurs within a frequently called method of that element. Element List depicted in Figure 3 is a candidate for such an element.
To inspect this element, the call matrix view is consulted to nd out more
information about called methods of this element (columns) as well as their
respective callers (rows). This is directly achieved by selecting the entries in the
matrix view since our implementation of Linking and Brushing [
While navigating through log le entries with the help of interactive
visualizations, our implementation of Linking and Brushing [
Regarding architecture, it is also possible that the tracer component receives the logging information remotely via network and interprets the information without further interventions of the user. Such connection between the system and the monitoring environment cannot be automated if logging messages follow an informal format, which is often the case in traditional logging. As a summary, the usage of interactive visualizations with runtime models exceeds manual root cause analysis in terms of logging e ort, navigability, traceability and reusability. 5
Our approach aims to combine the advantages of model-driven development with reusable software artefacts. Once the code generator is implemented, the system is extensible through the model repository and is fully transparent for the developer since manual actions are limited to logger calls where traceability is desired. All supporting artefacts (e.g. logger, database entries, annotations, aspects with join points and advices) are generated and integrated into the resulting system without further intervention. At runtime, interactive visualizations enable e cient tracing between log le entries and corresponding model artefacts.
If logging output is constantly observed, the models created at design time turn into runtime models which contribute to the overall understanding of the architecture while searching for anomalies. Changes to these models can be propagated to the running system to gain instant feedback if anomalies have been resolved. Our current prototype allows feedback to both structural and behavioural models, but is not yet able to propagate changes back to the running system. We plan to add this feature in the future, but we also see more potential in the idea of Linking and Brushing to reveal interrelationships between generic data sources and runtime models. For example, a time series interval of performance data could be selected which highlights model elements responsible for the selected data, ideally sorted by relative distribution (e.g., method sort produced 65% of the selected load) and accessible by selectable granularity (e.g., packages, classes and methods). Bottlenecks could be identi ed, repaired and changes be propagated to the running system. Interactive visualizations thus can be seen as abstraction/aggregation layer between running systems and their causally connected runtime models. They augment conventional runtime model, debugging and tracing approaches and help developers to understand the system, reproduce errors and nd the root cause of problems in models created at design time.
Nevertheless, the initial e ort must not be underestimated since the code generator must be implemented and produce proper output to see rst results. Furthermore, the overall performance of the system has yet to be tested since a large number of model elements and log le entries may result in increased e ort regarding code generation, log le analysis, repository lookup and interactive data presentation. This is especially the case when using techniques like Linking and Brushing where multiple models and analysis views must be traversed to highlight particular model elements. Further work is required to see if these concerns are reasonable and if so, to nd out how these problems can be solved. 6
Maoz [
Graf and Muller-Glaser [
Like in our approach, Fuentes and Sanchez [
An approach which also uses log les to coordinate model operations is
introduced by Bodensta et al. [
Regarding model access, Holmes et al. [
Dettmer [
Root cause analysis with the presence of huge amount of data during process
execution is also discussed by Rodriguez et al. [
In this paper we combined model-driven techniques with runtime models to augment root cause analysis of executing systems. We gave an overview of our approach to illustrate the development steps which cover software models created at design time and their runtime utilization when analyzing le output produced by the system under observation. We described our model-driven strategy which uses a repository to provide runtime access to model-relevant data for the system or other external applications. We introduced interactive visualizations of which we implemented a list view, a matrix view, an outline view and a call matrix view in our prototype. We analyzed advantages of our approach as well as potential performance drawbacks if models tend to increase in complexity.
We plan to perform an empirical evaluation of our approach to analyse its applicability. Furthermore, we aim to generalize our prototype to accept more meta-models than UML by adapting the code generator and to include additional runtime information sources like network tra c or workload of individual software parts. We especially see potential in improving the code generator to produce additional output, e.g. automated reports in case of unexpected behaviour. We will compare our work with existing transformation frameworks which provide basic tracing features and consider including code-to-model transformations to support existing implementations. Future versions will include better controlling of the log le output and propagation of changes to the running system.