[Context and Motivation] Existing approaches for specification mining and process mining allow to automatically identify requirementslevel system properties, which can be used for describing, verifying, or monitoring systems. We have developed an approach that can mine constraint candidates from event logs to support requirements monitoring. [Question/Problem] However, the usefulness of mining approaches is currently limited because of (i) weak support for adjusting the algorithms and settings to the current problem, (ii) the high number of properties mined from complex systems, and (iii) the typically high false positive rate. [Principal Ideas/Results] In this paper, we present ReMinds-CMT, a tool that guides domain experts throughout the mining process. [Contributions] The tool allows users to experiment with different thresholds and configurations of our mining, grouping, filtering, and ranking algorithms to ease the selection of useful constraints. We demonstrate the tool's features using constraints mined from event logs of a complex cyber-physical system controlling unmanned aerial vehicles.
Copyright c 2019 by the paper’s authors. Copying permitted for private and academic purposes.
a stand-alone tool, not an extension of ReMinds: events recorded with ReMinds [
Our constraint mining approach [3, 4] analyzes events and data recorded from systems (e.g., through monitoring) and derives candidates for constraints that can be used to check the compliance of these systems at runtime.
The input for our approach are event logs comprising recorded events and associated data, usually resulting
from multiple runs of different (sub-)systems completing different tasks. They contain multiple event sequence
types (patterns of multiple event types that have to occur in a given order) and event sequence instances (concrete
events matching an event sequence type). The information stored in event logs also has implications on the types
of constraints that can be mined. The minimal input needed for our mining approach is an event log containing
timestamped events. Event logs can be produced by monitoring tools such as ReMinds [
We use the ReMinds constraint DSL [
In the context of automation software and cyber-physical systems we have encountered the following types of constraints: Temporal constraints define a sequence of events that has to occur in a specific or arbitrary order and (optionally) within a specified amount of time. Such constraints typically describe a specific task of the monitored system, which consists of several steps, e.g., for the drone example a temporal constraint could check that a start event (the drone started flying a route) is followed by waypoint events and eventually an end event (the drone finished flying the route). Temporal constraints can be mined from an event log containing timestamped events without any further input. Value constraints specify the valid content of one or several event data elements – either as one explicit value (e.g., x == 1) or with thresholds (e.g., 5 < x 10). Often such data elements are grouped hierarchically, e.g., coordinates (X, Y, Z) are grouped under an element location sent as part of state events containing the status of the drone. Hybrid constraints combine temporal and value constraints. They include multiple events that have to occur in a given order and/or time, like in a temporal constraint, and additionally check event data elements of at least one of these events, like in a value constraint. Hybrid constraints can also check the relation between multiple event data elements, potentially related with multiple different events, e.g., to ensure that a certain ID remains unchanged throughout one particular event sequence. For the drone example (cf. Figure 1) a hybrid constraint could check that after a handshake event a state event occurs and that the state event contains the data element groundspeed with value 0, which checks that a new drone connecting to the system by performing a handshake is waiting for commands on the ground. 3
ReMinds-CMT
For complex software systems our mining approach potentially detects a large number of constraint candidates.
While many candidates can be removed from the list automatically, e.g., by removing duplicates, the selection
of constraint candidates cannot be fully automated and relies on domain knowledge. We have thus developed
ReMinds-CMT, an interactive tool guiding domain experts through the mining process and supporting them in
selecting the relevant constraints from the candidates. The tool provides a wizard-based interface (cf. Figure 1)
supporting end users throughout the mining process. It provides an interface to the algorithms of our approach
and allows users to select the event logs to be analyzed, to (optionally) configure all stages of the mining process,
and to review the constraint candidates for selection. Users can experiment with different filtering, ranking and
grouping algorithms and can also search in the list of candidates (cf. options on top of the screenshot shown in
Figure 1). The tool further allows users to fine-tune constraint candidates, e.g., change the value or operator
used in a value constraint (cf. the spinner and combo box controls after each data check and after each ‘within’
command for the constraints shown in Figure 1). It is possible to select individual constraints or whole groups
of constraints, which are then exported to a monitoring tool such as ReMinds [
When designing ReMinds-CMT we put special emphasis on flexibility and extensibility. The UI is therefore implemented in a two-level fashion: the outer level contains basic UI elements like the menu bar, whereas the inner level contains the individual pages as described below. This allows for easy exchange and extension of wizard pages. Further, regarding the extensibility in the model part of the application, we implemented all algorithms – such as the filtering, ranking, and grouping algorithms – in Java and provide an API that allows to configure thresholds and factors influencing the calculations. The tool solely relies on this API – meaning that all algorithms can easily be replaced and new algorithms or grouping factors can easily be added by implementing an interface.
Input file selection. The first page of the tool allows the user to choose one or multiple event logs that are used as input for the mining algorithms. The user also has to define how the logs define the event types (e.g., waypoint event, handshake event, etc.), timestamps, and sources of the events (e.g., a particular drone). For event logs recorded with our own monitoring tool ReMinds, this can be parsed automatically.
Input filters. Our tool allows to define filters on the events from the input event list read from the event logs. Options for these filters include filters on event types, sources, and time frames. Multiple filters can be combined and all filters can be defined inclusively or exclusively. Users can view statistics on how many events of the input list are filtered. Additionally, the tool shows examples to illustrate which kind of events pass the filter and which do not.
Configure algorithms. The tool then provides the possibility to choose and configure the mining algorithm, i.e., to skip certain parts of the selected algorithm. For example, the user can choose not to mine any constraints based on intervals calculated from the distribution of the observed event data items.
Constraint filtering. The tool allows to select and configure filters on the mined constraint candidates. Examples for such filters include filters on constraint types, accuracy (i.e., number of positive evaluations of a mined constraint in the event log), and constraints on specific event data items.
Grouping. To support the user during the selection of useful constraints from the (potentially very long) list of mined constraint candidates the tool groups the candidates (cf. Figure 1) based on multiple factors: the trigger event type (e.g., handshake event), the constraint type (e.g., hybrid constraint), the event sequence (e.g., start– waypoint–end), the event data item names (e.g., status, groundspeed), and the event data item values (e.g., 0, ‘active’). The groups are built such that constraints with high similarity – calculated as the weighted average of all factors – are grouped together. Figure 1 shows several constraints concerning handshake events and state events which have been automatically grouped. The weights for all factors and the threshold for grouping constraints are configurable.
Ranking. The user can select the ranking from a list of provided strategies (cf. Figure 1 top left). The tool ranks the constraint candidates within each of the groups first and selects the highest ranked constraint from each group. These constraints are then ranked to estimate the order of the groups.
Our tool supports the following ranking strategies [3]: (i) The first algorithm ranks constraints based on accuracy, calculated as the percentage of checks evaluating to true, among all evaluations of the given constraint for the input event log(s). (ii) The second strategy ranks constraints primarily based on their type. Temporal constraints depict the behavior of software systems and are often less specific. They are ranked first, followed by hybrid constraints, and value constraints. (iii) The third, combined ranking strategy sorts the constraints based on their average rank from the accuracy-based strategy and the type-based strategy. A fourth ranking strategy focuses on (iv) evaluations : it combines the accuracy (strategy (i)) with the relative number of positive evaluations, i.e., the number of positive evaluations for the given constraint candidate divided by the highest number of positive evaluations for any constraint candidate. Additional ranking strategies can be defined relying on the provided API.
Adapt and select constraints. Finally, the user can adapt the mined constraints and select the ones that should be exported. Possible adaptations include changing the duration and the thresholds of constraints and changing the used operators. For example, the maximum speed for a drone in a mined constraint is set close to the maximum speed encountered in the input data (6.1 m/s). However, in other scenarios drones can safely fly faster than this maximum and therefore this threshold can be increased before exporting the constraint. 4
We have presented ReMinds-CMT, an interactive tool that supports domain experts in mining constraints for requirements monitoring. The tool is flexible and extensible, e.g., it supports different input formats and new pages can be added in just a few steps. It allows end users to effectively mine constraints and customize the mining algorithm to their use case. So far, we have used our tool for constraints mined from event logs of two complex real-world systems [3] – a plant automation software system and a cyber-physical system controlling unmanned aerial vehicles. We recently presented our tool to several potential users of our industry partner during a workshop and received very positive feedback. In future work we plan to evaluate our tool for further systems and extend it based on feedback we will receive.
The financial support by the Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development, and Primetals Technologies is gratefully acknowledged. [1] Cleland-Huang, J., Vierhauser, M., Bayley, S.: Dronology: An incubator for cyber-physical systems research.
In: Proceedings of the 40th Int’l Conf. on Software Engineering: New Ideas and Emerging Results. pp. 109– 112. ACM (2018) [2] Gabel, M., Su, Z.: Javert: fully automatic mining of general temporal properties from dynamic traces.
In: Proceedings of the 16th ACM SIGSOFT Int’l Symposium on Foundations of Software Engineering. pp. 339–349. ACM (2008) [3] Krismayer, T., Kronberger, P., Rabiser, R., Grünbacher, P.: Supporting the Selection of Constraints for Requirements Monitoring from Automatically Mined Constraint Candidates. In: 25th Int’l Working Conf. on Requirements Engineering: Foundation for Software Quality. Essen, Germany (2019) [4] Krismayer, T., Rabiser, R., Grünbacher, P.: Mining Constraints for Monitoring Systems of Systems. In: 34th ACM/SIGAPP Symposium On Applied Computing. Limassol, Cyprus (2019) [5] Lemieux, C., Park, D., Beschastnikh, I.: General LTL Specification Mining (T). In: Proceedings of the 30th
IEEE/ACM Int’l Conf. on Automated Software Engineering. pp. 81–92. IEEE (2015) [6] Lo, D., Khoo, S.C., Han, J., Liu, C.: Mining Software Specifications: Methodologies and Applications. CRC
Press (2011) [7] Maita, A.R.C., Martins, L.C., Paz, C.R.L., Rafferty, L., Hung, P.C.K., Peres, S.M., Fantinato, M.: A systematic mapping study of process mining. Enterprise Information Systems 12(5), 505–549 (2018)