QPR ProcessAnalyzer (QPR Software Plc) Signavio Process Intelligence (Signavio GmbH) upfront to better understand what features and capabilities the vendors offer. Vendors were asked to grant access to all features in the demo environment to ensure all available features can be explored. The experimental approach also included the screening of all available knowledge bases and product documentations made accessible by the vendor. The derived criteria set was applied in three steps. In Phase 1, a test scenario was conducted for every tool using the same logs and files. In Phase 2, the results were compared with each other to identify inconsistent terminology and discrepancy in the level of detail. The final assessment was conducted in Phase 3. After testing, follow-up workshops were conducted with every vendor to clarify open questions and to get additional context for features. The exchange with the vendors also served as a quality gate for the correctness of the test results.
The software testing was conducted primarily using event logs of Purchase-to-Pay (P2P) processes with their respective “happy path” reference models in BPMN format.
In the course of the study, 16 tools capable of mining event log files were analysed, see Table I. The study was carried out in spring 2020.
The website is mainly built on three layers. While the homepage (first layer) introduces the discipline of process mining, typical use cases and our criteria overview, the “Tools” page (second layer) lists brief profiles of all tools which are linked to the detailed tool profile pages (third layer). An introductory paragraph briefly describes the vendor and the strengths of its software. Eight criteria categories examine the availability and extent of tested functionality while one criteria category provides general information. The “Distinctive Focus and Features” section provides additional context by highlighting outstanding functionality. In order to offer users visual impressions of a tool, every profile is enriched with a “featured video” provided by the vendor and up to seven screenshots, of which five are defined and two undefined (proprietary). Also, any two selected tool profiles can be contrasted with each other through a side-by-side comparison.
The software criteria derived from the literature review and experimental software testing represents the core of this study. The criteria were grouped into nine categories depicted in Tables II - X in the appendix.
Category General gives a brief overview of the vendor and key aspects of the tool. Data Management examines functionalities and factors related to the extraction, transformation and loading (ETL) of process data into the process mining tool. The Process Discovery category examines process graph capabilities and process analysis features such as benchmarking and rework analysis. Conformance Checking is a fundamental process mining feature to identify deviations between the actual “as is” process and an “a-priori” reference model. This category considers all relevant factors pertaining to conformance checking. The Operational Support criteria examine the availability of forward-looking capabilities to help users anticipate the outcome of running cases and facilitate decision making with the help of intelligent recommendations. Views, Monitoring and Reporting addresses the ability to monitor processes with the help of metrics and visualisations to support decision making. Additional criteria examine available languages and means of collaboration to share insights with other users. While process enhancement functionality such as performance metrics in the process graph is partly covered in the aforementioned criteria categories, Advanced Enhancement Capabilities investigates further capabilities that add a new perspective to the graph or the overall process. Lastly, Security & Compliance addresses role-based access control and the availability of audit logs.
IV. CONTRIBUTIONS, LIMITATIONS AND OUTLOOK The study of 16 process mining solutions with commercial licenses showed that the maturity level of the investigated software is highly varying. While some vendors offer basic discovery functionality without conformance checking in some cases, other vendors offer more elaborate features such as process simulation, predictive analytics and decision rule mining. We observe a potential trend: The boundaries between mere process mining functionality and other disciplines such as process modelling (BPMN), business intelligence and Machine Learning become more and more blurred.
The software selection is based on software listed in commercial reports and hence reflects a non-exhaustive picture of the market. Further, open-source software was not analysed. It is important to note that the software listing represents only a snapshot of the tools’ capabilities and features in terms of information timeliness. Vendors are continuously improving their products and extend the functionalities with periodic releases.
A follow-up study could examine the perspective of organisations on the relevance of the suggested criteria. Interviews may be conducted with organisations interested in process mining as well as organisations with already implemented process mining software.
The authors would like to express their gratitude to all vendors that participated in this study for their time and effort.
Criterion
(2 filtered sets of the same process can be compared with each other visually and metrically)
(Processes of 2 different event logs can be compared with each other visually and metrically)
Brief description
from as-is 37, Create new 37
be visualised in the process graph Yes/No - List of identified conformance violations for undesired activities, missing activities and non-compliant activity sequence
identified for selected conformance violations
Brief description
Yes/No
through a formula using own syntax, or by selection of any imported numerical attribute with the option of at least 5 different aggregation types, e.g. mean, median and percentiles Yes/No - User can define thresholds for metrics/KPIs or charts to emphasise (non-)acceptable values by colour highlighting
by attribute (e.g. country codes, city names) 37 (Visualisation of process-related locations in a world map graph)
(Sharing applied filter settings with other users; List of all additional means to collaborate and share insights, e.g. comment feature) Brief description
perspective by grouping of activities and org.
entities such as resources and departments [
Brief description
e.g. 2FA, LDAP, Active Directory