Organizations often store information about their process landscape in large process model collections. To utilize those knowledge bases, process model matchers assist experts in keeping track of model relationships by automatically identifying corresponding activities. Yet, state-of-the-art matchers achieve an overall low e ectiveness. That is, their results contain only a few existing, but many non-existing correspondences. Hence, practical application is far from being realized. In light of this, the thesis introduces ADBOT, an interactive matching technique that analyzes manually corrected matcher results to adapt itself to the characteristics of the respective model collection. At heart, ADBOT relies on BOT and OPBOT. BOT is a con gurable matcher that solely examines activity labels and OPBOT analyzes model collection characteristics to automatically con gure BOT without human intervention. While the three matchers pose di erent requirements on model and feedback availability, they all achieve a high e ectiveness in relation to the state-of-the-art, as shown in a comparative evaluation. ADBOT as the most e ective technique even yields improvements of up to 70%.
Process models are an essential tool to communicate, analyze, design and
continuously improve business processes. In organizations that use process models
large process model collections are not uncommon. To leverage the knowledge
that is hidden in such extensive collections, research has devised approaches for
model collection management [
Review Literature Phase I
Develop Propositions
Model Collections & Gold Standards
Develop Techniques
Assess
Effectiveness Matching Propositions
Technique Candidates
Matching
Techniques
Phase II
Fig. 1. Research design
of the process model matching contests in 2013 and 2015 [
The research methodology that the thesis is based on is oriented towards the
information systems research framework [
The rst phase deals with a literature review which follows guidelines for
literature reviews in information systems research [
The second phase attends to the development of matching techniques under the consideration of the shortcomings. The activities in this phase are carried out iteratively. Each iteration starts with the development of matching propositions where design decisions are examined with regard to their validity. Valid decisions are then considered as matching propositions. In the development of technique candidates the propositions are integrated into matching algorithms in di erent ways. Lastly, the e ectiveness is assessed and those candidates that achieve a high e ectiveness are proposed as matching techniques. It is important to note that due to this research approach, matching techniques are not treated as black boxes, i.e., as the sum of their parts. Instead, the design decisions are individually examined. Thus, the thesis does not only deliver matching techniques, but a catalogue of (in-)validated design decisions that future work can build upon.
The respective experiments rely on empirical datasets that contain model
collections, i.e., sets of model pairs, and the respective gold standards which
comprise manually identi ed correspondences. There are two types of datasets.
Development datasets are used in each iteration to evaluate the design decisions
and the e ectiveness. By contrast, evaluation datasets are exclusively applied in
a nal iteration to examine the generalizability of the ndings. In total, there
are four datasets, each containing 36 model pairs. While the birth registration
(BR) and the university admission (UA) dataset from the matching contest in
2013 [
In the experiments a variety of research methods is employed to assess the
design decisions and the e ectiveness of the matchers. Statistical measures such
as the Kolmogorov-Smirnov test [
The results of the thesis are summarized in four hypotheses. In the following, the hypotheses are outlined along with the respective evidence from the thesis.
H1: The identi cation of correspondences between business process models is a challenge for organizations which is not su ciently supported by existing approaches.
This hypothesis addresses the practical relevance which was already outlined in Section 1 where various problems related to model collection management are listed. Moreover, the scienti c demand for the development of more e ective matchers was veri ed through the literature review. A total of 17 publications that introduced matchers were identi ed and examined. In nine publications the applicability of matchers was restricted by assumptions that e.g., limit the supported modeling languages, that impose requirements on the labeling vocabulary, or that exclude situations where sets of activities correspond. Further, the matchers' e ectiveness is low and varies across datasets. Finally, prior work focused on illustrating the matchers' usage or on black-box evaluations that examined the e ectiveness, but not the validity of design decisions. These observations con rm the need for further research aimed at improving the e ectiveness.
The remaining hypotheses address di erent sources of information for
matching techniques. Each perspective is associated with its own matcher. Table 1
shows the e ectiveness of these matchers and of the top-performing matchers
from the matching contests [
H2: Label-based matching techniques yield a varying and generally insu cient e ectiveness.
Various natural language analysis techniques related to (i) the decomposition of labels into words, (ii) the syntactic and semantic comparison of words, and (iii) the reduction of di erences in label speci city were discussed and analyzed. The results were then used to devise BOT, a label-based matcher that can be adjusted to the model collection via ve parameters. BOT comprises a default con guration that performs nearly as good as the three best performing matchers from the matching contests. Finally, a qualitative analysis of BOT's results reveals that many falsely suggested correspondences are due to only small variations in the labels, e.g., when two labels only di er with regard to one word. Further, labels of correspondences that are not detected contain synonyms and homonyms or involve very generic descriptions. This analysis shows that to achieve a high e ectiveness, domain-speci c knowledge sources are required.
H3: The maximization of the e ectiveness of label-based matching techniques is enabled by the analysis of control ow information.
There are three scenarios for the use of control- ow information. First, activities are compared with regard to control- ow properties such as their position in the models. Second, candidates for complex correspondences can be established by extracting connected sub-graphs from the models. Third, there is the consistency analysis which assesses the degree to which control- ow dependencies between activities in one model resemble those of their corresponding counterparts in a second model. While experiments show that the rst two options are not generally valid, the third option is identi ed as a promising design decision. Consequently, OPBOT automatically executes many BOT con gurations and suggests a combination of the most consistent results. OPBOT generally detects con gurations that perform better or close to BOT's default con guration.
H4: The e ectiveness of matching techniques is improved by the utilization of expert feedback.
The process of feedback collection is iterative: a pair of models is selected, then automatically matched, and experts nally correct the suggestions. That way, a ground truth is established which can be used to adapt matchers and to prepare them for yet unmatched model pairs. In particular, ADBOT builds upon OPBOT and contains three strategies to learn from the feedback. First, the assessment of semantic word relations is adapted to better re ect the domainspeci c relationships. Second, it relies on transitivity which is a reliable indicator, i.e., two activities are matched, if they correspond to the same activity. Third, OPBOT's BOT con gurations are constantly re-adjusted. ADBOT outperforms the automated matchers from the contests by up to 70%. To reduce the expert workload, ADBOT also sorts the model pairs and feedback collection can be turned after 30 to 50% of the pairs, as the improvements level o at this point.
The results of the thesis are summarized in the main research hypothesis which was validated through the veri cation of the four sub-hypotheses H1 to H4: H0: The adaptation of business process model matching techniques to model collections is necessary to ensure a high e ectiveness and the analysis of the control ow as well as of expert feedback provides means to implement this adaptation.
An important aspect in future work is the extension of the datasets to cover further matching scenarios and achieve a broader evaluation. Moreover, more emphasis should be put on the expert involvement. As experts operate in di erent contexts, matchers should be adaptive to di erent views. To reduce human e orts, modeling tools must provide interfaces that allow for easily interpreting and correcting correspondences. This also includes strategies to derive expert feedback from other model collection tasks like model merging or querying.