Towards the Generalized Criterion for Evaluation of Business Process Model Quality

Abstract

Business process management has become the most widely-used and reliable approach to organizational management over the last decades. It is also considered as a part of quality management system in an organization. Business process modeling is the core of business process management, which is used for visualization, analysis, and improvement of organizational activities. Moreover, business process modeling plays an important role in the context of business process management maturity of an overall enterprise. Therefore, this paper is focused on the problem of business process model quality evaluation. Existing approaches based on the process modeling guidelines, measures and corresponding thresholds are reviewed, as well as the refined process modeling rules, corresponding quality criteria, the generalized quality criterion, and thresholds for its translation into linguistic values are proposed. The data model and software prototype are developed and the validation results are outlined.

Keywords

• Business Process Management
• Business Process Model
• Process Model Quality
• Process Model Metric
• Quality Criteria

Introduction

Business process modeling is considered as a key tool of Business Process Management (BPM). In paper [ 1 ] BPM is considered as both art and science of monitoring organizational activities in order to provide quality of produced products or services and find ways to improve these activities. According to Dumas et. al. [ 1 ], BPM is about managing entire chains of events, activities and decisions that ultimately add value to the organization and its customers. Whereas, business processes are considered as “chains of events, activities and decisions” that seems quite understandable and clear [ 1 ]. The BPM lifecycle itself starts with the modeling stage. More formally BPM is considered as the managerial discipline that uses the technologies for the process oriented management [ 2 ]. According to paper [ 3 ], a high-level overview of the BPM lifecycle includes the four key activities: model (create a business process model to be used for analysis or enactment), enact (use a process model to control and support concrete cases), analyze (analyze a process using a process model and/or event log), and manage (all other activities, e.g., adjusting the process, reallocating resources, or managing large collections of related process models). Business process models are mostly used in the design and analysis of information systems and are Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). considered as a good mechanism for communication among the stakeholders [ 4 ].

Therefore, it is important to ensure high quality of created business process models. A business process model of poor quality can disturb business process implementation and execution, as well as its performance [ 5 ]. While a high-quality model is expected to be accepted by stakeholders and thus prevents problems of business process implementation, deployment, execution, and continuous improvement [ 6 ].

Business process modeling is an essential part of understanding and improving the activities that an enterprise uses to achieve its business goals, however there was no generally accepted framework of process model quality [ 7 ]. To the best knowledge of the authors there is still no such quality framework nowadays. Therefore, this paper proposes evaluation criteria of business process model quality, which are based on existing best practices and guidelines together with measures and their corresponding thresholds. The objective of such criteria is to calculate a numerical degree to which a business process model fulfills requirements of process modeling guidelines. This definition of business process model quality is derived from ISO 9001’s definition of quality “degree to which a set of inherent characteristics fulfills requirements” [ 8 ].

Nowadays the Business Process Model and Notation (BPMN) is the leading standard for modeling business processes. It is provided by the Object Management Group (OMG) and it is used by business professionals as a standard notation allowing not only internal communication of the business procedures, but business-IT alignment and collaboration between business partners as well [ 9 ]. According to the latest survey in the domain of business process modeling [ 10 ], BPMN diagrams are used by 64% of respondents. After the BPMN 2.0 notation came up, and original name “Business Process Modeling Notation” was changed to “Business Process Model and Notation”, the graphic notation had been extended with the metamodel, the XML-based (eXtensible Markup Language) exchange file format, and the execution semantics.

Section 2 and its subsections describe related work in the field of business process quality management, BPM maturity models, and business process model quality evaluation. Business process modeling guidelines, corresponding measures, derived criteria of business process model quality, the generalized quality criterion and its threshold values are outlined in section 3. Section 4 describes how business information, which BPMN business process models communicate, might be processed for their further querying and reuse according to an ontology for organizing enterprise architectural artifacts (to which process models belongs) defined by the Zachman Framework [ 11 ]. This section also describes the developed software prototype used to validate the quality criteria proposed in section 3. Discussion of the obtained results is outlined, as well as conclusion and future work is formulated.

Related Work

Business Process Quality Management and Maturity Models

According to Tobias and Kern [ 12 ], BPM has its roots in Total Quality Management (TQM) appeared in late 1980s and in Business Process Reengineering (BPR) introduced in early 1990s by Hammer, Champy, and Davenport. One of BPM meanings is related to a four-step Plan-Do-Check-Act (PDCA) quality management cycle by Deming and Shewhart [ 12 ]. PDCA is a continuous cycle that consists of the following four successive stages: planning for change (plan), execution of the plan (do), evaluation of results (check) and standardization of the new, improved process (act) [ 13 ]. While TQM takes a scientific, statistical, and evidence-based approach for detecting and reducing quality anomalies through incremental quality improvement, BPR focuses on systematically reducing process anomalies through visualization and radical quality performance improvement [ 14 ]. Early BPM concept was focused largely on the technical aspects of process control, especially those related to production. Under the TQM paradigm, BPM gradually evolved from the narrow focus on technical issues toward a more management direction [ 13 ]. Authors of [ 12 ] define BPM as the management approach that focuses on business processes. A business process is defined as a horizontal sequence of activities that transform an input (need) to an output (result) to meet the needs of customers or stakeholders [ 12 ].

BPM took a more concrete form during the major revision of the ISO 9001 standard in 2000, and then in 2008 and 2015. These versions of the standard made the shift of focus from individual quality management requirements to a more holistic processoriented approach [ 13 ]. The ISO 9001:2015 standard considers process approach as the one of its quality management principles. A process approach means an organization manages their business as a system of processes. A process is described as a set of interrelated or interacting activities that use inputs to deliver an intended result. Process inputs and outputs might be tangible (e.g. materials, components or equipment) or intangible (e.g. data, information or knowledge) [ 8 ]. Since the revision of ISO 9001, BPM has had a central role in the implementation of all standards. Now it has a role of an integrated mechanism of managing the processes [ 13 ].

Since BPM is heterogeneously defined in the literature, a need of a standardized BPM framework has inspired appearance of BPM Maturity Models (BPMMM) [ 12 ]. Authors of review [ 15 ] refer to a maturity model as to the sequence of discrete maturity levels used to assess processes in one or multiple business domains. Maturity levels represent expected or typical evolution of these processes [ 15 ]. For example, Capability Maturity Model Integration (CMMI) in the area of software engineering emerged at the beginning of 1990s as a means to improve software development processes to achieve higher quality, and has been used since then by hundreds of organizations worldwide. The success of CMMI inspired the development of several maturity models in other domains, including BPM [ 15 ]. In paper [ 16 ], the importance and impact of an organization’s business process maturity on overall performance has been presented. BPMMM presented in this study is based on the maturity model that was defined for the maturity of software development capability [ 2 ]. According to [ 16 ], with the increasing level of an organization’s BPM maturity, the importance and impact of BPMN models (since BPMN is considered as de-facto standard in business process modeling) is increasing and becoming critical for overall BPM success.

Table 1 describes BPM maturity levels with respect to the dimensions of awareness of business processes (ABP), documentation of business processes (DBP), monitoring of business processes (MBP), and refinement of business processes (RBP). Therefore, to the best our knowledge, the maturity model presented in [ 16 ] is the only maturity model that, unlike BPMMMs reviewed in [ 15 ], presents the role of business process modeling and supporting tools in different process maturity levels. 2.2

Business Process Model Quality

The goal of business process modeling is the representation of organizational activities, so that current processes may be analyzed and improved. Business process modeling is not only a requirement for ISO 9001 quality standards or BPM maturity models; it plays an important role in the implementation of work-flow management [ 17 ].

Modeling helps visualize the important steps in a business process, how they are related to each other, which actors and information systems are involved in carrying out various activities, and where communication takes place with external parties. Business process models are usually described in a visual way, using figures that are connected to each other and supported by textual annotations [ 17 ]. Graphical notation of the BPMN 2.0 specification includes following elements to describe business processes (see Fig. 1) [ 18 ].

BPMN business process diagrams basically describe business processes in terms of events and actions connected through control flows that indicate valid sequences in the process execution. Gateways are special nodes connected through control flows that indicate whether the process executes in parallel (AND), alternatively (XOR) or optionally (OR). The beginning of the process is denoted by a start event and its conclusion by a set of end event nodes. Each pool represents a process itself, while each lane represents a human participant in the activity [ 18 ].

Authors of [ 17 ] proposed a business process model quality framework called SIQ (it is “Simple enough to be practically applicable, Integrates the most relevant insights from the BPM field, and deals with Quality” [ 17 ]). SIQ defines three quality categories of the business process model: syntactic quality (conformance to the rules of a modeling notation), semantic quality (conformance to a captured process), and pragmatic quality (understandability by readers) [ 17 ].

While syntactic quality of designed business process models is checked by modeling software tools and checking semantic quality is hardly possible without human involvement (it requires understanding the domain in question and the exact purpose of the process model [ 17 ]), checking pragmatic quality is of interest for research in terms of its automation. Mendling, Reijers, and van der Aalst defined Seven Process Modeling Guidelines (7PMG) framework for creating understandable models that are less error-prone or improving the quality of existing models [ 17, 19 ].

As for quantitative process model quality indicators, various metrics and thresholds for such metrics were proposed for BPMN process models [ 19 ], such as: • Size (number of events, gateways, tasks, and sequence flows). • Gateway mismatch (sum of gateway pairs that do not match each other). • Connectivity coefficient (ratio of the number of arcs to the number of nodes). • Control flow complexity (sum over all gateways weighted by their potential combinations of states after the split). 2.3

Conclusion on Related Work Review

Performed review demonstrates that, despite the lack of clear definition and standards, BPM is the dynamically growing management approach adopted by many organizations and considered as the one of quality management principles of ISO 9001 standard. Business process modeling, being the core technique of BPM, plays an important role in business process analysis and improvement, and also facilitates increasing of BPM maturity of an overall organization. Present State-of-the-Art shows there are heterogeneous metrics and corresponding thresholds used to measure business process model quality. The generalized measure that could combine viewpoints of size, gateway mismatch, nodes connectivity, and control flow complexity is not revealed yet. 3 3.1

Business Process Model Quality Evaluation

Business Process Model Measures

Out of the seven process modeling guidelines provided by the 7PMG framework, we have left only five by merging G1 and G7, and avoiding the guideline G6 [ 20 ]. It is suggested, that G7 “Decompose a model with more than 31 elements” might be applied as the consequence of G1 “Do not use more than 31 nodes”, so that is why these guidelines were merged into the single rule R1 “Do not use more than 31 nodes or decompose a model with more than 31 elements if possible”. The guideline G6 “Use verb-object activity labels” is avoided, since it belongs to the semantic process model quality, while the pragmatic quality is considered in this study. Remaining four guidelines G2 – G5 are adopted as rules R2 – R5 respectively (see Table 2). For the selected five BPMN modeling rules based on the 7PMG guidelines we need to define criteria that might quantifiably evaluate the degree of fulfillment of each of the modeling rules R1 – R5 by the analyzed BPMN process model. Such criteria might be derived from the following measures. nodes,

= | |.

Total number of nodes (TNN). The size of the collection of business process model Number of invalid elements (NIE). Number of nodes with invalid inputs or outputs (G2 recommendation should be applied for all types of nodes, not only gateways): = ∑ ∈ (| | ≠ 1 ∨ |

| ≠ 1) + ∑ ∈ (| | > 1 ∨ | | > 1) + + ∑ ∈ ¬[(| | = 1 ∧ | | > 1) ∨ (| | > 1 ∧ | | = 1)] . (1) Where: gateways ∈ ; puts of the node, ∈ = ∪ ∪ . ─ is the collection of tasks ∈ , is the set of events ∈ , is the collection of ─ | | is the size of the set of inputs of the node, | | is the size of the set of outNumber of start events (NSE). The size of the set of start events, which is the subset of the collection of events, of the collection of events, = | ⊆ |.

= | ⊆ |.

Number of end events (NEE). The size of the set of end events, which is the subset Number of mismatched gateways (NMG). The count of gateways that do not have matching gateways. It is calculated as the difference between numbers of split gateways (with 1 input and more than 1 output) and join gateways (with more than 1 input and 1 output): = |∑ ∈ (| | = 1 ∧ | | > 1) − ∑ ∈ (| | > 1 ∧ | | = 1)|. (2)

Total number of gateways (TNG). The size of the set of gateways,

= | |.

Total number of inclusive (OR) gateways (TNI). The size of the collection of inclu

sive (OR) gateways, which is the subset of the set of gateways, = | ⊆ |. 3.2

Criteria of Business Process Model Quality

Using business process model measures proposed in the previous subsection, we have formulated criteria of business process model quality , = ̅1̅,̅5̅ (see Table 2). These criteria are based on the set of modeling rules R1 – R5 derived from the 7PMG recommendations for business process modeling [ 20 ]. 1 − 1)2 } BPMN modeling rule R1: Do not use more than 31 nodes or decompose a model with more than 31 elements if possible (merged G1 and G7) R2: Avoid nodes with invalid inputs or outputs (refined G2) R3: Avoid usage of multiple start or multiple end events or missing events (G3) R4: Avoid gateways mismatch (G4) R5: Avoid inclusive (OR) gateways (G5)

Criteria equation 3 = min { 1 + (

1, 1 = { 31

≤ 31, , Each of these criteria , = 1̅̅,̅5̅ serves to quantifiably evaluate the degree to which the analyzed business process model fulfills corresponding rules R1 – R5. Therefore, calculated values of these criteria belong to the interval ∈ [ 0,1 ], = ̅1̅,̅5̅. By introducing these criteria we did contribute to the State-of-the-Art, however, the need for the generalized quality criterion of business process models still exists. Weighted sum model. In order to define the generalized criterion for BPMN process model quality evaluation, we used the weighted sum model (WSM) [ 21 ]. It is applicable in this case, since all the criteria are expressed in the same unit. Hence the generalized criterion of business process model quality might be defined as following: = ∑5=1 ⋅ . (3) Where , = ̅1̅,̅5̅ are the relative weights of importance of the criteria , = ̅1̅,̅5̅.

Values of the relative weights are based on the ranks of 7PMG recommendations defined in [ 20 ] (see Table 3). Therefore, the generalized quality criterion takes values in the interval ∈ [ 0,1 ], where 0 stands for the lowest quality, while 1 indicates the highest quality of a business process model. Quality criterion values translation into linguistic values. Inspired by the approach shown in [ 6 ], we propose the procedure of translation from crisp quality values represented by the criterion into linguistic values (see Table 4). However, while [ 6 ] proposes fuzzification of multiple business process model measures and inference procedure in order to obtain the linguistic values of process model understandability and modifiability, our approach is based on the translation of the generalized quality criterion into the single linguistic value according to the Harrington desirability scale (see Table 4) [ 22 ], which describes the quality of the business process model based on multiple criteria (see Table 2). Software Prototype. In order to store analyzed BPMN models in a way they can be queried and reused, we have applied the ontology defined in Zachman Framework and elaborated by authors of [ 11 ] (see Fig. 2). We also used practitioners experience in categorization of BPMN artifacts according to the Zachman Framework [ 23 ].

As it is shown in Fig. 2, the software prototype operates over the two-tier data storage. The lower layer is based on the in-memory H2 relational database that stores raw measures of process models, while higher layer is implemented in a form of RDF (Resource Description Framework) graph under the control of the Apache Jena framework that offers tools for querying linked data about business process models.

The overall architecture of the software prototype is quite simple (see Fig. 3); it is based on the Spring Boot Java framework on the server side and the single-page AngularJS web application on the front-end side. The Camunda BPMN Model API (Application Programming Interface) library is used to process XML-based files that contain BPMN 2.0 definitions of business process models.

In order to visualize summary results we used the Microsoft Power BI dashboard as the alternative for the web-interface under development (see Fig. 4).

Validation Results. At this moment the software prototype allows loading and processing of BPMN 2.0 documents in order to evaluate their quality in both numerical and linguistic forms. It also implements simple querying using the “subject-predicateobject” form over the stored business process models data. The software usage example is shown in Fig. 5. 3390 BPMN 2.0 definitions of business processes from different domains provided in the GitHub public repository of Camunda [ 24 ] were processed using the developed prototype.

Out of these models, 2606 are of very high quality (see Fig. 4), 470 are of high quality, and remaining 314 are of medium quality. However, only 1250 models are considered as those fully conform to the introduced quality criteria (see Fig. 4). It means, that 1250 models have = 1, while remaining 1356 very high-quality process models still could be improved ( < 1), as well as the 784 models of high and medium quality. As it is shown in Fig. 3, R2 is the most commonly violated business process modeling rule (avoid nodes with invalid inputs or outputs). Other commonly violated business process modeling rules are R4 (avoid gateways mismatch) and R3 (do not use multiple start or multiple end events). Examples of rules violations detected in the analyzed set of BPMN models are shown in Fig. 6.

Conclusion and Future Work

In this paper we have introduced the set of business process model measures used to evaluate the degree to which a business process model fulfills each of the business process modeling rules derived from the 7PMG framework. By applying the WSM model [ 21 ] and Harrington scale [ 22 ], we have obtained the generalized quality criterion, which crisp values could be translated into linguistic values. The data model and developed software prototype allow storing data about processed BPMN models in order to simplify their querying and reusing. Early results of business process model quality evaluation are demonstrated and discussed. Future work includes elaboration on the data model and the software solution (compliance with the Zachman Framework ontology, advanced querying and versions traceability of BPMN models), and new results analysis.

References

1. Dumas , M. , La Rosa , M. , Mendling , J. , Reijers , H.: Introduction to Business Process Management . In: Fundamentals of business process management . Heidelberg: Springer ( 2013 ). doi: 10 .1007/978-3- 642 -33143- 5 _ 1

2. Pavlicek , J. , Hronza , R. , Pavlickova , P. , Jelinkova , K. : The business process model quality metrics . Workshop on Enterprise and Organizational Modeling and Simulation , 134 - 148 . Springer, Cham ( 2017 ). doi: 10 .1007/978-3- 319 -68185-6_ 10

3. Van der Aalst , W. M.: Business process management: a comprehensive survey . Hindawi Publishing Corporation: ISRN Software Engineering ( 2013 ). doi: 10 .1155/ 2013 /507984

4. Kahloun , F. , Ghannouchi , S. A. : A Classification Algorithm for Assessing the Quality Criteria for Business Process Models . International Conference on Health Information Science , 71 - 81 . Springer, Cham ( 2017 ). doi: 10 .1007/978-3- 319 -76351- 4 _ 8

5. Oukharijane , J. , Yahya , F. , Boukadi , K. , Abdallah , H. B.: Towards an approach for the evaluation of the quality of business process models . AICCSA 2018 , 1 - 8 ( 2018 ). doi: 10 .1109/AICCSA. 2018 .8612811

6. Yahya , F. , Boukadi , K. , Ben-Abdallah , H. , Maamar , Z. : Application of Fuzzy Logic to Assess the Quality of BPMN Models . ICSOFT 2017 , 188 - 209 . Springer ( 2018 ). doi: 10 .1007/978-3- 319 -93641- 3 _ 9

7. de Oca, I. M. M. , Snoeck , M. , Reijers , H. A. , Rodriguez-Morffi , A. : A systematic literature review of studies on business process modeling quality . Information and software technology 58 , 187 - 205 ( 2015 ). doi: 10 .1016/j.infsof. 2014 . 07 .011

8. Tricker , R.: ISO 9001: 2015 for Small Business . Taylor & Francis ( 2016 )

9. Bonnet , F. , Decker , G. , Dugan , L. , Kurz , M. , Misiak , Z. , Ringuette , S. : Making BPMN a true lingua franca . BPM Trends ( 2014 )

10. Harmon , P. : The State of Business Process Management 2016 . BPTrends ( 2016 )

11. Lee , J. Y. , Ahn , K. , Lee , S. : Applying OWL Ontology to Zachman Framework for Requirement Analysis . Advanced Science and Technology Letters , 46 , 34 - 37 ( 2014 ). doi: 10 .14257/astl. 2014 . 46 .08

12. Roeser , T. , Kern , E. M.: Surveys in business process management - a literature review . Business Process Management Journal 21 ( 3 ), 692 - 718 ( 2015 ). doi: 10 .1108/BPMJ-07- 2014-0065

13. Chountalas , P. T., Lagodimos , A. G. : Paradigms in business process management specifications: a critical overview . BPM Journal 25 ( 5 ), 1040 - 1069 ( 2019 ). doi: 10 .1108/BPMJ01-2018-0023

14. Saab , N. , Helms , R. , Zoet , M. : Predictive quality performance control in BPM: proposing a framework for predicting quality anomalies . Procedia computer science , 138 , 714 - 723 ( 2018 ). doi: 10 .1016/j.procs. 2018 . 10 .094

15. Tarhan , A. , Turetken , O. , Reijers , H. A. : Business process maturity models: A systematic literature review . Information and Software Technology , 75 , 122 - 134 ( 2016 ). doi: 10 .1016/j.infsof. 2016 . 01 .010

16. Polancic , G.: Process Maturity and BPMN . Orbus Software ( 2013 )

17. Reijers , H. A. , Mendling , J. , Recker , J.: Business process quality management . In Handbook on Business Process Management 1 , 167 - 185 . Springer, Berlin, Heidelberg ( 2010 ). doi: 10 .1007/978-3- 642 -45100- 3 _ 8

18. Ceballos , H. G. , Flores-Solorio , V. , Garcia , J. P.: A Probabilistic BPMN Normal Form to Model and Advise Human Activities . International Workshop on Engineering Multi-Agent Systems , 51 - 69 . Springer, Cham ( 2015 ). doi: 10 .1007/978-3- 319 -26184- 3 _ 4

19. Corradini , F. , Ferrari , A. , Fornari , F. , Gnesi , S. , Polini , A , Re, B. , Spagnolo , G. O. : Quality Assessment Strategy: Applying Business Process Modelling Understandability Guidelines . University of Camerino, Italy ( 2015 )

20. Mendling , J. , Reijers , H. A., van der Aalst , W. M. P. : Seven process modeling guidelines (7PMG) . Information and Software Technology 52 , 127 - 136 ( 2010 ). doi: 10 .1016/j.infsof. 2009 . 08 .004

21. Ghosh , A. , Mal , P. , Majumdar , A. : Advanced Optimization and Decision-Making Techniques in Textile Manufacturing . CRC Press ( 2019 )

22. Voronin , A.: Multi-Criteria Decision Making for the Management of Complex Systems . IGI Global ( 2017 ). doi: 10 .4018/978-1- 5225 -2509-7

23. Deconstruct Business Process Model using Zachman Framework Principles, 5 part videos . https://sunilduttjha.wordpress.com/ 2012 /03/16/deconstruct-business -process-model-usingzachman-framework-principles-5-part-videos/

24. Camunda - BPMN for research. https://github.com/camunda/bpmn-for-research