-Nowadays, more and more industrial organizations are modeling their business processes and implementing business process management systems. Therefore, it is very important to choose a suitable business process management system. The main purpose of this paper is to propose and verify the framework for evaluating business process management systems. This method covers the most important aspects of a business process management systems from the design to the monitoring of the business process. The main advantage of the proposed approach is that it is independent from any specific domain and execution engine. The evaluation framework was tested by applying it to four different business process management systems: “Bonita BPM”, “Camunda”, “jBPM“, “Activiti”.
Keywords—Business process; Business Process modeling; Business Process Management System
Bas Steen suggested definition of business process: “A
business process is a chain of activities, performed by entities
that takes one or more kinds of inputs and creates an output
that is of value for one or more entities” [
Business process management system is a system that
completely defines, manages and executes “workflows” [
The remainder of this paper is structured as follows: Section II presents the proposed framework for evaluating business process management systems; Section III presents the comparison between open source business process managements systems; Section IV concludes the paper.
II. THE FRAMEWORK FOR EVALUATING BUSINESS PROCESS
MANAGEMENT SYSTEMS
Business process modeling criteria – evaluate systems ability to model clear and complete processes and to import and export them; Systems development criteria – evaluate business process execution capabilities and user interface design; Installation and performance criteria – evaluate versioning and aspects relevant to the processes execution; Monitoring and control criteria – evaluate aspects relevant to monitoring and controlling during the execution of processes; Processes and data analysis criteria – evaluate systems capabilities for users to analyze business processes and other relevant data in the simplest possible way; Common criteria – evaluate systems information accessible to its users and the use of the system.
The evaluation criteria and the systems has been selected on the basis of what is most relevant to small and mediumsized organizations. Evaluation conditions and weights were defined for each criterion. Weighting (1 to 3) reflects the importance in the whole range of the analysis. To simplify criteria evaluation for the people doing this work, each of the criterion can be assessed in a four-point scale: 0 – the system does not meet the criterion; 1 – the system meets the criterion minimally; 2 – the system meets the criterion at an average; 3 – the system fully meets the criterion. Weights and criteria identification is quite subjective (using analyzed literature and “know-how” principle) in this work, since no way to adapt formal methods was found.
Maximum score is calculated by multiplying weight by maximum possible points (3).
Part of the companies have already modeled (not necessarily formally) the current (“as-is”) organization model, before the development and/or implementation of the processes management system. Modeled existing process forms the basis for process improvements, which are then added to the desired (“to-be”) process [4]. It is therefore important to know whether it is possible to import already created models into the system.
One of the most important criteria for this group is the
modeling of processes, indicating that the system allows to
model processes using formal business process modeling
language [
In addition to the modeled active structure (processes) and
passive structure (data), it is also important to define business
rules [
To re-use process modeled in the system or to give it to
someone, ability to export it in various formats is required [
The simulation parameters and evaluation conditions are given in Table I.
One of the most important criteria in the field of this paper is the translation of modeled business process into executable business process model. This means – the system must support at least one business process execution language and, translation between business process modeling and execution languages. All the systems being compared, must have this functionality, since only the systems which execute business processes are relevant in this paper.
The following two criteria are related to users who interact
with business processes. Generally, users use forms (web
portals) to send data to business systems, and a tool is required
to specify user roles and rights to participate in the process [
Business process integration with different systems is also
important in systems development (e.g., databases, customer
relationship management systems) [
Systems design parameters and evaluation conditions given in Table II.
Business processes and other components version
management is important to maintain the traceability of
processes [
Some of the steps of the process cannot be completed
without user interaction, therefore, user information is required.
User information can be done in two ways [
User interface customization function allows user to change its parameters, like user interface color, time zone, language, etc. [9]
Modeling criteria and evaluation conditions given in Table
Real-time monitoring can be helpful in detecting process
bottlenecks and identifying emerging risks and interfering in to
the process before occurrence of the damage [
Monitoring and control criteria are given in Table IV.
To verify if business processes will be executed as
expected, it is convenient to run a business process simulation.
For example, when new version of current process is modeled,
it can be simulated before releasing into production to choose
between the versions [
Another important aspect of the analysis is modeled business process verification. Verification is used to verify modeled (being executed) business process validity against the semantics of business process modeling language being used (it can be done, if (executable) business process model was Weight 2 2
VI.
Multilingual
assistance
•
passive informing – user can view his activity log,
e.g., through the portal.
modeled using business process modeling/execution language
that has a clear semantics) [
Installation and performance criteria
Data visualization tools (various diagrams (column, pie and
histogram), graphs, scales) are critical when analyzing data [
Process and data analysis criteria and evaluation conditions given in Table V.
Other aspects, besides business process management systems provided functionality, are also important. It is important not only to choose the right system, but also to learn to use it as quickly as possible and to get the required information: get needed help and access learning material. For users to be able to help to improve the system and report found issues, it is important to have an easy and fast way to send suggestions to the system creators. This functionality helps to create a connection between systems users and creators.
Common criteria and evaluation conditions given in Table
The framework contains only recommendations for assessing business process management systems. The comparison criteria of business process management systems are flexible; therefore, anyone can easily change the weights according to what is most important for a specific project or product. Moreover, evaluation scale can also be changed: it can be narrowed (to two or three different grades) or expanded (for example, to introduce a ten-point rating scale). Furthermore, more criteria can be identified if required. The order of criteria evaluation is flexible. It is recommended to evaluate each system separately: to better evaluate specific system’s function, it is advised to fully evaluate the first system and only then proceed to the next one.
III. COMPARISON OF BUSINESS PROCESS MANAGEMENT SYSTEMS
Four open source candidates have been chosen according to projects currently being ran in university academic department: “Bonita BPM”, “Camunda”, “jBPM”, “Activiti”.
“Bonita BPM” is an open source business process
management and business process software collection, created
in 2001 [
“Camunda” is a lightweight, open-source platform for
business process management [
“jBPM” is a flexible business process management (BPM)
package, creating a bridge between business analytics and
system developers [
“Activiti” is a lightweight workflow and business process
management (BPM) platform for the actors, systems
developers and administrators [
Comparative analysis is summarized in Table VII. It contains evaluations of all four of the compared systems. Part of the criteria have been tested during this analysis. However, due to the complexity of some criteria (e.g., integrations with other systems), some are evaluated only considering the analyzed literature.
To demonstrate how systems were evaluated, an example is
given: evaluation criterion “Process import” has pre-assigned
weight of 1, maximum value is 3. “Bonita BPM” has a function
to import using XML, *.bpmn, *.ydx and other formats, so this
system gets maximum points – 3.
33
3
9
9
9
3
9
9
6
6
30
3
6
3
12
4
4
8
6
3
6
3
2
3
8
15
Comparison criteria
Process modeling criteria
“Bonita BPM” system allows importing diagrams, which
have been exported from “Bonita BPM” “BPMN 2.0”,
“Microsoft Visio 2010”, “ARIS BPM 7.x”, “XPDL 1.0” and
“jBPM 3.2” tools using either XML or other formats (i.e.:
*.bpmn, *.vdx) [
“Bonita BPM” system also allows to define business rules
while defining decision points [
The largest business process export possibilities have been
found on the “Bonita BPM” system, since process can be
exported not only to an image, BPMN 2.0, XLM, BOS format,
but also elements to be exported (e.g., data models,
configurations, dependencies, organizations) can be chosen
[
As mentioned earlier, all systems were required to have a business process execution tool, therefore, all of them were evaluated with a maximum score. Compared systems have form modeling tools, which allows to create required user interface forms using WYSIWYG principle. Moreover, they all allow to manage user roles. “Bonita BPM”, “Camunda” and “jBPM” at least part of the form can be generated.
On “Bonita BPM” specification page [
All systems under comparison have business process versioning, which ensures that former version can be restored if needed. Since this criterion was evaluated only considering the analyzed literature and versioning was described in all of them, all systems received maximum available score of 3.
“Bonita BPM”, “Camunda”, “Activiti” documentations state that email send events can be configured while modeling business processes. No other active information possibilities were identified during the analysis. Passive information about tasks to be done, etc. can be seen when logged in to any of the systems.
“Bonita BPM” system allows to define toolbar size (normal
or small), choose grid options (e.g., whether to show a grid on
new diagram) and to choose whether to suggest to change
diagram name when saving the first time (if this is enabled,
every time a new diagram is saved, a window is displayed
where the name of the diagram can be changed). Moreover,
“Bonita BPM” portal allows to apply a profile (e.g., create new
menu items) [
“Bonita BPM” allows to monitor during the execution of
processes and see information about them: their list, how many
errors occurred, how many cases execute without errors and
how many of them are open. User can only monitor processes
they control [
“Bonita BPM” and “Camunda” systems provide a
possibility to change processes in real-time. “jBPM” also
allows to do minor process changes (e.g., edit variables) [
All, except “Activiti”, systems allow to run a process simulation and review its results. It helps to identify what needs to be changed in the process to achieve the best results. “Activiti” does not have that kind of functionality, however, the user guide describes how it can be done using an experimental tool called “Activiti-Crystalball”. All systems documentations (or user guides) state that the system can verify business processes modeled using BPMN notation. Also, each system provides a possibility to visually review certain information, which helps to analyze data and make decisions about process modification.
Information on the “Bonita BPM” website [
Figure 1 shows a radar diagram management systems compared. business process
After doing a comparative analysis of business process management systems and considering an evaluation table (Table VII) and a radar diagram (Fig. 1) it was identified that the best match for defined criteria is the “Camunda” system.
This paper presented a framework which allows to comprehensively evaluate selected business process management systems. The evaluation framework itself consists of selected criteria and information on how to valuate these criteria. This evaluation framework is flexible, therefore, everyone can adapt it to their needs: choose the most important criteria or add more, change the weights, evaluation scale or change the order of criteria. For demonstration purposes, an example of implementation of the proposed framework on four business process management systems (“Bonita BPM”, “Camunda”, “jBPM“, “Activiti”) has been presented. Finally, the comparison of these systems was made.
This framework will be used in further research of university academic department and in author’s Final Degree Project where this framework will be used in creating information system for specific domain.
Available: