We describe in this paper an insurance company that has recently wanted to standardize on business process modeling language. To perform the evaluation, a generic framework for assessing the quality of models and modeling languages was specialized to the needs of the company. Three different modeling languages were evaluated according to the specialized criteria. The work illustrates the practical utility of the overall framework, where language quality features are looked upon as means to enable the creation of models of high quality. It also illustrates the need for specializing this kind of general framework based on the requirements of the specific organization.
There exists a large number of business process modeling languages. Deciding which modeling language to use for a specific task is often done in an ad-hoc fashion by different organizations. In this paper we present the work within an insurance company, which have a perceived need for using process modeling to support the integration of their business systems across different functions of the organization.
We have earlier developed a general framework for assessment of quality of
models, where one type of means to support different quality goals, is criteria for the
language to be used for modeling, also termed language quality
The next section describes the quality framework, with a focus on language quality. Section three describes the case in more detail, and is followed by the results of the evaluation. The conclusion highlights some of our experiences from using and specializing the quality framework for evaluating modeling languages for enterprise modeling.
The model quality framework
The model quality types are defined as relations between these sets. • Physical quality: The basic quality goals on the physical level are externalization, that the knowledge K of the domain D of some social actor has been externalized by the use of a modeling language, and internalizeability, that the externalized model M is persistent and available to the audience. • Empirical quality deals with HCI-ergonomics for documentation and modelingtools. • Syntactic quality is the correspondence between the model M and the language extension L of the language in which the model is written. • Semantic quality is the correspondence between the model M and the domain D.
The framework contains two semantic goals; Validity which means that all statements made in the model are correct relative to the domain and completeness which means that the model contains all the statements which is found in the domain. • Perceived semantic quality is the correspondence between the audience interpretation I of a model M and his or hers current knowledge K of D. • Pragmatic quality is the correspondence between the model M and the audience's interpretation of it (I). • The goal defined for social quality is agreement among audience members’ interpretations I. • The organizational quality of the model relates to that all statements in the model directly or indirectly contributes to fulfilling the goals of modeling (organizational goal validity), and that all the goals of modeling are being addressed through the model (organizational goal completeness).
Language quality relates the modeling languages used to the other sets. It is distinguished between two types of criteria:
Modeling domain
D
Modeller explicit knowledge
Km
Ideally, the language must be powerful enough to express anything in the domain, i.e.
not having construct deficit
Knowledge externalizability appropriateness
Organisational appropriateness Social actor explicit knowledge
Ks Modelling Goal
G
Model externalization
M Domain appropriateness
Technical
actor interpretation
T
Participant language knowledge appropriateness
Social actor interpretation
I
Comprehensibility appropriateness
Language extension
L Technical actor interpretation appropriateness
This area relates the knowledge of the stakeholder to the language. The conceptual basis should correspond as much as possible to the way individuals perceive reality. This will differ from person to person according to their previous experience, and thus will initially be directly dependent on the stakeholder or modeler. On the other hand the knowledge of the stakeholder is not static, i.e. it is possible to educate persons in the use of a specific language. In that case, one should base the language on experiences with languages for the relevant types of modeling, and languages that have been used successfully earlier in similar tasks. Participant language knowledge appropriateness is primarily a mean to help achieve physical and pragmatic quality.
This area relates the language to the knowledge of the modeler. The goal is that there are no statements in the explicit knowledge of the modeler that cannot be expressed in the language. Knowledge externalizability appropriateness is primarily a mean to achieve physical quality.
This area relates the language to the social actor interpretation. For the concepts of the language we have: • The concepts of the language should be easily distinguishable from each other.
(Vs. construct redundancy
As for the notation, the following aspects are important:
• Symbol discrimination should be easy.
• It should be easy to distinguish which of the symbols in a model any graphical
mark in the model is part of (What
Comprehensibility appropriateness is primarily a mean to achieve empirical and through that, pragmatic quality.
This area relates the language to the technical actor interpretation. For the technical actors, it is especially important that the language lend itself to automatic reasoning. This requires formality (i.e. both formal syntax and semantics. The formal semantics can be operational, logical, or both), but formality is not sufficient, since the reasoning must also be efficient to be of practical use. This is covered by what we term analyzability (to exploit the mathematical semantics) and executability (to exploit the operational semantics). Different aspects of technical actor interpretation appropriateness are a mean to achieve syntactic, semantic and pragmatic quality (through formal syntax, mathematical semantics, and operational semantics respectively).
relates the language to standards and other organizational needs within the organizational context of modeling. These are a mean to support organizational quality.
A number of sub-areas are identified for each of the six areas of language quality, and in (Østbø, 2000) approximately 70 possible criteria were identified. We will return to how this extensive list has been narrowed down and specialized for the task at hand.
The insurance company in our case has a large number of life insurance and pension insurance customers. The insurances are managed by a large number of systems of different age and using different technology. The business processes of the company go across systems, products and business areas, and the work pattern is dependant on the system being used. The company has modernized its IT-architecture. The ITarchitecture is service-oriented, based on a common communication bus and an EAIsystem to integrate the different system. To be able to support complete business processes in this architecture, there is a need for tools for development, evolution and enactment of business processes.
Before discussing the needs of the case organization specifically, we outline the main uses of enterprise process modeling. Five main categories for enterprise modeling can be distinguished: 1. Human-sense making and communication: To make sense of aspects of an enterprise and communicate this with other people. 2. Computer-assisted analysis: To gain knowledge about the enterprise through simulation or deduction. 3. Business process management e.g. in connection to following up ISO-certification. 4. Model deployment and activation: To integrate the model in an information system and thereby make it actively take part in the work performed by the organization 5. To give the context for a traditional system development project, without being directly deployed and activated.
The main goal of the insurance company was related to the fifth area above, to give the context for traditional systems development to support the integration of their business systems across different functions of the organization, and at the same time support human sense-making and communication (area 1). A general set of requirements to a modeling language based on the discussion of language quality in section 2 is outlined in (Østbø, 2000). These were looked at relative to the requirements of the case organization, and their importance was evaluated. The analysis together with the case organization resulted in the requirements in table 1.
The language should have an appropriate level of abstraction Concepts should be named similarly as it is in the domain The external representation of concepts should be intuitive to the stakeholders.
It should exist good guidelines for the use of the language It must be easy to differentiate between different concepts
The language must be consistent not having one symbol to represent several concepts, or more symbols that express the same concept.
One should strive for graphical simplicity It should be possible to group related statements The language should have a formal syntax
It must be possible to generate BPEL –documents from the model It must be possible to represent web-services in the model The language should lend itself to automatic execution and testing The language must be supported by tools that are either already available or can be made easily available in the organization The language should support traceability between the process model and any automated process support system The language should support the development of models that can improve the quality of the process.
The language should support the development of models that help in the follow-up of separate insurance cases “ “ “ “ " " " “ “ “ “ “ " Comp.app. “ “ Technical actor app. “ “
The overall approach to the evaluation was the following: First a short-list of relevant
languages was identified by us and the case organization in co-operation. The chosen
languages were then evaluated according to the selected criteria on a 0-3 scale. To
look upon this in more detail, all languages were used for the modeling of several real
cases from the insurance company using a modeling tool that could accommodate all
the selected languages (which in our case was METIS
Based on discussions with persons in the case-organization and experts on business
process modeling, three languages were selected for comparison. These will be briefly
described: For a longer description see the report
originally developed in the EU-project EXTERNAL as an extension of APM
The following main concepts are provided: • Task with input and output ports (which are specific types of decision-points) • General decision-points • Roles (Person-role, Organization-role, Tool-role, Object-role) • Resources (Persons, Organizations and groups of persons, Tools (manual and software), Objects (material and information)
A flow links two decision points and can carry a resource. A task has several parts: An in-port and an out-port, and potentially a set of roles and a set of sub-tasks. Roles 'is filled by' resources of the corresponding types. In addition EEML contains constructs for goal modeling, organizational modeling and data-modeling.
An activity-diagram can have the following symbols • Start, • End, • Activity, • Flow (between activities, either as control or as object-flows), • Decision-points, • Roles using swimlanes
In addition, a number of constructs is provided to support different kinds of control-flows.
BPMN
BPMN defines Business Process Diagram (BPD), which can be used to create graphical models especially useful for modeling business processes and their operations. It is based on a flowchart technique - models are networks of graphical objects (activities) with flow controls between them.
The four basic categories of elements are: • Flow Objects • Connecting Objects • Swimlanes • Artifacts (not included here)
Below the main result of the evaluation is summarized. For every language, every
requirement is scored according to the below scale on 0-3 (earlier evaluations of this
sort
The reasoning behind the grading can be found in
BPMN has the highest score in all categories, except for domain appropriateness, which is the category with highest weight, due to the importance of being able to express the relevant business process structures. EEML is found to have the best domain appropriateness, but loses to BPMN on technical actor appropriateness and participant knowledge appropriateness.
Comprehensibility appropriateness is the category which has the second highest weight, since the organization regards it to be very important that it is possible to use the language across the different areas of the organization, to improve communication between the IT-department and the business departments. In this category BPMN and Activity diagrams scores the same, which is not surprising given that they use the same kind of swimlane metaphor as a basic structuring mechanism. EEML got a lower score, primarily due to the graphical complexity of some of the concepts, combined with the fact that EEML has a larger number of concepts than the others.
Participant language knowledge appropriateness and technical actor appropriateness was scored equally high, and BPMN score somewhat surprisingly high on both areas. When looking at the evaluation not taking technical actor appropriateness into account, we see that the three languages score almost equal. Thus it is in this case the focus towards the relevant implementation platforms (BPEL and web services) that in this case is putting BPMN on top. On the other hand, we see that this focus on technical aspect do not destroy for the language as a communication tool between people, at least not as it is regarded in this case.
In the category organizational appropriateness, BPMN and Activity diagrams score almost the same. The organization had for some time used activity diagrams, but it also appeared that tools supporting BPMN was available for the organization. The organization concluded that it wanted to go forward using BPMN for this kind of modeling in the future.
We have in this paper described the use of a general framework for discussing the quality of models and modeling languages in a concrete case of evaluating business process modeling languages.
The case illustrates how our generic framework can (and must) be specialized to a
specific organization and type of modeling to be useful, which it was also found to be
by the people responsible for these aspects in the case organization. In an earlier use
of the framework, with a different emphasis, UML activity diagrams got a much
higher score than EEML, whereas here, they scored equally.
It can be argued that the actual valuation is somewhat simplistic (flat grades on a
03 scale that is summarized). On the other hand, different kinds of requirements are
weighted since the number of criteria in the different categories is different. An
alternative to flat grading is to use pair-wise comparison and AHP on the alternatives
This said, we should not forget that language quality properties are never more than means for supporting the model quality (where the modeling task typically has specific goals on its own). Thus instead of only evaluating modeling languages 'objectively' on the generic language quality features of expressiveness and comprehension, it is very important that these language quality goals are linked to model quality goals to more easily adapt such a generic frameworks to the task at hand. This is partly achieved by the inclusion of organizational appropriateness, which is not used in earlier work applying the framework. The evaluation results are also useful when a choice has been made, since those areas where the language does not score high can be supported through proper tools and modeling methodologies.