Semantic technologies promise fully leveraging the content of enterprise and business process models by applying reasoning techniques to query the process space or using ontological mappings for bridging the business-IT divide. Such an approach is known as semantic business process management. We conducted an empirical case study to explore semantic business process management. The case study was replicated 13 times with 17 participants from 8 different industrial and scientific organisations following a strict case study methodology to ensure validity and reliability of the results. The results show that participants embrace the possibilities of semantic technologies, but there are still open problems. In this paper, we present the case study design and discuss the results achieved.
models and languages allowing an easy transition. Following this idea, Abramowicz et al. [AFKK07] create a semantic representation of the BPMN modelling language, which allows relating BPMN activities to WSMO goals [FLP+06]. In a similar effort, Nitzsche et al. [NWvL07] provide an ontologised version of BPEL. Again, BPEL activities can be related to WSMO goals enabling semantic web service discovery during process execution. Dimitrov et al. [DSSK07] implemented a semantic business process modelling tool based on WSMO Studio, which allows modelling semantic business processes using semantic BPMN. Besides process design and execution, semantics are also used for process analysis [CAdMZ+07] and process mining [AdMPvdA+07]. Semantic compliance management [EKSMP08] is a more holistic approach ensuring compliance of semantic enterprise models with laws and regulations.
This overview shows sBPM is an active research area. So far, most efforts are fundamental research either defining the necessary languages [HR07a, AFKK07, NWvL07] or overall approaches [HLD+05, CAdMZ+07, AdMPvdA+07, EKSMP08]. However, empirical evaluations of the proposed technologies to validate the practical relevance of sBPM are still missing. Such work would provide feedback, which could be incorporated in research agendas to realign the research efforts with requirements from the industrial field. In this paper we1 present first work providing this missing feedback through an empirical case study conducted in the telecommunication domain. A prototype of a sBPM suite was provided to practitioners, consultants, and researchers not involved in sBPM research. In addition, a tutorial describing step-by-step how to use the prototype was provided. After conducting the tutorial, participants were interviewed and asked to reflect on the usage of semantics in BPM. This paper describes and discusses the case study’s results. 2
2.1
While designing our research, we first defined our research question as follows: Which benefits do today’s sBPM technologies provide to BPM experts and what obstacles exist hindering the adoption in industry? Our research interests were of explorative nature. According to Yin [Yin03], controlled experiments as well as case studies are possible research methods to answer how and why research questions. Kitchenham et al. [KPP95] add that experiments are usually used for research-in-the-small and case studies for research-in-the-typical. We were clearly focused on research-in-the-typical, since our intention was to investigate the usage of sBPM in real-world settings. Kitchenham et al. also state that case study research is often used to evaluate new technologies. This also applied to our case.
We decided to conduct an empirical case study following the case study research methodology defined by Yin [Yin03]. We augmented Yin’s methodology with ideas taken from 1Research is part of the integrated research project SUPER financed by the EU Commission. Kitchenham et al. [KPP95], because they describe specific practices for case study research in software engineering. According to Yin’s methodology, after defining the research question one has to identify the case study propositions. 2.2
Research questions are usually too abstract and too broad to be answered in a case study [Yin03]. Therefore, case study propositions (i. e. hypothesis) are defined to exactly know what to look at, to be aware of own preconceptions, and to prevent biased research. We focused on the top-down approach of business process automation. The use of semantics in this area promises a simplified generation of executable business process models, because an ontological mapping between the business and IT domain exists on a metalevel. Using data mediators suggests simplifying data handling, because semantically equal but structural different data objects can be transformed automatically. We expected that identifying appropriate services to automate functions is easier using semantics instead of selecting services manually. In contrast, we expected that business experts currently cannot motivate this use case. Services represent business partners, who are contracted carefully and not dynamically allocated at runtime. Another obstacle concerns the definition of the needed ontologies, because the business environment might evolve faster than ontologies can be adapted. Also, adding semantics to the enterprise computing stack means an increased learning curve and additional middleware to be supported. Abstracting from the different concrete propositions shows that we were sceptical about whether the use of semantics in BPM will pay off. Adding semantics will certainly simplify or even remove certain steps, but new steps are added. We estimated that an investment into semantics cannot be justified economically. 2.3
Yin [Yin03] requests to clearly define the unit of analysis meaning to define the case study’s case. As defined in the research question, we were interested in the benefits of semantics in BPM. In our case the unit of analysis was business process automation. We prepared two tutorials guiding a user through the process of implementing a business process. One tutorial covered the approach taken today without semantics. We call this the non-semantic tutorial or approach. The second tutorial used semantics and is therefore called semantic tutorial or approach. Both tutorials are based on business processes taken from real-world projects. The case of our case study comprises conducting at least the semantic tutorial if the participant is already familiar with the non-semantic approach. We gathered experience gained by the participants through semi-structured interviews based on 18 open-ended questions. The interviewees were asked to describe what they have done, how non-semantic and semantic approach differ, and to reflect on the usage of semantics. At the beginning of each interview, we elaborated on the background of our research effort and explained that the interview results are made anonymous and not publicly available assuring privacy. We emphasised that we are not trying to prove or disprove semantics as beneficial. We allowed participants to ask questions as well. The interviews were not recorded but instead conducted by two researchers. One researcher led the interview and the other researcher focused on taking notes. Each interviewer wrote a small summary immediately after the interview and both summaries were then exchanged. We also collected work artefacts such as the semantically annotated business process models to have multiple sources of evidence [Yin03]. 3
3.1
In this section we describe the different components of the case study. Before we started the case study, we prepared the semantic tutorial, the example business process, and semantically enabled prototypes. To ensure that all replications of the case study and all interviews are run and analysed in the same way, we created a case study protocol [Yin03]. The case study protocol defines the overall research goal, case study propositions, and unit of analysis. It also defines the questions to be asked during the interview, it describes how to conduct the study, and how to analyse the results. It also names possible participants and outlines the structure of the result report. To further increase the validity of our case study, we applied the case study checklist described in [HR07b]. 3.2
The case study is based on a non-semantic and a semantic tutorial. The non-semantic tutorial is available in two versions featuring two different business processes. One business process was taken from the automotive domain and the other one belongs to the egovernment domain. The tutorial uses ARIS to model the business process with the EPC notation. Each function is annotated with a software service using ARIS’ service discovery functionality. Afterwards, the business process is transformed by the tool into BPEL. Additional manual refinements are needed before the BPEL process can be deployed and executed on Oracle BPEL Process Server. The tutorial package consists of a database for ARIS, a set of implemented web services, and a detailed manual. The non-semantic tutorial was not prepared for the case study, but existed already before [SKD+08]. The semantic tutorial was prepared for the case study. It consists of semantically extended version of ARIS and uses Oracle BPEL Process Server (see section 3.4). A detailed stepby-step instruction explaining how to install the necessary extensions and how to conduct the tutorial was provided to the participants. The different parts of the semantic tutorial are described in the following subsections. 3.3
The business process used in the semantic tutorial was contributed by Telekomunikacja Polska (TP). TP is the dominant player in the Polish telecommunications market serving 10.6 million fixed-line subscribers and over 12 million mobile customers, employing about 28.000 people (as of Q1/2007).
The voice-over-IP (VoIP) ordering process was used in the tutorial, because it is a rather complex one involving internal and external parties. The business process is illustrated in Fig. 1 using a simplified EPC notation. Most events and all semantic annotations were removed from the process model so that it fits on one page. The VoIP ordering process allows TP’s customers to order the VoIP service for an existing contract. The process is initiated by the customer through TP’s web portal. After identifying the customer, the process first checks if all technical and formal requirements are fulfilled. A new order is created, which must be confirmed by the customer. A check is run to see if the customer already has the necessary hardware. If not, the hardware is sent together with the contract to the customer. After TP receives the signed contract, the contract is archived, the billing system is activated, and finally the VoIP service is activated. An ontology and belonging semantic web services existed already and were reused [FRS07]. 3.4
ARIS was used for modelling the semantic business process. The tool was extended as described in [SSEK08b]. Each function is annotated by a WSMO goal using an prototypical graphical user interface. On the left side of the screen, the user selects a goal. On the right side the belonging WSML description is shown. The WSML files on the left have no speaking names in order to force the participant to look at the WSML code. After confirming the selection of the goal, the WSMO goal and the ontological input/output instances are added to the function. The WSMO goals are later used during process execution to discover matching services. Another extension was developed allowing users to complete the data flow in the process model by mapping ontological instances produced by a function to the input of a later occurring function. The existing EPC to BPEL transformation was adapted, too. In case of a function annotated with a WSMO goal, a BPEL variable with the WSMO goal as value is created and a proxy service is invoked. Today’s process execution environments are not able to use semantic descriptions like WSMO goals to discover services during runtime. To still allow using semantic descriptions in the executed BPEL process, we implemented a proxy service. This proxy service consumes the semantic description through a standard web service interface and invokes a semantic execution environment to do semantic service discovery. The discovered semantic web service is invoked by the proxy service as well. A detailed description of our approach to service discovery during process execution can be found in [SSEK08a]. Prepare-Contract Activate-VoIP
Platform
Verify-FormalRequirements Create-Order
CustomerConfirms-Order
Send-viaCourier-Service
Archive-Contract
Verify-TechnicalRequirements
Notify-Customer There were participants from different organisations (see table 1). Research consulting institutes are research institutes, which are not solely financed through the public, but also offer commercial consulting services like the German Fraunhofer institutes. We distinguish between university and university of applied sciences, because the latter one focuses on practical application in contrast to theoretical education. None of the participants was part of the case study’s research team and most of them had no prior knowledge of semantic technologies. We provided no additional material for background reading to the participants besides the semantic tutorial. We paid special attention that the participants do not get aware of our own preconception of sBPM and the case study propositions. Participants were selected by the authors based on their BPM experience and their availability. At least 75% of the participants have more than 5 years of experience in BPM by either working as BPM experts or teaching advanced classes on BPM. 4
4.1
This section presents the case study results and a discussion of the outcomes. The results must be viewed as hypothesis to be investigated by future research activities. This section is structured around the main interview points and the most interesting discussions we had. The answers between participants from the different types of organisation were consistent if not discussed otherwise. 4.2
At the beginning of each interview, we asked participants to summarise the different steps of the tutorial. Most participants were able to name the steps and their order. During this summary, most of them used the word “semantic”. We asked them how they understand semantics and how they define the term ontology.
All participants said a semantic description is not a technical one, but instead business oriented. Interestingly, some of the participants said that a semantic description defines only what needs to be done but not how to achieve it. None of the participants provided one of the popular definitions of ontology like “shared conceptualisation”. Instead, all participants tried to describe what an ontology is like pointed out that an ontology is a collection of terms, concepts or classes. Some of them used the term “glossary” or “taxonomy”, but only a few called an ontology a “namespace”, a “domain”, a “classification” or a “domain specific language”. Some participants said an ontology not only defines terms, but also relationships between them. For example, one participant said an ontology describes “what exists and how everything is related to each other” and another said it is a “model of the world”. One participant pointed out that an ontology standardises the vocabulary used. Interestingly, some participants also talked about “business cases” while actually referring to concepts. However, only a few pointed out that an ontology is processable by machines. None of the participants seemed to be comfortable with the term “ontology”, because the term is not known from daily language usage, and seems artificial to them. We postulate that one should not use the term ontology while talking to business experts, but instead talk about “semantics” or “semantic descriptions”. To give a more detailed definition, one should talk about a glossary of business terms, which also has detailed relationships between terms in contrast to ordinary glossaries. One should also point out that semantic descriptions of services are business oriented, processable by computers, and used to describe what needs to be done, and not how it should be implemented. 4.3
The domain ontology developed by TP was presented to the reader. All participants confirmed to have studied it at the beginning of the tutorial, but only a few of them used it later. The example process was still simple enough to understand and the terminology used was also known to the participants. Many participants pointed out that it is unclear to them where the domain ontology comes from and who creates it. The domain ontology was only available in the printed tutorial, but it was not part of the tool. This was confusing for some participants, because they expected the ontology to be present in the tool, too. Many of them pointed out that for more complex ontologies an “ontology browser” is required to allow easy navigation between the different concepts. We postulate that having a domain ontology is useful even if no other semantic technologies are used. Such an ontology must be integrated in the business process modelling tool allowing easy access and navigation. 4.4
The domain ontology was presented in three different ways to the participants: a “star” of the main concepts generated by WSMO Studio, an UML class diagram with a class for each concept plus the belonging attributes and the main relationships, and the WSML code.
If participants were familiar with UML modelling like the participants not working in a company, they found the UML class diagram most useful. Participants said that the UML class diagram contains far more information compared to the star diagram. If participants were not familiar with UML, they preferred the star, because it provides an easy to understand overview of the domain ontology. All participants said the WSML code is not useful and readable. Some of them noted that it might be possible to understand the WSML syntax after training, but that it is definitely not useful for business experts. One participating business expert confirmed that by saying he refuses to look at “something” like the WSML code.
We postulate that a graphical representation along with a textual description of a domain ontology is required. Probably several graphical representations are necessary allowing the user to select the preferred one. 4.5
One important step of the tutorial was selecting a goal for each function. It turned out that all participants just identified the name of the WSMO Goal in the WSML code and based their decision mostly on the name. Only a few of them looked at additional details of the goal description such as pre-/postconditions. However, most participants recognised they must use the pre-/postconditions when goal selection is ambiguous.
Most participants were not satisfied with goal selection. Many pointed out that browsing a list of goals does not scale and more advanced search mechanisms are required. A participant suggested that it must be possible to filter the list of goals based on concepts taken from the ontology. Another participant proposed using the pre-/postconditions as filter criteria. It was also suggested to add a graphical representation for each goal. One participant suggested the visualisation shown in Fig. 2.
We postulate that goal selection is an important part and must be supported by a sophisticated tool. This requirement is amplified, because some participants pointed out that they cannot see any advantage compared to selecting a web service directly. Therefore, research should focus on ways to graphically visualise goals and semantic descriptions and evaluate the usefulness through empirical experiments. 4.6
After selecting a goal, participants completed the data flow by mapping output instances of a function to input instances of a later function. Even though all participants were able to complete this step, some concerns were raised. Participants pointed out that input/output instances are similar to variables, whereas business objects are normally used in business Goal Name
Order List Confirmation
Preconditions Order List is Created
Customer is
Identified process modelling. According to participants, those two concepts are not interchangeable, because a business object is always persistent whereas a variable must be stored in a data store explicitly. This is an interesting point, which must be further investigated. It seems that ontological instances defined by WSMO Goals are not abstract enough to be useful in business process modelling. 4.7
We asked participants to motivate dynamic service binding during process execution. We received a diverse set of answers with no clear conclusion. However, some answers proved that in contrast to our case study proposition some participants were able to motivate service binding during runtime. We explicitly asked participants for an economic motivation. If they provided such a motivation, they often mentioned failover scenarios. In our opinion, this problem can be already solved today with enterprise service bus (ESB) platforms, but we did not confront participants with our point of view. Participants said that instead of hard coding an endpoint URL into the executable process, only the service name is added to the process. This helps in case the service is moved to another server. Again, this seems to be a case where today’s technologies such as service registries can be applied. Some participants noted dynamic service binding only makes sense if there are several services for each goal. If there is only a 1 : 1 relationship between service and goal, participants were not able to justify dynamic service binding. Participants with research background also pointed out that dynamic service binding in a company might not be as relevant as e. g. in ubiquitous computing, because a company is able to better control and govern the portfolio of services. Other participants mentioned dynamic service binding is dangerous, because it adds a new error source and increases the complexity of the enterprise computing stack. This shows there is no consensus whether dynamic service binding is necessary in business process automation. 4.8
We asked participants about advantages and disadvantages of the semantic approach. Surprisingly, most of them mentioned a better separation of business and IT as the main advantage of the semantic approach, because the business process model is not polluted with technical details such as web service descriptions. Instead, the business expert only specifies the required capabilities. This helps business experts to concentrate on the business relevant part of process modelling instead of dealing with implementation details. It also allows using not yet existing services. In addition, technical service descriptions (WSDL) do not have to be available in the business process modelling tool, which prevents redundancy. Participants characterised the semantic approach as creating a process template, which can be flexibly enacted.
A significant problem is the conceptual mismatch between business objects and ontological input/output instances. Besides, several participants were not convinced that the investment in semantics will pay off, because ontologies must be defined and maintained. Some participants were reluctant about ontology modelling, because in their opinion similar efforts such as establishing an enterprise data model failed in the past. All participants agreed that business experts are not able to create ontologies. Graphical tools are required to overcome hurdles like WSML syntax and logical expressions. Using semantics might require having ontology engineers, which is a specific qualification. In general, the learning curve is increased, because semantics bring their own set of technologies, methods, and methodologies along. Also, the complexity of the enterprise computing stack is increased, which augments the probability of introducing errors and integration problems. Many participants pointed to the unbalanced distribution of efforts and benefits for using semantics as another major disadvantage. Ontologies, goals, and semantic descriptions must be defined by IT after consulting business experts, but those artefacts mainly help business people. This discrepancy must be carefully managed to ensure close cooperation between all involved parties. 4.9
We asked participants whether the semantic approach is possible to follow for business experts. Most of them agreed that it is feasible, but they also mentioned problems. Currently, technology is still too visible as discussed before.
Some participants were surprised that they were asked to annotate functions with semantic descriptions. They believed that a business process model already contains enough information. Those participants envisioned a more advanced way of using semantics in BPM. For example, one participant desired to have a repository of semantically described process fragments. Instead of defining the control flow of the business process, the participant expected to just define the pre- and postconditions as well as constraints and the control flow would be automatically created by an “intelligent component”. It will be interesting to see if such visionary approaches to sBPM will gain momentum. 5
We presented our findings of an empirical case study exploring the use of semantics in BPM. The case study was designed and conducted according to a strict methodology ensuring high validity and reliability. The case study was replicated several times using a real-world business process and typical end-user tools. Our results show that semantics promise moving BPM forward, but there are still many open questions. All results found are hypothesis, which must be validated by future research activities. Future research will show if the problems explored can be solved in order to leverage the advantages of sBPM. [AFKK07] [CAdMZ+07] [DSSK07] [EKSMP08]
W. Abramowicz, A. Filipowska, M. Kaczmarek, and T. Kaczmarek. Semantically enhanced Business Process Modelling Notation. In Workshop on Semantic Business Process and Product Lifecycle Management (SBPM), volume 251 of CEUR Workshop Proceedings, pages 88–91, Innsbruck, Austria, June 2007.
I. Celino, A. K. Alves de Medeiros, G. Zeissler, M. Oppitz, F. Facca, and S. Zo¨ller. Semantic Business Process Analysis. In Workshop on Semantic Business Process and Product Lifecycle Management (SBPM), volume 251 of CEUR Workshop Proceedings, pages 44–47, Innsbruck, Austria, June 2007.
M. Dimitrov, A. Simov, S. Stein, and M. Konstantinov. A BPMO Based Semantic Business Process Modelling Environment. In M. Hepp, K. Hinkelmann, D. Karagiannis, R. Klein, and N. Stojanovic, editors, Workshop on Semantic Business Process and Product Lifecycle Management (SBPM), volume 251 of CEUR Workshop Proceedings, pages 101–104, Innsbruck, Austria, June 2007.
M. El Kharbili, S. Stein, I. Markovic, and E. Pulvermu¨ller. Towards a Framework for Semantic Business Process Compliance Management. In The Impact of Governance, Risk, and Compliance on Information Systems (GRCIS), volume 339 of CEUR Workshop Proceedings, pages 1–15, Montpellier, France, June 2008. [SKD+08] [SS08] [SSEK08a] [SSEK08b] [Yin03]
D. Fensel, H. Lausen, A. Polleres, J. de Bruijn, M. Stollberg, D. Roman, and J. Domingue. Enabling Semantic Web Services: The Web Service Modeling Ontology. Springer, 2006.
J. Frankowski, P. Rubach, and E. Szczekocka. Collaborative Ontology Development in Real Telecom Environment. In 1st International Working Conference on Business Process and Services Computing (BPSC), volume 116 of LNI, pages 40–53, Leipzig, Germany, September 2007.
M. Hepp, F. Leymann, J. Domingue, A. Wahler, and D. Fensel. Semantic Business Process Management: A Vision Towards Using Semantic Web Services for Business Process Management. In IEEE International Conference on e-Business Engineering (ICEBE), pages 535–540, Beijing, China, 2005.
M. Hepp and D. Roman. An Ontology Framework for Semantic Business Process Management. In 8th International Conference Wirtschaftsinformatik, pages 423– 440, Karlsruhe, 2007.
M. Ho¨st and P. Runeson. Checklists for Software Engineering Case Study Research. In 1st International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 479–481, Madrid, Spain, September 2007. IEEE CS.
B. Kitchenham, L. Pickard, and S. L. Pfleeger. Case Studies for Method and Tool Evaluation. IEEE Software, 12(4):52–62, 1995.
J. Nitzsche, D. Wutke, and T. van Lessen. An Ontology for Executable Business Processes. In Workshop on Semantic Business Process and Product Lifecycle Management (SBPM), volume 251 of CEUR Workshop Proceedings, pages 52– 63, Innsbruck, Austria, June 2007.
S. Stein, S. Ku¨hne, J. Drawehn, S. Feja, and W. Rotzoll. Evaluation of OrViA Framework for Model-Driven SOA Implementations: An Industrial Case Study. In 6th International Conference on Business Process Management (BPM), volume 5240 of LNCS, pages 310–325, Milan, Italy, September 2008. Springer.
H. J. Schmelzer and W. Sesselmann. Gescha¨ftsprozessmanagement in der Praxis. Carl Hanser Verlag, Mu¨nchen, Germany, 6th revised edition, 2008.
C. Stamber, S. Stein, and M. El Kharbili. Prototypical Implementation of a Pragmatic Approach to Semantic Web Service Discovery During Process Execution. In W. Abramowicz and D. Fensel, editors, 11th International Conference on Business Information Systems (BIS), volume 7 of LNBIP, pages 201–212, Innsbruck, Austria, May 2008. Springer.
S. Stein, C. Stamber, and M. El Kharbili. ARIS for Semantic Business Process Management. In Workshop on Advances in Semantics for Web Services (semantics4ws), Milan, Italy, September 2008.