Collaborative or Group modeling [2,13] is a process that can enhance productivity in Information Systems Design and Business Process Re-engineering. Modeling has been observed to contain not only the models (end-products) but also the process that generates these models. Much attention has been paid to the models (end-products) and their associated quality (see, for example, [7]), but little attention has been paid to the process that generates these models. The only work known to have made attempts to look at the process of modeling within a communicative perspective is that of Hoppenbrouwers et al. [5], [10] and Rittgen [11,12].
Some of the most popular conceptual modeling assumptions include the following: modeling is product-oriented and design centered, the modeling process involves two roles: domain expert, and model builder (systems analyst) each playing their different roles at different times of conceptual modeling. Quality assessment and measurement is often restricted to only the end-products -the models (see, for example, [7]). We contend and hypothesize that conceptual modeling should be not only a product-oriented and design-centered approach but should also be a conversational activity and should be human-centered. There are also intermediary products that need to be analyzed and evaluated.
The process-oriented modeling approach assumes that the produced models should contain "shared knowledge" and the modeling process is governed and directed by a number of modeling rules and goals [15]. Modelers, however, do not concisely and explicitly perform "step-wise" thinking, in particular for nonexperts, in a product-oriented approach to incorporate such shared-knowledge in their models. Viewing modeling as consisting of the process and the products , helps us study the commitments, agreements, negotiations, decision making and consensus, etc. of the modelers and the rules and goals governing this process.
One of the problems identified in collaborative/group modeling is lack of an approach (and related tool-support) that can be used to study and improve the communicative acts that lead to the generation of the models. Taking a game-metaphorical approach to systems design (see, for example, [6]) has the potential of helping us determine the rules/goals driving the modeling games. To achieve this, we eventually aim to design collaborative modeling games(CMGs) in which the modeling process plays a significant role and human interaction and communication take center stage.
The following questions motivate our way of thinking in this research. The main research question is: How can we evaluate enacted modeling processes (in view of them achieving set goals)? To adequately answer this question, we raise the following sub-questions: What is the quality of modeling? How can we measure modeling process quality? Our long-term objectives is to analyze, evaluate and understand collaborative modeling games (CMGs) with the longer-term aim of supporting conceptual modeling with a tool. A more immediate objective is to evaluate currently available collaborative modeling sessions as if they are games to measure their effectiveness and efficiency.
Figure 1 shows the design, execution, evaluation and validation cycle of the CMGs.
It being such a broad and multi-disciplinary area, it is hard to review all the literature related to Collaborative (Group) modeling within the constraints of paper's size. We therefore highlight only works directly related to our own. Bostrom et al. [1] provide one of the earliest attempts to consider group facilitated meetings using a Group Support Systems (GSS) tool. This work is important since it shows how a GSS tool can be used to help stakeholders generate information, organize it, evaluate and select alternatives and finally communicate their actions. Although communication is one of the aspects talked about and the role of the facilitator is emphasized, communication is between the meeting participants and the facilitator. This is significantly different from our approach where communication plays a central role in the negotiation between the participants to reach agreement and a common shared understanding.
In [2] the authors draw on Electronic Meeting Systems (EMS) technology and re-engineering techniques to develop a method and a support tool for modeling business processes. This is a richer approach in user involvement and idea generation than other traditionally known collaborative modeling approaches and tools. It, however, lacks the theoretical rigour and underpinning for process modeling as it influences only the quality of the generated models. The approach is thus product-quality oriented. The work in [5] and [10] was the first attempt to critically analyze the role of communication in modeling and the modeling process. Our current work extends this work in emphasizing communication in the modeling process and trying to find out out how modelers generate their models. It, however, differs from it in that the current work employs the gaming approach to modeling to determine the rules and goals under which modeling processes take place.
The research work in this paper builds more on the work of Rittgen [11,12]. Rittgen observes that in a collaborative environment, participants engage in different types of conversations prior to the creation of an accepted model. Our work, however, differs from Rittgen's in that we take a more holistic approach that looks at modeling as a game-design theoretic approach.
In this section we present the basic conceptual framework to help us analyze the process of modeling. The developed framework is related to two previously developed frameworks: The Semiotic Quality (SEQUAL) framework of Krogstie et al. [7] and The Quality of Modeling framework QoMo of van Bommel et al. [15]. The basic concepts of our "RIM" (Rules, Interactions and Models) framework are shown in Fig. 2.
Fig. 2. Basic concepts for integrated analysis of interactions, rules and models.
The RIM framework is a three-tier framework that examines the communicative acts (interactions) in a modeling session, the rules/goals set, and the models produced as a result of the interaction and collaboration which is, metaphorically speaking, a sort of modeling game [6]. The different players work under a set of rules and goals. The rules/goals, interactions and models are all time-stamped to help us track and identify he interplay between any pair.
In addition to the framework above we use the collaborative evaluation (COME) framework given in Fig. 3 to evaluate the modeling process games (CMGs) using a number of artifacts to be evaluated in view of the CMGs. Employing the design science approach [4], we put these artifacts to use within the context of evaluation and improvement of the CMGs.
In this section, we present an analysis of the results obtained from a collaborative modeling session in a pilot study. In this first phase of our reserved project, the emphasis was on making the interactions and a few goals and rules using only the RIM framework.
The business process scenario given out to modelers, was about developing a Hazardous Material Management System (HMMS) by the Materials Management Department (MMD) of a city council. Two researchers and three modelers(two systems analysts (SA) and one domain expert (DE)) participated in the actual modeling. Figure 4 shows one of the screen-shots from the modeling session video recording. The session (which took 18 minutes) was video recorded with good sound quality. The modelers were also given a digital writing pad, which was recorded alongside the video. This provided us with a full, synchronized recording of all raw data we could wish for.
We first transcribed the video recordings, then made an annotation and categorization of the speech acts. We mainly drew on Language-Action Perspective (LAP) theory, Speech-Act Theory (SAT), Discourse Analysis and the Communicative Action Theory (CAT), see for example, [3]. Table 1 shows a sample categorization of the speech acts.
In this section we give some of our observations from the data obtained from the modeling session. The findings are given within the framework and methodological approach in Fig. 2.
• Setting the Agenda. It is noted that modelers, without the help of a facilitator, set their own agenda by structuring the modeling process in the following phases: (I) -Setting the main approach:choosing the language and subdivision of work (II) -Exploring and deciding which actors take part in the modeling process and (III) -Modeling the sub-processes.
• Categorization of the Conversations. It was noted from the video and the transcription that the communication among the modelers can broadly be categorized as a negotiation. This was the same conclusion reached in [11]. This comes from the argumentations (argue for/against) resulting in either acceptance,i.e.agreement (support) or rejection of the proposals. Rejection indicates disagreement. More details are found in [14].
• Categorization of Modeling Rules and Goals. From the transcription and observations it was noted that the rules and goals guiding the modeling process could be categorized as: imposed in the scenario or created within the modeling game. These rules and goals were further categorized as explicit-directly set and stated or implicit -indirectly stated and set. In [14] these rules and goals are explained in more detail. They include: rules that were set for the game are: Goal setting rule: creation goal, Goal setting rule: validation goal and rules that were set in the game are: Goal setting rule: grammar goal, Goal setting rule: creation goal and Goal setting rule: grammar goal.
This paper has looked at a research program aimed at shedding light on the process (act) of modeling. We employ an interactive and collaborative modeling approach, within the context of communicative modeling of business processes, to one modeling case. We have developed a three-tier conceptual framework and a methodological approach which can be used to analyze and understand the communicative process of modeling. Three key concepts: interactions, rules/goals and the modeling products have been identified.
• Contribution and Direction for Further Research. Our contribution in this research is a framework that can be used to analyze modeling games and an evaluation mechanism to measure the effectiveness and efficiency of these modeling games. We intend to focus on developing an evaluation mechanism and its requirements using the framework in Fig. 3 and to develop a methodology to enable us draw scientifically sound and definitive conclusions about collaborative modeling processes.