=Paper=
{{Paper
|id=Vol-2097/paper12
|storemode=property
|title=Empirical Insights into the Appraisal of Tool Support for Participative Enterprise
Modeling
|pdfUrl=https://ceur-ws.org/Vol-2097/paper12.pdf
|volume=Vol-2097
|authors=Anne Gutschmidt
|dblpUrl=https://dblp.org/rec/conf/emisa/Gutschmidt18
}}
==Empirical Insights into the Appraisal of Tool Support for Participative Enterprise
Modeling==
https://ceur-ws.org/Vol-2097/paper12.pdf
Empirical Insights into the Appraisal of Tool Support for
Participative Enterprise Modeling
An Experimental Comparison between Whiteboard and Multi-Touch Table
Anne Gutschmidt1
Abstract: The paper presents an experiment about modeling tools in the context of enterprise
modeling in teams. A goal modeling task was set where three teams of three persons worked with a
whiteboard, and three teams of three persons worked with a multi-touch table. The participants’ ratings
of the tool were compared between both manipulation groups. Moreover, the participants explained
their ratings in individual interviews. Results indicate that multi-touch tables can be expected to be
equally accepted by modeling teams, and uncover further potentials of the tool.
Keywords: enterprise modeling; multi-touch table; group work; tool appraisal; TAM; experiment
1 Introduction
The idea of participative enterprise modeling is to involve domain experts in the modeling
process [SPS07]. To support them in collaborative modeling, suitable tools are needed. This
study compares a modern tool, the multi-touch table (MTT), to a conventional tool, the
whiteboard, to gain more insight into users’ needs in the context of participative modeling.
A whiteboard is a low-cost modeling tool that is available almost anywhere in a company. It
allows participation in the modeling process of every stakeholder. A MTT represents a more
costly tool which, however, allows for saving models digitally in order to share and edit
them later on. The question is whether the MTT is likely to be equally accepted by modelers
as is a whiteboard, such that the use of MTTs could be recommended for collaborative
modeling. According to the Technology Acceptance Model (TAM), the intention to use a
technology and its actual use are influenced by perceived usefulness, perceived ease of use,
and perceived enjoyment [VB08]. This paper presents an exploratory comparison between
whiteboard and MTT for enterprise modeling in teams focusing on the following research
question: Does the MTT reach similar or better rankings for perceived usefulness, ease of
use and enjoyment than the whiteboard such that it can be expected to be equally accepted
by users in the context of participative enterprise modeling? Secondly, determinants of
these constructs are examined in order to gain concrete hints on how to support modelers.
1 University of Rostock, Department of Computer Science, Albert-Einstein-Str. 22, 18059 Rostock, Germany
anne.gutschmidt@uni-rostock.de
71 Anne Gutschmidt
2 Background
According to the TAM, perceived usefulness and perceived ease of use are major predictors
of the intention to use and the actual use of a technology where ease of use is an antecedent
of usefulness [VB08, Da89]. While usefulness refers to the product created by using the
technology, the process of using the technology is addressed by enjoyment. Thus, usefulness
can be considered as extrinsic motivation whereas enjoyment represents intrinsic motivation
[Da89]. The latter is an important determinant of ease of use [VB08]. Thus, all three
constructs serve in predicting the acceptance of a technology such as MTT.
The advantages of MTTs have been shown in previous studies, especially in the educational
domain [PH09, MH14], where they are appreciated by students for enabling easy sketching
and changing. Although there have already been several studies on MTT, there is still a
need for research, especially with regard to functionality (usefulness) and the design of user
interfaces (ease of use) for different kinds of tasks. Modeling tasks can take quite different
forms. [Ba13] examined teams who were modeling UML state charts either on MTT or PC.
He observed that input via touch keyboard was more laborious and time-consuming. [LW11]
compared the use of a MTT with structured interviews for assessing business processes
involving domain experts. They found that using the MTT was more fun and brought more
insights in process understanding although the ANOVA showed no significance. [Bu12]
compared MTT with flipchart and pen-and-paper equipment on brainstorming tasks. All
teams produced a similar amount of ideas, however the pen-and-paper equipment seemd
better-suited for categorizing ideas. The authors claim that the users’ assessment of a
tool was dependent on the kind of task at hand, e.g., the MTT seemed best suited for
mindmapping. Enterprise modeling comprises a variety of models, thus, it is probably
connected with different kinds of tasks. The study presented here will focus on one of these
models, questioning the suitability of a MTT for collaborative goal modeling by comparing
it to a conventional tool, i.e. a whiteboard.
3 Method
Experimental Design and Data Evaluation Method: To examine the influence of the tool
on the users’ subjective appraisal the tool was manipulated as independent variable. The
participants of the study were invited to a multimedia lab where they worked either with a
MTT or a whiteboard, based on random choice. In a group of three persons, the participants
had to draw a goal model, which is a very basic model of the 4EM method, for a pizza
delivery service. Every team had 30 minutes time to create the model. Afterwards, individual
interviews were conducted and the participants filled out a questionnaire to assess the
dependent variables. Perceived usefulness and perceived ease of use were measured using
scales by [Da89]. A scale by [DBW92] was used to measure enjoyment. The questionnaire
also contained questions about demographics and experience with modeling and MTT. In
the interviews, the participants were asked why they found the tool useful or not useful etc.
This lead to a list of advantages and disadvantages of the respective medium. Since the
Empirical Insights into the Appraisal of Tool Support for Participative Enterprise Modeling 72
Fig. 1: Average values depicted as bars, and standard deviations of perceived usefulness, perceived
ease of use and enjoyment (items rated on a 7-point scales with 1 = lowest), separated by tool, i.e.
multi-touch table (MTT) and whiteboard (WB).
sample was too small for more sophisticated statistical analysis, only descriptive statistics
were used to explore for a possible influence of the tool on the users’ appraisal. To get
overall values of perceived usefulness, perceived ease of use and enjoyment, mean values of
all items of the respective scales were calculated. Afterwards, mean values of the respective
constructs were determined for all whiteboard and all MTT users separately. Qualitative
content analysis was applied to the interview transcriptions. To get a deeper understanding
of the participants’ appraisal of the tool, the interviews were scanned for statements on
reasons for their appraisals of the tool they used in the modeling session. Subsequently, the
text units were again categorized.
Sample: 18 persons took part in the study, three of them female. Three teams used the
whiteboard and three teams used the MTT, with one gender-mixed team in each group.
They were students of business information systems or computer science. In the whiteboard
group, participants were 24.9 years old on average (σ = 2.2, max = 28, min = 22), in the
MTT group, participants were 23.6 years old on average (σ = 1.9, max = 28, min = 22).
The participants came from Russia, India and Germany. The level of experience of the
4EM notation, measured with a 5-point scale with 1 representing no experience, was at 2
on average for the MTT group (σ = 1, max = 3, min = 1) and at 3.7 on average for the
whiteboard group (σ = 0.6, max = 4, min = 3). On a 5-point scale, the participants of
the MTT group estimated their experience with MTTs at an average value of 1.7 with 1
representing no experience (σ = 1.3, max = 5, min = 1).
4 Results
Figure 4 shows the average values, represented as bars, of perceived usefulness, perceived
ease of use and enjoyment, divided into MTT and whiteboard. Standard deviations can be
found above the bars.
When asked to explain their judgement on usefulness, ease of use and enjoyment of
the whiteboard or MTT, respectively, the participants named several advantages and
disadvantages of the tools of which only the most frequently named will be listed here. With
73 Anne Gutschmidt
regard to advantages of the whiteboard, six whiteboard users said that they appreciated
that changes could be easily made at the whiteboard, e.g., “What I like so much is that
you can wipe off everything, can write again” (5,1,1,63, translated from German).2 Five
whiteboard users mentioned that whiteboards were easy to handle, using them was intuitive
and simple. Five persons said that with the witeboard they got a good overview. A major
disadvantage of a whiteboard seems to be that with growing complexity models get more
confusing, as stated by four whiteboard users. Three participants criticized that models
cannot be saved. Moreover, three participants complained that changes would be hard to
make to a model. Two participants said that the space on the whiteboard was restricted.
Concerning advantages of the MTT, six of the respective participants said that the MTT
was easy to handle, e.g., “it was a lot easy. Begin easily, drag here and there, set up the links.
It was very friendly” (3,2,1,75). Furthermore, five participants appreciated that changes
could be made easily, such as deleting or moving content, with the software automatically
adapting the model. Four participants said they especially liked the touch interface. They
found it natural and easy, or they described themselves as used to it from smartphones. The
possiblity of saving and thus being able to share models was mentioned by four participants.
Three participants liked that the MTT enabled parallel working which may make the group
faster. Two persons said the MTT let them get a good overview. Two participants said, with
the MTT, one was able to build big models. Shortcomings of the software represented a
major point of criticism for the MTT. Seven participants complained about problems
with the software, e.g., when using the keyboard. This category also included missing
functions, i.e. one team would have liked a zoom function. Two participants mentioned
general problems using the MTT, e.g., “Maybe I have to get used to using this, because my
finger is so fat” (1,2,2,2).
5 Discussion
The participants of the study judged whiteboard and MTT very similarly with regard to
usefulness, ease of use and enjoyment. The MTT was very slightly rated higher. As far
as can be derived from the interviews, most participants consider both tools as easy to
handle. The touch interface seems to entail some attractiveness. Moreover, the opportunity
of parallel working at the MTT seems to be appealing. Users are aware of typical advantages
of the MTT, such as being able to share digital copies of models. The corresponding
disadvantage of the whiteboard was named by whiteboard users. Space seems to be a
challenge on both tools, however, MTT allow for intelligent software solutions that would
possibly enable appropriate space management including zooming. The study has shown
that users may be irritated or even frustrated if the software does not work properly or as
expected. Thus, companies who want to apply enterprise modeling with MTT should care
for usable and functional software that fits to their purposes as suggested by the TAM. All
in all, users seemed to be equally comfortable with a MTT as with a whiteboard leading to
the assumption that MTT will be equally accepted as a tool for participative modeling.
2 Citations from interviews are given with number of trial, number of participant, page and paragraph.
Empirical Insights into the Appraisal of Tool Support for Participative Enterprise Modeling 74
The small sample size is one limitation of the study, however, it was considered as acceptable
fo enabling an exploratory insight into tool support for collaborative modeling. Future
studies should involve greater samples including different domains and less technophile
persons. Additionally, further models and varying model complexity must be examined.
The goal models in this study comprised at most 30 components. With growing complexity
and size of models, more shortcomings of MTT might appear. That is why the software
prototype used in this study will be improved and tested in future studies with particular
focus on space management and supporting parallel working. MTTs offer advantages in
terms of quick digitalization but they also have huge potentials with regard to participative
enterprise modeling, enabling the creation of models based on the collaboration of several
stakeholders. It might be useful if all stakeholders had a chance to participate in drawing,
possibly resulting in increased satisfaction and commitment with the model as [LW11]
already hypothesized. The connection between active participation and subjective appraisals
such as commitment with the model will also be part of future studies.
References
[Ba13] Basheri, Mohammed: Multi-touch table for enhancing collaboration during software
Design. PhD thesis, Durham University, 2013.
[Bu12] Buisine, Stéphanie; Besacier, Guillaume; Aoussat, Améziane; Vernier, Frédéric: How
do interactive tabletop systems influence collaboration? Computers in human behavior,
28(1):49–59, 2012.
[Da89] Davis, Fred D.: Perceived Usefulness, Perceived Ease of Use, and User Acceptance of
Information Technology. MIS Quarterly, 13(3):319–340, 1989.
[DBW92] Davis, Fred D.; Bagozzi, Richard P.; Warshaw, Paul R.: Extrinsic and Intrinsic Motivation
to Use Computers in the Workplace1. Journal of Applied Social Psychology, 22(14):1111–
1132, 1992.
[LW11] Luebbe, Alexander; Weske, Mathias: Tangible Media in Process Modeling – A Controlled
Experiment. In (Mouratidis, Haralambos; Rolland, Colette, eds): Advanced Information
Systems Engineering. Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 283–298, 2011.
[MH14] Mercier, Emma; Higgins, S: Creating joint representations of collaborative problem solving
with multi-touch technology. Journal of Computer Assisted Learning, 30(6):497–510,
2014.
[PH09] Piper, Anne Marie; Hollan, James D: Tabletop displays for small group study: affordances
of paper and digital materials. In: Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems. ACM, pp. 1227–1236, 2009.
[SPS07] Stirna, Janis; Persson, Anne; Sandkuhl, Kurt: Participative Enterprise Modeling: Expe-
riences and Recommendations. In (Krogstie, John; Opdahl, Andreas; Sindre, Guttorm,
eds): Advanced Information Systems Engineering. Springer Berlin Heidelberg, Berlin,
Heidelberg, pp. 546–560, 2007.
[VB08] Venkatesh, Viswanath; Bala, Hillol: Technology Acceptance Model 3 and a Research
Agenda on Interventions. Decision Sciences, 39(2):273–315, 2008.