=Paper=
{{Paper
|id=None
|storemode=property
|title=Alaska Simulator Toolset for Conducting Controlled Experiments
|pdfUrl=https://ceur-ws.org/Vol-592/PaperDemo03.pdf
|volume=Vol-592
|dblpUrl=https://dblp.org/rec/conf/caise/WeberPZW10b
}}
==Alaska Simulator Toolset for Conducting Controlled Experiments==
https://ceur-ws.org/Vol-592/PaperDemo03.pdf
Alaska Simulator Toolset for Conducting
Controlled Experiments
– Tool Paper –
Barbara Weber1 , Jakob Pinggera1 , Stefan Zugal1 , and Werner Wild2
1
Quality Engineering Research Group, University of Innsbruck, Austria
{Barbara.Weber, Jakob.Pinggera, Stefan.Zugal}@uibk.ac.at
2
Evolution Consulting, Innsbruck, Austria
werner.wild@evolution.at
Abstract. Alaska Simulator Toolset is an interactive software suite de-
veloped at the University of Innsbruck which allows to explore differ-
ent approaches to process flexibility by using a familiar metaphor, i.e.,
travel planning and execution. In particular, Alaska Simulator Toolset is
used for studying research questions in the context of business process
management and other related fields. For this, Alaska Simulator Toolset
provides integrated support of different approaches to process flexibil-
ity fostering their systematic comparison. Moreover, Alaska Simulator
Toolset facilitates the design and execution of controlled experiments
through experimental workflow support.
Providing effective IT support for business processes has become an essential
activity of enterprises in order to stay competitive in today’s market [1]. When
assessing the usability of BPM approaches, however, enterprises have to rely
on vendor promises or qualitative data rather than on empirical or experimen-
tal research [2]. This is rather surprising as these research methods have been
successfully applied in similar research areas like software engineering (e.g., [3,
4]). Alaska Simulator Toolset (AST) has been developed to address this need
and allows the investigation of strengths and weaknesses of different approaches
for process flexibility through the execution of controlled experiments. Due to
the many similarities between business processes modeling and execution and
journey planning, AST uses a journey as a metaphor1 . Furthermore, the used
metaphor provides an attractive context to be engaged in, thus increasing the
willingness of subjects to participate in experiments. AST consists of three ma-
jor components: Alaska Configurator, Alaska Simulator and Alaska Analyzer. In
the following we describe the main functionalities of AST and how design and
execution of controlled experiments is supported using the experiment described
in [5] as example (cf. Fig. 1 for the experimental design).
1
For a detailed description of the journey metaphor visit the simulator’s website:
http://www.alaskasimulator.org
Group 1 Factor Level 1: Group 1 Factor Level 2:
n/2 Participants Few Constraints
California A n/2 Participants Many Constraints
Alaska B
Group 2 Factor Level 2: Group 2 Factor Level 1:
n/2 Participants Many Constraints
California B n/2 Participants Few Constraints
Alaska A
First Run Second Run
Experiment Eindhoven
Fig. 1.Experiment
Experimental Design
Innsbruck (replication)
Alaska Configurator was used to design two journey configurations (Califor-
nia and Alaska) including locations, actions, events, constraints as well as vari-
ability of weather conditions. For each journey configuration two variants were
created, one for each factor level (i.e., few and many constraints). To gather the
participants’ demographic information Alaska Configurator was used for design-
ing a survey. The journey configurations and the survey were then assembled to
an experimental workflow.
During experiment execution participants were guided by the experimental
workflow. After presenting them with a survey, half of the students obtained
configuration California with few constraints, while the second half obtained
the same configuration with many constraints. The students then planned and
executed a journey to California. Each step that was performed while planning
and executing was logged for later investigation and detailed analysis. Having
completed their California journeys, subjects planned and executed a journey to
Alaska.
After the planning session researchers were supported in analyzing the jour-
neys by enabling them to replay journeys step by step using Alaska Analyzer.
Alaska Simulator, including a test configuration, extensive documentation
and screencasts can be downloaded from http://www.alaskasimulator.org. For
detailed information on the results of a controlled experiments which was con-
ducted using Alaska Simulator we refer to [5].
Acknowledgements: We thank Felix Schöpf for his work on the experimental work-
flow support.
References
1. Antonucci, Y.L.: Using Workflow Technologies to Omprove Organizational Com-
petitiveness. Int’l. J. of Management 14 (1997) 117–126
2. Weber, B., Mutschler, B., Reichert, M.: Investigating the Effort of Using Business
Process Management Technology: Results from a Controlled Experiment. Science
of Computer Programming 75 (2010) 292–310
3. Myers, G.J.: A Controlled Experiment in Program Testing and Code Walk-
throughs/Inspections. Commun. ACM 21 (1978) 760–768
4. Lott, C.M., Rombach, H.D.: Repeatable Software Engineering Experiments for
Comparing Defect-Detection Techniques. Empirical Software Engineering 1 (1996)
241–277
5. Weber, B., Reijers, H., Zugal, S., Wild, W.: The Declarative Approach to Business
Process Execution: An Empirical Test. In: Proc. CAiSE’09. (2009) 470–485