=Paper=
{{Paper
|id=Vol-1370/paper_7
|storemode=property
|title=The Importance of Teaching Goal-Oriented Analysis Techniques: an Experience Report
|pdfUrl=https://ceur-ws.org/Vol-1370/paper_7.pdf
|volume=Vol-1370
|dblpUrl=https://dblp.org/rec/conf/caise/PajaHM15
}}
==The Importance of Teaching Goal-Oriented Analysis Techniques: an Experience Report==
https://ceur-ws.org/Vol-1370/paper_7.pdf
1st International iStar Teaching Workshop (iStarT 2015)
The importance of teaching goal-oriented
analysis techniques: an experience report
Elda Paja1 , Jennifer Horkoff2 , and John Mylopoulos1
1
University of Trento, Italy – {paja, jm}@disi.unitn.it
2
City University London, UK – horkoff@city.ac.uk
Abstract. In this paper, we report on our experience in teaching i*
and related goal-oriented techniques at a master-level course at the Uni-
versity of Trento. In our experience, we have observed that analysis is
an important factor that influences learning and understanding of i*.
Analysis allows students to not only evaluate the satisfaction of goals
in their model, but also to better understand their models, helping to
refine models until they are more meaningful and more likely to fulfill
their intended purpose.
1 Introduction
For goal models to be used in practice, goal-oriented requirements languages and
techniques must be taught through university and professional courses. They are
often taught as topics in the syllabus of courses on requirements engineering,
sometimes at the graduate level. As instructors, we want students to be capable
to build useful models, effectively using models for system comprehension and
decision making. More importantly, we want students to value their learning
experience. How can we improve our teaching practices to achieve these goals?
One of the features of goal-oriented models is their capability to facilitate
systematic satisfaction analysis of model elements. It is possible to use existing
analysis techniques, such as [7], to answer “What if?” or ”Is this achievable?”
questions. These procedures allow students to explore the meaning of the con-
nections within their models, understanding the consequences of alternative se-
lection. Such analysis is intended to help find problematic areas in the model,
and to help choose the best mitigation or alternative for such problems. Similar
analysis procedures have been introduced for complementary frameworks, such
as the Business Intelligence Model (BIM) [5].
In this paper, we argue that the teaching of goal models should be coupled
with one or more systematic analysis procedures. It is through such analysis that
students understand whether their models are meaningful and whether or not
they fulfill their purpose. In our experience, it is analysis that makes students
appreciate the benefits of models and modeling.
Our thesis has been tested with a qualitative study involving a masters-
level course at the University of Trento. The course is titled “Organizational
Information Systems” and has been offered for more than 10 years with 20-35
37
enrollments per year. The course covers enterprise modeling, strategic objectives
modeling and analysis with i* [10], business process modeling and simulation
with Adonis [8]. Course requirements included a course project worked on in
teams involving modeling and analysis of an enterprise of each team’s choice.
Recent editions of the course have placed more emphasis on systematic goal
model analysis.
The rest of the paper is structured as follows. In section 2 we present more
details about the course syllabus, while in section 3 we describe the requirements
for the students’ projects. Section 4 summarizes course project results by drawing
our lessons learned, and section 5 concludes.
2 Course syllabus
The Organizational Information Systems (OIS) course is taught at the master
level with the objectives of (i) teaching students basic concepts about modeling
business organizations and business processes; (ii) teaching information system
technologies and architectures that support the operation of organizations; (iii)
understanding how to manage organization information systems; and (iv) intro-
ducing new trends in organizational information systems.
Students are required to have a general knowledge of software engineering,
including knowledge of UML, as well as a general knowledge of databases and
information systems.
The course is organized as follows. First, it provides an introduction to orga-
nizations and organizational information systems, organizational structures, and
organizational business processes. Second, it presents students with modeling ap-
proaches for organizations, standards and reference architectures. Among others,
it presents approaches for organizational modeling such as i * [10] and strategic
business modeling such as BIM [5] and Tactical BIM (TBIM) [2]. Emphasis is
placed on teaching systematic analysis for both i* and BIM [5,7]. Third, the
course covers modeling and analysis approaches for business processes, based
on Adonis 3 or BPMN 2.0 4 . Fourth, it discusses OISs Management – plan,
implement, deliver, monitor, evaluate, and improve organizational information
systems. Finally, the course discusses information assurance, presenting methods
for IT Goal-Risk-Compliance [1], and Information Security [3,4,9].
It is worth emphasizing that over the years the syllabus of the course has been
continuously updated to accommodate new and emerging techniques. Although
many goal model analysis procedures exist ([6]) we have chosen to teach the stu-
dents qualitative, interactive analysis as this type of analysis is relatively simple,
does not require detailed domain information and comes with relatively stable
tool support. Similarly, we chose [5] to teach quantitative BIM analysis using
indicators, supporting business decision making, in part due to it’s similarity to
i*-specific techniques from [6].
3
http://www.boc-group.com/it/products/adonis/
4
http://www.omg.org/spec/BPMN/2.0/
38
3 Students’ roles and course projects
Student evaluation is performed via a course project, completed by teams of
two or three. Students choose their own teammates. Each team project involves
using state-of-art tools to model and analyze an organization of their choice,
representing the organizational structure, the organizational goals and strategic
objectives. The student projects model and analyze business processes within
that organization to design (or improve) an information system that supports
some of these processes. The project is discussed in a final oral exam.
Roles. Students are required to play interchangeably the roles of the require-
ments analyst and stakeholders, in order to capture both perspectives when
building and analyzing the models. Students have interacted directly with real
stakeholders and customers in only a few cases.
Project description. Student projects are divided into two assignments. The
first assignment is focused on modeling, while the second focuses on analysis,
intended to support the improvement of the organization.
A1. In the first assignment, students report on the problem by initially de-
scribing the organization in natural language (English/Italian). This description
provides an overview of the organization (sector, size, location, services, etc.),
specific features (what makes it different from competitors), and hypothetical
plans about the future of the organization. Students should define the scope of
the project (especially for big organizations), i.e. the parts of the organization
for which they will design an information system.
In the rest of the first assignment students are required to model the exist-
ing organization (within the defined scope), representing important actors, their
goals and interdependencies. This modeling is performed with i*, for which stu-
dents are free to use state-of-art tools of their choice. Moreover, an important
step of the modeling activities involves capturing strategic goals of the organi-
zation, including relevant situations and indicators, typically represented with
BIM or TBIM models. Finally, students are required to identify and model at
least three complex business processes with BPMN or Adonis.
A2. The purpose of the second assignment is twofold: (1) to analyze the chosen
organization in order to identify weaknesses, bottlenecks, and under-performance.
Emphasis is placed on the instruction of strategic analysis for i* and BIM/TBIM,
as well as business process analysis and simulation with ADONIS components;
(2) to improve the current organization by designing part of an organizational
information system. Ideally, the system will overcome the identified limitations.
This should be demonstrated using further analysis.
To achieve these objectives, students should analyze their i* models and
BIM/TBIM models to determine goal satisfaction or denial. Most importantly,
they are required to describe how these changes affect the identified business
processes. As far as business process analysis is concerned, students are required
to execute: (i) consistency queries for all the models, to show that the models
are syntactically correct and complete; (ii) they should run some queries to elicit
useful information from the business processes (path analysis, capacity analysis).
39
4 Lessons learned
As stated earlier, over the years the course has been reshaped to accommodate
new emerging state-of-the-art techniques. While initially it focused mainly on
enterprise architecture and business process modeling and analysis, using goal
models only to describe the chosen organization, in the last two course instan-
tiations, we have required students to make more use of systematic analysis
techniques to assess their goal models. We report on our observations of how the
use of goal model analysis influenced students’ understanding.
We noticed qualitatively that the outcomes of the projects were much im-
proved. In the oral exam and presentations in the recent year we observed that
students they demonstrate a better understanding of goal models after applying
analysis, which allowed them to iteratively and incrementally build goal models
that adequately capture the intended domain and fulfill their purpose. We have
observed that the results of the goal-analysis help students to understand what
organizational changes can be made to better achieve goals. Moreover, they have
used BIM analysis to identify the strategies that support those goals. Out of the
9 projects from previous year 8 performed extensive goal analysis over i * and
BIM models, and only 1 (single student) failed to do so. Of the 8 projects, 6
reran analysis over the i * models which were improved by previous analysis re-
sults, while 2 provided suggestions for improvement, without evaluating these
improvements with systematic analysis. We illustrate improvements made by
students providing excerpts from a representative student project that
applied analysis to iteratively improve goal-models. They have constructed the i *
models, performing in total 7 iterations and have performed evaluations running
both forward (“what if?”) and backward (“is this possible?”) analysis.
Fig. 1 shows part of the i * model proposed by the students and the results of
the backward analysis. After checking the analysis results, students noticed that
for instance the goal Make profit (circled in bold) for their chosen organization
is not satisfiable neither completely nor partially based upon backward analysis.
They revised the model, following these intuitions for this goal: “Make profit de-
pends on cost saving in our goal diagram. However, this goal model lacks which
tasks really help in increasing income and better profits. So, it might be good to
add goals in Marketing Manager that help in generating income, such as obtain-
ing new projects. Also building software will help in generating income, hence a
link should be added from building software to Make Profit.” Then, they reran the
backward analysis over the revised model, see Fig. 2. Now the goal Make profit
is satisfied. Similarly they performed changes for the softgoal Quality Software.
They have performed three other iterations to make the improvements, fol-
lowing forward analysis results as well. We do not present the models for those
iterations here due to lack of space.
5 Discussion and conclusions
In this paper we have discussed our experience in teaching goal-modeling tech-
niques at a graduate level course. We noticed that the use of each type of analysis
40
Fig. 1: Student i * model and analysis results (zoom to see details)
Fig. 2: Resulting improved i * model after analysis
41
offered students a deeper understanding of the activities they performed, from
the modeling of the organization at a high level (with i *), to strategic modeling
(with BIM/TBIM), to business process modeling, and information system de-
sign. Our experiences so far are reported based on our informal recollections and
observations. We are currently working to quantify and qualify such observations
more precisely.
The presented results are from the previous two course instances; thus, we
need to test our thesis further in the future course offerings. Our conclusions
have some threats to validity: (1) the groups of students might have been better
than those of previous years; (2) we had method designers teaching the analysis
techniques.
Acknowledgments
This research was partially supported by the ERC advanced grant 267856,
‘Lucretius: Foundations for Software Evolution’, www.lucretius.eu. Jennifer
Horkoff is supported by an ERC Marie Skodowska-Curie Intra European Fellow-
ship (PIEF-GA-2013-627489) and by a Natural Sciences and Engineering Re-
search Council of Canada Postdoctoral Fellowship (Sept. 2014 - Aug. 2016).
References
1. Y. Asnar, P. Giorgini, and J. Mylopoulos. Goal-driven risk assessment in require-
ments engineering. REJ, 16(2):101–116, 2011.
2. F. Francesconi, F. Dalpiaz, and J. Mylopoulos. Tbim: A language for modeling
and reasoning about business plans. In Proc. of the 32nd ER Conference, pages
33–46. Springer, 2013.
3. P. Giorgini, F. Massacci, J. Mylopoulos, and N. Zannone. Modeling security re-
quirements through ownership, permission and delegation. In Proc. of the 13th
IEEE International Conference on RE, pages 167–176, 2005.
4. P. Giorgini, F. Massacci, J. Mylopoulos, and N. Zannone. Requirements engineer-
ing for trust management: model, methodology, and reasoning. IJIS, 5:257–274,
October 2006.
5. J. Horkoff, D. Barone, L. Jiang, E. Yu, D. Amyot, A. Borgida, and J. Mylopoulos.
Strategic business modeling: representation and reasoning. Software & Systems
Modeling, 13(3):1015–1041, 2014.
6. J. Horkoff and E. Yu. Analyzing goal models: different approaches and how to
choose among them. In SAC, pages 675–682. ACM, 2011.
7. J. Horkoff and E. Yu. Interactive goal model analysis for early requirements engi-
neering. REJ, pages 1–33, 2014.
8. D. Karagiannis, S. Junginger, and R. Strobl. Introduction to business process
management systems concepts. In BPM, pages 81–106. Springer, 1996.
9. E. Paja, F. Dalpiaz, and P. Giorgini. Managing security requirements conflicts
in socio-technical systems. In Proc. of the 32nd ER Conference, volume 8217 of
LNCS, pages 270–283, 2013.
10. E. Yu, P. Giorgini, N. Maiden, and J. Mylopoulos. Social Modeling for Require-
ments Engineering. MIT Press, 2010.
42