=Paper=
{{Paper
|id=Vol-1370/paper_1
|storemode=property
|title=Teaching Goal Modeling in Undergraduate Education
|pdfUrl=https://ceur-ws.org/Vol-1370/paper_1.pdf
|volume=Vol-1370
|dblpUrl=https://dblp.org/rec/conf/caise/Dalpiaz15
}}
==Teaching Goal Modeling in Undergraduate Education==
https://ceur-ws.org/Vol-1370/paper_1.pdf
1st International iStar Teaching Workshop (iStarT 2015)
Teaching Goal Modeling in Undergraduate Education
Fabiano Dalpiaz
Utrecht University, the Netherlands
Abstract. Goal modeling in general, and i* in particular, are typically taught
in specialized courses that are part of postgraduate programs. In this paper, we
report on our experience concerning teaching i* and its basic, essential dialect
called simple i* to over 130 first-year students of a bachelor degree in information
science. We present the intended learning outcomes and activities, we introduce
the simple i* dialect that was used in the labs, we discuss the gained knowledge
was tested in the final exam, and we discuss the obtained results.
1 Introduction
Despite requirements engineering has seldom been part of the standard computer and
information science programs [1], there has been an increasing amount of attention
on requirements engineering education starting in 2004 [8], also through the regular
organization of the Requirements Engineering Education and Training workshop series.
We focus on the teaching of goal modeling in general, and i* [10] in particular, as
a specific technique to model the problem domain in requirements engineering. More
specifically, we report on our employment of i* in a first-year course in a bachelor de-
gree in information science (Utrecht University, the Netherlands) with over 130 enrolled
students with no background knowledge in requirements engineering or modeling.
To the best of our knowledge, the common practice in higher education is to teach
goal modeling in master-level courses, or in advanced bachelor courses. Our hypothesis
is that the basic intentional and social primitives are also suitable for first-year students.
On the other hand, we believe that more advanced themes, such as formal goal modeling
languages like KAOS [9], are better suited for later phases in higher education, when
the necessary prerequisites on formal languages are achieved.
In this paper, we share our experience and make the main findings available to the
requirements engineering research community. The following sections describe:
– The intended learning outcomes (ILOs) [2] related to i* modeling, within the con-
text of the considered course, and the relevant learning activities that were held
throughout the course to enable the students reach the ILOs (Section 2);
– A simplified version of i* (that we called simple i*) intended for first-year students,
and a discussion of on its usage in the workshop sessions of the course (Section 3);
– How the knowledge of i* was tested in the exam, and a preliminary analysis about
how well the different concepts were learned, based on the exam results (Section 4);
– A discussion of our overall experience with respect to the ILOs, and a presentation
of our future directions in the field of i* and goal modeling education (Section 5).
1
2 Intended Learning Outcomes and Activities
The course where i* modeling was included is the 2014/2015 edition of Organizations
and ICT (http://oictuu.wordpress.com/), and it is a compulsory first-year course in the
Information Science bachelor degree at Utrecht University, the Netherlands. The main
purpose of the course is to introduce the students to the interplay between organizations
and Information & Communication Technology (ICT).
Intended Learning Outcomes. The successful student is one that, upon passing the exam
and the practical assignments, achieves the following intended learning outcomes:
1. Can use the main organizational theories to describe how an organization functions;
2. Can explain fundamentals and challenges of integrating ICT in an organization;
3. Knows the key types of ICT systems, and can show how they support the operation
of an organization;
4. Can critically analyze the ICT systems that exist within one organization;
5. Can effectively study an organization using modeling techniques and frameworks.
The i* language was taught as part of the learning activities towards the fifth of these
learning outcomes, i.e., as a modeling technique for effectively studying an organiza-
tion, alongside mainstream frameworks such as the Business Process Modeling No-
tation [6], Entity-Relationship diagrams [3], and the Business Model Canvas [7]. We
defined ILOs concerning i* so that the successful student:
a. Can explain how i* compares to other organizational modeling techniques;
b. Can recognize the modeling primitives and their meaning in an existing model;
c. Can choose the most adequate primitive to denote an organizational phenomenon;
d. Can create useful organizational models with the fundamental primitives of i*.
Outcome a. requires the students to understand that i* enables modeling the why be-
hind actor behavior, social dependencies among them, and alternative ways of fulfilling
the actors’ goals. Outcome b. focuses on the ability to adequately read an i* model. Out-
come c. concerns the capability of selecting the modeling primitive that is best suited
to represent a state of affairs within an organization. Finally, outcome d. focuses on the
ability to create useful models, with usefulness being defined as fit for purpose [5]: for
i*, the model should correctly convey the rationale behind actor’s behavior, their social
reliance on one another, thereby informing the design and evaluation of the business
processes of an organization.
Learning Activities. In order to achieve the learning outcomes a–d, several learning
activities were devised in the course, each contributing to one or more ILOs, as indicated
between square brackets:
– A 2-hour lecture was given to present the i* framework, and to introduce the simple
i* dialect that we devised for the OICT course [a,b,d];
– A 1-hour workshop was conducted on the same day of the lecture to get students
acquainted with simple i* modeling [b,c];
– A group homework assignment was created, where students had to model part of a
real-world organization [c,d];
– The final exam included one question focusing on the i* framework [c].
2
3 The simple i* Language
The simple i* language was devised to facilitate first-year bachelor students in the cre-
ation of models for their group homework assignment, without employing the full ontol-
ogy of i*, which is still a research theme [4]. The main differences between the simple
i* language and the traditional i* are as follows:
– Two types of actors exist: agent and role. There are no positions and generic actors.
– Three types of intentional elements exist: goal, soft-goal, and task. Resources are
not included, with the aim to keep the language minimal and easier to use.
– Refinement links: goals are organized in acyclic AND/OR graphs where a high-
level goal is decomposed through the AND-refinment and OR-refinement relation-
ships to lower-level goals and to tasks. Tasks cannot be further refined. However, a
model does not necessarily have all goals refined to tasks.
– Simple dependency links are employed, connecting a goal or task (dependum) of an
actor (depender) to another agent or role (dependee). Within its scope, the dependee
must have the dependum. Other types of dependency are not possible.
– A Contribution connects a goal or a task to a soft-goal (other contributions are
ruled out). Four levels of contribution exist: fully negative (- -), partially negative
(-), partially positive (+), and fully positive (++).
– There are no separate actor and rationale diagrams. A single diagram exists.
Research Scientific
Fabiano published publisher
AND
PhD
AND AND PhD student
Paper obtained
published AND
Paper Paper AND
written reviewed OR OR
Paper written Project
OR Upload to Submit to by PhD supervised
OR
website publisher AND AND
Paper written
Paper written +
by PhD Experiment Experiment
by Fabiano
performed corrected
+ -
- + OR
OR
Experiment
Minimize Paper conducted by Experiment
effort quality PhD conducted
by MSc
MSc
Reviewer
student
Fig. 1. An example of simple i*: publishing a paper
Fig. 1 illustrates simple i* on an example concerning paper publishing. Compared to
i*, note the refinement links (with AND and OR annotations) that link goals to sub-goals
and to tasks. For example, goal “research published” is AND-refined to “paper written”,
“paper reviewed”, and “paper published”; goal “paper published” is OR-refined to tasks
“Upload to website” and “Submit to publisher”. For the latter task, agent “Fabiano”
depends on a role “Scientific publisher”, whose scope is omitted in the figure.
3
Simple i* in practice. 46 groups of students applied simple i* to model part of a chosen
organization that was studied from multiple angles throughout the course. The modeling
was done through the collaborative online tool Lucidchart (http://www.lucidchart.com/),
and using a custom drawing palette that we provided to the students. The quality of the
models was good, especially considering that the students were in the early phase of
their higher education. Most of the errors concerned the use of AND/OR-refinements,
which were sometimes used to connect tasks to goals, or to represent a dependency (the
link indicated a dependency for that goal on an actor). Moreover, we found that sev-
eral groups replicated very similar patterns to those in the example that was used in the
lecture (Fig. 1), especially the orthogonal contributions of two alternative goals/tasks to
two soft-goals (see the bottom left of Fig. 1).
4 Assessing the Gained Knowledge in the Final Exam
To test the knowledge that individual students gained about i*, we included one associa-
tive question in the final exam where the student was required to link a statement with
an i* element chosen from a list that we created (see Table 1). We made this choice to
align with ILO c. in Section 2.
Table 1. Exam statements about i*, with possible elements to associate being: agent, role, goal,
soft-goal, resource, plays, task, AND-decomposition, OR-decomposition, positive contribution,
negative contribution, goal dependency, task dependency, resource dependency
ID Statement Associated Element
A doctor requires a nurse to take a blood sample from a patient
S1 Task Dependency
following a specific procedure
S2 A “nurse” of the Utrecht Medical Center Role
S3 Every patient of the hospital has a birthdate -
S4 The well-defined process of making a “Pizza Margherita” Task
To teach lecture 8 of this course, Fabiano has not only to prepare
S5 AND-decomposition
the slides, but also to present the slides to the students
Creating a Q&A page on the course website as being useful for
S6 Positive contribution
Fabiano to save time
The possibility to create the topping of a pizza either by adding
S7 OR-decomposition
either Mozzarella or Gouda Cheese
S8 An “insurance policy” for a driver who has provoked a car accident Resource
The rector relying on professor Mike for teaching a course on
S9 Goal Dependency
databases
S10 A patient’s desire to “have his broken knee repaired” Goal
The relationship between the “Department of Computing Science
S11 -
of Utrecht University” and “Utrecht University”
The “Department of Information and Computing Sciences” of
S12 Agent
Utrecht University
S13 Utrecht University’s aim of “improving the students satisfaction” Soft-goal
Publishing the grades within 2 days from the exam to reduce the
S14 Positive contribution
anxiety of students
S15 A student’s desire of obtaining high-quality education Soft-goal
4
123 students participated in the exam; on average, 59% of the statements about i*
were associated correctly, compared to an average of 63.2% for the other 8 exam ques-
tions. Thus, the average percentage is slightly lower for the i* question, yet comparable.
Table 2 presents more detailed results from the exam. Although we have not run
advanced statistical analyses yet, our preliminary results provide interesting insights:
Table 2. Findings from the exam, N=123
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15
50% 85% 69% 32% 76% 67% 89% 36% 21% 65% 60% 76% 78% 27% 56%
a. Correct answers percentage per individual statement
Correct
Element type Related Statements
answers
AND/OR-decomposition S5 , S7 83%
Role/Agent S2 , S12 80%
Not in i* S3 , S11 65%
Goal/Task/Softgoal/Resource S4 , S8 , S10 , S13 , S15 53%
Contributions S6 , S14 47%
Dependency S1 , S9 36%
b. Correct answers percentage per element type
50 50
Nr. of students
40 29
30 25
20 11
10 3 5
0
0
-3 QT -2 QT -1 QT same QT +1 QT +2 QT +3 QT
Quartile difference
c. Students per-quartile allocation: the +X QT (-X QT) bar indicates the number of students that
gained (lost) X quartiles considering the i* question, compared to the other exam questions
a. Concerning the individual statements, S7 and S2 are those that the students asso-
ciated more accurately; these statements concerned OR-decomposition (89%) and
role (85%), respectively. The statements that presented more difficulties were S9
(21%), S14 (27%), S4 (32%), and S8 (36%), which described a goal dependency,
a positive contribution, a task, and a resource, respectively. Our experience shows
that some link types are harder to grasp than the i* entities; however, we also hy-
pothesize that the phrasing of the sentences may have had an impact as well.
b. Analyzing the association correctness per element type, statements on actors and
decompositions were well addressed, while most problems occurred with depen-
dency links (36%), and with contributions (47%).
c. We classified the individual student results for the i* question and for the other
questions according to the corresponding quartile. The aim was to detect if the
performance of a student in the i* question deviates from her performance in the
other questions. For example, if a student lay in the first quartile for i*, and in the
second quartile for the other questions, that student would have lost one quartile (-1
QT). The chart shows that there is no visible tendency towards gaining or losing
quartiles, despite a minor skewing towards losing quartiles.
5
5 Discussion and Outlook
Our usage of i* in a first-year bachelor course was positive, although some problems
arose, as expected when teaching advanced topic to students at the beginning of their
journey in higher education. Looking at the ILOs from Section 2, we conclude that:
a. Comparison of i* to other organizational modeling techniques. This outcome was
largely achieve, as shown by our qualitative analysis (omitted, due to space limits)
of the simple i* models that the students have created for their projects.
b. Recognition of the modeling primitives and their meaning in an existing model. We
did not explicitly assess this ILO, and no conclusion can be therefore drawn.
c. Choice of the most adequate primitive to denote an organizational phenomenon.
This objective was partially achieved, as shown in Section 4. Some constructs were
more difficult than others, especially contributions and dependencies. However, this
may also be due to a misinterpretation of the statements in Table 1.
d. Creation of useful organizational models with the fundamental primitives of i*.
Our qualitative analysis of the project outcomes show a satisfactory use of simple
i*, with models conveying the rationale of the actors and their relationships, despite
some modeling errors that do not hinder communication. Higher-quality modeling
would require substantial training, which was outside the scope of this course.
Our future work on goal modeling in higher education comprises several lines: (i)
obtaining results from multiple student cohorts; (ii) defining validated tests to assess
i* knowledge; (iii) using automated reasoning to detect patterns in the created models;
and (iv) employing gamification during learning to improve students motivation.
References
1. B. Berenbach. A hole in the curriculum. In Proceedings of the 1st International Workshop
on Requirements Engineering Education and Training (REET05), Paris, France, 2005.
2. J. Biggs and C. Tang. Teaching for quality learning at university. Open University Press, 4
edition, 2007.
3. P.P. Chen. The entity-relationship model—toward a unified view of data. ACM Transactions
on Database Systems, 1:9–36, 1976.
4. R.S.S. Guizzardi, X. Franch, G. Guizzardi, and R. Wieringa. Ontological distinctions be-
tween means-end and contribution links in the i* framework. In Proc. of ER, pages 463–470.
Springer, 2013.
5. M.S. Martis. Validation of simulation based models: a theoretical outlook. The Electronic
Journal of Business Research Methods, 4(1):39–46, 2006.
6. Object Management Group. Business Process Model and Notation (BPMN) 2.0.
http://www.omg.org/spec/BPMN/2.0, January 2011.
7. A. Osterwalder and Y. Pigneur. Business Model Generation. Wiley, 2010.
8. S. Ouhbi, A. Idri, J.L. Fernández-Alemán, and A. Toval. Requirements engineering educa-
tion: a systematic mapping study. Requirements Engineering, pages 1–20, 2013.
9. A. van Lamsweerde. Requirements Engineering: From System Goals to UML Models to
Software Specifications. Wiley, 2009.
10. E. Yu. Modelling Strategic Relationships for Process Reengineering. PhD thesis, University
of Toronto, 1996.
6