For accomplished users of goal modelling, the diculties experienced with learning the concepts, semantics and practical skills for applying the techniques are often distant memories. Teaching such approaches provides valuable insights into the learnability of the techniques. In this paper we discuss our experiences of teaching i⇤ goal modelling for the first time, as part of a requirements engineering module for undergraduate and and postgraduate students. More specifically, we describe a set of guidelines designed to help students produce i⇤ models more e↵ectively, and evaluate these using data from 134 coursework assignments. We make some suggestions to help i⇤ teachers convey these concepts and also provide some thoughts on how i⇤ as a technique could become more learnable.
i⇤ provides conceptual and visual means to model the functional and nonfunctional goals of a socio-technical systems. Producing a good i⇤ model is a complex task that requires good understanding of the concepts and semantics of the i⇤ framework as well as practical skills. These understandings and skills are often gained through long run practice and application to di↵erent real-world projects. Some guidelines have been proposed to facilitate the production of i⇤ Strategic Dependency (SD) and Strategic Rationale (SR) models. However, the question remains open on how to best convey these principles and condense this learning process within a requirements engineering course. It is unclear what concepts and conventions are the most dicult for students to grasp and how to facilitate the learning process, especially when these students have di↵erent backgrounds and experiences.
In this paper we report on our experience of using some guidelines to teach i⇤ modelling in the context of a requirements engineering course for the first time. We analyse the results of 134 coursework assignments, the primary form of the student’s assessment, in order to assess whether the students were able to identify the main concepts and links of the i⇤ SD and SR models correctly and apply the aforementioned guidelines e↵ectively. The motivation behind this analysis is to identify the i⇤ concepts that are easy for the students to understand and the guidelines that they can easily apply to produce e↵ective models. In addition, potential ways to improve the teaching of i⇤ are suggested to be used by others.
The paper is structured as follows. Section 2 introduces the Amazon drone delivery coursework. Section 3 introduces the guidelines and conventions we used for teaching i⇤ modelling. Section 4 reports on the results of the use of these guidelines by students to complete the coursework. Finally, Section 5 summarises our findings and concludes the paper. 2
The students were given a use case specification describing how a shopper may use the Amazon Prime Air service to receive a package delivered by a drone. They were also provided with more general background information on the new delivery system to provide them with the necessary context. They were then asked to produce an i⇤ SD model for a set of given actors, and to produce an SR model for the shopper actor. To help the students with an example, it was specified that the model should include the goal dependency the shopper depends on the drone to satisfy the goal package unlocked. 3
We taught the students a number of guidelines, developed by City University
London [
Guideline 1: Degree of delegation to determine the type of dependency.
One of the most dicult decisions in SD modelling is to determine the type
of dependency. The diculty might be due to the confusion between the data
flow and the responsibilities and expectations of the actors involved. We used a
guideline based on the degree of delegation from the depender to the dependee
in order to guide the choice of dependency [
Package Shopper
Col ect package
Drone
Shopper
Guideline 4: Pre-defined operators for soft goals. To help the students
think about how to specify soft goals, we introduced the goal operators used in the
KAOS goal modelling approach [
Guideline 5: Task structure for SR models. There is a lack of explicit
process guidance about how to start creating an SR model. Therefore, we applied
the guideline from [
Guideline 6: Task decomposition to soft goal. To guide the modelling of
task decompositions, we used the guideline that states: a sub-soft goal of a task
must be satisfied by the completion of the task, and is a post-condition. Therefore,
Place order
we encouraged students to model desirable properties or states of a task using
task decomposition links. For example, Figure 4 shows the di↵erence between
using a task decomposition link and a contributes-to soft goal link.
The need for tooling. As reported in [
We reviewed the complete set of 134 coursework submissions and assessed the i⇤ SD and SR models based on a set of criteria related to the basic semantics of i⇤ modelling as well as the correct application of the guidelines described in Section 3, to which they have been introduced during the lectures and tutorials. 59 of the assignments are from undergraduate (UG) students and 75 from postgraduate (PG) students. Figure 5 summarises the criteria and results, which we describe in the following.
Correct SD elements. We checked if the students had understood the definitions of the goal, soft goal, task and resource elements and whether they had applied them correctly. We found that 61% of UGs and 72% of PGs made no errors specifying the 4 di↵erent element types. The most common mistake was misunderstanding the subtleties between some soft goals and goals. For example, expressing package delivered on time as a soft goal when it expresses a boolean condition akin to a goal. The next most common error was confusing tasks and goals, which seemed to be caused by the students not following the suggested wording conventions. Correct SD links. We checked if the SD models were constructed correctly with respect to actors and linked dependencies. The outcome was encouraging, with 93% of UGs and 99% of PGs successfully constructing their SD models. Correct SR elements. Interestingly, the students performed better at choosing the correct process elements for the SR models than for the SD models. 68% of UGs and 87% of PGs made no errors with the 4 types of process elements. Correct SR links. We looked at the di↵erent i⇤ link types to check if they had been applied and connected properly. The results showed that the students did not construct SR models as successfully as they did SD models, which is Correct"modelling"of"SR"elements"within"the"actor"boundary"
Correct"type"of"dependency"
Correct"direc6on"of"the"dependency"
Correct"applica6on"of"naming"conven6on"
Correct"use"of"KAOS"operators"to"name"so>"goals" 7# 17# Correct"applica6on"of"the"recommended"task"structure"for"SR"model""
Correct"applica6on"of"the"sub=so>"goal"task"decomposi6ons""
Correct"SD"elements"
Correct"SD"links" Correct"SR"elements"
Correct"SR"links" 61# 72# 68# 65# 87#
93#99# 36# 34# 43# 34# 25# 22# 24# 32# 54# 56# 53# 56# 48# PG" UG" " s c 6 rs i u e H " s c 6 n a m e s *" i 0" 10"
20" 30" 40" 50" 60" 70" 80" 90" 100" not surprising due to the additional links and richer semantics. For the UGs, 54% applied the links correctly as compared with 65% for the PGs. Common mistakes were contributes-to soft goal links connected to non-soft goals, and task decomposition links used to decompose goals.
Correct modelling of SR elements within the actor boundary. We checked to see if the elements within the actor boundary specified only what the actor could accomplish themselves. We found that only 36% of UGs correctly identified which elements should be contained with the Shopper boundary, with the corresponding percentage for PGs at 56%. There was a strong tendency for the students to model certain tasks and goals owned by the drone e.g. the task unlock package. Correct type of dependency. We reviewed the SD models to check if the correct dependency types had been used and whether duplication of dependencies was avoided. The results were disappointing, as only 22% of UGs and 25% of PGs correctly applied the delegation guideline. The most common error was duplicating dependencies, usually by specifying resource dependencies as well as the higher level goals and tasks to which they relate.
Correct direction of the dependency. As with delegation, this guideline was not applied successfully the majority of the time. The guideline was applied correctly by 34% of UGs and 43% of PGs. It was common for students to express that the depender actor depends on the dependee to undertake a task. For example, Drone depends on Shopper to collect package, rather than Shopper depends on Drone to collect package. In reality, Shopper collects the package, whilst Drone provides the package.
Correct application of naming convention. Looking at both the SD and SR models, we found that the naming conventions were only applied correctly by 34% of UGs and 53% of PGs. Many students failed to use the past participle when expressing goals and soft goals.
Correct use of KAOS operators to name soft goals. This was the least successfully applied guideline, with only 7% of UGs and 17% of PGs using the operators on all of the soft goals in their i⇤ models. It was clear that whilst these formal operators may help specify certain types of soft goals, they are not appropriate for all. Correct application of the recommended task structure for SR model. This was the most subjective of the criteria, but nonetheless we made a boolean assessment of the criterion based on the SR model structure we were trying to mandate (see Figure 3). We found that 24% of UGs correctly followed our guidelines, with a more encouraging number of PGs at 48%.
Correct application of the sub-soft goal task decompositions. We checked whether the ‘qualities of the task’ had been modelled using task decomposition links as we had advised. For UGs, 32% correctly applied the guideline, whilst this percentage was higher for PGs at 56%.
In this paper we discussed the e↵ectiveness of existing guidelines for teaching i⇤ modelling. While the main elements and links of the models appeared easy to grasp, applying most of the guidelines had only qualified success.
For teaching the basic concepts and conventions of i⇤ , we found the most e↵ective method was using trivial real-world examples, e.g., organising a Halloween party. Therefore, we would recommend using a non-software engineering example to convey the basics of i⇤ , and then move onto more complex examples.
In terms of the guidelines, given that there is a heavy payload learning the i⇤ elements and semantics, introducing guidance on modelling too soon appears to overwhelm many of the students. However, the students who grasped the semantics quickly managed to apply the guideline correctly and create some very impressive i⇤ models. Overall, this was more apparent with the PG students. Whilst identifying process elements for the SD model through the delegation principles, it was evident through our teaching that the students may have benefitted from making a draft SR model at the same time. Rather than follow the sequential development of SD and SR models, we believe that an iterative process where both models are developed concurrently can be more ecient.
We believe that some of the students found the suggested naming conventions counterintuitive, e.g., using the past participle for goals while specifying a system to be. Therefore, teachers should make it clear that goals are about desirable states of the system to be. Also, we still feel that these conventions are helpful and worth teaching, as many of the students use the wording to understand the di↵erences between tasks and goals, and goals and soft goals. Finally, visual tools can provide assistance and improve substantially the models produced by the students.