Introduction

For goal models to be used in practice, goal-oriented requirements languages and techniques must be taught through university and professional courses. They are often taught as topics in the syllabus of courses on requirements engineering, sometimes at the graduate level. As instructors, we want students to be capable to build useful models, effectively using models for system comprehension and decision making. More importantly, we want students to value their learning experience. How can we improve our teaching practices to achieve these goals?

One of the features of goal-oriented models is their capability to facilitate systematic satisfaction analysis of model elements. It is possible to use existing analysis techniques, such as [7], to answer "What if?" or "Is this achievable?" questions. These procedures allow students to explore the meaning of the connections within their models, understanding the consequences of alternative selection. Such analysis is intended to help find problematic areas in the model, and to help choose the best mitigation or alternative for such problems. Similar analysis procedures have been introduced for complementary frameworks, such as the Business Intelligence Model (BIM) [5].

In this paper, we argue that the teaching of goal models should be coupled with one or more systematic analysis procedures. It is through such analysis that students understand whether their models are meaningful and whether or not they fulfill their purpose. In our experience, it is analysis that makes students appreciate the benefits of models and modeling.

Our thesis has been tested with a qualitative study involving a masterslevel course at the University of Trento. The course is titled "Organizational Information Systems" and has been offered for more than 10 years with 20-35 enrollments per year. The course covers enterprise modeling, strategic objectives modeling and analysis with i* [10], business process modeling and simulation with Adonis [8]. Course requirements included a course project worked on in teams involving modeling and analysis of an enterprise of each team's choice. Recent editions of the course have placed more emphasis on systematic goal model analysis.

The rest of the paper is structured as follows. In section 2 we present more details about the course syllabus, while in section 3 we describe the requirements for the students' projects. Section 4 summarizes course project results by drawing our lessons learned, and section 5 concludes.

Course syllabus

The Organizational Information Systems (OIS) course is taught at the master level with the objectives of (i) teaching students basic concepts about modeling business organizations and business processes; (ii) teaching information system technologies and architectures that support the operation of organizations; (iii) understanding how to manage organization information systems; and (iv) introducing new trends in organizational information systems.

Students are required to have a general knowledge of software engineering, including knowledge of UML, as well as a general knowledge of databases and information systems.

The course is organized as follows. First, it provides an introduction to organizations and organizational information systems, organizational structures, and organizational business processes. Second, it presents students with modeling approaches for organizations, standards and reference architectures. Among others, it presents approaches for organizational modeling such as i * [10] and strategic business modeling such as BIM [5] and Tactical BIM (TBIM) [2]. Emphasis is placed on teaching systematic analysis for both i* and BIM [5,7]. Third, the course covers modeling and analysis approaches for business processes, based on Adonis3 or BPMN 2.04 . Fourth, it discusses OISs Management -plan, implement, deliver, monitor, evaluate, and improve organizational information systems. Finally, the course discusses information assurance, presenting methods for IT Goal-Risk-Compliance [1], and Information Security [3,4,9].

It is worth emphasizing that over the years the syllabus of the course has been continuously updated to accommodate new and emerging techniques. Although many goal model analysis procedures exist ( [6]) we have chosen to teach the students qualitative, interactive analysis as this type of analysis is relatively simple, does not require detailed domain information and comes with relatively stable tool support. Similarly, we chose [5] to teach quantitative BIM analysis using indicators, supporting business decision making, in part due to it's similarity to i*-specific techniques from [6].

Students' roles and course projects

Student evaluation is performed via a course project, completed by teams of two or three. Students choose their own teammates. Each team project involves using state-of-art tools to model and analyze an organization of their choice, representing the organizational structure, the organizational goals and strategic objectives. The student projects model and analyze business processes within that organization to design (or improve) an information system that supports some of these processes. The project is discussed in a final oral exam.

Roles. Students are required to play interchangeably the roles of the requirements analyst and stakeholders, in order to capture both perspectives when building and analyzing the models. Students have interacted directly with real stakeholders and customers in only a few cases.

Project description. Student projects are divided into two assignments. The first assignment is focused on modeling, while the second focuses on analysis, intended to support the improvement of the organization. A1. In the first assignment, students report on the problem by initially describing the organization in natural language (English/Italian). This description provides an overview of the organization (sector, size, location, services, etc.), specific features (what makes it different from competitors), and hypothetical plans about the future of the organization. Students should define the scope of the project (especially for big organizations), i.e. the parts of the organization for which they will design an information system.

In the rest of the first assignment students are required to model the existing organization (within the defined scope), representing important actors, their goals and interdependencies. This modeling is performed with i*, for which students are free to use state-of-art tools of their choice. Moreover, an important step of the modeling activities involves capturing strategic goals of the organization, including relevant situations and indicators, typically represented with BIM or TBIM models. Finally, students are required to identify and model at least three complex business processes with BPMN or Adonis. A2. The purpose of the second assignment is twofold: (1) to analyze the chosen organization in order to identify weaknesses, bottlenecks, and under-performance. Emphasis is placed on the instruction of strategic analysis for i* and BIM/TBIM, as well as business process analysis and simulation with ADONIS components;

(2) to improve the current organization by designing part of an organizational information system. Ideally, the system will overcome the identified limitations. This should be demonstrated using further analysis.

To achieve these objectives, students should analyze their i* models and BIM/TBIM models to determine goal satisfaction or denial. Most importantly, they are required to describe how these changes affect the identified business processes. As far as business process analysis is concerned, students are required to execute: (i) consistency queries for all the models, to show that the models are syntactically correct and complete; (ii) they should run some queries to elicit useful information from the business processes (path analysis, capacity analysis).

Lessons learned

As stated earlier, over the years the course has been reshaped to accommodate new emerging state-of-the-art techniques. While initially it focused mainly on enterprise architecture and business process modeling and analysis, using goal models only to describe the chosen organization, in the last two course instantiations, we have required students to make more use of systematic analysis techniques to assess their goal models. We report on our observations of how the use of goal model analysis influenced students' understanding.

We noticed qualitatively that the outcomes of the projects were much improved. In the oral exam and presentations in the recent year we observed that students they demonstrate a better understanding of goal models after applying analysis, which allowed them to iteratively and incrementally build goal models that adequately capture the intended domain and fulfill their purpose. We have observed that the results of the goal-analysis help students to understand what organizational changes can be made to better achieve goals. Moreover, they have used BIM analysis to identify the strategies that support those goals. Out of the 9 projects from previous year 8 performed extensive goal analysis over i * and BIM models, and only 1 (single student) failed to do so. Of the 8 projects, 6 reran analysis over the i * models which were improved by previous analysis results, while 2 provided suggestions for improvement, without evaluating these improvements with systematic analysis. We illustrate improvements made by students providing excerpts from a representative student project that applied analysis to iteratively improve goal-models. They have constructed the i * models, performing in total 7 iterations and have performed evaluations running both forward ("what if?") and backward ("is this possible?") analysis.

Fig. 1 shows part of the i * model proposed by the students and the results of the backward analysis. After checking the analysis results, students noticed that for instance the goal Make profit (circled in bold) for their chosen organization is not satisfiable neither completely nor partially based upon backward analysis. They revised the model, following these intuitions for this goal: "Make profit depends on cost saving in our goal diagram. However, this goal model lacks which tasks really help in increasing income and better profits. So, it might be good to add goals in Marketing Manager that help in generating income, such as obtaining new projects. Also building software will help in generating income, hence a link should be added from building software to Make Profit." Then, they reran the backward analysis over the revised model, see Fig. 2. Now the goal Make profit is satisfied. Similarly they performed changes for the softgoal Quality Software.

They have performed three other iterations to make the improvements, following forward analysis results as well. We do not present the models for those iterations here due to lack of space.

Discussion and conclusions

In this paper we have discussed our experience in teaching goal-modeling techniques at a graduate level course. We noticed that the use of each type of analysis offered students a deeper understanding of the activities they performed, from the modeling of the organization at a high level (with i *), to strategic modeling (with BIM/TBIM), to business process modeling, and information system design. Our experiences so far are reported based on our informal recollections and observations. We are currently working to quantify and qualify such observations more precisely.

The presented results are from the previous two course instances; thus, we need to test our thesis further in the future course offerings. Our conclusions have some threats to validity: (1) the groups of students might have been better than those of previous years; (2) we had method designers teaching the analysis techniques.