1st International iStar Teaching Workshop (iStarT 2015) Teaching Goal Modeling in Undergraduate Education Fabiano Dalpiaz Utrecht University, the Netherlands Abstract. Goal modeling in general, and i* in particular, are typically taught in specialized courses that are part of postgraduate programs. In this paper, we report on our experience concerning teaching i* and its basic, essential dialect called simple i* to over 130 first-year students of a bachelor degree in information science. We present the intended learning outcomes and activities, we introduce the simple i* dialect that was used in the labs, we discuss the gained knowledge was tested in the final exam, and we discuss the obtained results. 1 Introduction Despite requirements engineering has seldom been part of the standard computer and information science programs [1], there has been an increasing amount of attention on requirements engineering education starting in 2004 [8], also through the regular organization of the Requirements Engineering Education and Training workshop series. We focus on the teaching of goal modeling in general, and i* [10] in particular, as a specific technique to model the problem domain in requirements engineering. More specifically, we report on our employment of i* in a first-year course in a bachelor de- gree in information science (Utrecht University, the Netherlands) with over 130 enrolled students with no background knowledge in requirements engineering or modeling. To the best of our knowledge, the common practice in higher education is to teach goal modeling in master-level courses, or in advanced bachelor courses. Our hypothesis is that the basic intentional and social primitives are also suitable for first-year students. On the other hand, we believe that more advanced themes, such as formal goal modeling languages like KAOS [9], are better suited for later phases in higher education, when the necessary prerequisites on formal languages are achieved. In this paper, we share our experience and make the main findings available to the requirements engineering research community. The following sections describe: – The intended learning outcomes (ILOs) [2] related to i* modeling, within the con- text of the considered course, and the relevant learning activities that were held throughout the course to enable the students reach the ILOs (Section 2); – A simplified version of i* (that we called simple i*) intended for first-year students, and a discussion of on its usage in the workshop sessions of the course (Section 3); – How the knowledge of i* was tested in the exam, and a preliminary analysis about how well the different concepts were learned, based on the exam results (Section 4); – A discussion of our overall experience with respect to the ILOs, and a presentation of our future directions in the field of i* and goal modeling education (Section 5). 1 2 Intended Learning Outcomes and Activities The course where i* modeling was included is the 2014/2015 edition of Organizations and ICT (http://oictuu.wordpress.com/), and it is a compulsory first-year course in the Information Science bachelor degree at Utrecht University, the Netherlands. The main purpose of the course is to introduce the students to the interplay between organizations and Information & Communication Technology (ICT). Intended Learning Outcomes. The successful student is one that, upon passing the exam and the practical assignments, achieves the following intended learning outcomes: 1. Can use the main organizational theories to describe how an organization functions; 2. Can explain fundamentals and challenges of integrating ICT in an organization; 3. Knows the key types of ICT systems, and can show how they support the operation of an organization; 4. Can critically analyze the ICT systems that exist within one organization; 5. Can effectively study an organization using modeling techniques and frameworks. The i* language was taught as part of the learning activities towards the fifth of these learning outcomes, i.e., as a modeling technique for effectively studying an organiza- tion, alongside mainstream frameworks such as the Business Process Modeling No- tation [6], Entity-Relationship diagrams [3], and the Business Model Canvas [7]. We defined ILOs concerning i* so that the successful student: a. Can explain how i* compares to other organizational modeling techniques; b. Can recognize the modeling primitives and their meaning in an existing model; c. Can choose the most adequate primitive to denote an organizational phenomenon; d. Can create useful organizational models with the fundamental primitives of i*. Outcome a. requires the students to understand that i* enables modeling the why be- hind actor behavior, social dependencies among them, and alternative ways of fulfilling the actors’ goals. Outcome b. focuses on the ability to adequately read an i* model. Out- come c. concerns the capability of selecting the modeling primitive that is best suited to represent a state of affairs within an organization. Finally, outcome d. focuses on the ability to create useful models, with usefulness being defined as fit for purpose [5]: for i*, the model should correctly convey the rationale behind actor’s behavior, their social reliance on one another, thereby informing the design and evaluation of the business processes of an organization. Learning Activities. In order to achieve the learning outcomes a–d, several learning activities were devised in the course, each contributing to one or more ILOs, as indicated between square brackets: – A 2-hour lecture was given to present the i* framework, and to introduce the simple i* dialect that we devised for the OICT course [a,b,d]; – A 1-hour workshop was conducted on the same day of the lecture to get students acquainted with simple i* modeling [b,c]; – A group homework assignment was created, where students had to model part of a real-world organization [c,d]; – The final exam included one question focusing on the i* framework [c]. 2 3 The simple i* Language The simple i* language was devised to facilitate first-year bachelor students in the cre- ation of models for their group homework assignment, without employing the full ontol- ogy of i*, which is still a research theme [4]. The main differences between the simple i* language and the traditional i* are as follows: – Two types of actors exist: agent and role. There are no positions and generic actors. – Three types of intentional elements exist: goal, soft-goal, and task. Resources are not included, with the aim to keep the language minimal and easier to use. – Refinement links: goals are organized in acyclic AND/OR graphs where a high- level goal is decomposed through the AND-refinment and OR-refinement relation- ships to lower-level goals and to tasks. Tasks cannot be further refined. However, a model does not necessarily have all goals refined to tasks. – Simple dependency links are employed, connecting a goal or task (dependum) of an actor (depender) to another agent or role (dependee). Within its scope, the dependee must have the dependum. Other types of dependency are not possible. – A Contribution connects a goal or a task to a soft-goal (other contributions are ruled out). Four levels of contribution exist: fully negative (- -), partially negative (-), partially positive (+), and fully positive (++). – There are no separate actor and rationale diagrams. A single diagram exists. Research Scientific Fabiano published publisher AND PhD AND AND PhD student Paper obtained published AND Paper Paper AND written reviewed OR OR Paper written Project OR Upload to Submit to by PhD supervised OR website publisher AND AND Paper written Paper written + by PhD Experiment Experiment by Fabiano performed corrected + - - + OR OR Experiment Minimize Paper conducted by Experiment effort quality PhD conducted by MSc MSc Reviewer student Fig. 1. An example of simple i*: publishing a paper Fig. 1 illustrates simple i* on an example concerning paper publishing. Compared to i*, note the refinement links (with AND and OR annotations) that link goals to sub-goals and to tasks. For example, goal “research published” is AND-refined to “paper written”, “paper reviewed”, and “paper published”; goal “paper published” is OR-refined to tasks “Upload to website” and “Submit to publisher”. For the latter task, agent “Fabiano” depends on a role “Scientific publisher”, whose scope is omitted in the figure. 3 Simple i* in practice. 46 groups of students applied simple i* to model part of a chosen organization that was studied from multiple angles throughout the course. The modeling was done through the collaborative online tool Lucidchart (http://www.lucidchart.com/), and using a custom drawing palette that we provided to the students. The quality of the models was good, especially considering that the students were in the early phase of their higher education. Most of the errors concerned the use of AND/OR-refinements, which were sometimes used to connect tasks to goals, or to represent a dependency (the link indicated a dependency for that goal on an actor). Moreover, we found that sev- eral groups replicated very similar patterns to those in the example that was used in the lecture (Fig. 1), especially the orthogonal contributions of two alternative goals/tasks to two soft-goals (see the bottom left of Fig. 1). 4 Assessing the Gained Knowledge in the Final Exam To test the knowledge that individual students gained about i*, we included one associa- tive question in the final exam where the student was required to link a statement with an i* element chosen from a list that we created (see Table 1). We made this choice to align with ILO c. in Section 2. Table 1. Exam statements about i*, with possible elements to associate being: agent, role, goal, soft-goal, resource, plays, task, AND-decomposition, OR-decomposition, positive contribution, negative contribution, goal dependency, task dependency, resource dependency ID Statement Associated Element A doctor requires a nurse to take a blood sample from a patient S1 Task Dependency following a specific procedure S2 A “nurse” of the Utrecht Medical Center Role S3 Every patient of the hospital has a birthdate - S4 The well-defined process of making a “Pizza Margherita” Task To teach lecture 8 of this course, Fabiano has not only to prepare S5 AND-decomposition the slides, but also to present the slides to the students Creating a Q&A page on the course website as being useful for S6 Positive contribution Fabiano to save time The possibility to create the topping of a pizza either by adding S7 OR-decomposition either Mozzarella or Gouda Cheese S8 An “insurance policy” for a driver who has provoked a car accident Resource The rector relying on professor Mike for teaching a course on S9 Goal Dependency databases S10 A patient’s desire to “have his broken knee repaired” Goal The relationship between the “Department of Computing Science S11 - of Utrecht University” and “Utrecht University” The “Department of Information and Computing Sciences” of S12 Agent Utrecht University S13 Utrecht University’s aim of “improving the students satisfaction” Soft-goal Publishing the grades within 2 days from the exam to reduce the S14 Positive contribution anxiety of students S15 A student’s desire of obtaining high-quality education Soft-goal 4 123 students participated in the exam; on average, 59% of the statements about i* were associated correctly, compared to an average of 63.2% for the other 8 exam ques- tions. Thus, the average percentage is slightly lower for the i* question, yet comparable. Table 2 presents more detailed results from the exam. Although we have not run advanced statistical analyses yet, our preliminary results provide interesting insights: Table 2. Findings from the exam, N=123 S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 50% 85% 69% 32% 76% 67% 89% 36% 21% 65% 60% 76% 78% 27% 56% a. Correct answers percentage per individual statement Correct Element type Related Statements answers AND/OR-decomposition S5 , S7 83% Role/Agent S2 , S12 80% Not in i* S3 , S11 65% Goal/Task/Softgoal/Resource S4 , S8 , S10 , S13 , S15 53% Contributions S6 , S14 47% Dependency S1 , S9 36% b. Correct answers percentage per element type 50 50 Nr. of students 40 29 30 25 20 11 10 3 5 0 0 -3 QT -2 QT -1 QT same QT +1 QT +2 QT +3 QT Quartile difference c. Students per-quartile allocation: the +X QT (-X QT) bar indicates the number of students that gained (lost) X quartiles considering the i* question, compared to the other exam questions a. Concerning the individual statements, S7 and S2 are those that the students asso- ciated more accurately; these statements concerned OR-decomposition (89%) and role (85%), respectively. The statements that presented more difficulties were S9 (21%), S14 (27%), S4 (32%), and S8 (36%), which described a goal dependency, a positive contribution, a task, and a resource, respectively. Our experience shows that some link types are harder to grasp than the i* entities; however, we also hy- pothesize that the phrasing of the sentences may have had an impact as well. b. Analyzing the association correctness per element type, statements on actors and decompositions were well addressed, while most problems occurred with depen- dency links (36%), and with contributions (47%). c. We classified the individual student results for the i* question and for the other questions according to the corresponding quartile. The aim was to detect if the performance of a student in the i* question deviates from her performance in the other questions. For example, if a student lay in the first quartile for i*, and in the second quartile for the other questions, that student would have lost one quartile (-1 QT). The chart shows that there is no visible tendency towards gaining or losing quartiles, despite a minor skewing towards losing quartiles. 5 5 Discussion and Outlook Our usage of i* in a first-year bachelor course was positive, although some problems arose, as expected when teaching advanced topic to students at the beginning of their journey in higher education. Looking at the ILOs from Section 2, we conclude that: a. Comparison of i* to other organizational modeling techniques. This outcome was largely achieve, as shown by our qualitative analysis (omitted, due to space limits) of the simple i* models that the students have created for their projects. b. Recognition of the modeling primitives and their meaning in an existing model. We did not explicitly assess this ILO, and no conclusion can be therefore drawn. c. Choice of the most adequate primitive to denote an organizational phenomenon. This objective was partially achieved, as shown in Section 4. Some constructs were more difficult than others, especially contributions and dependencies. However, this may also be due to a misinterpretation of the statements in Table 1. d. Creation of useful organizational models with the fundamental primitives of i*. Our qualitative analysis of the project outcomes show a satisfactory use of simple i*, with models conveying the rationale of the actors and their relationships, despite some modeling errors that do not hinder communication. Higher-quality modeling would require substantial training, which was outside the scope of this course. Our future work on goal modeling in higher education comprises several lines: (i) obtaining results from multiple student cohorts; (ii) defining validated tests to assess i* knowledge; (iii) using automated reasoning to detect patterns in the created models; and (iv) employing gamification during learning to improve students motivation. References 1. B. Berenbach. A hole in the curriculum. In Proceedings of the 1st International Workshop on Requirements Engineering Education and Training (REET05), Paris, France, 2005. 2. J. Biggs and C. Tang. Teaching for quality learning at university. Open University Press, 4 edition, 2007. 3. P.P. Chen. The entity-relationship model—toward a unified view of data. ACM Transactions on Database Systems, 1:9–36, 1976. 4. R.S.S. Guizzardi, X. Franch, G. Guizzardi, and R. Wieringa. Ontological distinctions be- tween means-end and contribution links in the i* framework. In Proc. of ER, pages 463–470. Springer, 2013. 5. M.S. Martis. Validation of simulation based models: a theoretical outlook. The Electronic Journal of Business Research Methods, 4(1):39–46, 2006. 6. Object Management Group. Business Process Model and Notation (BPMN) 2.0. http://www.omg.org/spec/BPMN/2.0, January 2011. 7. A. Osterwalder and Y. Pigneur. Business Model Generation. Wiley, 2010. 8. S. Ouhbi, A. Idri, J.L. Fernández-Alemán, and A. Toval. Requirements engineering educa- tion: a systematic mapping study. Requirements Engineering, pages 1–20, 2013. 9. A. van Lamsweerde. Requirements Engineering: From System Goals to UML Models to Software Specifications. Wiley, 2009. 10. E. Yu. Modelling Strategic Relationships for Process Reengineering. PhD thesis, University of Toronto, 1996. 6