=Paper= {{Paper |id=None |storemode=property |title=Using the Tropos Approach to Inform the UML Design: An Experiment Report |pdfUrl=https://ceur-ws.org/Vol-708/sqm2011-capiluppi-boldyreff-11-tropos.pdf |volume=Vol-708 }} ==Using the Tropos Approach to Inform the UML Design: An Experiment Report== https://ceur-ws.org/Vol-708/sqm2011-capiluppi-boldyreff-11-tropos.pdf
                       Using the Tropos Approach to Inform the UML Design:
                                      An Experiment Report


                                         Andrea Capiluppi          Cornelia Boldyreff
                                School of Computing, Information Technology and Engineering
                                                 University of East London
                                                  London, United Kingdom
                                             {a.capiluppi,c.boldyreff}@uel.ac.uk



   Abstract—Tropos is an agent-oriented software engineering           Among the goal-based approaches, Tropos is an AOSE
(AOSE) methodology, based on the notion of actors, with goals       methodology based on two key ideas: agents and their
and plans, and spanning all the phases of software development,     interactions within the system environment. The main aim
from the very early phases of requirements analysis down to the
actual implementation. The effectiveness of such methodology        of Tropos is to produce a better understanding of the
in the production of better design documents is evaluated in this   application domain where a system will operate, and of the
study, by investigating the null hypothesis “using the Tropos       kind of interactions that should occur between such system
Methodology before the analysis and design phases can produce       and the human agents. Within Tropos, the notion of agent,
a more accurate and complete set of UML diagrams than when          together with their goals and plans, are used since the early
no such technology is used”.
   The evaluation of a real case scenario was given as part of a
                                                                    analysis of requirements elicitation: in the early phase of
coursework in a BSc module at the University of East London,        such analysis, the organizational setting is studied for the
and the Tropos and UML diagrams were requested as part of           purpose of better understanding the scenario. In the late
the deliverables. The results of how students performed in such     phase of requirements gathering and elicitation, the system
tasks, and how the Tropos approach helped in the drawing of         is also inserted in the operational environment as one actor:
the UML diagrams, are presented here.
                                                                    the dependencies with the other actors represent the system’s
   The results show that generally, and confined to this exper-
iment, the Tropos methodology has not helped in the design of       functional and non-functional requirements.
the UML diagrams, and that students failed in understanding            In both phases, the actor and goal diagrams are produced
the link between the two methodologies.                             as outcomes, with the system being inserted in the diagrams
  Keywords-Software Quality; UML; Tropos Methodology                in the late phase, but not in the early phase. The actor
                                                                    diagram represents the overall view of all the actors with
                                                                    their high-level dependencies to other actors, while the goal
                      I. I NTRODUCTION
                                                                    diagram is a refinement of the former with emphasis on the
   Among the core skills employed during the phase of               goals of a single actor (see Figure 1).
requirements gathering and elicitation, is that of being able          The focus of this work is on the early and late phases of
to identify and model the basic concepts of the application         requirements elicitation covered by the Tropos methodology,
domain upon which the software system will be built.                where the business entities are identified as actors, their
Such activity has been named conceptual modelling, and              goals assessed, and their inter-dependencies defined. In the
it serves the purpose of glueing together the requests by the       UML notation instead, as summarised in Table I, these
customers, and the expertise of designers and developers,           two phases correspond to the production of a model in
providing a platform to ease communication, meet users              the problem space (MOPS [3]). Such a model comprises
expectations and distribute knowledge [1]. Two techniques           of a set of use cases and business class diagrams (i.e.,
have recently been considered and compared for the mod-             diagrams documenting business entities, their attributes and
eling of such concepts, one based on scenarios of how               behaviors). When the business entities are converted into
the system is going to behave (or how the users will                implementable entities, the UML notation produces the
interact with it (e.g., the UML approach [2]); the other            Model of Solution Space (MOSS) with the aim of feeding
expressing what are the needs that the built system will            such model to the design phase.
fulfill, relating the business goals of the stakeholders with the      The aim of this paper is to compare the UML outcomes
functional and non-functional requirements of such system.          from the MOPS phase (use cases and business class di-
The latter has been termed goal-based approach, and the             agrams) as produced by undergraduate and postgraduate
agent-oriented software engineering (AOSE) methods have             students, when combining (or not) the Tropos methodology
been one the more developed branches of such approach in            as a “treatment”. The rationale of such experiments is
the requirements elicitation.                                       to determine through evaluation whether the joint use of
              Early Require-        Late Requirements             to fulfill with the use of such system. Such goals could
              ments (ER)            (LR)                          be “hard” or “soft”, depending on whether it is clear what
    Tropos    ER actors and         LR actors and goals           actions and plans (or resources) should be performed (or
              goals                 (with system)                 used) in order to achieve such goals. A Tropos “actor
    UML       MOPS                  MOPS + MOSS                   diagram” details the overall connections between all the
                          Table I                                 actors in the scenario, where a dependee (e.g. actor 3 in
 T ROPOS AND UML DELIVERABLES IN THE EARLY AND LATE PHASES        Figure 1) fulfills the goal(s) of a depender (e.g., actor 1 in
                 OF REQUIREMENTS GATHERING
                                                                  Figure 1). A Tropos “goal diagram” focuses more precisely
                                                                  on one actor, and tries and elaborates on what plans, actions
                                                                  and resources should be performed to achieve each goal, and
                                                                  which actors are needed to fulfill these goals.
goal-based (Tropos) and scenario-based (UML) approaches              In the literature, the effectiveness of goal-oriented and
should be preferred to the use of only a scenario-based           scenario-based approaches is analyzed in several works
approach in the production of quality UML diagrams.               illustrating the application of different methods to case
   This paper details one experiment where BSc students at        studies (e.g., [9], [10], [11] or comparing the strengths and
the university of East London, UK, produced both Tropos           limitations of the approaches according to different criteria
and UML diagrams towards the assessment of a scenario             (e.g., [12], [13]). However, to the best of our knowledge,
where a software system has to be built. The UML and              experimental comparisons of these requirements modelling
Tropos diagrams were assessed against the benchmark pro-          paradigms using different visualization methods are rare [5].
duced as a marking scheme, and it is questioned whether           Such comparisons may raise insights and help decide which
the presence of the Tropos methodology has helped in the          modelling paradigm to adopt for a given software devel-
completeness of the resulting UML diagrams. This paper is         opment project. The “quality” of UML models, comprised
the first of two experiments, where the Tropos methodology        guidelines for the aesthetic quality, have also been evalu-
is used to inform the UML design: we plan to replicate            ated [14].
this experiment in the semester starting in February 2011,           One important factor for comparison or evaluation is
without the Tropos “treatment”: students will be required to      the immediacy in understanding the respective models by
work on the same scenario, but no Tropos diagrams will be         projects stakeholders, for instance by requirements ana-
required (or taught), therefore allowing for the comparison       lysts [15], who have to understand a given model dur-
of two different sets of UML diagrams. This will provide          ing analysis and refinement tasks to accommodate new or
the basis for comparing the effectiveness (or not) of the two     changed requirements.
combined approaches.
                                                                                  III. E MPIRICAL A PPROACH
         II. BACKGROUND AND R ELATED W ORK                           This section introduces the definitions used in the fol-
   This paper builds upon the scenario-based and the goal-        lowing empirical study and presents the general objective
based approaches as two viable tools in the requirements          of this work, and it does that in the formal way proposed
elicitation phase and for validation purposes. As a practi-       by the Goal-Question-Metric (GQM) framework [16]. The
cal exemplification of the scenario-based requirement en-         GQM approach evaluates whether a goal has been reached,
gineering method, we have used the Jacobson’s Use Case            by associating that goal with questions that explain it from
technique, which has been lately incorporated into the UML        an operational point of view, and providing the basis for
notation language [2]. Such a model is based on the notion of     applying metrics to answer these questions. This study
“scenario” which is a sequence of interaction events between      follows this approach by developing, from the wider goal
a system and its environment in the restricted context of         of this research, the necessary questions to address the goal
achieving some implicit purposes [4], [5].                        and then determining the metrics necessary for answering
   On the other hand, this paper relies on the concepts of        the questions.
agents and the agent-oriented paradigm (AOSE), as one                Goal: The long term goal of this research is to evalu-
example of goal-oriented approach [6], [7], [8]. This second      ate whether the Tropos methodology (as an experimental
approach is based on agents interacting as a group within a       “treatment”), jointly with the UML MOSS notation, produce
system, not just reacting to stimuli, but also communicating,     higher standards of conceptual modelling than the UML
coordinating, and cooperating as an autonomous and social         notation alone.
entity that can to achieve individual and organizational goals.      Question: In this paper, and considering a given scenario
   The main notations of UML (as a scenario-based method-         as a case study, the following research questions will be
ology) and Tropos (as goal-based) are summarised in Fig-          evaluated:
ure 1 (taken from [5]). Specifically for the Tropos notation,        1) Are the models produced by the students with the Tro-
every system can be thought of several actors, having goals              pos notation “complete” against a given benchmark?
                                  Figure 1.   Main UML and Tropos concepts and notations (from [5])



      Rationale: the aim of this question is to check whether        minimum number of functionalities that are expected for
      the diagrams produced with the Tropos notation are             (and from) this system, corresponding to both functional
      compliant with a minimum list of actors and goals              and non-functional requirements (see Table II). Also, the
      directly derived from the scenario. Such list of actors        minimum set of UML use cases has been developed and it
      and goals should be considered as the “absolutely              served as a benchmark to evaluate how students assessed
      mandatory” in a typical requirements elicitation and           the scenario (see Table III). Each group coursework was
      gathering phase.                                               evaluated against these two lists, and the number of correct
  2) Are the models produced with UML complete against               diagrams produced by the students evaluated against these
      a given benchmark?                                             baselines.
      Rationale: similarly to the question above, the aim of
      this question is to check whether the diagrams pro-                              IV. E XPERIMENTAL D ESIGN
      duced with a UML editor (Rational Rose, ArgoUML,                  The first part of the experiment was set up at the Univer-
      etc) can be mapped to a minimum list of use case               sity of East London, during the Level 3 module “Advanced
      diagrams, necessary to describe the how the users of           Information Systems Development”. The experiment popu-
      the system interact with its functionalities.                  lation comprised some 65 students, divided in 17 groups of
  3) Can students map the Tropos actors and goals to UML             3 to 4 members1 . Each group was in charge of producing
      use cases?                                                     two sets of diagrams: the Tropos goal and actor diagrams
      Rationale: the aim of this question is to evaluate             (for both the early and late phase of the requirements); and
      whether the use of “goals” and “actors” can help               the UML use cases and class diagrams. All the students in
      in focusing on the main functionalities of the system,         the module had already studied the basic UML concepts in a
      expressed as UML use cases. Given the set of Tropos            previous module, while the Tropos concepts were introduced
      diagrams produced by any group of students, and a              during several lectures, and their practical implementation
      benchmark mapping of “Goals-to-use-cases” (see last            was assisted in the lab sessions. The scenario was distributed
      column of Table III), it will be evaluated how the             to students on week 4 (out of 12 weeks in the module),
      Tropos diagrams have informed the specified group              and it represents the coursework needed to pass the module,
      of students in the creation of use cases.                        1 Since the selection of students and groups was not random, the study
  Metrics: The Tropos actor and goal diagrams for this               should be referred to as a quasi-experiment. We will use the term “experi-
scenario have been listed in their minimum form, i.e., the           ment” as a synonym throughout the study
together with the final exam. The students had 9 weeks to        B. Expected Outcomes – Tropos Marking Scheme
complete the task.                                                  In order to assess the courseworks produced by the
  In order to produce the Tropos diagrams, the OME tool,         students, a list of “model solutions” was produced, and
implementing the i* notation2 , was taught and demonstrated      checked against the delivered set of diagrams. In partic-
during the lab sessions. In order to produce the UML class       ular, a minimum list of the Tropos actors present in the
and use case diagrams, students could select the UML editor      scenario was produced and their main goals were identified:
of their choice (e.g., the IBM Rational Rose toolkit, or the     the following Table II was therefore used as the baseline
Open Source ArgoUML tool3 , etc.).                               for marking the assignment. These goals and actors were
                                                                 prepared by one of the authors (running the module) and
A. Scenario                                                      the assistant, a PhD student whose focus is on the secure
                                                                 aspects of Tropos.
   The following problem statement was provided to the
                                                                    Three main actors (Client, Company and Employee) were
students, with the request for modeling such scenario via
                                                                 identified as expressing goals within the interaction with the
a scenario-based approach and a goal-based approach. This
                                                                 system, while other two (the System, and the HM Revenue
is based on a previous job placement where a student effec-
                                                                 and Customs agency – HMRC) are also present, acting as
tively designed and developed the system outlined below.
                                                                 dependees in one or more of those goals by the three main
    A company has supplied and supported its clients             actors.
    in the area of Tax and Returns Automation for                   The marks available for the completion of such task were
    more than 10 years. This involves an employee                25 out of 50.
    going to the client sites and inspecting the rev-
    enues that each of the client companies claim in                           Goal-based approach – TROPOS
                                                                  Actor             Goals – (H)ard or (S)oft         Dependee
    a given year and giving advices and filling the
    necessary forms for Tax Return purposes. Once                              GCo1 Schedule periodic meetings       Client
                                                                                      (H)
    the employee has filled the relevant forms (on a                           GCo2 Get data to fill forms (H)       Client
    per-client basis), these forms need to be saved to                         GCo3 Get up-to-date Returns           HMRC
    a couple of paper copies, one to be kept by the               Company
                                                                                      rules (H)
    client, one to be archived within the company.                             GCo4 Secure data based on client      System
    The company is seeking to streamline and au-                                      or employee (H)
                                                                               GCo5 Provide a better service to      Self
    tomatise its systems for record keeping, therefore                                clients (S)
    enabling the business to offer their clients a bet-                        GCo6 Rationalise forms (S)            Self
    ter service. The aim of this project would be to                           GE1    Get training on up-to-date     Company
    develop a system allowing data collection during                                  procedures (H)
    site visits to be entered onto an online application,                      GE2    Get online access during       Client
    that sits on the web: the employee visiting the site’s        Employee            visits (H)
                                                                               GE3    Access clients details on      System
    premises would input the data to a specified form                                 system (H)
    (which can be extended by a System Administrator                           GE4    Log activity or duration (H)   System
    to contain more fields and input data, it could be                         GCl1 Obtain copies of job per-        Employee
    reused from existing form, and new forms can be                                   formed (H)
    created ad-hoc). The data once collected would                Client       GCl2 Get Tax Return advices (H)       Employee
    be synchronized with the companys database, but                            GCl3 Browse activity logs (H)         System
                                                                               GCl4 Get secure service (S)           Company
    during the initial trial period, the paper-based
                                                                  System
    system, and the on-line system, would need to run
                                                                  HMRC
    together, and be synchronised.
    The data collected would be used to keep the                                             Table II
                                                                           M ARKING SCHEME – T ROPOS ACTORS AND GOALS
    clients informed of the results of the employee’s
    visits and the next visit’s date. This upgrade
    project would be expected to cover the following
    areas: data acquisition using online, secure sys-
    tems; synchronizing of data; a database to store             C. Expected Outcomes – UML Marking Scheme
    the data of clients; and a PC based management                  The following Table III summarises instead the list of
    tool for the data-captured database.                         UML use cases that were set up as a baseline for marking
                                                                 the scenario-based part of the assignments: three main UML
 2 OME3, available online at http://www.cs.toronto.edu/km/ome/   actors were expected to be interacting with the system, with
 3 ArgoUML, freely available at http://argoUML.tigris.org/       increasing amount and type of provileges: the clients of
                                                                          Tropos      Groups         Perc      Average
the Tax Revenue company (ci in Table III, i = 1..5), its
                                                                          Goal        delivering
employees (ei in the same Table, with i = 1..7) and the                   GCo1        12            70.59%
system administrator (si in the same Table, with i = 1..6).               GCo2        14            82.35%
   The UML use cases listed, and intended as a “model                     GCo3        8             47.06%
                                                                                                               70.59%
solution”, are a subset of what was documented during a                   GCo4        11            64.71%
business consultancy,where the described system was ac-                   GCo5        17           100.00%
                                                                          GCo6        10            58.82%
tually implemented by a student in a job placement. The
                                                                          GE1         7             41.18%
listed UML use cases should be inferred by reading the                    GE2         15            88.24%
problem statement of the scenario, and they should also                                                        58.82%
                                                                          GE3         13            76.47%
become clearer after working on the Tropos goals and actors.              GE4         5             29.41%
Albeit more specified UML actors could be identified (e.g.,               GCl1        12            70.59%
the ISP administrator, the project manager in charge of                   GCl2        14            82.35%
                                                                                                               58.82%
delivering the requested system, the Tax Revenue company                  GCl3        8             47.06%
                                                                          GCl4        6             35.29%
owner, etc.), the above three provide the minimum set
                                                                          Grand                                63.87%
of scenarios that fulfill most of the functional and non-                 Average
functional requirements of the scenario. In some of these,
                                                                                            Table V
one UML use case could be the extension, or being included                            R ESULTS – B Y GROUP
in some other use case (for instance, the “log-in” use case
is typically included in any interaction with the system,
independently from the privileges).
                                                                          UML use     Groups         Perc     Average
   The marks available for the completion of the UML task                 case        delivering
were also 25 out of 50. This was decided to balance the                   c1          9             52.94%
relative importance of both Tropos and UML tasks.                         c2          3             17.65%
                                                                          c3          4             23.53%    27.06%
D. Results                                                                c4          3             17.65%
   As said above, the results obtained from the marking                   c5          4             23.53%
of the presented coursework represent the first part of                   e1          7             41.18%
                                                                          e2          11            64.71%
this study: the second part will be based on assessing the                e3          5             29.41%
diagrams produced by the students when the “treatment”                    e4          15            88.24%    38.66%
Tropos is not taught or requested.                                        e5          2             11.76%
   Table IV shows how each group coursework (G1 to G17)                   e6          2             11.76%
was evaluated against the list of Tropos goals and UML                    e7          4             23.53%
use cases, gathered around the main actors expressing their               a1          12            70.59%
                                                                          a2          3             17.65%
requirements, either in a goal-based approach, or a scenario-             a3          13            76.47%
based approach.                                                                                               43.53%
                                                                          a4          5             29.41%
   At a first glance, the results found in the table show that            a5          1              5.88%
the students found easier to assess the Tropos actors and                 a6          15            88.24%
goals, rather than producing the relative UML diagrams to                 Grand                               38.56%
                                                                          Average
describe how the actors are interacting with the system.
Even when breaking down the aggregated results in the                                       Table VI
                                                                                      R ESULTS – B Y GROUP
main components and actors, it is visible that some actors
were assessed better than others: the Tropos models for
the Company providing the Tax Revenue service are more
complete than other actors (as visible in Table V where on
average 70% of the groups assessed the benchmark goals              These discrepancies are also visible when considering
from the Company actor).                                         single students groups:
   The striking difference with such a finding is visible           • among the Tropos goals, 4 goals out of 6 were on av-
by observing the results of the UML cases, summarised                 erage correctly identified, with regards to the Company
in Table VI, where on average, only 38% of the groups                 goals (average 4.23 goals); among the Employee goals,
assessed the “Employee” use cases, and only 43% delivered             2 goals out of 4 were on average assessed (average 2.35
the “administrator” cases. As a grand average, some 64%               goals); finally, among the Client goals, 2 out of 4 goals
of groups successfully assessed the set of Tropos goals               were identified (average 2.35 goals);
proposed as a baseline, while only 38% of students assessed         • with respect to the UML cases, 1 out of 5 cases were
the set of UML cases of the benchmark.                                identified for the client (average 1.35 cases); 2 out of 7
                                       Scenario-based approach – UML
        UML actor                          UML use cases                             Via Tropos goal(s)
                        c1   Can log-in                                              GCl3, GCl4, GCo4, GCo5
                        c2   Can update their details                                GCl4
        Client          c3   Can browse the log of activity                          GCL3, GE4
                        c4   Can browse relevant documentation                       GCo3, GCl4
                        c5   Has sole access to private area                         GCl4
                        e1   Can schedule visit                                      GCo1
                        e2   Can log-in                                              GE2
                        e3   Can select appropriate forms based on client            GE1, GCo6
        Employee        e4   Can fill in forms                                       GCo2, GE4
                        e5   Can fill in the log of activity                         GE4, GCl3
                        e6   Can upload relevant documentation                       GCo3
                        e7   Has privileged access to all clients private area       GCo4, GCo5, GE3
                        a1   Can log-in                                              GCo5
                        a2   Can create/update/remove employees                      GCo4, GE1
                        a3   Can create/update/remove forms                          GCo6
        Administrator
                        a4   Can create/update/remove clients                        GCo4
                        a5   Can monitor the activity of employees                   GE1, GE4
                        a6   Can synchronise the database                            GE1, GE4, GCo6
                                                  Table III
                                  M ARKING SCHEME – L IST OF UML USE CASES



       G1
       √    G2
            √      G3
                   √    G4   G5
                             √      G6
                                    √     G7
                                          √      G8    G9    G10      G11
                                                                       √     G12
                                                                              √        G13
                                                                                        √     G14   G15
                                                                                                     √    G16
                                                                                                           √      G17
                                                                                                                   √
GCo1   √    √      √    √    √      √     √      √     √               √      √                √           √       √
GCo2        √      √    √                 √                            √      √         √            √
GCo3        √      √    √     √           √      √                     √      √         √                     √   √
GCo4   √    √      √    √     √     √     √      √     √      √        √      √         √      √     √        √   √
GCo5        √           √     √           √      √                            √         √            √        √   √
GCo6
            √           √                 √                           √       √                      √        √
GE1    √           √    √     √     √     √      √     √              √       √         √      √     √        √   √
GE2    √    √           √     √     √     √      √     √              √                 √            √        √   √
GE3         √                       √     √      √                                                            √
GE4
            √      √    √     √     √     √                           √          √      √            √        √   √
GCl1   √    √           √     √     √     √            √      √       √          √      √            √        √   √
GCl2   √           √          √           √      √                    √                                       √   √
GCl3               √          √                  √            √       √                              √
GCl4
       √    √                                    √            √       √          √                   √        √   √
c1                                               √                    √                              √
c2     √                      √                                                                               √   √
c3                            √                                                                               √   √
c4          √                                                 √                  √                            √
c5
            √           √     √     √                         √                  √      √
e1     √    √      √          √           √            √      √       √          √                   √        √
e2     √           √                                          √                         √      √
e3     √    √      √    √     √     √     √      √     √      √       √          √      √      √              √
e4                                                     √      √
e5          √                                          √
e6     √                                         √            √       √
e7
       √    √      √    √           √     √                           √          √             √     √        √   √
a1                                               √     √                         √
a2     √    √                 √     √     √      √                    √          √      √      √     √        √   √
a3          √           √                 √            √                         √
a4          √
a5     √    √      √    √     √     √     √      √     √              √          √             √     √        √   √
a6
                                                     Table IV
                                               R ESULTS – B Y GROUP
     cases were identified for the employee (average 2.70           in formulating the Tropos goals and actors. The authors
     cases); and 2 out of 6 cases were assessed for the             would also like to thank the anonymous reviewers, since
     administrator of the system (average 2.88 cases)               many improvements were added to the text, based on their
  Relatively to the experiment performed with the students          suggestions.
of the University of East London, we can conclude that the                  VII. C ONCLUSION AND F URTHER W ORK
use of the Tropos approach was not effective to inform the
UML conceptual model.                                                  The usage of visual modelling tools has become a com-
                                                                    mon support for the design of a software system’s capabili-
                 V. T HREATS TO VALIDITY                            ties; the use of such tools has become more valuable in the
                                                                    early phase of requirement gathering, where the interaction
   Like any other empirical study, the validity of ours is
                                                                    with non-technical stake-holders requires jargon-free and
subject to several threats. In the following, threats to internal
                                                                    easily usable approaches. Among these techniques, this pa-
(whether confounding factors can influence your findings),
                                                                    per has considered the goal-based (Tropos) and the scenario-
external (whether results can be generalized), and construct
                                                                    based (UML) methodologies, trying to assess whether the
validity (relationship between theory and observation) are
                                                                    use of the first could be useful to inform the definition of
illustrated.
                                                                    more complete use cases.
   • Internal validity – the terminology “quality of UML               A set of “model solutions” was prepared for a given
      models” was used to define whether “better” models            scenario, that was handed out as part of a coursework at
      could be obtained with the use of the additional Tropos       the University of East London, UK. A baseline set of actors
      analysis. Obviously the quality of UML diagrams is a          was prepared for the Tropos approach, and one for the UML
      multi-faceted dimension of several possible: aesthetic        use cases. Each coursework was assessed against these two
      aspects could be considered, but also others based            baselines. Contrarily to what was expected, a larger number
      on design metrics of UML diagrams, as coupling,               of students correctly assessed a larger amount of Tropos
      complexity, etc).                                             goals, whereas the UML cases were delivered less often, and
   • External validity – the following threats to external          more erroneously. Although the correct UML cases were
      validity have been identified:                                assessed where the relevant Tropos actors were identified,
        1) these findings cannot be generalised by one sce-         this was not always the case: students found it difficult to
            nario, distributed to some 70 students, and based       connect the two approaches, and synchronise the actors and
            on one observation only. Replications are needed        goals with how the system was supposed to perform.
            not only regarding the presence or absence of the          These results are interesting, but we need to produce
            Tropos “treatment”, but also with more students         a similar set of observations when removing the Tropos
            involved.                                               approach from the experiment: we plan to replicate this
        2) Despite the initial results, a stronger statistical      experiment in a course starting in February 2011, where
            formalism cannot be used for investigating the          the same scenario will be provided, and where only the
            research questions: this is because since there         UML use cases will be requested. This will help in assessing
            is no comparison with a null hypothesis, such           whether the use of the Tropos approach can be considered to
            analysis cannot be properly performed. The results      play a difference in the requirements gathering phase, when
            will become much more reliable when the second          coupled to the UML notation.
            part of the experiment will be carried out.
                                                                                            R EFERENCES
   • Construct validity: the minimum set of actor and
      goal diagrams, and the minimum set of UML use                  [1] A. van Lamsweerde, “Requirements engineering in the year
      cases derived for the construction of the benchmark                00: a research perspective,” in Proceedings of the 22nd
                                                                         International Conference on Software engineering (ICSE 00).
      could play an important part in the outcomes of this               New York, NY, USA: ACM, 2000, pp. 5–19.
      experiment. The reason of choosing these use cases, and
      the relative Tropos actors and goals, are of a practical       [2] G. Booch, J. Rumbaugh, and I. Jacobson, The Unified Mod-
      nature: the proposed one is a real scenario of a past job          eling Language User Guide, 2nd Edition (Addison-Wesley
      placement, where a student designed and implemented                Object Technology Series). Addison-Wesley Professional,
                                                                         2005.
      the system to be delivered: the “model answers” are a
      subset of the diagrams implemented for the deployment          [3] B. Unhelkar, Verification and Validation for Quality of UML
      of such system.                                                    2.0 Models. Wiley-Interscience, 2005.

                VI. ACKNOWLEDGEMENTS                                 [4] A. Sutcliffe, “Scenario-based requirements engineering,” in
                                                                         Proceedings of the 11th IEEE International Conference on
 The authors would like to thank Dr H. Mouratidis and                    Requirements Engineering. Washington, DC, USA: IEEE
Michalis Pavlidis for the extensive comments, and the help               Computer Society, 2003, pp. 320–.
 [5] I. Hadar, T. Kuflik, A. Perini, I. Reinhartz-Berger, F. Ricca,
     and A. Susi, “An empirical study of requirements model
     understanding: Use Case vs. Tropos models,” in Proceedings
     of the 25th ACM Symposium on Applied Computing. New
     York, NY, USA: ACM, 2010, pp. 2324–2329.

 [6] J. Mylopoulos, “Information modeling in the time of the
     revolution,” Information Systems, vol. 23, pp. 127–155, May
     1998.

 [7] E. S. Yu, “Modelling strategic relationships for process
     reengineering,” Ph.D. dissertation, Toronto, Ont., Canada,
     Canada, 1996.

 [8] A. Dardenne, A. van Lamsweerde, and S. Fickas, “Goal-
     directed requirements acquisition,” Science of Computer Pro-
     gramming, vol. 20, pp. 3–50, April 1993.

 [9] P. Bresciani, P. Giorgini, F. Giunchiglia, J. Mylopoulos, and
     A. Perini, “TROPOS: An agent-oriented software develop-
     ment methodology,” Autonomous Agents and Multi-Agent
     Systems, vol. 8, no. 3, pp. 203–236, May 2004.

[10] J. Castro, M. Kolp, and J. Mylopoulos, “Towards
     requirements-driven information systems engineering:
     the tropos project,” Information Systems, vol. 27, pp.
     365–389, September 2002.

[11] M. Kim, S. Park, V. Sugumaran, and H. Yang, “Managing
     requirements conflicts in software product lines: A goal and
     scenario based approach,” Data & Knowledge Engineering,
     vol. 61, pp. 417–432, June 2007.

[12] J. L.M.C. Filho, V.Werneck and E. Yu, “Agentgoal orientation
     vs object orientation for requirements engineering: A practical
     evaluation using an exemplar,” in Proceedings of the 8th
     Workshop on Requirements Engineering, 2005, pp. 123–134.

[13] C. Rolland, G. Grosz, and R. Kla, “Experience with goal-
     scenario coupling in requirements engineering,” in Proceed-
     ings of the 4th IEEE International Symposium on Require-
     ments Engineering. Washington, DC, USA: IEEE Computer
     Society, 1999, pp. 74–.

[14] H. Eichelberger and K. Schmid, “Guidelines on the aesthetic
     quality of UML class diagrams,” Information and Software
     Technology, vol. 51, no. 12, pp. 1686 – 1698, 2009.

[15] C. F. J. Lange, “Improving the quality of UML models in
     practice,” in Proceedings of the 28th International Conference
     on Software engineering. New York, NY, USA: ACM, 2006,
     pp. 993–996.

[16] V. R. Basili, G. Caldiera, and D. H. Rombach, “The goal
     question metric approach,” in Encyclopedia of Software En-
     gineering. John Wiley & Sons, 1994, pp. 528–532, see also
     http://sdqweb.ipd.uka.de/wiki/GQM.