-Tropos is an agent-oriented software engineering (AOSE) methodology, based on the notion of actors, with goals and plans, and spanning all the phases of software development, from the very early phases of requirements analysis down to the actual implementation. The effectiveness of such methodology in the production of better design documents is evaluated in this study, by investigating the null hypothesis “using the Tropos Methodology before the analysis and design phases can produce a more accurate and complete set of UML diagrams than when no such technology is used”. The evaluation of a real case scenario was given as part of a coursework in a BSc module at the University of East London, and the Tropos and UML diagrams were requested as part of the deliverables. The results of how students performed in such tasks, and how the Tropos approach helped in the drawing of the UML diagrams, are presented here. The results show that generally, and confined to this experiment, the Tropos methodology has not helped in the design of the UML diagrams, and that students failed in understanding the link between the two methodologies.
Among the core skills employed during the phase of
requirements gathering and elicitation, is that of being able
to identify and model the basic concepts of the application
domain upon which the software system will be built.
Such activity has been named conceptual modelling, and
it serves the purpose of glueing together the requests by the
customers, and the expertise of designers and developers,
providing a platform to ease communication, meet users
expectations and distribute knowledge [
Among the goal-based approaches, Tropos is an AOSE methodology based on two key ideas: agents and their interactions within the system environment. The main aim of Tropos is to produce a better understanding of the application domain where a system will operate, and of the kind of interactions that should occur between such system and the human agents. Within Tropos, the notion of agent, together with their goals and plans, are used since the early analysis of requirements elicitation: in the early phase of such analysis, the organizational setting is studied for the purpose of better understanding the scenario. In the late phase of requirements gathering and elicitation, the system is also inserted in the operational environment as one actor: the dependencies with the other actors represent the system’s functional and non-functional requirements.
In both phases, the actor and goal diagrams are produced as outcomes, with the system being inserted in the diagrams in the late phase, but not in the early phase. The actor diagram represents the overall view of all the actors with their high-level dependencies to other actors, while the goal diagram is a refinement of the former with emphasis on the goals of a single actor (see Figure 1).
The focus of this work is on the early and late phases of
requirements elicitation covered by the Tropos methodology,
where the business entities are identified as actors, their
goals assessed, and their inter-dependencies defined. In the
UML notation instead, as summarised in Table I, these
two phases correspond to the production of a model in
the problem space (MOPS [
The aim of this paper is to compare the UML outcomes from the MOPS phase (use cases and business class diagrams) as produced by undergraduate and postgraduate students, when combining (or not) the Tropos methodology as a “treatment”. The rationale of such experiments is to determine through evaluation whether the joint use of goal-based (Tropos) and scenario-based (UML) approaches should be preferred to the use of only a scenario-based approach in the production of quality UML diagrams.
This paper details one experiment where BSc students at the university of East London, UK, produced both Tropos and UML diagrams towards the assessment of a scenario where a software system has to be built. The UML and Tropos diagrams were assessed against the benchmark produced as a marking scheme, and it is questioned whether the presence of the Tropos methodology has helped in the completeness of the resulting UML diagrams. This paper is the first of two experiments, where the Tropos methodology is used to inform the UML design: we plan to replicate this experiment in the semester starting in February 2011, without the Tropos “treatment”: students will be required to work on the same scenario, but no Tropos diagrams will be required (or taught), therefore allowing for the comparison of two different sets of UML diagrams. This will provide the basis for comparing the effectiveness (or not) of the two combined approaches.
This paper builds upon the scenario-based and the
goalbased approaches as two viable tools in the requirements
elicitation phase and for validation purposes. As a
practical exemplification of the scenario-based requirement
engineering method, we have used the Jacobson’s Use Case
technique, which has been lately incorporated into the UML
notation language [
On the other hand, this paper relies on the concepts of
agents and the agent-oriented paradigm (AOSE), as one
example of goal-oriented approach [
The main notations of UML (as a scenario-based
methodology) and Tropos (as goal-based) are summarised in
Figure 1 (taken from [
In the literature, the effectiveness of goal-oriented and
scenario-based approaches is analyzed in several works
illustrating the application of different methods to case
studies (e.g., [
One important factor for comparison or evaluation is
the immediacy in understanding the respective models by
projects stakeholders, for instance by requirements
analysts [
This section introduces the definitions used in the
following empirical study and presents the general objective
of this work, and it does that in the formal way proposed
by the Goal-Question-Metric (GQM) framework [
Goal: The long term goal of this research is to evaluate whether the Tropos methodology (as an experimental “treatment”), jointly with the UML MOSS notation, produce higher standards of conceptual modelling than the UML notation alone.
Question: In this paper, and considering a given scenario as a case study, the following research questions will be evaluated: 1) Are the models produced by the students with the Tropos notation “complete” against a given benchmark? Rationale: the aim of this question is to check whether the diagrams produced with the Tropos notation are compliant with a minimum list of actors and goals directly derived from the scenario. Such list of actors and goals should be considered as the “absolutely mandatory” in a typical requirements elicitation and gathering phase. 2) Are the models produced with UML complete against a given benchmark? Rationale: similarly to the question above, the aim of this question is to check whether the diagrams produced with a UML editor (Rational Rose, ArgoUML, etc) can be mapped to a minimum list of use case diagrams, necessary to describe the how the users of the system interact with its functionalities. 3) Can students map the Tropos actors and goals to UML use cases? Rationale: the aim of this question is to evaluate whether the use of “goals” and “actors” can help in focusing on the main functionalities of the system, expressed as UML use cases. Given the set of Tropos diagrams produced by any group of students, and a benchmark mapping of “Goals-to-use-cases” (see last column of Table III), it will be evaluated how the Tropos diagrams have informed the specified group of students in the creation of use cases.
Metrics: The Tropos actor and goal diagrams for this scenario have been listed in their minimum form, i.e., the minimum number of functionalities that are expected for (and from) this system, corresponding to both functional and non-functional requirements (see Table II). Also, the minimum set of UML use cases has been developed and it served as a benchmark to evaluate how students assessed the scenario (see Table III). Each group coursework was evaluated against these two lists, and the number of correct diagrams produced by the students evaluated against these baselines.
The first part of the experiment was set up at the University of East London, during the Level 3 module “Advanced Information Systems Development”. The experiment population comprised some 65 students, divided in 17 groups of 3 to 4 members1. Each group was in charge of producing two sets of diagrams: the Tropos goal and actor diagrams (for both the early and late phase of the requirements); and the UML use cases and class diagrams. All the students in the module had already studied the basic UML concepts in a previous module, while the Tropos concepts were introduced during several lectures, and their practical implementation was assisted in the lab sessions. The scenario was distributed to students on week 4 (out of 12 weeks in the module), and it represents the coursework needed to pass the module, 1Since the selection of students and groups was not random, the study should be referred to as a quasi-experiment. We will use the term “experiment” as a synonym throughout the study together with the final exam. The students had 9 weeks to complete the task.
In order to produce the Tropos diagrams, the OME tool, implementing the i* notation2, was taught and demonstrated during the lab sessions. In order to produce the UML class and use case diagrams, students could select the UML editor of their choice (e.g., the IBM Rational Rose toolkit, or the Open Source ArgoUML tool3, etc.).
The following problem statement was provided to the students, with the request for modeling such scenario via a scenario-based approach and a goal-based approach. This is based on a previous job placement where a student effectively designed and developed the system outlined below.
A company has supplied and supported its clients in the area of Tax and Returns Automation for more than 10 years. This involves an employee going to the client sites and inspecting the revenues that each of the client companies claim in a given year and giving advices and filling the necessary forms for Tax Return purposes. Once the employee has filled the relevant forms (on a per-client basis), these forms need to be saved to a couple of paper copies, one to be kept by the client, one to be archived within the company.
The company is seeking to streamline and automatise its systems for record keeping, therefore enabling the business to offer their clients a better service. The aim of this project would be to develop a system allowing data collection during site visits to be entered onto an online application, that sits on the web: the employee visiting the site’s premises would input the data to a specified form (which can be extended by a System Administrator to contain more fields and input data, it could be reused from existing form, and new forms can be created ad-hoc). The data once collected would be synchronized with the companys database, but during the initial trial period, the paper-based system, and the on-line system, would need to run together, and be synchronised.
The data collected would be used to keep the clients informed of the results of the employee’s visits and the next visit’s date. This upgrade project would be expected to cover the following areas: data acquisition using online, secure systems; synchronizing of data; a database to store the data of clients; and a PC based management tool for the data-captured database. 2OME3, available online at http://www.cs.toronto.edu/km/ome/ 3ArgoUML, freely available at http://argoUML.tigris.org/ B. Expected Outcomes – Tropos Marking Scheme
In order to assess the courseworks produced by the students, a list of “model solutions” was produced, and checked against the delivered set of diagrams. In particular, a minimum list of the Tropos actors present in the scenario was produced and their main goals were identified: the following Table II was therefore used as the baseline for marking the assignment. These goals and actors were prepared by one of the authors (running the module) and the assistant, a PhD student whose focus is on the secure aspects of Tropos.
Three main actors (Client, Company and Employee) were identified as expressing goals within the interaction with the system, while other two (the System, and the HM Revenue and Customs agency – HMRC) are also present, acting as dependees in one or more of those goals by the three main actors.
The marks available for the completion of such task were 25 out of 50.
Actor Company Employee Client System HMRC
Goal-based approach – TROPOS
Goals – (H)ard or (S)oft GCo1 Schedule periodic meetings
(H) GCo2 Get data to fill forms (H) GCo3 Get up-to-date Returns
rules (H) GCo4 Secure data based on client
or employee (H) GCo5 Provide a better service to
clients (S) GCo6 Rationalise forms (S) GE1 Get training on up-to-date
procedures (H) GE2 Get online access during
visits (H) GE3 Access clients details on
system (H) GE4 Log activity or duration (H) GCl1 Obtain copies of job per
formed (H) GCl2 Get Tax Return advices (H) GCl3 Browse activity logs (H) GCl4 Get secure service (S) Dependee Client Client HMRC System Self Self Company C. Expected Outcomes – UML Marking Scheme
The following Table III summarises instead the list of UML use cases that were set up as a baseline for marking the scenario-based part of the assignments: three main UML actors were expected to be interacting with the system, with increasing amount and type of provileges: the clients of Tropos Goal GCo1 GCo2 GCo3 GCo4 GCo5 GCo6 GE1 GE2 GE3 GE4 GCl1 GCl2 GCl3 GCl4 Grand Average UML use case c1 c2 c3 c4 c5 e1 e2 e3 e4 e5 e6 e7 a1 a2 a3 a4 a5 a6 Grand Average the Tax Revenue company (ci in Table III, i = 1..5), its employees (ei in the same Table, with i = 1..7) and the system administrator (si in the same Table, with i = 1..6).
The UML use cases listed, and intended as a “model solution”, are a subset of what was documented during a business consultancy,where the described system was actually implemented by a student in a job placement. The listed UML use cases should be inferred by reading the problem statement of the scenario, and they should also become clearer after working on the Tropos goals and actors. Albeit more specified UML actors could be identified (e.g., the ISP administrator, the project manager in charge of delivering the requested system, the Tax Revenue company owner, etc.), the above three provide the minimum set of scenarios that fulfill most of the functional and nonfunctional requirements of the scenario. In some of these, one UML use case could be the extension, or being included in some other use case (for instance, the “log-in” use case is typically included in any interaction with the system, independently from the privileges).
The marks available for the completion of the UML task were also 25 out of 50. This was decided to balance the relative importance of both Tropos and UML tasks.
As said above, the results obtained from the marking of the presented coursework represent the first part of this study: the second part will be based on assessing the diagrams produced by the students when the “treatment” Tropos is not taught or requested.
Table IV shows how each group coursework (G1 to G17) was evaluated against the list of Tropos goals and UML use cases, gathered around the main actors expressing their requirements, either in a goal-based approach, or a scenariobased approach.
At a first glance, the results found in the table show that the students found easier to assess the Tropos actors and goals, rather than producing the relative UML diagrams to describe how the actors are interacting with the system. Even when breaking down the aggregated results in the main components and actors, it is visible that some actors were assessed better than others: the Tropos models for the Company providing the Tax Revenue service are more complete than other actors (as visible in Table V where on average 70% of the groups assessed the benchmark goals from the Company actor).
The striking difference with such a finding is visible by observing the results of the UML cases, summarised in Table VI, where on average, only 38% of the groups assessed the “Employee” use cases, and only 43% delivered the “administrator” cases. As a grand average, some 64% of groups successfully assessed the set of Tropos goals proposed as a baseline, while only 38% of students assessed the set of UML cases of the benchmark.
These discrepancies are also visible when considering single students groups: • among the Tropos goals, 4 goals out of 6 were on average correctly identified, with regards to the Company goals (average 4.23 goals); among the Employee goals, 2 goals out of 4 were on average assessed (average 2.35 goals); finally, among the Client goals, 2 out of 4 goals were identified (average 2.35 goals); • with respect to the UML cases, 1 out of 5 cases were identified for the client (average 1.35 cases); 2 out of 7 cases were identified for the employee (average 2.70 cases); and 2 out of 6 cases were assessed for the administrator of the system (average 2.88 cases) Relatively to the experiment performed with the students of the University of East London, we can conclude that the use of the Tropos approach was not effective to inform the UML conceptual model.
Like any other empirical study, the validity of ours is subject to several threats. In the following, threats to internal (whether confounding factors can influence your findings), external (whether results can be generalized), and construct validity (relationship between theory and observation) are illustrated.
• Internal validity – the terminology “quality of UML models” was used to define whether “better” models could be obtained with the use of the additional Tropos analysis. Obviously the quality of UML diagrams is a multi-faceted dimension of several possible: aesthetic aspects could be considered, but also others based on design metrics of UML diagrams, as coupling, complexity, etc). • External validity – the following threats to external validity have been identified: 1) these findings cannot be generalised by one scenario, distributed to some 70 students, and based on one observation only. Replications are needed not only regarding the presence or absence of the Tropos “treatment”, but also with more students involved. 2) Despite the initial results, a stronger statistical formalism cannot be used for investigating the research questions: this is because since there is no comparison with a null hypothesis, such analysis cannot be properly performed. The results will become much more reliable when the second part of the experiment will be carried out. • Construct validity: the minimum set of actor and goal diagrams, and the minimum set of UML use cases derived for the construction of the benchmark could play an important part in the outcomes of this experiment. The reason of choosing these use cases, and the relative Tropos actors and goals, are of a practical nature: the proposed one is a real scenario of a past job placement, where a student designed and implemented the system to be delivered: the “model answers” are a subset of the diagrams implemented for the deployment of such system.
The authors would like to thank Dr H. Mouratidis and Michalis Pavlidis for the extensive comments, and the help in formulating the Tropos goals and actors. The authors would also like to thank the anonymous reviewers, since many improvements were added to the text, based on their suggestions.
The usage of visual modelling tools has become a common support for the design of a software system’s capabilities; the use of such tools has become more valuable in the early phase of requirement gathering, where the interaction with non-technical stake-holders requires jargon-free and easily usable approaches. Among these techniques, this paper has considered the goal-based (Tropos) and the scenariobased (UML) methodologies, trying to assess whether the use of the first could be useful to inform the definition of more complete use cases.
A set of “model solutions” was prepared for a given scenario, that was handed out as part of a coursework at the University of East London, UK. A baseline set of actors was prepared for the Tropos approach, and one for the UML use cases. Each coursework was assessed against these two baselines. Contrarily to what was expected, a larger number of students correctly assessed a larger amount of Tropos goals, whereas the UML cases were delivered less often, and more erroneously. Although the correct UML cases were assessed where the relevant Tropos actors were identified, this was not always the case: students found it difficult to connect the two approaches, and synchronise the actors and goals with how the system was supposed to perform.
These results are interesting, but we need to produce a similar set of observations when removing the Tropos approach from the experiment: we plan to replicate this experiment in a course starting in February 2011, where the same scenario will be provided, and where only the UML use cases will be requested. This will help in assessing whether the use of the Tropos approach can be considered to play a difference in the requirements gathering phase, when coupled to the UML notation.