ion Model (RAM). We evaluated our results via a pilot controlled experiment and the results show a statistically significant improvement in the time required to assess the impact of changes by 37% with the same accuracy.
Model Driven Engineering [
The models should graphically visualize requirements at different abstraction levels and the relationships between them.
The models should be fully integrated with the existing requirements engineering
tools, e.g. RequisitePro [
The models should support both forward and reverse engineering – i.e. modelling the requirements and generating text specifications (forward) and vice-versa (reverse).
By using the models, business analysts and developers should be able to assess the impact of requirement change in a shorter time, identify contradicting and missing requirements in a shorter time and thus increase the quality of the final software product.
All requirements should be traceable to the design documents (e.g. UML or Simulink models) to support assessment of changes in the design (as an effect of optimizations) on the requirement specification.
These requirements resulted in developing a new modelling notation as a domainspecific modelling language gRAM. The initial evaluation of this language via a controlled experiment shows that such a notation fulfils the requirements for shortening the time required for assessing the impact of change.
This paper is structured as follows; Section 2 presents work related to this research. Section 3 briefly introduces the requirements specification format used in this evaluation. Section 4 reports the design and result of the evaluation and section 5 concludes the paper.
The work presented in this paper is part of our ongoing research outlined in [
Existing graphical modelling languages which include requirements modelling are
the UML [
Modelling of requirements has also been advocated in the context of MDE/MDA,
e.g. in [
Studies to validate the effectiveness of the RAM approach have been done in e.g.
[
In this section we present the graphical modelling language for structuring requirements and summarize the basis for it – the textual format of Requirement Abstraction Model.
The Requirement Abstraction Model (RAM) [
RAM ensures traceability between requirements through all levels of abstraction by enforcing that, with the exception of the product level, no requirement may exist without a link to the more abstract requirement. The rationale is that no requirement may exist unless there is a clear and unambiguous reason for its existence motivated by higher-level requirements, and conversely, high-level requirements should be traceable to the lower-level requirements that satisfy them.
gRAM is a Model-Driven Engineering (MDE) approach to requirements specification with the purpose of creating an easy to use environment for direct on-screen manipulation of a requirements structure, from which other documents can be automatically generated. The gRAM is a formalized graphical Domain Specific Language2 (DSL) complying with the RAM, where validation rules (i.e. static semantics) built into the gRAM ensures that the model and the resulting requirement specification are syntactically correct and well-formed according to the RAM. gRAM defines traceability links according to the RAM with its Owns/Satisfies link between requirements at adjacent abstraction levels, and adds the Depends-on traceability link, which indicates that there is a dependency between two requirements within the same abstraction level.
The definition of the gRAM follows the approach for language engineering as
advocated by Evans et al. [
Visual Studio 2008 DSL Toolkit [
The above elements are defined in the following sub-sections.
The main component in the abstract syntax, shown as a UML meta-model in Fig. 1, is the concept of abstraction level as defined in RAM. The top node in the abstract 2 In this context the term “domain” is requirement engineering, not a vertical application domain like automotive or telecom syntax is the notion of the model (Model), which contains abstraction levels (Abstraction Level). The abstraction levels contain requirements (Requirement). These two containment relationships are denoted by the name “BelongsTo”.
Each requirement, with the exception of ones at Product level, must be linked to a requirement one abstraction level higher – denoted by relationship Owns/OwnedBy. Any given requirement can also depend on other requirements – as defined by the relationship DependsOn/UsedBy. Furthermore, requirements at the lowest level of abstraction (Component in our case) also have a link to the module in the logical design that implements the requirement.
This abstract syntax defines models that contain requirements grouped in abstraction levels. The constraint that each requirement must link to a requirement at a higher abstraction level is not expressed in the abstract syntax, but is expressed as static semantics – validation rules. This particular design choice is dictated by the need to be able to move requirements between abstraction levels during requirement engineering processes.
The concrete syntax is defined using a set of geometric shapes in MS DSL Toolkit Domain Designer. The concrete syntax is shown as an example gRAM requirements structure presented in the following subsection. The semantic parts of the gRAM are (i) static semantics, in the form of validation rules in order to ensure that the diagram complies with the specifications of RAM and (ii) translational semantics for information interchange with other tools.
The static semantics are validation rules ensuring that the model complies with the RAM. The validation rules in gRAM is written in a general purpose programming language (C#) which allows for the creation of custom complex rule sets.
Although our proposed DSL as shown here is based on the four abstraction levels
suggested in [
The translational semantics of the gRAM allows information exchange with other
tools. The version of gRAM proposed in this paper, three such semantic sets were
defined; ability to export the requirements structure to a format compatible with
IBM/Rational RequisitePro [
The purpose of this example is to illustrate how a set of requirements can be designed using gRAM.
Let us consider a roadside speed surveillance camera, which should be able to identify and take a picture of a speeding vehicle, and from that image identify the vehicle model or type, the registration plate number and extract a cropped image of the drivers face. An excerpt of a simplified requirement structure is shown in Fig. 2.
In the requirements structure, the level a requirement belongs to is automatically assigned to the requirement depending on where it is placed. The tool allows requirements to be dragged and dropped into another abstraction level, with automatic integrity checks. An invalid model cannot be saved and the modeller is asked to correct the problems.
When a valid requirement structure is considered ready for use in an external tool, it is exported to the textual interchange Fig. 4 A generated gRAM requirement (full version) format, which can be used for import into the desired tool. Fig. 3 and Fig. 4 show two Fig. 3 A generated gRAM requirement (short version) versions of a requirement exported to Microsoft Word format (these versions were used in the evaluation reported in the following section).
The purpose of this initial evaluation is to examine how the use of a graphical requirements specification model, such as gRAM, affects change impact assessment. The study examined whether there is any statistically significant impact on the efficiency, with respect to speed and accuracy, of assessing change impact on a specified system.
In the following subsections, the experiment design is briefly outlined and then the results are reported.
The evaluation of the gRAM was conducted through one initial and one replication experiment using the same basic instrumentation. Two groups of subjects were introduced to the requirements specification and logical design of a toy software system. One group was presented with the graphical requirements structure together with a short textual requirements specification, as shown in figure Fig. 2 and Fig. 3, while the other group with a the full textual requirements specification as shown in Fig. 4. Both of these requirements specifications were generated from the same model to ensure consistency. The subjects were then asked to perform a number of change impact assessment tasks, common to both groups, using the provided material.
The following subsections details the experiment design.
The population of this experiment is software designers working with implementation of software requirement specifications and systems analysts creating/maintaining these specifications.
In the initial experiment, the participants were 14 first and second year master students (i.e. in their 4th and 5th year of university studies) attending Software Engineering and Management programme and four 3rd year bachelor students (i.e. their 3rd year of education) from the same programme. Most of the participating master students had over one year of industrial experience prior to their studies.
In the replication experiment, 12 bachelor students, attending the first year of the Software Engineering and Management programme, participated.
In each experiment, the test subjects were introduced to the context of a toy software system by the following scenario:
A vehicle manufacturing company has designed and implemented a simulator for a new type of drive-by-wire power steering system. The intention of the simulator is to provide a realistic environment for engineers to experiment with, e.g. different algorithms for solving certain tasks related to the power steering system. The design of the simulator has the goals of providing a realistic physical environment, a realistic software architecture and easy visualization of the vehicle; in particular, the parts related to steering. The simulator was implemented in Java prior to the experiment and the requirements were traced to the software components of the simulator (via the requirements at the lowest level of abstraction). Two versions of software requirements specifications were prepared – a textual and a gRAM-based one, both generated from the same gRAM model, by using the translational semantics built into gRAM to ensure consistency. The toy system was inspired by the real-world systems our partners work with, and which could not be used due to confidentiality and the complexity of the systems.
The experiment objects were: (i) written experiment instructions, (ii) a requirements specification, and (iii) logical view of the system. An introductory lecture was given to each group separately, with the only difference in contents being the requirements specification format; where one group was presented with the textual RAM format (an example requirement is shown in Fig. 4), while the other with the gRAM format (an excerpt of a graphical example requirements structure is shown in Fig. 2, and an example requirement is shown in Fig. 3). The written experiment instructions and requirements specification differed between the groups in the same way. The requirements specification consisted of 56 requirements, of which 29 were at the lowest (component) level of abstraction.
The logical view, showing a high-level class diagram of the implemented power
steering simulator, was identical for both groups, and consisted of 10 logical modules
and 15 inter-module dependencies. The full set of experiment material is available at
[
The measurements collected for each task include the time taken, the score as a percentage of correct answers, number of false positives and the subjects’ perceived confidence. The following variables were derived from the collected variables for each subject: − AVG_SCORE − TOT_FP − TOT_TIME − AVG_CONF − EFF
The subject’s average score over all tasks (%) The subject’s total number of false positives for all tasks The total amount of time the subject spent on the tasks The average of the subject’s confidence level over all tasks The efficiency of the change impact assessment process, calculated as AVG_SCORE / TOT_TIME The null hypotheses posed are that there is no difference in mean values in the derived variables between the two groups of subjects.
The main threats to the validity of the study are external and conclusion validity, as
described by Wohlin et al. [
The main threat to the external validity is the use of student subjects, which may limit
the ability to generalize the result to an industrial situation. The study was done
mainly to evaluate the impact of the format of the requirements specification, and we
do not make any conclusions about its applicability in an industrial situation yet. An
industrial evaluation of gRAM is planned for the future in the same way as an
industrial evaluation presented in [
The statistical power of the conclusions is quite low due to the small sample size. This threat to validity limits the strength of the conclusions drawn from the study. Rather than stating firm conclusion, we limit ourselves to indications and tendencies.
Testing of the collected data showed that many of the variables did not fit to the normal distribution. Of this reason, non-parametric tests were chosen, which further decreases the statistical power of the result but does avoid the risk of violating assumptions and introducing further threats to the validity of the conclusions.
Variable Mean Std. Dev. Sign. (p)
No (0.424) Table 1 shows the No descriptive statistics (0.351) for the derived Yes variables in the (0.002) experiment. The No results indicate (with (0.501) statistical significance No as shown in Table 1) (0.183) that the gRAM group was 37% faster than Table 1 Descriptive statistics the text group (TOT_TIME). Although not significant, the results also show that the text group scores 12% better (AVG_SCORE) and produce 22% less false positives (TOT_FP) than the gRAM group, indicating that the textual specification improves the accuracy of change impact assessment. A hypothesis for this is that the graphical notation quickly gives a sense of overview and understanding of the structure of the requirements, which in turn leads to the subject being confident more quickly and hence satisfied with the answer more quickly.
This hypothesis was supported by post-experiment interviews with a sample of the test subjects. There is an indication of a difference in the perceived confidence of the answers (TOT_CONF) with slightly higher confidence in the gRAM group, further supporting this hypothesis.
The efficiency (EFF), calculated as score over time, is 44% higher for the gRAM group indicating a great improvement in efficiency, considered as the number of correctly given answers per time unit. The difference in efficiency could however not be shown statistically significant. Fig. 5 and Fig. 6 show the boxplots of the time and efficiency variables, where group 1 was provided with the textual requirements specification and group 2 with the graphical version.
One of the more prominent problems with realizing the visions of model-driven
development is limited traceability between requirements and design models [
Our preliminary conclusions from using the gRAM are that it makes it easier to develop a requirement specification, provides means for automatic verification of the specifications, and provides more flexibility in the process of creating the specification. The visualization of requirements gives a clear overview of the requirements structure and allows for direct manipulation and on-screen feedback of the implications of the manipulations (e.g. automatically changing the requirement properties when moving the requirement between abstraction levels), thus making graphical requirements modelling a more powerful approach than a textual-based one. Using gRAM enables automatic verification of requirements format and structure by validating static semantics as defined by the RAM. Preliminary observations that using gRAM results in more complete and less inconsistent requirement specifications were confirmed during the development of the experiment material reported in this paper, when several inconsistencies in the requirements specification were detected; the inconsistencies had not been spotted when inspecting the text format prior to that. During the development of experiment materials, the gRAM allowed for quick restructuring of requirements in order to try different approaches, allowing for flexibility that is not as easily achieved in a pure textual format.
The goal of information exchange with other established requirement management tools was partly accomplished by the using translational semantics in gRAM, which converts the gRAM requirements model into a textual format understandable by the desired tool (e.g. IBM/Rational Requisite Pro)
We performed an experiment conducted in two sessions to evaluate what effect the use of gRAM has on the efficiency of assessing the impact of a requested change. The conclusions from the evaluation shows that using the gRAM visual requirement structure improves the speed of assessing the impact of a requested change, in our case by 37%.
In our future research, as part of the evaluating this approach a study at a company is planned. The study is expected to discover whether this approach adequately addresses the challenge of producing complete and structurally sound requirements documents in industry.
This research is partially sponsored by VINNOVA under the V-ICT program and the ASIS (Algorithms and Software for Improved Safety) project.