This paper reports on experiences of integrating Agile and Model-Driven Development, for the development of code generators and nancial systems. We evaluate the benets of the Agile MDD approach by comparing Agile non-MDD and Agile MDD developments of code generators, and an agile MDD development of a nancial application with three other independent versions of the same application developed using dierent approaches. We also compare the functionality of the systems and a variety of technical debt metrics measuring the quality of the code and its design. Based on the case study results, we have found evidence that the use of Agile MDD leads to reductions in development eort, and to improvements in software quality and eciency.
Our research goal is to evaluate the impact of integrating Agile development and MDD. For this goal we ask the following research questions: RQ1: What is the impact of integrating Agile development and MDD on the software product? RQ2: What is the impact of integrating Agile development and MDD on the software development process?
In order to answer RQ1, the properties of quality, eciency and
maintainability of the case study versions were compared. For quality and maintainability,
we use measures of technical debt [
We use threshold values based on those used in the PMD technical debt analysis tools (pmd.github.io): EAS 1000 LOC; EHS 100 LOC; EP L 10 parameters; CC 10; EN O 10 per class. 3
As part of the UML-RSDS toolset (www.nms.kcl.ac.uk/kevin.lano/uml2web) a
number of code generators are provided to map UML and OCL to program
code. Translators to Java (3 versions), C# and C++ were manually coded in
Java using an agile approach over the period 2003 to 2015, whilst more
recent code generators, UML2C for ANSI C and UML2Python for Python, have
been developed using our Agile MDD process [
A systematic requirements engineering process was followed for the UML2C and UML2Python generators, involving document mining, scenario analysis, goal decomposition, exploratory and evolutionary prototyping, inspection and refactoring.
For UML2C, the development was organised into 5 iterations, one iteration for each of the following main functional requirements:
Scenario UML element e
C representation e’
Table 2 shows the corresponding mapping for Python. There were 2 iterations with F1.1 and F1.2 considered in the rst, and F1.3, F1.4, F1.5 in the second.
Scenario UML element e
Python representation e’
In contrast to the large eort expended on writing code in the case of the manually-coded code generators, the principal eort in the UML2C and UML2Python developments was the formal specication denition and review/testing of this specication. In UML-RSDS, the construction, review, testing, optimisation, correction and refactoring of the application specication replaces the corresponding code construction processes in a conventional non-MDD approach. No manual coding was used for either UML2C or UML2Python, and a declarative/functional style of specication was used for both.
The formal specication consists of a class diagram and use cases, dened by OCL constraints. An example constraint from UML2C is the following, which expresses formally the informal mapping F1.1.3:
CPointerType->exists( p | p.ctypeId = typeId &
CStruct->exists( c | c.name = name & c.ctypeId = name & p.pointsTo = c ) ) The constraints map instances of the UML metamodel into instances of a C metamodel. Additional constraints dene how concrete C text is produced from the C model. In the case of UML2Python, text was directly produced from the UML model.
The design structure of the UML2C synthesiser is an external transformation chain of two transformations: uml2Ca, which produces the app.h header le for the C output, and uml2Cb, which produces the app.c main code le. F1.1 and F1.2 are implemented in uml2Ca (an internal chain of 2 subtransformations), and F1.3, F1.4 and F1.5 are implemented in uml2Cb as a chain of 3 subtransformations. For UML2Python a single uml2py transformation was delivered, with a hierarchical organisation: model2py (implementing F1.1, F1.2, F1.5) is a client to stat2py (F1.4), which is a client of exp2py (F1.3). 3.1
We compare UML2C and UML2Python to the manually-coded C++ code generator. The developer had low experience in C++, C and Python, and hence the three generators are comparable in each requiring signicant background research and experimentation with dierent generation strategies. UML2C was the rst generator to target a procedural language, and presented greater conceptual diculties than previous generators (eg., how to represent inheritance). UML2Python was the rst generator for a dynamically-typed language.
Table 3 compares the overall development eort in person days between the C++, C and Python code generators.
Stage
C++ generator UML2C UML2Python Requirements Elicitation 10 Evaluation/Negotiation 10 Specication 60 Review/Validation 50 Implementation/ 180 Testing A major factor in this dierence is the much simpler and more concise transformation specication of the C and Python code generators (expressed in UMLRSDS) compared to the Java code of the C++ code generator. Not only is the UML2C specication 7 times shorter than the UML to C++ Java code, but the latter is scattered over multiple source les (eg., Attribute.java, Association.java, Entity.java, etc) and mixed with the code of other parts of UML-RSDS, making debugging and maintenance more complex compared to the C translator, which is dened in 2 self-contained specication les. In total, the core code of the UML-RSDS tools is 90,500 lines of Java code, of which approximately 20% (18,100 lines) is the C++ code generator. In contrast the UML2C specication is 874 (uml2Ca) and 1576 (uml2Cb) lines, in total 2450 lines. The UML2Python generator is a single le of 605 lines. The OCL specication of UML2C is highly declarative and corresponds directly to the informal requirements, hence it is easier to understand and modify compared to a programming language implementation. UML2Python reused the structure of the UML2C informal specication, and substantial parts of the UML2C formal specication, with simplications due to the higher level of the Python language compared to C.
Whilst UML2C is explicitly divided into 5 sequential stages, each subdivided into model to model and model to text modules, the C++ generator has a monolithic structure. UML2Python has a hierarchical structure of 3 strongly separated subsystems. Only two design patterns (Iterator and Visitor) are used in the C++ generator, whilst 13 are used to organise UML2C and 5 in UML2Python.
Table 4 summarises the dierences in software quality aspects between the C++ generator and UML2C and UML2Python.
Measure
C++ generator UML2C
UML2Python Size (LOC) 18,100 2,450 605 Abstraction level Low (code) High (specication) High (specication) Software architecture Partial Detailed Detailed Modularity Low (1 module) High (10 modules) Medium (3 modules) Cohesion Low High High Coupling Low Low Low Design patterns 2 13 5 Table 4: Software quality aspects of C++ and C, Python code generators We can also compare the level of design aws or technical debt in the C++ translator and in UML2C and UML2Python. For the C++ translator the data has been calculated using the PMD tool (https://pmd.github.io). For UML2C and UML2Python we used the technical debt metrics built in to UML-RSDS.
Measures of fan-out, duplicate code and coupling between operations are not measured by PMD, so are omitted in the comparison. Table 5 compares the TD gures for UML2C, UML2Python and the C++ generator. For the latter, the gures for EHS/ERS, CC and EPL are produced by dividing the total numbers of awed operations in the entire UML-RSDS toolset by 5. For EAS and ENO+ENR we count the number of classes (C++ generator) or subtransformations (UML2C and UML2Python) with these aws. It can be seen that the gures for UML2C and UML2Python are generally lower than for the C++ translator. We also compared the eciency of the code generators applied to example models of varying size up to 100 classes with 100 features in each class. 4
In this case study we compared the Agile MDD approach with three other
approaches: MDD, Agile, and hand-coded approaches. The four applications all
implemented the same problem and were completed independently using
dierent development approaches by dierent teams. The case study problem is to
calculate and evaluate the risk of nancial instruments known as Collateralized
Debt Obligation s (CDOs) [
To evaluate eciency, we measured the total execution time of each application version on a sample dataset input of 16 sectors and output of P(S = s) for all values of s 20.
In order to answer RQ2, we used qualitative measures through an opinion survey, in which the developers were asked to report the main benets they perceived, along with the issues they faced in each approach.
Agile MDD approach The CDO application was implemented using the
UMLRSDS Agile MDD approach [
Original CDO application The CDO application was previously developed in C++ and used in a nancial company. It was developed using a traditional code-centric non-agile approach. The developer had over 20 years experience in C++ programming and was a nancial analyst in that company.
Table 6 presents the execution time and the size in LOC of the four application versions
Agile MDD MDD Agile
Original
Direct communication with the customer representative during development was very important in reducing errors and correcting misunderstandings in the Agile and Agile MDD cases. The major challenge for the developers of the Agile MDD, MDD and Agile versions was in understanding the domain and formalising the required functionality correctly.
For the code generator versions, there were no signicant dierences in eciency between the Agile MDD C and Python code generators and the Agile non-MDD C++ code generator. In terms of product quality, there were clear gains in reduced application size and reduced technical debt, and improved system structuring. In turn, these benets make the ongoing maintenance of the UML2C and UML2Python code generators more eective than for the manually-coded generators. For example, we recently implemented a source metamodel enhancement within 30 minutes for each of these generators, the corresponding change for the C++ generator was around 2 hours. Lessons learnt from the UML2C development were applied to UML2Python, resulting in a much reduced development eort (5% of the UML2C eort) in the case of UML2Python. Another factor here is that the language gap between UML and Python is much smaller than between UML and C (however the gap between UML and C is higher than between UML and C++). Substantial reuse of the UML2C specication in UML2Python was facilitated by the concise and semantically-transparent nature of this specication. Table 8 summarises the dierence in TD levels between the three code generator versions.
Transformation aws aws/LOC
Although the aw density is lower for the non-MDD approach, the number of aws is much higher. The overall functionality of the C++ and C code generators is comparable, yet the C++ generator has 3 times the number of aws of the C generator. For the nancial case study, in terms of eciency, the application developed using Agile MDD was the fastest. With regard to the product quality, the Agile MDD application had consistently better metrics than the other versions.
Table 9 and Table 10 present the average of the values of the metrics for the two Agile versus the two non-Agile approaches and for the two MDD approaches versus the two non-MDD approaches. Agile approaches therefore have better values than non-Agile approaches in 8 of the 9 measures while MDD approaches have better values than non-MDD approaches in 7 of the 9 measures.
Ec. LOC ENO MHS MCC CBR DC aws aws/LOC MDD 73ms 118.5 10 13.5 2.5 12(2) 0.5 3 0.0253 Non-MDD 135ms 162.5 11.5 27.5 6 9(1) 2 4 0.0246
Table 10: MDD versus non-MDD approaches for CDO 5.3
For RQ1, we found improvements in the quality aspects and TD measures for the Agile MDD solutions in both case studies, and in eciency of the nancial case.
For RQ2 we found a 40% reduction in development eort for UML2C compared to the C++ code generator, despite the greater challenges of generating C compared to C++. Specication-level reuse was highly benecial in the UML2Python case, whilst only code-level reuse was possible for the C++ generator.
Errors in deliverables were reduced in the Agile MDD and Agile versions of the CDO case due to direct customer collaboration in development. In both the code generator and CDO case studies, the use of incremental development and iterations helped in the organisation and decomposition of the Agile MDD developments. 5.4
A signicant issue concerns the development team size. Agile methods emphasise communication and team collaboration. In this research, the code generator application versions and MDD and original CDO versions were implemented by single developers, whilst the Agile MDD and Agile CDO versions involved a main developer and an advising customer representative. Therefore it is dicult to generalise our conclusions for larger projects.
Secondly, only one form of Agile MDD is used (UML-RSDS) and only one form of Agile (Scrum), so our analysis does not necessarily hold for other varieties of these approaches.
Our results in these case studies show some evidence that Agile MDD has improved eciency and quality of delivered software. In future work, we intend to extend this study using larger case studies with larger development teams.