Since the launch of the Agile Manifesto there has been numerous propositions in how to combine agile practices with Model-driven Development. Ideally the combination would give the bene ts of agile { e.g. rapid response to changes and shorter lead times { with the promises of Model-driven development { such as high-level designs and automation. A commonality among the proposals is that they lack in empirical evaluation. Our contribution is a systematic literature review to nd out to what experiences there are of Model-driven Agile Development, MAD, from an empirical context. Among our conslusions is that MAD is sitll an immature research area and that more experience reports from industry are needed before we can claim to have understood the possibilities and drawbacks of MAD.
Model-driven Agile Development, MAD, which aims to combine the bene ts
of agile practices with the positives of Model-driven development, MDD, has
been a dream for many years. Mellor et. al. [
To see what empirical evidence there is for MAD we decided to conduct a
systematic literature review [
The rest of our contribution is structured as follows; in the next section we will explain our methodology, including the identi cation and analysis of relevant publications. In section 3 we present our ndings based on the papers that passed both our inclusion and exclusion criteria as well as the quality criteria. Section 4 synthesises the ndings in relation to the research questions while threats to validity are given in section 5. Finally, in section 6 we conclude and propose future work. 2
Following the guidelines presented by Kitchenham et al. [
We used three di erent digital libraries { IEEE explore, ACM digital library and the SpringerLink library applying the search string (\agile" AND \model") to the listed titles. Publications explicitly mentioning MAD in an empirical setting where included while publications discussing the combination of agile development and MDD as a theoretical contribution were excluded. Only contributions published after 2001, after the Agile manifesto was launched, were included while those not written in English were excluded. If a publication was published in multiple forms we selected the most comprehensive one, meaning that journals had precedence over book chapters, chapters over conference preceedings which in turn had precedence over workshops. Only peer reviewed publications were considered.
The publications were then included or excluded in a three-step process.
First, titles and abstracts were analysed according to our inclusion/exclusion
criteria. A total of 291 publications were found { 86 publications originated
from Springerlink, ACM Digital Library yielded 84 publications while IEEE
Xplore contained 121 publications. Second, the introduction and conclusion
sections of the included papers were studied to further re ne the selection. This
resulted in 78 included publications. Third, the criteria were applied to the full
publications. After removing eleven duplicates and ve contributions that were
published at multiple venues we ended up with thirteen papers that mention
empirically documented cases of MAD. We then went through all references in the
thirteen included papers, without nding any secondary studies that matched
our inclusion and exclusion criteria. The included publications are presented in
Table 1 together with their respective bibliographic reference number and type
of publication venue.
To ensure that we can get quality data from the selected papers we need su
cient contextual and methodological information [
In this section we present our ndings based on the seven publications that
passed both the inclusion, exclusion and quality criteria. The focus in this section
is on the strategies that were employed, the challenges in adopting MAD and
As seen in Table 3, publication P9 deploys Scrum, Model-Driven Architecture
(MDA [
Publication P1 is interesting in two di erent comparisons; First, P1 and P13 have both applied MAD within the telecom industry { which is not surprising since Zhang is an author of both publications. However, the publications di er in their motivations for MAD as well as how MAD was implemented in terms of agile and MDD practices. The second comparison is between publications P1 and P3 which are similar in context { aim and chosen practices { but not in domain. From the publications that passed the three sets of inclusion, exclusion and quality criteria there is no common theme in how and why MAD was adapted in relation to contextual factors. 3.2
The strategy used to achieve agile model-driven development di ers among the selected publications. P9 and P11 suggest to build mockup models, as a mean of communication and for easy requirement gathering and handling requirement changes. P11 promotes an agile way of building models, with speci c tools to easily transform the models under development. P1 suggests using iterations for MDD and P5 wants sprints and iterations but to still have a high level design. P13's strategy is to have both agility and quality built into their development process and P3 suggests a modi ed agile method and presents a tailored approach to address the need of managing evolution using model-based techniques. P11 and P13 use a Test-driven development approach combined with modeling. Our ndings tell us that the common trend among the reviewed papers is to include a more agile way into existing model-driven engineering practices. Especially because of agile development's advantages regarding rapid response to change and close stakeholder involvement { a claim of MDD but where the included publications see agile more successful.
The most interesting and comprehensive strategy we found is in P1. They use a methodology called System Level Agile Process (SLAP). SLAP is a Scrumbased agile methodology, constructed by Motorola that includes XP practices. In SLAP the software lifecycle is split into short iterations, where each iteration includes three sprints { requirements, architecture and development { and then system integration feature testing. They conclude that \From MDD perspective the key to success is to maximize automation using the MDD tools chain to enable mistake-free (high-quality) development and signi cant productivity increase" and \From the Agile perspective, the key is to e ciently achieve endto-end iterations, from system engineering all the way to system testing. This requires streamlining di erent process activities such as system engineering, development, and testing". 3.3
Zhang et al. state in P1 that MAD \is still relatively new in real software development. The learning curve is sharp for any new organization to adopt due to process, culture, methodology, and other related changes. Thus, adopting a new agile MDD process is not likely to produce a short-term bene t. But, for the long-term, it's ultimately worth it for large projects with multiple releases". P12 also mentions the steep learning curve, this time for web modelling tools.
To provide lightweight-agility in development seems to be a solution for Kulkarni et al. who in P3 argues that traditional agile development is not suited for larger teams or projects. 3.4
When it comes to impact, publications P1, P2, P3 and P9 state that they successfully combined agile practices and MDD. P1 and P2 argue that the combination could be useful for future development teams, due to the fact that the learning curve is steep it is not likely to produce short-time bene ts. P2 and P13 noticed an increase in commitment, quality and productivity from the developers { despite having di erent aims and strategies { while P11 report on improvements regarding both time and satisfaction. P5 state that their approach would not be e ective on large projects while P13 states that their approach could be benecial and possible for any-size projects. Still, the overall assessment is that the authors fail to provide detailed descriptions on what was successful and why. 4
The data drawn from seven publications can now be further re ned into a synthesis answering the research questions that motivated our systematic literature review. 4.1
Based on our ndings, we can conclude that the area around MAD is immature when it comes to empirical evidence and industrial experiences. To answer this research question, it is not enough to look at the theoretical part, we need evidence to prove any best practice or state of the art, though the evidence seems to be lacking. The strategies used among the papers we found is in many cases also contradictory to each other { providing information about di erent domains, goals and strategies, making it di cult to claim any best practice or success over someone else. Though the most common goals seem to be the agile value of rapid response to change and contact to external stakeholders. Another commonality is that a majority of the included publications want to include agility into their MDD practices { and not extend agile practices with MDD features { a situation that is seen in other areas since agile is the trend in current software development.
Only when the area is more mature will it be possible to claim a state of the art regarding the combinations of agile practices and MDD. Though claiming that the area is not yet mature does not mean it is not possible to adapt. As Zhang and Patel say in P1, MAD requires investments in both learning and technology and the success will not be immediate. Other authors such as Lee et al. argue in P5 that a pure agile methodology or a pure MDD approach is not longer enough \we can not apply agile processes to web application development directly. UML is not su cient for modeling the navigation of Web applications". 4.2
After removing articles based on our inclusion and exclusion criteria we only had thirteen papers left mentioning any concrete empirical evidence. This is a rather low value for an systematic literature review, which in turn suggests that the area is still immature. According to the publication venues, this conclusion is supported by the fact that there are only three articles published in journals, suggesting again that the more mature and empirical research contributions regarding MAD are still to be written. The search for secondary studies did not result in any additional publications that matched our inclusion, exclusion nor quality criteria { again suggesting that more research in the area is needed.
There are authors (e.g. Ambler [
Future empirical publications would contribute substantially to our understanding of MAD if they provided information about: Teams/project: What was the size of the team? Which responsibilities did the team members have? Did the team members ahve any earlier experience of agile or MDD? For how long did the project run? Used practices: Which agile and MDD practices were used? Strategy: How were the agile and MDD practices combined? Were the practices easy to combine? Which where the challenges of introducing MAD? Tools: Which tools were used to achieve MAD? Where speci c tools for MAD develop or could o -the-shelf tools be used? Impact: Was MAD a success or a failure? What succeeded/failed and why?
As Ambler himself indicates \The use of Agile methodology in model-driven
development is not prevalent yet, except tailored Agile approaches, such as Agile
model driven development" [
Our results may face validity issues as our search was restricted to ACM digital library, Springerlink and IEEE explore. There are other digital libraries which are widely used in the software engineering eld. However, these three libraries include the journals and conferences with the highest impact factor which should imply that they represent the most mature research. Another threat to the validity is our search string, as we did a title search on the words agile and model, it is possible that we have missed publications discussing the combination of agile and model-driven development while not explicitly mentioning the fact in their title. We tried to mitigate this by searching for secondary studies among the references in our primary studies but after applying our inclusion and exclusion criteria we could not nd any additional publications. This gives us some con dence that we could not have missed many articles in our initial search. Our inclusion and exclusion criteria and our quality assessment can also be a threat to validity as we were searching for actual empirical cases where the authors mention certain key information that we as researchers see as empirical evidence, such as teams, project success, MDD and agile practices among others.
There are many other papers that discuss how and why you should combine the two development methods but they lack the key information, thus not being identi ed as relevant publications for our systematic literature review. The publications not passing our criteria might have important information regarding MAD, but could not be included in our data extraction and synthesis. Also, the publications that passed all three sets of criteria could still contribute data of poor quality as they might pass our criteria but fail to deliver detailed information. 6
In this paper we have presented the results of a systematic literature review about empirical evidence on Model-Driven Agile Development. In the seven papers which passed all our quality criteria, the authors wanted to combine agile methods and MDD in a way that could draw bene ts from both worlds and at the same time avoid their respective shortcomings. In the result section, we have presented di erent integrations of agile and MDD practices for various purposes. There are multiple authors discussing di erent theoretical approaches for MAD but just a handful publications describing detailed information how a company or a team adopted such a practice into their existing context. Six out of the seven publications that passed our quality criteria also fail to deliver comprehensive information how they performed MAD. They barely mention their team-setup, what practices they combined and what tools they used.
Discovering that the empirical contributions regarding MAD are immature gives plenty of opportunities for future research, especially when it comes to detailed experience reports. The empirical experiences regarding MAD are still few and lacking details. We aim to extend this study by including more publication databases and by providing a more comprehensive analysis of the found publications. Such an anlsyis would also seek to map the theoretical proposals to the empirical evidence for MAD.