=Paper=
{{Paper
|id=Vol-2019/edusymp_5
|storemode=property
|title=Influence of Programming Style in Transformation Bad Smells: Mining of ETL Repositories
|pdfUrl=https://ceur-ws.org/Vol-2019/edusymp_5.pdf
|volume=Vol-2019
|authors=Nicolás Bonet,Kelly Garcés,Rubby Casallas,María Elsa Correal,Ran Wei
|dblpUrl=https://dblp.org/rec/conf/models/BonetGCCW17
}}
==Influence of Programming Style in Transformation Bad Smells: Mining of ETL Repositories==
https://ceur-ws.org/Vol-2019/edusymp_5.pdf
Influence of programming style in transformation
bad smells: Mining of ETL Repositories
Nicolás Bonet∗ , Kelly Garcés∗ , Rubby Casallas∗ , Marı́a Elsa Correal† , and Ran Wei‡
∗ Universidad de los Andes, Bogota D.C. Colombia, School of Engineering.
Department of Systems and Computing Engineering
{n.bonet2476, kj.garces971, rcasalla}@uniandes.edu.co
† Universidad de los Andes, Bogota D.C. Colombia, School of Engineering.
Departament of Industrial Engineering
{ mcorreal}@uniandes.edu.co
‡ The University of York. UK. Department of Computer Science
{ran.wei}@cs.york.ac.uk
Abstract—Bad smells affect maintainability and performance We have identified several studies [1]–[6] aiming to find a
of model-to-model transformations. A number of studies have de- relation between traditional software quality attributes and
fined a set of transformation bad smells, and proposed techniques M2M transformations, in effort to determine which aspects
to recognize and —according to their complexity— fix them in
a (semi-)automated way. In education, it is necessary to make of the languages have a direct impact on these attributes.
students aware of this subject and provide them with guidelines to Additionally, we have found research that studies how bad
improve the quality of their transformations. This paper presents smells (bad software practices) can affect the maintainability
some common bad smells in model transformations written by and performance of transformations [7]–[11]. Most of the
master students from Universidad de los Andes and compares available studies target transformations written in ATL, but
them with that of publicly available repositories of ETL trans-
formations, for the purpose of knowing whether programming a few tackle other languages such as Epsilon Transformation
style affects the incidence of smells. Three contributions are Language (ETL)1 , QVT and Henshin.
presented: i) Two new bad smell patterns enriching the existing To the best of our knowledge, there has been no research
catalogs; ii) A process that includes the automated extraction
of transformation metrics and bad smells metrics from the
that studies how coding styles (either imperative or declarative)
repositories, and a statistical analysis that helps in identifying may affect the number of bad smells found in transformations.
the relations between such metrics; and iii) A tool that supports Thus, we propose an approach (Section II) that: i) statically
the process. By applying our approach on the datasets, we discuss analyzes repositories of transformations; ii) extracts transfor-
whether it is easier for students with imperative programming mation metrics (metrics of the language constructs); iii) detects
language background to make use of appropriate declarative
constructs of a transformation language compared to imperative
bad smells based on a pre-defined catalog; and iv) establishes a
ones. We conclude that students must be encouraged and guided relation between these two kind-of metrics, based on statistical
to use declarative constructs whereas possible when developing techniques (Section IV). Other contributions of our work are:
declarative transformations, that results in artifacts that are more two new bad smells that enrich the existing catalog found in
maintainable and with a better performance. the literature and a tool that operationalizes the process and is
Index Terms—Model-Driven Engineering, Model Transfor-
mation, Quality, Metric, Bad Smells, Epsilon Transformation
built on top of the Epsilon languages and Haetae2 .
Language (ETL), Educational Purpose. We have applied the approach to a case study (Section
III) that includes ETL transformations developed by students
I. I NTRODUCTION from Universidad de los Andes and by other stakeholders (the
Model-Driven Engineering (MDE) has gained some popu- contributors of ETL from University of York and Github users)
larity in industry because companies and developers are begin- for the sake of comparison. From the results (Section V), we
ning to adopt MDE approaches to reduce software complexity were able to identify the quality of the transformations made
and improve its maintainability. In this context, universities by our students, and code styles to increase such a quality.
play a major role since they teach MDE to students, who Section VI introduces the implications that the analysis has
become software developers after they graduate. It is essential for instructors and students, and summaries future work.
to teach and students about the best practice of MDE for them
to become better software developers/architects in the future.
1 Epsilon Transformation Language (ETL). URL: https://www.eclipse.org/
Whilst MDE becomes popular, it has pointed out that
epsilon/doc/etl/. July 13, 2017.
measuring the quality of model management programs, es- 2 Epsilon Labs, haetae. URL: https://github.com/epsilonlabs/haetae. July 13,
pecially model-to-model (M2M) transformations, is crucial. 2017.
II. M INING ETL R EPOSITORIES : P ROCESS OVERVIEW in reverse and forward MDE projects. This dataset is publicly
The repository mining consists of three steps. In the first available4 and can be used for further investigation. In the
step, a set of transformation metrics are calculated and bad York dataset, transformations are built by developers with
smells are identified on an available corpus of files. In the vast experience in ETL, they were retrieved from the official
second step, a report is automatically generated discriminating Epsilon Website.
the information obtained for each transformation of the corpus. Finally, the transformations of the Github dataset are the
Finally, the collected data is analyzed to identify relations ones found in public GitHub repositories. The metamodels
between transformation metrics and bad smells to advise involved in these transformations are unknown as well as the
instructors about which could be the weakest points with experience of developers. With respect to the datasets’ size,
respect to quality on their students work. In the remaining Uniandes has 191 files and 27526 LOC, York 23 files and
sections of this paper, we present how this process has been 1550 LOC, and Github 72 files and 26948 LOC.
instantiated in a case study.
IV. BAD SMELLS DETECTION
III. T HE CASE STUDY SETUP
A. Catalog of bad smells
A. Case study context
We followed 4 steps to build our bad smells catalog: i) Re-
This study is part of a research which aims at improving
view the literature that categorizes transformation bad smells,
the programming skills of masters-degree students that attend
i.e., [7], [8], [13] ; ii) Adapt bad smells in related works to
MDE lectures, a vast majority of the students (20 in average
ETL, since there are few differences between ETL constructs
each semester since 2014) have no experiences on MDE. This
and the ones from other M2M transformation languages (e.g.
lecture has been part of the MSc Software Engineering course
ATL); iii) Perform code review of the dataset to identify
and we are constantly looking for better ways of improving the
new smells out of the scope of previous work; iv) Build a
teaching methodologies to increase the quality of the software
consolidated catalog.
produced by students. In this context, the following research
Reviewing previous catalogs of bad smells [7], [8], [13]
questions were asked:
generates 27 bad smells (overlapped smells are omitted). In our
• RQ1: What is the distribution of bad smells across
work, we consider 6 of these previously categorized bad smells
datasets? and contribute 2 new bad smells observed in our datasets. The
• RQ2: What is the variety of code smells in the model
reason for including just 6 of 27 is that these can be discovered
transformations involved in the case study? via static analysis, in a fully automatic way. In contrast, for the
• RQ3: How the total number of bad smells are influenced
discovery of the rest of bad smells, it is necessary to have more
by programming style (i.e., declarative or imperative)? information such as execution traces (e.g., that indicate when
The transformation metrics used as the base line of the an operation is computational intensive) or some knowledge
study are the following: the number of: matched/lazy rules, about the transformation context (e.g., to know if rules have
operations with/without context, calls to lazy rules per rule, meaningful names).
calls to operations per rule, variables per rule, if, loops, The 8 bad smells of our catalog can be classified into
unused operations and unused parameters3 . two categories, defined in [7]: i) Restructuring (these bad
These metrics were chosen as we wanted to identify how smells are related with how the transformation is built and
the transformation strategy was relevant to the amount of bad structured); and ii) OCL Optimization (As ETL is an OCL-
smells that could be found on the dataset. Based on [12] and based language, optimization of those queries is extremely
our own experience, we have identified that transformations important to improve the transformation performance). For
developed using an imperative programming style tend to each bad smell we give an abbreviation, a description, and
majorly use operations, if, and loops. This also involves how it is discovered. In addition, for existing bad smells we
transformations that have few matched rules, but the body of mention the identifier managed in the original source [7] for
those rules is built using plenty of imperative constructs. On cross-referencing purposes, and for new bad smells we give
the other hand, transformations that contain matched rules, an example. We summarized bad smells taken from previous
guards, and OCL queries show a more declarative approach. work in a Table available in this link5 , this section focuses on
B. Characteristics of the dataset new smells.
To perform the study we extracted a total of 286 ETL B. New bad smells
transformations from multiple sources, they were grouped in
three different datasets referred to as Uniandes, York, and 1) Restructuring: Imperative filtering:
Github. In the Uniandes dataset, the transformations are used • Abbreviation: IEO.
• Description: A for statement has an if to prevent the
3 Both matched and lazy rules specify the way in which target model
elements must be generated from source model elements. The execution of
logic being applied to all elements, this is less efficient
matched rule is scheduled by the engine depending on the type of elements
that are applicable, whereas the execution of lazy rules is determined by the 4 GITHUB, ETLMetrics. URL: https://github.com/NicolasBonet/
programmer. ETLMetrics/. July 13, 2017.
than an OCL first-order logic expression and is also to as TOC (trivial operations called once) and REB (rule body
harder to understand. is embedded into if blocks) were the less common ones, each
• Discover Algorithm: The discoverer checks each for one with less than 60 occurrences. When the data is interpreted
statement made on the transformation and if the for logic dataset by dataset, it can be seen that for Uniandes the error
is wrapped inside an if, then it increases the count for DOE (duplicated and complex OCL expressions) is the most
this bad smell. common one. For York, the most common smell is TOC (trivial
6
• Example: An example is available at this link . operations called once). Note that the discoverer allows one
2) OCL Optimization: Imperative element creation in to configure the number of lines that an operation should have
loops: to be considered are trivial. For this experiment, we set this
• Abbreviation: CNL.
threshold to 1-3 lines based on what we have observed in the
• Description: Creating elements inside loops are discour-
developed code. Finally, in Github the occurrences are quite
aged as the creation of multiple elements can be achieved similar to the overall result.
by rules, the use of guards is useful to prevent it from C. Influence of programming style on bad smells incidence
happening over all elements if necessary.
• Discovery Algorithm: The discoverer checks each loop
RQ3: How the total number of bad smells are influenced
statement (i.e., while and for) and verifies if at least by the programming style (declarative or imperative)?. To
one occurrence of the reserved word ”new” —which is answer this question we made a Negative Binomial Regression
intended to create new elements— is found in the block. between the transformation metrics defined in Section III
• Example: An example is available at this link .
6 (matched/lazy rules, operations, etc.) and bad smells metrics.
This regression helps us to measure the influence of a particu-
V. S TATISTICS AND DATA ANALYSIS lar construct on the occurrence-rate of bad smells (i.e., λt ) by
In this chapter, we present the results of our case study means of a Relative Risk (RR). In our context, a RR is a way to
taking into account the stated research questions. compare occurrence-rate of bad smells in transformations that
uses a certain language construct with that of transformations
A. Distribution of bad smells not using it. We calculated RR values for each construct, one at
RQ1: What is the distribution of bad smells across the a time, based on a Negative Binomial Regression Model. Since
datasets?. We found that 118 out of 286, or 41.26% of the RR is a quotient, an RR of 1.20 for a construct means that
transformations, have no bad smells, while 13.64% present using that particular construct increases bad smells by 20%,
more than 10 occurrences. We also established the occurrence- while a RR of 1 means that the construct has not influence in
rate of bad smells by transformation (λt ), which is defined as bad smells. The influence is assessed testing if RR is equal to
follows: PI 1, significantly.
badSmells(i) We applied the statistical model to metrics coming from
λt = i=0 i ; I = T ransf ormations
the three datasets —as a whole set— and from each dataset,
We saw that Uniandes presents rates lower than Github and
separately. Table I shows the results for the entire set and
higher than York. From these results, and taking into account
they are similar to the results of each particular dataset. The
that the main promoters of ETL are at the University of York,
table consists of the following three columns: the language
it is worth to define patterns that favor quality in conjunction
construct, RR and significant effect. In fact, it is the latter
with them (Section VI describes three patterns as a starting
that indicates a strong relation between the number of bad
point).
smells and the use of variables per rules, control statements
B. Variety of bad smells like while, for, if and calls to operations per rule. There is
RQ2: What is the variety of smells found in the transfor- a trade-off between the transformations quality and the time
mations involved in the case study?. The discoverer detected to produce a fully functional MDE code generator. For the
the occurrences of the bad smells in the datasets and the students, it is easier to use an imperative style (which is
results follow: There are three types of bad smells with the the their background programming style) than a declarative
highest occurrences across all datasets (see Table7 for a brief approach that they do not manage to master in 4 months
list of these smells): i) CSF (a chain of select/first in OCL (duration of the course).
is less efficient than using selectFirst); ii) IEO (if statements Table I also shows a strong relation between unused pa-
embedded into a for are less efficient than OCL filters), and iii) rameters/operations and bad smells, this is due to the project
CNL (creation of new elements in loops). Each of these bad of the course is developed in an incremental way and the
smells with more than 200 occurrences found in the entire set students were going through the learning curve. Therefore,
of transformations. On the other hand, the bad smells referred early versions of the transformation includes operations and
parameters that become unused in the subsequent versions,
5 GITHUB, Bad smell catalog. URL: https://github.com/phillipus85/ however, these operations remain in the code resulting an
ETLMetrics/blob/master/EduSymp2017/resources/Catalog.pdf. July 13, 2017. increase of bad smells which hampers maintainability.
6 GITHUB, IEO and CNL code. URL: https://github.com/phillipus85/
ETLMetrics/blob/master/EduSymp2017/resources/IEOandCNL.pdf. July 13, We have the following metrics which are not related with
2017. bad smells: matched rules, lazy rules, operations without
TABLE I generalized to identify bad smells in transformations written in
I NFLUENCE OF LANGUAGE CONSTRUCTS ON BAD SMELLS other languages such as QvT, ATL, ASF+SDF. Finally, since
Metric IRR Significant Effect most of transformation developers use specific IDEs, it would
Matched rules 1.0093 No be useful to integrate the discoverer to these IDEs in order to
Lazy rules 1.0130 No
Operations With Context 1.1219 Yes
provide feedback to the programmers at implementation time.
Operations Without Context 1.0236 No In addition, it would be useful to integrate the previous work
Ifs 1.0427 Yes made around (semi-)automated refactoring.
Loops 1.1631 Yes
Variables per rule 1.1989 Yes R EFERENCES
Calls to operations per rule 1.0788 Yes
Calls to lazy rules per rule 1.0035 No [1] L. Kapová, T. Goldschmidt, S. Becker, and J. Henss, Evaluating
Unused operations 1.1527 Yes Maintainability with Code Metrics for Model-to-Model Transformations.
Unused parameters 2.4112 Yes Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 151–166.
[2] A. Rentschler, Q. Noorshams, L. Happe, and R. Reussner, Interactive
Visual Analytics for Efficient Maintenance of Model Transformations.
Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 141–157.
context, and calls to lazy rules per rule. In the case of matched [3] M. F. van Amstel, C. F. Lange, and M. G. van den Brand, “Using metrics
rules, this should not come as a surprise since transformation for assessing the quality of asf+ sdf model transformations,” in Inter-
languages are built to be used with this declarative construct. national Conference on Theory and Practice of Model Transformations.
Springer, 2009, pp. 239–248.
In addition, it was interesting to notice that lazy rules have [4] M. F. van Amstel and M. van den Brand, “Using metrics for assessing
no significant impact on smells either, this could be because the quality of atl model transformations,” in Workshop on Model
lazy rules are being invoked from matched rules and the lazy Transformation with ATL. Springer, 2010, pp. 19–33.
[5] A. Vignaga, “Metrics for measuring atl model transformations, tech
code is not necessarily imperative. The latter is necessary report,” 2009.
in some contexts, for example, when an input element has [6] M. F. Van Amstel and M. G. J. Van Den Brand, “Model transformation
to be transformed into several output elements that share no analysis: Staying ahead of the maintenance nightmare,” in Proceedings
of the 4th International Conference on Theory and Practice of Model
references. Finally, the influence of OWC (operations without Transformations, ser. ICMT’11. Berlin, Heidelberg: Springer-Verlag,
context) on smell incidence does come as an interesting fact. In 2011, pp. 108–122.
principal, the use of context should lead to a proper invocation [7] M. Wimmer, F. Jouault, and J. Cabot, “A catalogue of refactorings for
model-to-model transformations.” Journal of Object Technology, vol. 11,
of the operation to avoid errors. However, some students abuse no. 2, pp. 2–1, 2012.
this kind-of operation to create output elements, instead of [8] R. Wei and D. S. Kolovos, “Automated analysis, validation and
using matched/lazy rules. suboptimal code detection in model management programs.” in Big-
MDE@STAF, ser. CEUR Workshop Proceedings, vol. 1206. CEUR-
VI. C ONCLUSIONS AND FUTURE WORK WS.org, 2014, pp. 48–57.
[9] M. Tichy, C. Krause, and G. Liebel, “Detecting performance bad smells
It is important that instructors sensitize students about tak- for henshin model transformations.” in AMT@MoDELS, ser. CEUR
ing into consideration not only transformations functionality Workshop Proceedings, vol. 1077. CEUR-WS.org, 2013.
[10] B. Alkhazi, T. Ruas, M. Kessentini, M. Wimmer, and W. I. Grosky,
but also the quality of transformations for maintenance and “Automated refactoring of atl model transformations: A search-based
performance whilst they develope model management pro- approach,” in Proceedings of MODELS ’16. New York, NY, USA:
grams. This work gives empirical evidence that a declarative ACM, 2016, pp. 295–304.
[11] J. S. Cuadrado, E. Guerra, and J. de Lara, “Static analysis of model
programming style influences positively the quality because transformations,” IEEE Transactions on Software Engineering, vol. PP,
the transformations are less bad-smell-prone. In this context, no. 99, pp. 1–1, 2016.
instructors should present the catalog of bad smells to stu- [12] J. Di Rocco, D. Di Ruscio, L. Iovino, and A. Pierantonio, “Mining
correlations of atl model transformation and metamodel metrics,” in
dents and explain them how the imperative style increase Proceedings of the Seventh International Workshop on Modeling in
the probability of smells occurrences in code. In addition, Software Engineering, ser. MiSE ’15. Piscataway, NJ, USA: IEEE
instructors should consider the quality in the evaluation criteria Press, 2015, pp. 54–59.
[13] J. S. Cuadrado, F. Jouault, J. G. Molina, and J. Bézivin, “Optimization
of practical work to motivate students to avoid bad smells. patterns for ocl-based model transformations,” in International Confer-
Finally, the following concrete tips can be given to students: ence on Model Driven Engineering Languages and Systems. Springer,
i) Favor lazy rules compared to operations if matched rules do 2008, pp. 273–284.
not allow to express certain mappings (e.g., a source element
that have to be transformed to different target elements that
share no references); ii) Prefer guards and declarative rules
instead of if, loops, and creation of new elements in the
statement block of rules; and iii) Keep the code updated and
avoid dead code (such as unused operations/parameters).
The proposed discoverer is able to found occurrences of 8
types of bad smells. It would be worth to extend our discoverer
to cover the remaining 21 bad smells categorized in the state-
of-art [7], [8], [13]. In addition, our discoverer could also be
7 GITHUB, Bad smell metrics. URL: https://github.com/phillipus85/
ETLMetrics/blob/master/EduSymp2017/resources/Metrics.pdf. July 13, 2017.