<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Comparing Reuse Mechanisms for Model Transformation Languages: Design for an Empirical Study</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Daniel Strüber</string-name>
          <email>strueber@uni-koblenz.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Anthony Anjorin</string-name>
          <email>anthony.anjorin@upb.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Universität Koblenz-Landau</institution>
          ,
          <addr-line>Germany and, Philipps-Universität Marburg</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Paderborn</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Reuse mechanisms for model transformation languages help avoid duplication, thereby increasing maintainability and enabling a more systematic overall development process. As the introduction of such reuse mechanisms to model transformation languages is still in its preliminary stages, however, language designers are currently faced with the challenge of choosing from amongst diverse proposed approaches. Although there are a few surveys comparing syntactic and semantic di erences, there is still a need for empirical studies on the actual e ectiveness of di erent reuse mechanisms for transformation developers. In this paper, therefore, we present a study design for a controlled experiment to investigate the bene ts and drawbacks of two speci c reuse mechanisms for model transformation languages: rule re nement and variability-based rules. Both mechanisms are tailored to graph-based model transformation languages, yet represent two contrasting reuse paradigms: modularizing rules by composing them from smaller, shared fragments, versus maintaining a single, integrated representation via variability annotations. We propose to compare these two approaches using comprehension and bug- nding tasks to investigate understandability, and bug- xing and modi cation tasks to study changeability.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. INTRODUCTION</title>
      <p>
        Model transformations play a fundamental role in
ModelDriven Engineering (MDE), where software systems are
produced by the continuous re nement, translation, and
synchronization of models [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ].
      </p>
      <p>
        As the eld matures and MDE approaches are faced with
increasingly complex development tasks, the size and
complexity of required model transformations also increases.
Developing and especially maintaining such large
transformations is challenging without dedicated support for reusing
parts of existing transformations [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. As such reuse
mechanisms for model transformation languages are just starting
to emerge [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], however, transformation developers are often
forced to resort to code or pattern duplication, well-known
for its manifold maintainability-related drawbacks.
      </p>
      <p>
        But how can language designers decide which reuse
mechanisms to integrate in their model transformation
languages? While some surveys exist comparing syntactic and
semantic di erences of various proposed reuse mechanisms
This research was partially supported by the research
project Visual Privacy Management in User Centric Open
Environments (supported by the EU's Horizon 2020
programme, Proposal number: 653642).
(cf., e.g., [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]), there exist no empirical studies of various
transformation reuse mechanisms and their actual
usefulness for transformation developers.
      </p>
      <p>
        In this work, therefore, we present our design for a
controlled experiment that aims to explore the bene ts and
drawbacks of the proposed reuse mechanisms for model
transformation languages. Speci cally, we intend to compare two
recently emerging reuse mechanisms: rule re nement [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] and
variability-based rules [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ]. This choice is motivated by the
fact that both mechanisms address the same technical scope
- the reuse of graph-based model transformations [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] - while
contrasting in the employed reuse paradigm: rule re nement
reuses transformation rules by breaking them into smaller
fragments that can be assembled using a composition
mechanism, while a variability-based rule represents a family of
multiple, similar rules in an integrated form, using
variability annotations to denote individual parts. This distinction
between composition- and annotation-based approaches is
well-known from the domain of software product lines [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
      </p>
      <p>
        We are interested in the in uence of these reuse
mechanisms on maintainability, a main quality aspect of software
artifacts [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] that is particularly important in the face of
continuous development and evolution. More speci cally, our
study targets the maintainability aspects of
understandability and changeability of model transformations [
        <xref ref-type="bibr" rid="ref18 ref26">18, 26</xref>
        ].
Given a group of participants, understandability can be
quanti ed in terms of correctness, completion time, and
perceived e ort during bug- nding and multiple-choice style
comprehension tasks. To measure changeability, we plan to use
bug- xing and modi cation tasks, again using correctness,
completion time, and perceived e ort for quanti cation.
      </p>
      <p>The rest of the paper is structured as follows: Section 2
discusses related work. Section 3 introduces a motivating
running example that is used to explain the two reuse
mechanisms for the comparison. Section 4 presents our study
design with an outline of the experimental variables of the
set-up, while Section 5 concludes.
2.</p>
    </sec>
    <sec id="sec-2">
      <title>RELATED WORK</title>
      <p>
        The planned study is the rst empirical investigation of
transformation reuse mechanisms. Empirical studies in the
area of model transformation have so far been con ned to
the evaluation of transformation languages and approaches.
Acretoaie et. al [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] have compared their usability-oriented
Visual Model Transformation Language to traditional visual
and textual languages. They found that their approach
establishes a compromise between e ort and correctness. In
an evaluation of their transformation-by-example approach,
Avazpour et al. [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] found that developers perceive the
provided interaction-based tool support as useful during the
development of transformations. Guana and Stroulia [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] propose
a study design to understand developer performance during
the development of model-to-code transformations and
measure the e ect of traceability visualizations. Conversely, only
reuse mechanisms for code have been investigated so far,
mostly from a product line perspective [
        <xref ref-type="bibr" rid="ref15 ref8">8, 15</xref>
        ].
applications where possibly hundreds of rules are required,
such pattern duplication can quickly lead to a maintenance
nightmare as changes, e.g., bug xes, have to be consistently
applied to multiple, independent rules without tool support.
3.1
      </p>
    </sec>
    <sec id="sec-3">
      <title>Rule refinement</title>
      <p>
        The basic idea of rule re nement [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] is to enable a exible
sharing and composition of common rule fragments.
Pattern duplication can be avoided in this way by including the
3. MOTIVATING SCENARIO same fragment as a part of multiple rules. Rule re nement
Our motivating scenario (adapted from [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]) is a simpli- can be applied to our running example to result in a network
ed move method refactoring, implemented via three trans- of rule fragments as depicted in Fig. 2. Rule A remains
unformation rules depicted in Fig. 1: The rst rule A transfers changed and is regarded as \resolved" as it does not re ne
ethod(src,trg,ma) method m from its current containing class s to another any other rule fragment (it has no outgoing \inheritance"
class t. This rule is a graph mtroavnesMfoerthmodaAtidodnWrrauplepetrh(sarct,tprgr,ems)er- arrows). Rule B is reduced to a re nement of A, i.e., it is
preserve» «prevseersvea»ll e«lepmreseenrtvse»marked with pres«eprrveese,rdvee»let«epsreaslelrveel»eme«nptrseserve»produced by merging rule A with the speci ed re nement.
fields :Fieldmarked wittyhpedelete, and creates allfieelldemsent:Fsiemldarked withtype This merge operator is relatively straightforward: The
disrve» create. Su«cphrersuerlvees»are on:«lCyplareassspeprvliec»able if all preserved and :«Cplaresssejrovien»t union of both patterns (in general of multiple patterns)
deleted ele m:Celanssts can be matncahmede=sinrc a hos«tdmeleotde»el. namies=tfrogrmed, and then all elements with the same identi er are
e=src Rules B anndamCe=rtergpresent var«iacnretasteo»f A:mBethcordesates a wra- glued together. Similarly, rule C is speci ed as a re nement
deetlheoted»s «prespeprveer»meth«mocedrtehawotedf»sor simple delegmaettihoondsto the «mpreetsheerocvtdes»mofththaet«mcertehaoted»ssoof lvBi,nrgesruullteinBgainnd tthheensarmepeeaptaitntgertnheassaimneFmiger1gibnyg
prrsotcerses:Methisodto be moved. This can b:«Mecreeutahsteoedd» to limit the e
      </p>
      <p>:Method
namreef=amctoring and avoid having to change all exniasmtine=gmclients</p>
      <p>name=m
of the class s. Rule C additionally creates an annotation
that marks w as deprecated to discourage future usage of
mthoevewMreathpopdeArddmDeetphreocda,tewdWhircahppcear(nsrcb,terge,vme)ntually deleted.</p>
      <p>with the speci ed re nement for rule C. Note that the merge
operator requires the manual speci cation of identical
elements, which is done by assigning names (e.g., s, me) to
pattern variables, a concept that is not to be confused with
parameters (e.g., src, trg, m), which are used to propagate
values across name attributes in this example.
«preserve»
:Class
name=trg
«create»
methods «create»
:Method
name=m</p>
      <p>«preserve» «preserve»
:«2Cplnareasssme:«AeCrp:v=larmeses»rosscevrev(esr»cf,ietrldg«,spmrf)eies:«leFdprisvereeld»se:«rFvpieerel»dserve» type «preserve»
«crneaamtee»=src «delete»
methods «create» methods
annotation «delete»
«create» method«spreserve»
:Annotation :Method
value="Dep" name=:«mMpreetsheordve»</p>
      <p>name=m
«pr:eCslearsvse» «preserve»
typneame=trg
:Class
«crneaamtee»=trg
methods
«create»
methods</p>
      <p>B: moveAndCreateDelegate(src, trg, m)
«preserve» 5 «preserve»
:Class :Class
name=trg moveMentahmode=(ssrrcc,trg,m)
«preserve» «preserve»</p>
      <p>fields :Field
«create» «preserve»
methods :Class
««pcrreesaetrev»e» «preserve»
mfeitehldodss :Field
name«=csrrecate»</p>
      <p>:Method
«mcertehanotaedm»se=m
«preserve» «preserve»</p>
      <p>type :Class
deprecate -&gt; wrappenrame=trg</p>
      <p>This simple example illustrates how multiple
transformation rules can \share" a common part that must be repeated
in each variant if no reuse mechanism is provided. In realistic</p>
      <p>
        In stark contrast to rule re nement, variability-based rules
[
        <xref ref-type="bibr" rid="ref21 ref23 ref24">21, 23, 24</xref>
        ] enable an integrated representation of multiple
rules in a single diagram. The variability-based rule for our
running example is depicted in Fig. 3. The rule comprises
two con guration options that are denoted via variability
annotations, e.g., the annotation [wrapper] is used to indicate
which edges and nodes in the diagram are only present when
con guration option wrapper is set to true. Note that
dependencies between con guration options can be speci ed, e.g.,
the dependency deprecate -&gt; wrapper (upper-left corner)
is used to indicate that the con guration option deprecate
requires wrapper, i.e., all elements annotated with wrapper
are also present in the deprecate variant of the rule.
      </p>
      <p>moveMethod(src,trg,m)</p>
      <p>deprecate -&gt; wrapper
«preserve» «preserve» «preserve» «preserve» «preserve»
:Class fields :Field type :Class
name=src name=trg
the in uence of confounding factors (e.g., participant
expertise) is reduced as each participant serves as their own
control. A main threat to this kind of study design are
learning e ects: over the course of the experiments, participants
become more experienced and might carry over gained
experience among tasks. A viable mitigation to this threat is to
vary the order of treatments among participants. We discuss
additional threats and mitigation strategies in Sec. 4.3.</p>
      <p>To discuss the di erent dimensions that in uence our
experiment, we rst outline its variables in the following. We
then specify our research plan as one possible path through
the design space spanned by these variables. Finally, we
describe potential threats to validity in our set-up, and our
means of mitigating these threats.
4.1</p>
    </sec>
    <sec id="sec-4">
      <title>Experimental variables</title>
      <sec id="sec-4-1">
        <title>4.1.1 Independent variables</title>
        <p>As the single independent variable in the planned
experioveMethod(src,trgT,mh)is scenario illustrates a set of maadidnWtarainppaebri(lsirtcy,mt)rade-odesprecate(mme)nextt,ewndes measure the e ects of the employed reuse
mecha«preservteh»at we aim to«perxepselorvree»in our studeyx.tends</p>
        <p>moveMethocdo(nsrsci,dtregr,med) maed-dWrappneirs(smrc.,mF)or RQ1 and RQ3, only the data points of the two
fieldschan:«Fpiisermeldsesrhvee»lp avoitdyppeattern duplicationB, owthhich may simplify proposed reuse mechanisms are considered, whereas RQ2
inpreserve» the consistent editin«gproefsemrveu»ltiple va«rpiraensetrsvea»t once. In ad«-create»corporates the data points for the no-reuse case as well.
Class dition, if applied skil:lCfluasllsy, rule re ne:Cmlaessnt can lead to com:-Method
name=src prehensible and maintnaaimnea=btlreg modulanramrue=lesrcf«rcargematee»nts, re- name=m4.1.2 Dependent variables
«delete» ect«ipnrgesaenrvein»depe«ncdreeantte»handling and separatiomnetohfocdosncerns a«ncnTroehtaaetteivo»anriables we want to observe as a result of the
expemethodisn th:Me ettrhaondsformatmioenth.odVsariability-ba«scerdeartue»les improve nota«-create»riment are the following:</p>
        <p>
          name=m
tional compactness drastically, there:bMyetphootdentially reducin:gAnnotationCorrectness score. We plan to measure correctness
bareading e ort: it is well-known that hnuammea=nms struggle with value="sDeedp"on how well participants are able to solve certain tasks.
processing more than 7 2 units of information at once [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. As these tasks might not be directly comparable, counting
On the downside, as they require developers to derive the the number of correctly solved tasks only provides ordinal
included rule variants in their minds, both approaches may values.
increase cognitive overhead. In particular, variability-based Perceived e ort. Participants will be asked to rate the
rules may su er from their increased amount of informa- perceived e ort it took to solve all tasks using a suitable
tion in each individual diagram, comparable to the case of ordinal scale ranging from, e.g., 0 to 10 to quantify e ort.
variability-annotated code, where a decrease in developer Completion time. For all tasks, completion times will
performance was found [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]. be measured for all participants. This qualitative value can
be contrasted with correctness scores and perceived e ort to
4. STUDY DESIGN yield further insights.
        </p>
        <p>
          User preference. Finally, regardless of correctness,
pertheTofoilnlovwesitniggarteesetahrechideqnuteisteiodntsr:ade-o s further, we propose rceeuivseedmeecohrat,niasnmd tchoemypplerteifoenr atinmdee,nwjoeysehdalulsainskg uthseermswoshti.ch
RQ1: Which reuse mechanism is most bene cial for For statistical analysis, we intend to apply the Wilcoxon
developers in terms of understandability and changea- signed-rank test [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ], which is a robust non-parametric test
bility? to identify signi cant di erences between two treatments. It
assumes data measured at least on an ordinal scale, which
RQ2: How do the studied reuse mechanisms compare applies to all of our dependent variables.
to o ering no reuse at all? A remaining issue concerns the fact that we are interested
RQ3: Can the understandability and changeability of in two di erent quality goals that should ideally be studied
the selected reuse mechanisms be further improved via as separate entities. Yet, as changeability clearly depends
the available tool support? on understandability, it is not obvious how these two di
erent goals can be studied in isolation. To this end, a viable
method is to collect performance data for both goals, and
to subtract the understandability from the changeability
results. For instance, the time required to actually change
the a ected rules can be obtained from subtracting the time
required to understand the rules from the full completion
time. A prerequisite for this method is that subtraction
is supported by the underlying scale, which is the case for
completion times.
        </p>
        <p>To address these research questions, we plan to administer
a set of comprehension and implementation tasks to
participants who are familiar with graph-based model
transformations. Such participants will be recruited from research
groups, as well as master-level model-driven engineering
courses.</p>
        <p>
          We plan to apply a crossover trial, a variant of the
withinsubject design [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]: in this design, all subjects are
sequentially exposed to each treatment. This choice is motivated
by the statistical e ciency of the design: it allows the
number of required participants to be minimized. Moreover,
        </p>
        <p>The design space of our controlled variables comprises the
4.1.3</p>
      </sec>
      <sec id="sec-4-2">
        <title>Controlled variables</title>
        <sec id="sec-4-2-1">
          <title>Independent variable Controlled variables</title>
        </sec>
        <sec id="sec-4-2-2">
          <title>Dependent variables</title>
        </sec>
        <sec id="sec-4-2-3">
          <title>Reuse mechanism Transformation language Example type Task type</title>
          <p>Environment
Correctness score
Perceived e ort
Completion time
User preference</p>
          <p>Possible values
f Rule re nement, variability-based rules, no reuse g
f Henshin, ATL, EOL, ... g
f queries, endogenous, exogenous g
f bug- nding, quiz, bug- xing, modi cation task g
f pen-and-paper, tool g
Ordinal
Ordinal
Interval</p>
          <p>Ordinal
following dimensions:</p>
          <p>
            Transformation language. The variety of available
model transformation languages is usually classi ed along a
number of main paradigms, most prominently imperative,
hybrid, and graph-based model transformations [
            <xref ref-type="bibr" rid="ref16 ref7">7, 16</xref>
            ]. In the
planned study, we aim to investigate reuse in the context
of Henshin [
            <xref ref-type="bibr" rid="ref5">5</xref>
            ], a graph-based language. The reason for this
choice is twofold: rst, examples and tool support for one
of the supported reused mechanisms are readily available in
Henshin (convenience sampling). Second, the graphical
syntax of Henshin closely resembles that of a group of languages
including eMo on [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ], EMorF [
            <xref ref-type="bibr" rid="ref12">12</xref>
            ], and QVTs [
            <xref ref-type="bibr" rid="ref29">29</xref>
            ],
rendering it a promising choice to obtain results which might also
generalize to these languages. In addition, Henshin is under
active development and in widespread use.1 Our study will
be based on bug- nding and bug- xing tasks, since these
tasks can avoid lengthy textual task descriptions that might be
hard to match to the presented visual representations [
            <xref ref-type="bibr" rid="ref27">27</xref>
            ].
Our example rules will mostly be endogenous, such as those
shown in the running example, allowing us to establish a
compromise between external and internal validity.
          </p>
          <p>
            Example types. In the choice of example
transformations, we need to account for the inherent trade-o between
external and internal validity [
            <xref ref-type="bibr" rid="ref20">20</xref>
            ]: by using simple and
easyto-understand examples, we can mitigate the e ect of
participant heterogeneity, but risk ending up with an experiment
that is not realistic enough. The simplest transformation
that can be expressed using rules is a query { a rule
performing no modi cations of the input model. Queries are
generally easy to understand; yet in realistic settings they are
likely to be expressed using dedicated query languages.
Endogenous transformations { transformations that update the
input model or translate it to a model of the same language
{ are a typical use-case for graph-based model
transformation, but require an understanding of its underlying
replacement mechanism. Exogenous transformations, creating an
output model of a di erent language, are most complicated
as they require the understanding of two meta-models and
their relations; yet they are particularly important for the
typical forward-engineering process in model-driven
engineering. Two other sub-dimensions of this dimension are the
number of examples and the size of the individual examples.
          </p>
          <p>We intend to conduct a pilot study to nd a suitable
collection of examples that avoids participant fatigue, using a
di erent set of participants than in the actual study to avoid
1For a selection of publications describing Henshin's use
in numerous research projects, see https://www.eclipse.org/
henshin/publications.php
history e ects. Other goals of the pilot study are to
determine if we can extract any meaningful results (e.g. maybe
all task are too di cult and we do not gather su cient
useful data), and to see if our assumptions about participant's
abilities hold true.</p>
          <p>
            Task types. To study understandability and
changeability, we plan to administer tasks allowing us to quantify these
properties. The understandability of a software artifact can
be investigated using bug- nding tasks or quiz-style
comprehension questions. In the example, a bug- nding task
would be to swap the delete and create edges and have the
user nd and mark this error on a printout. A
comprehension question could be \How many individual rules can be
derived from this representation?". Changeability refers to
the ease of modifying or extending the functionality of a
system. Two possible kinds of tasks to study changeability
are the xing of a preconceived bug, such as the one
described above, and the implementation of a feature request
such as \I need a set of transformation rules with the same
behavior as the given ones, but with one modi cation: the
target class must have a eld whose type is the source class".
A challenge for the study design lies in choosing a feasible
subset of all tasks so as to avoid exhausting the participants.
To this end, we plan to use bug- nding and bug- xing tasks,
since these tasks can avoid lengthy textual task descriptions
that might be hard to match against the presented visual
representations [
            <xref ref-type="bibr" rid="ref27">27</xref>
            ].
          </p>
          <p>
            Environment. Another balancing act between control
and remaining adequately realistic concerns the
experimental environment. By setting up the participants in a
controlled pencil-and-paper environment, we can eliminate
confounding factors related to the tool implementations, for
instance, di erences in visual appearance. Such a controlled
environment is, however, not really comparable to the
realworld environment that actual users of the reuse mechanisms
nd themselves in, especially in the case of changeability
tasks. Moreover, RQ3 can only be studied by comparing the
reuse mechanisms with and without the actual tool support
provided. As of now, tool support for rule re nement [
            <xref ref-type="bibr" rid="ref13">13</xref>
            ]
and VB rules [
            <xref ref-type="bibr" rid="ref25">25</xref>
            ] exists, but is implemented within di erent
model transformation languages and corresponding tool
environments { eMo on and Henshin. To ensure optimal
control during the experimental comparison, both mechanisms
should be implemented for the same language and tool
environment.
4.2
          </p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Quantitative data</title>
      <p>To gain additional insight about the perceived bene ts
and pitfalls of both reuse approaches, we also plan to
collect quantitative data, using text elds in the task sheet to
collect participants' opinions after their completion of the
experiment.
4.3</p>
    </sec>
    <sec id="sec-6">
      <title>Potential threats to validity</title>
      <p>In addition to the threats already described in the
previous discussion, we plan to mitigate researcher bias by
striving to ensure that participants are unable to directly map
reuse mechanisms to researchers, e.g., by anonymising tool
names.</p>
      <p>To avoid selection bias in favor of volunteers, whose level
of motivation does not re ect that of the whole population,
we will o er a small prize among participants.</p>
      <p>Finally, the participants' performance can be confounded
by relevant prior knowledge, e.g., from exercise in the area
of software product lines and other annotation-based model
families. To study the e ect of prior knowledge, we plan to
ask participants to rate their own prior knowledge of relevant
topics.</p>
    </sec>
    <sec id="sec-7">
      <title>CONCLUSION</title>
      <p>In this paper, we present our study design for an
empirical comparison of two reuse mechanisms for graph-based
model transformation languages. The selected reuse
mechanisms exemplify the contrasting reuse paradigms of
modularizing a set of rules and merging similar rules in a
variabilityannotated representation.</p>
      <p>
        Our prior expectations are as follows: we expect re
nements to be (perhaps only slightly) more understandable
but less changeable than variability rules (RQ 1), both
reuse mechanisms to substantially improve changeability but
(at least without substantial training) reduce
understandability (RQ 2), and that appropriate tool support plays a
crucial role in both cases (RQ 3). It remains to be seen if
our experiment results will support or refute these initial
hypotheses. Addressing the trade-o between di erent forms
of experimental validity [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], our set-up aims to establish a
compromise between internal and external validity. A series
of follow-up studies moving the spotlight in either direction
can lead to a more comprehensive picture.
      </p>
      <p>The results of our study will provide evidence to
designers of model transformation languages who are planning
to include a reuse mechanism within their language. In a
broader context, we hope to shed a new light on the
discussion of when and how software reuse can be established in a
manner that is most bene cial to all involved stakeholders.</p>
      <p>Acknowledgements: The authors wish to thank Vlad
Acret,oaie and the anonymous reviewers for their careful
reading of this paper and their detailed comments and
suggestions.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1] ISO/IEC 25010:
          <year>2011</year>
          .
          <article-title>Systems and software engineering { Systems and software Quality Requirements and Evaluation (SQuaRE) { System and software quality models</article-title>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>V.</given-names>
            <surname>Acretoaie</surname>
          </string-name>
          , H. Storrle, and
          <string-name>
            <surname>D.</surname>
          </string-name>
          <article-title>Struber. VMTL: a language for end-user model transformation</article-title>
          .
          <source>Software &amp; Systems Modeling</source>
          , pages
          <volume>1</volume>
          {
          <fpage>29</fpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A.</given-names>
            <surname>Anjorin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Lauder</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Patzina</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Schu</surname>
          </string-name>
          <article-title>rr</article-title>
          .
          <source>eMo on: Leveraging EMF and Professional CASE Tools. Informatik</source>
          ,
          <volume>192</volume>
          :
          <fpage>281</fpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>A.</given-names>
            <surname>Anjorin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Saller</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Lochau</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Schu</surname>
          </string-name>
          <article-title>rr. Modularizing Triple Graph Grammars Using Rule Re nement</article-title>
          . In International Conference on Fundamental Approaches to Software Engineering, pages
          <volume>340</volume>
          {
          <fpage>354</fpage>
          . Springer,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>T.</given-names>
            <surname>Arendt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Biermann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Jurack</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Krause</surname>
          </string-name>
          , and
          <string-name>
            <given-names>G.</given-names>
            <surname>Taentzer</surname>
          </string-name>
          . Henshin:
          <article-title>Advanced Concepts and Tools for In-Place EMF Model Transformations</article-title>
          .
          <source>In International Conference on Model Driven Engineering Languages and Systems</source>
          , pages
          <fpage>121</fpage>
          {
          <fpage>135</fpage>
          . Springer,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>I.</given-names>
            <surname>Avazpour</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Grundy</surname>
          </string-name>
          , and
          <string-name>
            <given-names>L.</given-names>
            <surname>Grunske</surname>
          </string-name>
          .
          <article-title>Specifying model transformations by direct manipulation using concrete visual notations and interactive recommendations</article-title>
          .
          <source>Journal of Visual Languages and Computing</source>
          ,
          <volume>28</volume>
          :
          <fpage>195</fpage>
          {
          <fpage>211</fpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>K.</given-names>
            <surname>Czarnecki</surname>
          </string-name>
          and
          <string-name>
            <given-names>S.</given-names>
            <surname>Helsen</surname>
          </string-name>
          .
          <article-title>Feature-based survey of model transformation approaches</article-title>
          .
          <source>IBM Systems Journal</source>
          ,
          <volume>45</volume>
          (
          <issue>3</issue>
          ):
          <volume>621</volume>
          {
          <fpage>645</fpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>J.</given-names>
            <surname>Feigenspan</surname>
          </string-name>
          , C. Kastner, S. Apel,
          <string-name>
            <given-names>J.</given-names>
            <surname>Liebig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Schulze</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Dachselt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Papendieck</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Leich</surname>
          </string-name>
          , and
          <string-name>
            <given-names>G.</given-names>
            <surname>Saake</surname>
          </string-name>
          .
          <article-title>Do background colors improve program comprehension in the #ifdef hell? Empirical Software Engineering</article-title>
          ,
          <volume>18</volume>
          (
          <issue>4</issue>
          ):
          <volume>699</volume>
          {
          <fpage>745</fpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>V.</given-names>
            <surname>Guana</surname>
          </string-name>
          and
          <string-name>
            <given-names>E.</given-names>
            <surname>Stroulia</surname>
          </string-name>
          .
          <article-title>How Do Developers Solve Software-engineering Tasks on Model-based Code Generators? An Empirical Study Design</article-title>
          . In First International Workshop on Human Factors in Modeling, pages
          <volume>33</volume>
          {
          <fpage>38</fpage>
          . CEUR-WS.org,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>B.</given-names>
            <surname>Jones</surname>
          </string-name>
          and
          <string-name>
            <given-names>M. G.</given-names>
            <surname>Kenward</surname>
          </string-name>
          .
          <article-title>Design and analysis of cross-over trials</article-title>
          . CRC Press,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>C.</given-names>
            <surname>Ka</surname>
          </string-name>
          stner, S. Apel, and
          <string-name>
            <given-names>M.</given-names>
            <surname>Kuhlemann</surname>
          </string-name>
          .
          <article-title>Granularity in software product lines</article-title>
          .
          <source>In International Conference on Software Engineering</source>
          , pages
          <volume>311</volume>
          {
          <fpage>320</fpage>
          . ACM,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>L.</given-names>
            <surname>Klassen</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Wagner</surname>
          </string-name>
          .
          <article-title>EMorF-A tool for model transformations</article-title>
          .
          <source>Electronic Communications of the EASST</source>
          ,
          <volume>54</volume>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>G.</given-names>
            <surname>Kulcsar</surname>
          </string-name>
          , E. Leblebici,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Anjorin</surname>
          </string-name>
          .
          <article-title>A Solution to the FIXML Case Study Using Triple Graph Grammars and eMo on</article-title>
          .
          <source>In Transformation Tool Contest</source>
          , pages
          <volume>71</volume>
          {
          <fpage>75</fpage>
          . CEUR-WS.org,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>A.</given-names>
            <surname>Kusel</surname>
          </string-name>
          , J. Schonbock, M. Wimmer, G. Kappel,
          <string-name>
            <given-names>W.</given-names>
            <surname>Retschitzegger</surname>
          </string-name>
          , and
          <string-name>
            <given-names>W.</given-names>
            <surname>Schwinger</surname>
          </string-name>
          .
          <article-title>Reuse in model-to-model transformation languages: are we there yet?</article-title>
          <source>Software &amp; Systems Modeling</source>
          ,
          <volume>14</volume>
          (
          <issue>2</issue>
          ):
          <volume>537</volume>
          {
          <fpage>572</fpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>J.</given-names>
            <surname>Melo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Brabrand</surname>
          </string-name>
          ,
          <article-title>and A. Wa,sowski. How Does the Degree of Variability A ect Bug Finding</article-title>
          ? In International Conference on Software Engineering, ICSE '
          <volume>16</volume>
          , pages
          <fpage>679</fpage>
          {
          <fpage>690</fpage>
          , New York, NY, USA,
          <year>2016</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>T.</given-names>
            <surname>Mens</surname>
          </string-name>
          and
          <string-name>
            <given-names>P. V.</given-names>
            <surname>Gorp</surname>
          </string-name>
          .
          <article-title>A taxonomy of model transformation</article-title>
          .
          <source>Electr. Notes Theor. Comput. Sci.</source>
          ,
          <volume>152</volume>
          :
          <fpage>125</fpage>
          {
          <fpage>142</fpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>G. A.</given-names>
            <surname>Miller</surname>
          </string-name>
          .
          <article-title>The magical number seven, plus or minus two: Some limits on our capacity for processing information</article-title>
          .
          <source>Psychological review</source>
          ,
          <volume>63</volume>
          (
          <issue>2</issue>
          ):
          <fpage>81</fpage>
          ,
          <year>1956</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>S.</given-names>
            <surname>Nalchigar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Salay</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Chechik</surname>
          </string-name>
          .
          <article-title>Towards a catalog of non-functional requirements in model transformation languages</article-title>
          .
          <source>In Workshop on the Analysis of Model Transformations. CEUR-WS.org</source>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>S.</given-names>
            <surname>Sendall</surname>
          </string-name>
          and
          <string-name>
            <given-names>W.</given-names>
            <surname>Kozaczynski</surname>
          </string-name>
          . Model Transformation:
          <article-title>The Heart and Soul of Model-Driven Software Development</article-title>
          .
          <source>IEEE Software</source>
          ,
          <volume>20</volume>
          (
          <issue>5</issue>
          ):
          <volume>42</volume>
          {
          <fpage>45</fpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>J.</given-names>
            <surname>Siegmund</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Siegmund</surname>
          </string-name>
          , and
          <string-name>
            <surname>S. Apel.</surname>
          </string-name>
          <article-title>Views on internal and external validity in empirical software engineering</article-title>
          .
          <source>In International Conference on Software Engineering</source>
          , volume
          <volume>1</volume>
          , pages
          <fpage>9</fpage>
          <lpage>{</lpage>
          19. IEEE,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>D.</given-names>
            <surname>Stru</surname>
          </string-name>
          <article-title>ber. Model-Driven Engineering in the Large: Refactoring Techniques for Models and Model Transformation Systems</article-title>
          .
          <source>PhD thesis</source>
          , University of Marburg,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>D.</given-names>
            <surname>Stru</surname>
          </string-name>
          ber, T. Kehrer,
          <string-name>
            <given-names>T.</given-names>
            <surname>Arendt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Pietsch</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Reuling</surname>
          </string-name>
          .
          <article-title>Scalability of Model Transformations: Position Paper and Benchmark Set</article-title>
          . In Workshop on Scalable Model Driven Engineering, pages
          <volume>21</volume>
          {
          <fpage>30</fpage>
          . CEUR-WS.org,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>D.</given-names>
            <surname>Stru</surname>
          </string-name>
          ber, J. Rubin,
          <string-name>
            <given-names>T.</given-names>
            <surname>Arendt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Chechik</surname>
          </string-name>
          , G. Taentzer, and
          <string-name>
            <given-names>J.</given-names>
            <surname>Plo</surname>
          </string-name>
          <article-title>ger. RuleMerger: Automatic Construction of Variability-Based Model Transformation Rules</article-title>
          . In International Conference on Fundamental Approaches to Software Engineering, pages
          <volume>122</volume>
          {
          <fpage>140</fpage>
          . Springer,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>D.</given-names>
            <surname>Stru</surname>
          </string-name>
          ber, J.
          <string-name>
            <surname>Rubin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Chechik</surname>
            , and
            <given-names>G.</given-names>
          </string-name>
          <string-name>
            <surname>Taentzer</surname>
          </string-name>
          .
          <article-title>A Variability-Based Approach to Reusable and E cient Model Transformations</article-title>
          . In International Conference on Fundamental Approaches to Software Engineering, pages
          <volume>283</volume>
          {
          <fpage>298</fpage>
          . Springer,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>D.</given-names>
            <surname>Stru</surname>
          </string-name>
          <article-title>ber and</article-title>
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz</surname>
          </string-name>
          .
          <article-title>A Tool Environment for Managing Families of Model Transformation Rules</article-title>
          .
          <source>In International Conference on Graph Transformation</source>
          , pages
          <volume>89</volume>
          {
          <fpage>101</fpage>
          . Springer,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>E.</given-names>
            <surname>Syriani</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Gray</surname>
          </string-name>
          .
          <article-title>Challenges for addressing quality factors in model transformation</article-title>
          .
          <source>In International Conference on Software Testing, Veri cation and Validation</source>
          , pages
          <volume>929</volume>
          {
          <fpage>937</fpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>I.</given-names>
            <surname>Vessey</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Galletta</surname>
          </string-name>
          .
          <article-title>Cognitive t: An empirical study of information acquisition</article-title>
          .
          <source>Information systems research</source>
          ,
          <volume>2</volume>
          (
          <issue>1</issue>
          ):
          <volume>63</volume>
          {
          <fpage>84</fpage>
          ,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>F.</given-names>
            <surname>Wilcoxon</surname>
          </string-name>
          .
          <article-title>Individual Comparisons by Ranking Methods</article-title>
          .
          <source>Biometrics Bulletin</source>
          ,
          <volume>1</volume>
          (
          <issue>6</issue>
          ):
          <volume>80</volume>
          {
          <fpage>83</fpage>
          ,
          <string-name>
            <surname>Dec</surname>
          </string-name>
          .
          <year>1945</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>E. D.</given-names>
            <surname>Willink</surname>
          </string-name>
          . Optimized Declarative Transformation:
          <article-title>First Eclipse QVTc Results</article-title>
          . In Workshop on Scalable Model Driven Engineering, pages
          <volume>47</volume>
          {
          <fpage>56</fpage>
          . CEUR-WS.org,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>