=Paper=
{{Paper
|id=Vol-1301/ontocomodise2014_1
|storemode=property
|title=Analyzing the Behavior of Modelers in Interpreting Relationships in Conceptual Models: An Empirical Study
|pdfUrl=https://ceur-ws.org/Vol-1301/ontocomodise2014_1.pdf
|volume=Vol-1301
}}
==Analyzing the Behavior of Modelers in Interpreting Relationships in Conceptual Models: An Empirical Study==
https://ceur-ws.org/Vol-1301/ontocomodise2014_1.pdf
Analyzing the Behavior of Modelers in Interpreting
Relationships in Conceptual Models: An Empirical Study
Maria das Graças da Silva Teixeira, Ricardo de Almeida Falbo, Giancarlo Guizzardi,
Ontology and Conceptual Modeling Research Group (NEMO)
Federal University of Espírito Santo, Vitoria/ES, Brazil.
maria.teixeira@ufes.br, {gguizzardi,falbo}@inf.ufes.br
Abstract. This paper presents an experiment analyzing the behavior of model-
ers in interpreting conceptual model fragments developed using an Ontology-
Driven Conceptual Modeling Language (OntoUML). Our goal is to evaluate the
effect of two ontological constructs (relator and role) in modeling relationships.
Our hypotheses are: (i) the use of these constructs for modeling relations in-
creases the clarity of the models; (ii) as a consequence, this increases the per-
formance of modelers in interpreting these models. The behavior of the model-
ers is evaluated considering the participants´ answers to predetermined ques-
tions asked about two model fragments (compared with a template response).
We also consider the rationale underlying the answers given by the participants.
We have collected indications that both the role and the relator constructs in-
fluence positively in the quality of the resulting interpretations.
Keywords: Conceptual Model, Empirical Study, OntoUML, Relationship.
1 Introduction
Despite its fundamental importance in conceptual modeling, there is evidence that the
relationship construct and its modeling implications are still not fully understood by
modelers. Wand, Storey and Weber [1] point out that, while both entities and relation-
ships are fundamental to conceptual modeling, relationships prove to be more difficult
to use. The lack of rigorously defined meaning of conceptual modeling constructs,
especially in the case of the relationships, precludes the effective use of these con-
structs. Empirical evidence shows that using relationships as a way to communicate
the meaning of an application domain is often problematical [1] [2].
Most modelers have an intuitive understanding of what a relationship is. However,
recognizing relationships in a real-world context, or correctly representing them in a
model has proven to be challenging. There are various ways to represent a relation-
ship [1], and modelers have to decide which one to use.
In [3], Guizzardi provides a number of ontological theories, giving rise to the Uni-
fied Foundational Ontology (UFO). UFO was used as basis for evaluating the Unified
Modeling Language (UML) 2.0 metamodel, and for defining an ontologically well-
founded UML profile for class diagrams, called OntoUML.
Concerning relationships, UFO's theory of relations makes a fundamental distinc-
tion between two main types of relationships, namely: formal and material relations.
Whilst the former holds directly between two entities without any further intervening
individual, the latter is induced by the presence of mediating entities called relators.
Relators are individuals with the power of connecting entities. For example, an en-
rollment connects a student with an educational institution, an employment connects
an employee with an employer [3]. OntoUML has a construct for modeling relator
universals. Every instance of a relator universal is existentially dependent of at least
two distinct entities. The formal relations that take place between a relator universal
and the object classes it mediates are termed mediation relations [3][4].
Role is another object type strongly related to the modeling of relationships, since
it represents relationally dependent entities. For instance, the role student is played by
a person, when she is enrolled in an educational institution. Every role class must be
connected to an association end of a mediation relation. Thus, roles and relators are
important and heavily used constructs for modeling relationships in OntoUML.
In this paper, in order to evaluate the behavior of modelers in interpreting relation-
ships in a conceptual model, we partially describe an experiment whose goal is to
collect early indications about the interpretation of the constructs relator and role in
OntoUML conceptual model fragments. Our research hypothesis is that the presence
of these constructs increases the clarity of the models and, hence, improves the per-
formance of modelers in interpreting model fragments using both constructs (consid-
ering correctness of answers according to a template, as well as the rationale given by
the participants) when compared to the behavior of participants interpreting model
fragments without them. The subjects are Computer Science students that have some
experience in conceptual modeling. The results contain indications that the hypothesis
is true. The findings indicate that role is a valuable assistance to avoid the problematic
use of optional associations, while relator influences positively the participants´ per-
formance. It is important to highlight that, although initially we assumed that the
modelers’ level of experience could be easily balanced, the results show that this as-
pect interfered in the interpretations made by the participants.
The remainder of this paper is organized as follows. Section 2 contains the back-
ground about the concepts involved in the experiment. Section 3 discusses the empiri-
cal study itself. Section 4 describes some related works. Section 5 presents our final
considerations.
2 Background
Conceptual Modeling refers to formally describing in diagrammatic notation aspects
of the physical and social world for the purposes of understanding, communication
and problem-solving [5]. It is a difficult task, even if appropriate methods and lan-
guages are applied. Among the available concepts, one of the most problematic is the
relationship, that seems to be difficult to use in a clear and unambiguous way [1].
Olivé [2] comments that both entities and relationships are abstractions from a do-
main at a certain time, but while we can imagine entities as isolated objects in a do-
main, it is impossible to imagine a relationship without the presence of the objects.
Thus, identifying a relationship requires us to record not only the relationship itself,
but also its relata. Another difficulty in representing relationships comes from the
existence of multiple alternative constructs that could be used to represent the same
type of phenomenon in the real world. For instance, in UML, a binary relation can be
represented as an association, as an association class, or as an attribute. The more
options we have, the more complicated it is for a modeler to systematically choose the
most appropriate construct for a given circumstance (as well for justifying his choice).
There are mandatory relationships (always exist) and optional relationships (may
or may not exist). A mandatory relationship is identified by assigning the minimum
cardinality of 1 to the opposite association end of a given related entity, while an op-
tional relationship is identified by a minimum cardinality of 0 to that position. The use
of optional relationships is controversial. Some researchers argue that it should be
avoided. Weber et al. [6] argue that an alternative approach is to eliminate optional
attributes and relationships from a conceptual model by using subtypes that have only
mandatory attributes and relationships.
Researchers have proposed languages and methods that aim to support creating
quality conceptual models. In this context, foundational ontologies have been ex-
plored as a means to improve conceptual modeling languages. For instance, in [7],
Evermann and Wand report the results they obtained by mapping UML constructs to
the BWW ontology. In [3], Guizzardi incorporates in the UML 2.0 metamodel some
ontological distinctions and axioms put forth by the Unified Foundational Ontology,
giving rise to the so-called OntoUML language. OntoUML is, thus, an ontologically
well-founded version of UML 2.0 for conceptual modeling. Table 1 presents a brief
summary of some OntoUML's constructs used in the experiment described here.
Table 1. Some OntoUML stereotypes [3]
Stereotype Description
A stereotype <> is a representation of a Substante Sortal whose
<>
A instances are funcional complexes. E.g.: Person.
A <> represents an anti-rigid and relationally dependent universal.
<>
B Every <> must be connected to an association end of a
<> relation. E.g.: Student.
A <> universal is a relational moment universal. Every
<>
C <> must be (directly or indirectly) connected to an association
end on at least one <> relation. E.g.: Marriage.
<> A <> is a formal relation that takes place between a relator
universal and the endurant universal(s) it mediates. E.g.: the relator
universal Marriage mediates the role universals Husband and Wife.
derivation A derivation relation represents the formal relation of derivation that exists
relation between a material relation and the relator universal this material relation is
derived from. E.g.: the material relation married to, derived from the
relator universal Marriage.
<> A <> relation is a relational universal which is induced by a
relator universal. E.g.: a person is married to another person.
3 The Empirical Study
In this section, we describe the empirical study performed. This experiment was con-
ducted following the guidelines presented in [8]. Due to space limitations, we de-
scribe only part of the experiment (considering design and results).
The experiment goal is to collect indications of the impact of different representa-
tion strategies for modeling relationships in the interpretation of model fragments.
The objects of study are conceptual model fragments developed using OntoUML in
different domains. The research hypothesis is that the presence of relators and roles
increases the models clarity and, hence, improves the performance of modelers in
interpreting model fragments using both constructs (considering correctness of an-
swers according to a template, as well as the rationale given by the participants) when
compared to the behavior of participants interpreting model fragments without them.
The experiment has qualitative and quantitative strategies. The experimentation
level was in-vitro (it was conducted in a controlled environment). The research ap-
proach is primarily analytical, to collect early indications for further experiments. The
first model fragment focus on representing a reflexive relationship, while the second
focus on representing a binary relationship between two different concepts.
The subjects are Computer Science students, which attend classes of a Conceptual
Modeling course. The minimum requirement expected for participating in the experi-
ment is having basic knowledge of UML and OntoUML. A questionnaire was applied
to capture the participants’ profile. Regarding the sample size, there were 22 partici-
pants. They were divided into four groups (GA, GB, GC and GD) randomly. GA and
GB have 6 participants each, and GC and GD have 5 participants each.
The participants’ profile is of students acting as model users (readers) with some
level of knowledge in conceptual modeling, specifically on using UML and On-
toUML relations. The educational level (undergraduate, master and doctoral students)
of the four groups was balanced (most participants were postgraduate students, but all
groups have at least one undergraduate student). Regarding experience time in con-
ceptual modeling using UML, all members of GA and GC have more than one year of
experience, while GB has around 80% of its members in this range, and GD has
around 60% of its members in this range. Concerning knowledge in OntoUML prior
to the Conceptual Modeling course, around 80% of the participants in GA and GC
already had prior knowledge in the language. That number drops to around 30% in
GB and 40% in GD. However, over the course, OntoUML was studied and all the
participants had access to the minimum knowledge necessary for performing this
activity. At first, we considered that this would cause equilibrium between the distinct
groups. The results, however, showed us that this was not the case, and the level of
experience of the modelers influenced their interpretation of the diagrams.
The factor of the experiment is the representation of relationships using relator and
role constructs in conceptual models. The alternatives are: i) representing only the
material relation, without relator and role (see Figure 1, Group A); ii) representing
roles and material relation, without relator (see Figure 1, Group B and Figure 2,
Group C); iii) representing relator, roles and the corresponding mediation relations
(see Figure 1, Group D and Figure 2, Group B); iv) representing relator and the corre-
sponding mediation relations, without roles (see Figure 2, Group D); iv) representing
role, relator, the corresponding mediation relations, and the material relation derived
from the relator (henceforth termed “complete representation”) (see Figure 1, Group
C and Figure 2, Group A). The task is to interpret two conceptual model fragments,
each one regarding a different domain, using different representations. Figures 1 and
2 depict the representations of the model fragments for each domain and group. The
interpretation is done by means of answering four questions relative to each model
fragment. Each response should include an explanation of how the participant arrived
at the answer.
Fig. 1. Model fragments of Domain 1 interpreted by each group
Fig. 2. Model fragments of Domain 2 interpreted by each group
The selected domains were artificially designed. In the first domain, the model
fragments intend to capture a relation between Witches involved in a Bewitchment. In
this fictional world, witches can put spells (bewitchments) on each other. In the sec-
ond domain, the fragments are designed to represent a relation between a Knight and
a Tamed Dragon. In this (again) fictitious domain, knights and their tamed dragons
are connected by a loyalty bond. The intention behind the choice of model fragments
in fictional domains has the goal of minimizing the use of previous domain
knowledge in answering the questions.
The questions, which are the same for all groups, are:
• Domain 1 (Bewitching Witches bewitch Bewitched Witches): Q1) How many
Witches can be bewitched by a Bewitchment? Q2) How many Witches can be the
Bewitching Witch of a Bewitchment? Q3) Can a Witch be bewitched by her own
Bewitchment? Q4) Can a Bewitchment exist without affecting any Bewitched
Witch?
• Domain 2 (Bonds between Knights and Tamed Dragons): Q1) Can a Tamed Drag-
on exist without a Knight? Q2) How many Bonds might exist between a Dragon
and a Knight? Q3) Can a person be a Knight without a Bond with a Dragon? Q4)
How many Bonds a Knight may participate at the same time?
Q2 in Domain 2 includes an implicit condition being tested. Instead of asking for
"Tamed Dragon" (a role) it asks for "Dragon" (a kind). There is nothing in the model
specifying that all dragons must be tamed. We wanted to verify if the participants are
aware of this difference.
The dependent variable is correctness of answers, considering also the rationale
followed by the participant. This variable is measured by comparing the fragments of
the participants’ answers to the corresponding fragments of a template. If they are the
same, then the fragment is correct; otherwise, the fragment is considered incorrect.
Collected Data
Table 2 presents the tabulated results in terms of percentage of correct answers for
each question per group. As we can notice, GA and GC (groups containing the most
experienced modelers) presented better performance (over 50% of correct answers),
than GB and GD (less experienced groups). These latter groups answered less than
50% of the questions right. In both domains, the highest success rate (100% correct)
was obtained by groups GC and GA. We had also cases in which no correct answers
were given (0% correct). The latter case happened in groups GA, GC and GD. It is
worthwhile to remember that Q2 of Domain 2 had an implicit condition being tested.
This could justify the poor performance of the groups, including GA and GC.
Another aspect useful for evaluating the groups´ performance are the cases of par-
ticipants who indicated difficulties in answering a question due to the absence of a
relator (see table 2). For instance, consider Q1 in Domain 1. In our template, we es-
tablished as an ideal response: Minimum = “1”. Maximum = “it cannot be determined
with certainty”. An instance of the relationship “bewitches” must exist for the roles
“Bewitching Witch” and “Bewitched Witch” exist too. If such relationship exists,
then there is at least one “Bewitching Witch” and one “Bewitched Witch” instances.
The relationship´s cardinality also allows to infer that the same “Bewitching Witch”
can “bewitches” several “Witch” (“Bewitched Witch”), but this may occur through
one “Bewitchment” or even several “Bewitchment” - which cannot be deduced from
the model where such concept is not present. We notice that, in comparison with
group GB, a significant percentage of GA’s members indicated that the absence of an
explicitly represented relator prevented them in answering these questions. An unex-
pected outcome was noticed in the behavior of GC (more experienced group), which
has less than 30% of participants indicating difficulties to answer a question due to
relator absence. It is worthwhile to point out that this aspect did not apply to GD,
since all model fragments read by them contained relators in both domains.
Table 2. Correctness of Answers (Aproximated Percentage - %)
Domain
Group
Question 1 Question 2 Question 3 Question 4
A B C D A B C D A B C D A B C D
1 33 16 100 40 33 33 100 60 100 83 20 0 16 33 100 80
2 100 83 100 20 0 16 0 40 100 66 60 20 50 33 0 0
Tot. GA: 54.17 GB: 45.83 GC: 60.0 GD: 32.5
Rel* GA: 54.17 GB: 29.17 GC: 26.67 GD: -
Rel* = Relator Absence Identified (complementary information).
Table 3 also presents the results regarding correctness, but focusing on the differ-
ent representation strategies. We noticed that the best performance was achieved by
groups using the complete representation. The second best result is when both relator
and roles are presented, but the derived material relation is not. However, we should
reinforce that, in general, we perceived an influence of the experience level in the
results. Thus, it is more adequate to compare the performances of a same group using
different representations. This comparison provides an indication about the influence
of the presence/absence of relator and/or role constructs. For instance, consider GC.
In the complete representation, GC obtained 80% of correct answers. However, using
a representation without relator, the same group answered only 40% of the answers
correctly.
Table 3. Average Percentage of Correct Answers by Representation Strategy
With relator
Domain
and roles / Without
Without ma- With relator / relator / Without rela-
Complete terial relation Without roles With roles tor and roles
Group % Group % Group % Group % Group %
1 C 80.00 D 45.00 - - B 41.67 A 45.83
2 A 62.50 B 50.00 D 20.00 C 40.00 - -
Total 70.45 47.73 20.00 40.91 45.83
Table 4 shows an interesting aspect regarding the use of optional associations ver-
sus the use of roles and mandatory associations. This observation involves questions
of type "how many" (Q1 and Q2 in D1; Q2 and Q4 in D2) concerning the minimum
value indicated in the answers. To simplify we compare only two groups per domain.
In D1, we compare GA (without role, with optional relationship) and GC (with role,
with mandatory relationship). In D2, we compare GD (without role, with optional
relationship) and GB (with role, with mandatory relationship). For these questions,
the minimum cardinality should be 1, except for the case of Q2 in D2, which, as pre-
viously mentioned, has an implicit condition being tested, and because of this, the
minimum 0 is acceptable. In both domains we noticed better performances by the
groups that worked with representations that explicitly show the role construct.
In general, the results of the experiment point out that: 1) The experience level in-
terferes in the modelers' performance in interpreting models. Most experienced groups
presented better performance; 2) The representation strategy affects the modelers'
performance in interpreting models. Participants reading model fragments that explic-
itly show the constructs relator and role presented better performance.
Table 4. Percentage of Responses in Questions of the Type "How many" Separated by
Minimum Cardinality Indicated per Group
Domain GA GB GC GD
Question Min. 0 Min. 1 Min. 0 Min. 1 Min. 0 Min. 1 Min. 0 Min. 1
D1 Q1 66.67% 0.00% - - 0.00% 100.00% - -
D1 Q2 66.67% 0.00% - - 0.00% 100.00% - -
D2 Q2 - - 16.67% 33.33% - - 40.00% 0.00%
D2 Q4 - - 0.00% 66.67% - - 20.00% 0.00%
For answering the experiment research questions, we used the collected data pre-
sented above, as well as analyses of participants’ responses. To illustrate these anal-
yses, next we present the analysis done for question Q4 of domain D1. Similar anal-
yses were performed on the other questions of the experiment, but only the main find-
ings are identified in this paper.
D1. Q4: Can a bewitchment exist without affecting any Bewitched Witch?
Table 5 presents the performance of the participants in this question.
Table 5. Performance of groups in D1.Q4
Group Clearly Correct Answers Declared impossibility to precisely answer Q4
A 33.33% 16.67%
B 50% 50%
C 100% None
D 80% None
In GA, 3 out of 6 answered the question incorrectly (“yes”, due to the zero mini-
mum cardinality constraint). The correct answer is: “no”, given that a Bewitchment is
an existentially dependent entity and it depends on at least one individual playing the
role of a Bewitched Witch as well as at least one (distinct) individual playing the role
of Bewitching Witch. A 4th participant answered that it was impossible to answer the
question precisely without the explicit representation of the Bewitchment. Despite
sensible, we consider this answer as incorrect since: (i) in order to be a Bewitched
Witch one has to participate in at least one relationship; (ii) since this is a material
relation, participating in this relationship means being mediated by a relator (a Be-
witchment); (iii) given that a relator is an existentially dependent entity, it must con-
nect at least one instance of Bewitched Witch (and at least one distinct instance of
Bewitching Witch). A 5th participant answered the question stating that if a Be-
witchment is interpreted as a pair, then it should contain an instance of Bewitched
Witch in the pair. Notice that although showing a sensible reasoning, this answer
highlights a conceptual mistake, namely, that a Bewitchment is an instance of the
relationship. In contrast, given the cardinalities of the model, it is clearly possible for
the same Bewitchment instance involves a number of Bewitching Witches and Be-
witched Witches. Anyway, we consider in this context the answer as a correct one. A
6th participant answered: “the role indicates that an individual is a Bewitching Witch
when participating in a relationship”. We envisage that the participant followed the
same reasoning of the previous one.
In GB, 3 out of 6 participants stated that it was impossible to answer the question.
However, as explained about for the 4th participant of GA, we consider this answer
incorrect. Three of the participants answered the question correctly. Moreover, one of
them made an explicit reference to the relator, while the other two based their answer
solely on the cardinality constraint (minimum of 1).
In GC, all participants answered the question uniformly and correctly. A similar
tendency can be observed for GD in which 4 out of 5 participants gave the same cor-
rect answer. A 5th participant of GD mentioned that the model did not contain enough
information for the question to be answered (which is an incorrect answer).
In summary, the groups with an explicit representation of the relator performed
better in answering the question – observe GC and GD in Table 5. Moreover, the
group with the complete representation (GC) was the only group with unanimously
correct and justified answers. GA had the worst performance, with the majority of the
participants showing a shallow reasoning process by reasoning only on the directly
represented cardinality of the represented relation and, hence, answering the question
incorrectly (50% of incorrect answers for this reason). The participants of GA and GB
that answer the question correctly did so by reasoning on the instance of the relation
and on the role at hand. It is no surprise that this performance was then better in GB
where the roles were explicit represented (50% correct answers) than in GA
(33.33%). Finally, half of the participants in GB and one participant in GA answered
the question incorrectly by being unsuccessful in reconstructing the relation between
roles, material relations, the relator. Furthermore, the only participant in GA that
clearly answered the question correctly also made a conceptual mistake in equating
the instance of the relation with the relator. This is a common mistake. However, we
interpret this mistake as also being influenced by the non-explicit representation of
the relator and its connection to the material relation at hand.
Research Questions
RQ1) What are the effects of optional relationship in interpreting model fragments?
Regarding the use of the role construct to prevent the occurrence of optional asso-
ciations, we conclude that it leads to a better performance of the participants. This
evidence can be perceived by contrasting model fragments with optional association
(without role) and with mandatory association (with role), as Table 4 shows.
Contrasting groups with equivalent experience, in Domain 1, we see that GC had
100% of correct answers (minimum 1) in both questions (Q1 and Q2), while GA indi-
cated more than 65% of incorrect answers (minimum 0) in both questions. The re-
mainder of the responses were also incorrect, but due to some other reasons that we
refrain from discussing here (due to space limitations). This is a clear indication of the
influence of the role construct in guiding the participants to answer questions correct-
ly. In Domain 2, question Q4, we realize that GB's participants indicated the correct
answer (minimum 1) more than 65% of cases, while GD's participants indicated 20%
of incorrect answers (minimum 0) and no correct answers. The other occurrences of
errors were due to other reasons. Here, we once more notice the influence of role to
guide the correct answer, although not as significantly as in Domain 1. Regarding
question Q2, the minimum 0 is the correct answer, for those who noticed the implicit
condition being tested in this question. However, only 3 participants explicitly men-
tion that they have perceived this implicit condition being tested, and thus the results
of this question mask the effect of using the role construct.
In summary, the evidence points out that in questions involving minimum cardinal-
ities interpretation, optional relations confuse the reader. Using roles, thus represent-
ing only mandatory relations, clearly leads to a better performance of the participants.
RQ2) What are the effects of the presence / absence of relators in interpreting model
fragments?
The findings of the experiment show that in model fragments where relators are
present, the tendency of the participants is to reason on them to interpret the question.
Moreover, in general, relators contribute positively to a quality interpretation.
Table 3 gives us some interesting indications: (i) There was better performance of
groups with the complete representation (total above 70%, while other representation
forms did not reach 50% of correctness); (ii) When contrasting the results of GA and
GC (the most experienced groups, which used the complete representation and a rep-
resentation without relator), we also realize a better performance of each in cases
when they interpreted model fragments that had the presence of relators. The result of
GB also provides evidence of the positive influence of relator, even though less sig-
nificantly; (iii) The representations with relator (first two columns of table) had better
performance than representations without relator (last two columns of table). One
exception occurred in the representation with relator, without role (the middle col-
umn), which seems to have been affected by the experience level of GD (there was a
significant number of partially correct answers, which we count as error); (iv) We
notice that a small change in the representation strategy (between columns 1 and 2, by
showing the material relation derived from the relator) apparently causes a large
variation in the result (correctness percentage). In this case, however, it seems to be
an influence of the experience level more than of the representation strategy (GA and
GC have more experience than GB and GD).
Another aspect should be highlighted: Were the participants able to notice when a
question could not be satisfactorily answered by a representation? Without relator,
there were attempts to follow other paths of reasoning, and usually only the most
experienced modelers realized the implications of the absence of the relator construct
in questions where it was required. This situation should have been noted by the par-
ticipants of GA and GB in the four questions of Domain 1, as well as by the partici-
pants of GC in three questions (Q2, Q3, Q4) of Domain 2. The last line of Table 2
contains some values related to it. The values are lower than expected, especially in
the more experienced groups. This may be an indication that the participants tried to
answer the questions with the information provided, without thinking enough about
what might be missing.
Limitations and Validity Threats
During the experiment, we identified some limitations and validity threats. Firstly, we
invited to participate in the experiment students of a course that involved different
student levels (undergraduate, master and doctoral), with different experience levels
in conceptual modeling. We tried to balance this diversity along the course. Thus, the
participants were aware of the concepts necessary to execute the proposed activity.
However, the results showed us that the different experience levels affected the exper-
iment outcome. Secondly, we selected some unusual problem domains to be interpret-
ed through the models, in order to reduce the influence of background knowledge. So,
the modeler could concentrate on the model to answer the questions. However, the
use of such domains may have had an inverse effect, distracting some participants
instead of allowing them to focus on the model. Thirdly, we considered the interpreta-
tion of small fragments of conceptual models. The understandability in larger models
can lead to different results from those obtained in this experiment. This is not clear
yet, as the use of relator and role constructs in these models could contribute for en-
larging the models and increasing their complexity. Fourthly, the number of partici-
pants was small, and thus we had not a representative sample. Because of that, we
could not to apply statistical hypotheses tests. Fifthly, each group analyzed different
relationship representations in different problem domains, thus analyzing different
relationship types (for example, reflexive and binary relationships). The results may
have been influenced by the familiarity of the participants with such aspects. Sixthly,
as the number of participants was small and we were ambitious in the quantity of
representation strategies that we intended to evaluate, not all strategies identified were
analyzed by all groups. This fact complicated the analysis of the collected data, be-
cause not all possible variations (groups versus representation strategies) were availa-
ble for analysis. Finally, the participants knew that the experiment aims to seek for
evidence of the usefulness of relator and role constructs. The awareness of the im-
portance of such concepts may have induced them to reason in a certain way. Indeed,
as aforementioned, some participants detached the relator absence and pointed out
such situation as a factor that hindered the model interpretation (which was expected).
4 Related Works
Several works have been undertaken aiming at studying the use of one of the most
basic constructs in conceptual modeling: relationship. Regarding the difficulty of
representing relationships, Wand, Storey and Weber [1] said that “users of conceptual
modeling methodologies are frequently confused about whether to show an associa-
tion between things via a relationship, an entity, or an attribute”. They developed an
ontological analysis based on BWW Ontology, providing a precise definition of sev-
eral conceptual modeling constructs. Moreover, they defined derived rules for the use
of relationships in entity-relationship (ER) conceptual models, and showed how these
rules solve ambiguities that exist in the practice of conceptual modeling. This work
and ours have in common the fact that both focus on relationship representation in
conceptual models and on the use of a foundational ontology to improve it. In fact, we
consider the work of Wand, Storey and Weber as a basis. However, their work does
not contain empirical results, just comments about some possibilities and examples.
Furthermore, their focus was on the development of models, while we were interested
on interpretation of models.
Regarding empirical studies related to the relationship construct, Poels et al. [9]
applied an experiment to test the readers’ performance in the interpretation of rela-
tionship cardinalities. The focus was on many-to-many relations, which are represent-
ed in UML class diagram in different manners (via association class or via object
class, the latter being an objectification of the relationship). They concluded that, for
users without experience in modeling, the use of association class is better than object
class. Analogously, our work concentrates on the use of the relator construct (not
specifically in cardinalities), which, in a simplified manner can be seen as an objecti-
fication of a relationship. However, Poels et al.'s experiment focuses on relationship
cardinalities and business user, while ours focuses on the representation itself and
model users possessing UML basic knowledge. We believe that the domain require-
ments, among other aspects, should guide the relationship representation strategy
chosen. Using an association class, we can represent two pairs of cardinalities only
(the simplest case). Using an object class, we can represent four pairs of cardinalities.
Using a relator plus the material relation derived from it, we can represent six pairs of
cardinalities, enabling a more detailed but complex representation. This belief could
explain the different results in both experiments. Anyway, the different results ob-
tained in both studies detach the need for more studies regarding the interpretation of
relationships.
Weber et al. [6] conducted some experiments to test the use of optional attributes
and relationships in conceptual schema diagrams (ER diagrams). As an indication of
that study, the authors state that, when users require a deep-level understanding, op-
tional attributes and relationships should not be used, because they undermine users’
abilities to grasp important domain semantics. By avoiding the use of optional proper-
ties, replacing them by subclasses, the semantic of model fragments at hand becomes
clearer. We also investigated this aspect, and our findings are in agreement with the
conclusion of Weber et al. The use of the role construct, always a subclass in a class
diagram, conducted to better interpretations of the participants in several questions.
5 Final Considerations
The experiment partially presented in this paper collected evidence on the use of the
relator and role constructs in conceptual models fragments developed in OntoUML
and interpreted by modelers. The findings are in favor of our hypothesis that the use
of such constructs increases the models quality, allowing better interpretation of them.
Some evidences identified are: (i) The use of relator and role constructs influences
the quality of the resulting models, making them clearer; (ii) The role construct con-
tributes positively in avoiding the use of optional associations; (iii) The relator con-
struct assists both novice and experienced modelers. However, such assistance is
manifested differently; (iv) The experience level interferes in the performance of the
participants. We were not able to isolate this variable in the experiment, as we thought
at the beginning. It is necessary to contrast the influence of experience level versus
representation strategy in another experiment, to better understand the ways each
aspect interferes in the performance of the participants.
Concerning the use of roles and mandatory relations in place of optional relations,
our findings are in line with the ones achieved by Weber et al. [6].
Regarding relators, although the results also confirm our hypothesis, the evidences
are more subtle. In fact, we perceived an influence of the experience level in these
results. Although we have tried to balance the experience level of the modelers in
each group, we did not really achieve such balance. Thus, further studies should be
undertaken for improving our understanding regarding modelers' profile. In this sense,
we intend to perform other experiments, taking as basis the design used by Hadar in
[10]. In this work, she presents an empirical study on the difficulties that the object
oriented paradigm presents to designers (even experienced ones) and points out the
psychological motivations behind these difficulties. She intended to establish how the
individuality of each professional interferes in design, with some indications of the
reasons behind it. Her empirical findings suggest that a tension between intuitive and
logical thinking modes may lead to simple mistakes software engineers exhibit when
practicing object-oriented analysis and design.
This experiment collected evidence that we intend to use to guide future studies,
more rigorous and focused. We are aware of some limitations and threats to validity
of this study, as discussed in Section 3. So, we intend to use these results to orientate
future efforts, improving the design of the experiments and deepening in the evidence
identified. Some future works include: (i) Evidence points positively to the proposed
hypothesis, but further studies should be developed to check them. For instance:
Should the results be the same with larger and varied groups? Should the results be
similar in the interpretation of larger models? (ii) We need to evaluate some limita-
tions and threats to validity that we identified and did not explore yet. For instance,
further studies should be undertaken for improving our understanding regarding mod-
elers' profile.
Due to the lack of space we described the experiment partially in this paper. We in-
tend to publish the complete result soon, also making possible the experiment replica-
tion by other researchers interested in the field.
Acknowledgments
This work was partially supported by the Lucretius ERC Advanced Grant #267856
and FAPES/Brazil.
References
1. Wand, Y., Storey, V.C., Weber, R.: An ontological analysis of the relationship construct in
conceptual modeling. ACM Trans. Database Syst. 24, 494–528 (1999).
2. Olivé, A.: Conceptual Modeling of Information Systems. Springer-Verlag New York, Inc.,
Secaucus, NJ, USA (2007).
3. Guizzardi, G.: Ontological Foundations for Structural Conceptual Models. Centre for
Telematics and Information Technology, University of Twente, The Netherlends (2005).
4. Guizzardi, G., Wagner, G.: What’s in a Relationship: An Ontological Analysis. 27th
International Conference on Conceptual Modeling - ER2008. pp. 83–97. Springer-Verlag,
Berlin, Heidelberg (2008).
5. Mylopoulos, J.: Conceptual Modeling and Telos. In: Loucopoulos, P. and Zicari, R. (eds.)
Conceptual Modeling, Databases, and Case: An Integrated View of Information Sytems
Development. pp. 49–68. Wiley (1992).
6. Weber, R., Sim, M., Patel, A., Bodart, F.: Should Optional Properties be Used in
Conceptual Modelling? A Theory and Three Empirical Tests. Information Systems
Research. 12, 384–405 (2001).
7. Evermann, J., Wand, Y.: Towards Ontologically Based Semantics for UML Constructs.
20th International Conference on Conceptual Modeling - ER2001. pp. 27–30. Springer-
Verlag, Yokohama, Japan (2001).
8. Juristo, N., Moreno, A.M.: Basics of Software Engineering Experimentation. Springer
Publishing Company, Incorporated (2001).
9. Poels, G., Gailly, F., Maes, A., Paemeleire, R.: Object Class or Association Class? Testing
the User Effect on Cardinality Interpretation. In: Akoka, J., Liddle, S.W., Song, I.-Y.,
Bertolotto, M., Comyn-Wattiau, I., Cherfi, S.S.-S., van den Heuvel, W.-J., Thalheim, B.,
Kolp, M., Bresciani, P., Trujillo, J., Kop, C., and Mayr, H.C. (eds.) ER (Workshops). pp.
33–42. Springer (2005).
10. Hadar, I.: When intuition and logic clash: The case of the object-oriented paradigm.
Science of Computer Programming. 78, 1407–1426 (2013).