Despite its fundamental importance in conceptual modeling, there is evidence that the relationship construct and its modeling implications are still not fully understood by modelers. Wand, Storey and Weber [1] point out that, while both entities and relationships are fundamental to conceptual modeling, relationships prove to be more difficult to use. The lack of rigorously defined meaning of conceptual modeling constructs, especially in the case of the relationships, precludes the effective use of these constructs. Empirical evidence shows that using relationships as a way to communicate the meaning of an application domain is often problematical [1] [2]. Most modelers have an intuitive understanding of what a relationship is. However, recognizing relationships in a real-world context, or correctly representing them in a model has proven to be challenging. There are various ways to represent a relationship [1], and modelers have to decide which one to use.
In [3], Guizzardi provides a number of ontological theories, giving rise to the Unified Foundational Ontology (UFO). UFO was used as basis for evaluating the Unified Modeling Language (UML) 2.0 metamodel, and for defining an ontologically wellfounded UML profile for class diagrams, called OntoUML.
Concerning relationships, UFO's theory of relations makes a fundamental distinction between two main types of relationships, namely: formal and material relations. Whilst the former holds directly between two entities without any further intervening individual, the latter is induced by the presence of mediating entities called relators. Relators are individuals with the power of connecting entities. For example, an enrollment connects a student with an educational institution, an employment connects an employee with an employer [3]. OntoUML has a construct for modeling relator universals. Every instance of a relator universal is existentially dependent of at least two distinct entities. The formal relations that take place between a relator universal and the object classes it mediates are termed mediation relations [3] [4].
Role is another object type strongly related to the modeling of relationships, since it represents relationally dependent entities. For instance, the role student is played by a person, when she is enrolled in an educational institution. Every role class must be connected to an association end of a mediation relation. Thus, roles and relators are important and heavily used constructs for modeling relationships in OntoUML.
In this paper, in order to evaluate the behavior of modelers in interpreting relationships in a conceptual model, we partially describe an experiment whose goal is to collect early indications about the interpretation of the constructs relator and role in OntoUML conceptual model fragments. Our research hypothesis is that the presence of these constructs increases the clarity of the models and, hence, improves the performance of modelers in interpreting model fragments using both constructs (considering correctness of answers according to a template, as well as the rationale given by the participants) when compared to the behavior of participants interpreting model fragments without them. The subjects are Computer Science students that have some experience in conceptual modeling. The results contain indications that the hypothesis is true. The findings indicate that role is a valuable assistance to avoid the problematic use of optional associations, while relator influences positively the participants´ performance. It is important to highlight that, although initially we assumed that the modelers' level of experience could be easily balanced, the results show that this aspect interfered in the interpretations made by the participants.
The remainder of this paper is organized as follows. Section 2 contains the background about the concepts involved in the experiment. Section 3 discusses the empirical study itself. Section 4 describes some related works. Section 5 presents our final considerations.
Conceptual Modeling refers to formally describing in diagrammatic notation aspects of the physical and social world for the purposes of understanding, communication and problem-solving [5]. It is a difficult task, even if appropriate methods and languages are applied. Among the available concepts, one of the most problematic is the relationship, that seems to be difficult to use in a clear and unambiguous way [1].
Olivé [2] comments that both entities and relationships are abstractions from a domain at a certain time, but while we can imagine entities as isolated objects in a domain, it is impossible to imagine a relationship without the presence of the objects.
Thus, identifying a relationship requires us to record not only the relationship itself, but also its relata. Another difficulty in representing relationships comes from the existence of multiple alternative constructs that could be used to represent the same type of phenomenon in the real world. For instance, in UML, a binary relation can be represented as an association, as an association class, or as an attribute. The more options we have, the more complicated it is for a modeler to systematically choose the most appropriate construct for a given circumstance (as well for justifying his choice).
There are mandatory relationships (always exist) and optional relationships (may or may not exist). A mandatory relationship is identified by assigning the minimum cardinality of 1 to the opposite association end of a given related entity, while an optional relationship is identified by a minimum cardinality of 0 to that position. The use of optional relationships is controversial. Some researchers argue that it should be avoided. Weber et al. [6] argue that an alternative approach is to eliminate optional attributes and relationships from a conceptual model by using subtypes that have only mandatory attributes and relationships.
Researchers have proposed languages and methods that aim to support creating quality conceptual models. In this context, foundational ontologies have been explored as a means to improve conceptual modeling languages. For instance, in [7], Evermann and Wand report the results they obtained by mapping UML constructs to the BWW ontology. In [3], Guizzardi incorporates in the UML 2.0 metamodel some ontological distinctions and axioms put forth by the Unified Foundational Ontology, giving rise to the so-called OntoUML language. OntoUML is, thus, an ontologically well-founded version of UML 2.0 for conceptual modeling. Table 1 presents a brief summary of some OntoUML's constructs used in the experiment described here.
In this section, we describe the empirical study performed. This experiment was conducted following the guidelines presented in [8]. Due to space limitations, we describe only part of the experiment (considering design and results). The experiment goal is to collect indications of the impact of different representation strategies for modeling relationships in the interpretation of model fragments. The objects of study are conceptual model fragments developed using OntoUML in different domains. The research hypothesis is that the presence of relators and roles increases the models clarity and, hence, improves the performance of modelers in interpreting model fragments using both constructs (considering correctness of answers according to a template, as well as the rationale given by the participants) when compared to the behavior of participants interpreting model fragments without them.
The experiment has qualitative and quantitative strategies. The experimentation level was in-vitro (it was conducted in a controlled environment). The research approach is primarily analytical, to collect early indications for further experiments. The first model fragment focus on representing a reflexive relationship, while the second focus on representing a binary relationship between two different concepts.
The subjects are Computer Science students, which attend classes of a Conceptual Modeling course. The minimum requirement expected for participating in the experiment is having basic knowledge of UML and OntoUML. A questionnaire was applied to capture the participants' profile. Regarding the sample size, there were 22 participants. They were divided into four groups (GA, GB, GC and GD) randomly. GA and GB have 6 participants each, and GC and GD have 5 participants each.
The participants' profile is of students acting as model users (readers) with some level of knowledge in conceptual modeling, specifically on using UML and On-toUML relations. The educational level (undergraduate, master and doctoral students) of the four groups was balanced (most participants were postgraduate students, but all groups have at least one undergraduate student). Regarding experience time in conceptual modeling using UML, all members of GA and GC have more than one year of experience, while GB has around 80% of its members in this range, and GD has around 60% of its members in this range. Concerning knowledge in OntoUML prior to the Conceptual Modeling course, around 80% of the participants in GA and GC already had prior knowledge in the language. That number drops to around 30% in GB and 40% in GD. However, over the course, OntoUML was studied and all the participants had access to the minimum knowledge necessary for performing this activity. At first, we considered that this would cause equilibrium between the distinct groups. The results, however, showed us that this was not the case, and the level of experience of the modelers influenced their interpretation of the diagrams.
The factor of the experiment is the representation of relationships using relator and role constructs in conceptual models. The alternatives are: i) representing only the material relation, without relator and role (see Figure 1 The selected domains were artificially designed. In the first domain, the model fragments intend to capture a relation between Witches involved in a Bewitchment. In this fictional world, witches can put spells (bewitchments) on each other. In the second domain, the fragments are designed to represent a relation between a Knight and a Tamed Dragon. In this (again) fictitious domain, knights and their tamed dragons are connected by a loyalty bond. The intention behind the choice of model fragments in fictional domains has the goal of minimizing the use of previous domain knowledge in answering the questions.
The questions, which are the same for all groups, are:
• Q2 in Domain 2 includes an implicit condition being tested. Instead of asking for "Tamed Dragon" (a role) it asks for "Dragon" (a kind). There is nothing in the model specifying that all dragons must be tamed. We wanted to verify if the participants are aware of this difference.
The dependent variable is correctness of answers, considering also the rationale followed by the participant. This variable is measured by comparing the fragments of the participants' answers to the corresponding fragments of a template. If they are the same, then the fragment is correct; otherwise, the fragment is considered incorrect.
Table 2 presents the tabulated results in terms of percentage of correct answers for each question per group. As we can notice, GA and GC (groups containing the most experienced modelers) presented better performance (over 50% of correct answers), than GB and GD (less experienced groups). These latter groups answered less than 50% of the questions right. In both domains, the highest success rate (100% correct) was obtained by groups GC and GA. We had also cases in which no correct answers were given (0% correct). The latter case happened in groups GA, GC and GD. It is worthwhile to remember that Q2 of Domain 2 had an implicit condition being tested. This could justify the poor performance of the groups, including GA and GC.
Another aspect useful for evaluating the groups´ performance are the cases of participants who indicated difficulties in answering a question due to the absence of a relator (see table 2). For instance, consider Q1 in Domain 1. In our template, we established as an ideal response: Minimum = "1". Maximum = "it cannot be determined with certainty". An instance of the relationship "bewitches" must exist for the roles "Bewitching Witch" and "Bewitched Witch" exist too. If such relationship exists, then there is at least one "Bewitching Witch" and one "Bewitched Witch" instances. The relationship´s cardinality also allows to infer that the same "Bewitching Witch" can "bewitches" several "Witch" ("Bewitched Witch"), but this may occur through one "Bewitchment" or even several "Bewitchment" -which cannot be deduced from the model where such concept is not present. We notice that, in comparison with group GB, a significant percentage of GA's members indicated that the absence of an explicitly represented relator prevented them in answering these questions. An unexpected outcome was noticed in the behavior of GC (more experienced group), which has less than 30% of participants indicating difficulties to answer a question due to relator absence. It is worthwhile to point out that this aspect did not apply to GD, since all model fragments read by them contained relators in both domains. Table 3 also presents the results regarding correctness, but focusing on the different representation strategies. We noticed that the best performance was achieved by groups using the complete representation. The second best result is when both relator and roles are presented, but the derived material relation is not. However, we should reinforce that, in general, we perceived an influence of the experience level in the results. Thus, it is more adequate to compare the performances of a same group using different representations. This comparison provides an indication about the influence of the presence/absence of relator and/or role constructs. For instance, consider GC. In the complete representation, GC obtained 80% of correct answers. However, using a representation without relator, the same group answered only 40% of the answers correctly. Table 4 shows an interesting aspect regarding the use of optional associations versus the use of roles and mandatory associations. This observation involves questions of type "how many" (Q1 and Q2 in D1; Q2 and Q4 in D2) concerning the minimum value indicated in the answers. To simplify we compare only two groups per domain. In D1, we compare GA (without role, with optional relationship) and GC (with role, with mandatory relationship). In D2, we compare GD (without role, with optional relationship) and GB (with role, with mandatory relationship). For these questions, the minimum cardinality should be 1, except for the case of Q2 in D2, which, as previously mentioned, has an implicit condition being tested, and because of this, the minimum 0 is acceptable. In both domains we noticed better performances by the groups that worked with representations that explicitly show the role construct.
In general, the results of the experiment point out that: 1) The experience level interferes in the modelers' performance in interpreting models. Most experienced groups presented better performance; 2) The representation strategy affects the modelers' performance in interpreting models. Participants reading model fragments that explicitly show the constructs relator and role presented better performance. For answering the experiment research questions, we used the collected data presented above, as well as analyses of participants' responses. To illustrate these analyses, next we present the analysis done for question Q4 of domain D1. Similar analyses were performed on the other questions of the experiment, but only the main findings are identified in this paper.
Table 5 presents the performance of the participants in this question. In GA, 3 out of 6 answered the question incorrectly ("yes", due to the zero minimum cardinality constraint). The correct answer is: "no", given that a Bewitchment is an existentially dependent entity and it depends on at least one individual playing the role of a Bewitched Witch as well as at least one (distinct) individual playing the role of Bewitching Witch. A 4th participant answered that it was impossible to answer the question precisely without the explicit representation of the Bewitchment. Despite sensible, we consider this answer as incorrect since: (i) in order to be a Bewitched Witch one has to participate in at least one relationship; (ii) since this is a material relation, participating in this relationship means being mediated by a relator (a Bewitchment); (iii) given that a relator is an existentially dependent entity, it must connect at least one instance of Bewitched Witch (and at least one distinct instance of Bewitching Witch). A 5th participant answered the question stating that if a Bewitchment is interpreted as a pair, then it should contain an instance of Bewitched Witch in the pair. Notice that although showing a sensible reasoning, this answer highlights a conceptual mistake, namely, that a Bewitchment is an instance of the relationship. In contrast, given the cardinalities of the model, it is clearly possible for the same Bewitchment instance involves a number of Bewitching Witches and Bewitched Witches. Anyway, we consider in this context the answer as a correct one. A 6th participant answered: "the role indicates that an individual is a Bewitching Witch when participating in a relationship". We envisage that the participant followed the same reasoning of the previous one.
In GB, 3 out of 6 participants stated that it was impossible to answer the question. However, as explained about for the 4 th participant of GA, we consider this answer incorrect. Three of the participants answered the question correctly. Moreover, one of them made an explicit reference to the relator, while the other two based their answer solely on the cardinality constraint (minimum of 1).
In GC, all participants answered the question uniformly and correctly. A similar tendency can be observed for GD in which 4 out of 5 participants gave the same correct answer. A 5 th participant of GD mentioned that the model did not contain enough information for the question to be answered (which is an incorrect answer).
In summary, the groups with an explicit representation of the relator performed better in answering the question -observe GC and GD in Table 5. Moreover, the group with the complete representation (GC) was the only group with unanimously correct and justified answers. GA had the worst performance, with the majority of the participants showing a shallow reasoning process by reasoning only on the directly represented cardinality of the represented relation and, hence, answering the question incorrectly (50% of incorrect answers for this reason). The participants of GA and GB that answer the question correctly did so by reasoning on the instance of the relation and on the role at hand. It is no surprise that this performance was then better in GB where the roles were explicit represented (50% correct answers) than in GA (33.33%). Finally, half of the participants in GB and one participant in GA answered the question incorrectly by being unsuccessful in reconstructing the relation between roles, material relations, the relator. Furthermore, the only participant in GA that clearly answered the question correctly also made a conceptual mistake in equating the instance of the relation with the relator. This is a common mistake. However, we interpret this mistake as also being influenced by the non-explicit representation of the relator and its connection to the material relation at hand.
Regarding the use of the role construct to prevent the occurrence of optional associations, we conclude that it leads to a better performance of the participants. This evidence can be perceived by contrasting model fragments with optional association (without role) and with mandatory association (with role), as Table 4 shows.
Contrasting groups with equivalent experience, in Domain 1, we see that GC had 100% of correct answers (minimum 1) in both questions (Q1 and Q2), while GA indicated more than 65% of incorrect answers (minimum 0) in both questions. The remainder of the responses were also incorrect, but due to some other reasons that we refrain from discussing here (due to space limitations). This is a clear indication of the influence of the role construct in guiding the participants to answer questions correctly. In Domain 2, question Q4, we realize that GB's participants indicated the correct answer (minimum 1) more than 65% of cases, while GD's participants indicated 20% of incorrect answers (minimum 0) and no correct answers. The other occurrences of errors were due to other reasons. Here, we once more notice the influence of role to guide the correct answer, although not as significantly as in Domain 1. Regarding question Q2, the minimum 0 is the correct answer, for those who noticed the implicit condition being tested in this question. However, only 3 participants explicitly mention that they have perceived this implicit condition being tested, and thus the results of this question mask the effect of using the role construct.
In summary, the evidence points out that in questions involving minimum cardinalities interpretation, optional relations confuse the reader. Using roles, thus representing only mandatory relations, clearly leads to a better performance of the participants.
The findings of the experiment show that in model fragments where relators are present, the tendency of the participants is to reason on them to interpret the question. Moreover, in general, relators contribute positively to a quality interpretation.
Table 3 gives us some interesting indications: (i) There was better performance of groups with the complete representation (total above 70%, while other representation forms did not reach 50% of correctness); (ii) When contrasting the results of GA and GC (the most experienced groups, which used the complete representation and a representation without relator), we also realize a better performance of each in cases when they interpreted model fragments that had the presence of relators. The result of GB also provides evidence of the positive influence of relator, even though less significantly; (iii) The representations with relator (first two columns of table) had better performance than representations without relator (last two columns of table). One exception occurred in the representation with relator, without role (the middle column), which seems to have been affected by the experience level of GD (there was a significant number of partially correct answers, which we count as error); (iv) We notice that a small change in the representation strategy (between columns 1 and 2, by showing the material relation derived from the relator) apparently causes a large variation in the result (correctness percentage). In this case, however, it seems to be an influence of the experience level more than of the representation strategy (GA and GC have more experience than GB and GD).
Another aspect should be highlighted: Were the participants able to notice when a question could not be satisfactorily answered by a representation? Without relator, there were attempts to follow other paths of reasoning, and usually only the most experienced modelers realized the implications of the absence of the relator construct in questions where it was required. This situation should have been noted by the participants of GA and GB in the four questions of Domain 1, as well as by the participants of GC in three questions (Q2, Q3, Q4) of Domain 2. The last line of Table 2 contains some values related to it. The values are lower than expected, especially in the more experienced groups. This may be an indication that the participants tried to answer the questions with the information provided, without thinking enough about what might be missing.
During the experiment, we identified some limitations and validity threats. Firstly, we invited to participate in the experiment students of a course that involved different student levels (undergraduate, master and doctoral), with different experience levels in conceptual modeling. We tried to balance this diversity along the course. Thus, the participants were aware of the concepts necessary to execute the proposed activity. However, the results showed us that the different experience levels affected the experiment outcome. Secondly, we selected some unusual problem domains to be interpreted through the models, in order to reduce the influence of background knowledge. So, the modeler could concentrate on the model to answer the questions. However, the use of such domains may have had an inverse effect, distracting some participants instead of allowing them to focus on the model. Thirdly, we considered the interpretation of small fragments of conceptual models. The understandability in larger models can lead to different results from those obtained in this experiment. This is not clear yet, as the use of relator and role constructs in these models could contribute for enlarging the models and increasing their complexity. Fourthly, the number of participants was small, and thus we had not a representative sample. Because of that, we could not to apply statistical hypotheses tests. Fifthly, each group analyzed different relationship representations in different problem domains, thus analyzing different relationship types (for example, reflexive and binary relationships). The results may have been influenced by the familiarity of the participants with such aspects. Sixthly, as the number of participants was small and we were ambitious in the quantity of representation strategies that we intended to evaluate, not all strategies identified were analyzed by all groups. This fact complicated the analysis of the collected data, because not all possible variations (groups versus representation strategies) were available for analysis. Finally, the participants knew that the experiment aims to seek for evidence of the usefulness of relator and role constructs. The awareness of the importance of such concepts may have induced them to reason in a certain way. Indeed, as aforementioned, some participants detached the relator absence and pointed out such situation as a factor that hindered the model interpretation (which was expected).
Several works have been undertaken aiming at studying the use of one of the most basic constructs in conceptual modeling: relationship. Regarding the difficulty of representing relationships, Wand, Storey and Weber [1] said that "users of conceptual modeling methodologies are frequently confused about whether to show an associa-tion between things via a relationship, an entity, or an attribute". They developed an ontological analysis based on BWW Ontology, providing a precise definition of several conceptual modeling constructs. Moreover, they defined derived rules for the use of relationships in entity-relationship (ER) conceptual models, and showed how these rules solve ambiguities that exist in the practice of conceptual modeling. This work and ours have in common the fact that both focus on relationship representation in conceptual models and on the use of a foundational ontology to improve it. In fact, we consider the work of Wand, Storey and Weber as a basis. However, their work does not contain empirical results, just comments about some possibilities and examples. Furthermore, their focus was on the development of models, while we were interested on interpretation of models.
Regarding empirical studies related to the relationship construct, Poels et al. [9] applied an experiment to test the readers' performance in the interpretation of relationship cardinalities. The focus was on many-to-many relations, which are represented in UML class diagram in different manners (via association class or via object class, the latter being an objectification of the relationship). They concluded that, for users without experience in modeling, the use of association class is better than object class. Analogously, our work concentrates on the use of the relator construct (not specifically in cardinalities), which, in a simplified manner can be seen as an objectification of a relationship. However, Poels et al.'s experiment focuses on relationship cardinalities and business user, while ours focuses on the representation itself and model users possessing UML basic knowledge. We believe that the domain requirements, among other aspects, should guide the relationship representation strategy chosen. Using an association class, we can represent two pairs of cardinalities only (the simplest case). Using an object class, we can represent four pairs of cardinalities. Using a relator plus the material relation derived from it, we can represent six pairs of cardinalities, enabling a more detailed but complex representation. This belief could explain the different results in both experiments. Anyway, the different results obtained in both studies detach the need for more studies regarding the interpretation of relationships.
Weber et al. [6] conducted some experiments to test the use of optional attributes and relationships in conceptual schema diagrams (ER diagrams). As an indication of that study, the authors state that, when users require a deep-level understanding, optional attributes and relationships should not be used, because they undermine users' abilities to grasp important domain semantics. By avoiding the use of optional properties, replacing them by subclasses, the semantic of model fragments at hand becomes clearer. We also investigated this aspect, and our findings are in agreement with the conclusion of Weber et al. The use of the role construct, always a subclass in a class diagram, conducted to better interpretations of the participants in several questions.
The experiment partially presented in this paper collected evidence on the use of the relator and role constructs in conceptual models fragments developed in OntoUML and interpreted by modelers. The findings are in favor of our hypothesis that the use of such constructs increases the models quality, allowing better interpretation of them. Some evidences identified are: (i) The use of relator and role constructs influences the quality of the resulting models, making them clearer; (ii) The role construct contributes positively in avoiding the use of optional associations; (iii) The relator construct assists both novice and experienced modelers. However, such assistance is manifested differently; (iv) The experience level interferes in the performance of the participants. We were not able to isolate this variable in the experiment, as we thought at the beginning. It is necessary to contrast the influence of experience level versus representation strategy in another experiment, to better understand the ways each aspect interferes in the performance of the participants. Concerning the use of roles and mandatory relations in place of optional relations, our findings are in line with the ones achieved by Weber et al. [6].
Regarding relators, although the results also confirm our hypothesis, the evidences are more subtle. In fact, we perceived an influence of the experience level in these results. Although we have tried to balance the experience level of the modelers in each group, we did not really achieve such balance. Thus, further studies should be undertaken for improving our understanding regarding modelers' profile. In this sense, we intend to perform other experiments, taking as basis the design used by Hadar in [10]. In this work, she presents an empirical study on the difficulties that the object oriented paradigm presents to designers (even experienced ones) and points out the psychological motivations behind these difficulties. She intended to establish how the individuality of each professional interferes in design, with some indications of the reasons behind it. Her empirical findings suggest that a tension between intuitive and logical thinking modes may lead to simple mistakes software engineers exhibit when practicing object-oriented analysis and design.
This experiment collected evidence that we intend to use to guide future studies, more rigorous and focused. We are aware of some limitations and threats to validity of this study, as discussed in Section 3. So, we intend to use these results to orientate future efforts, improving the design of the experiments and deepening in the evidence identified. Some future works include: (i) Evidence points positively to the proposed hypothesis, but further studies should be developed to check them. For instance: Should the results be the same with larger and varied groups? Should the results be similar in the interpretation of larger models? (ii) We need to evaluate some limitations and threats to validity that we identified and did not explore yet. For instance, further studies should be undertaken for improving our understanding regarding modelers' profile.
Due to the lack of space we described the experiment partially in this paper. We intend to publish the complete result soon, also making possible the experiment replication by other researchers interested in the field.
This work was partially supported by the Lucretius ERC Advanced Grant #267856 and FAPES/Brazil.