=Paper=
{{Paper
|id=Vol-1859/emisa-02-paper
|storemode=property
|title=Design and Evaluation of a Web-based Modeling Platform to Support the Learning of Conceptual Modeling and of Studying the Corresponding Learning Processes
|pdfUrl=https://ceur-ws.org/Vol-1859/emisa-02-paper.pdf
|volume=Vol-1859
|authors=Benjamin Ternes
|dblpUrl=https://dblp.org/rec/conf/emisa/Ternes17
}}
==Design and Evaluation of a Web-based Modeling Platform to Support the Learning of Conceptual Modeling and of Studying the Corresponding Learning Processes==
https://ceur-ws.org/Vol-1859/emisa-02-paper.pdf
138
Design and Evaluation of a Web-based Modeling Platform to
Support the Learning of Conceptual Modeling and of
Studying the Corresponding Learning Processes
Benjamin Ternes1
Abstract: The research presented in this dissertation proposal aims at designing, developing, evaluating
and studying a web-based modeling platform to support the learning of conceptual modeling and of
studying the corresponding learning processes. More specifically, this research project is situated in a
distance learning and teaching context and pursues the primary objective of creating a learning and
teaching software for guiding and supporting the learning of conceptual modeling and of providing
the instructor with means to observe individual and aggregate learning processes. Learner interaction
with graphical model editors is tracked and analytics allow for insights at the individual and aggregate
learner levels.
Keywords: Construction-oriented research, learning conceptual modeling, learning process of
conceptual modeling, teaching conceptual modeling
1 Motivation, Background, and Objectives
Conceptual modeling as an activity involves a complex web of cognitive performances
and human actions including communicating, interpreting, abstracting, conceptualizing,
visualizing and justifying. Despite its proliferation in practice and its history in scientific
research spanning more than four decades, the learning process of conceptual modeling
has received only limited attention so far. Further knowledge about the learning process
of conceptual modeling—from the beginning modeler to the intermediate to the advanced
modeler—promises to assist in shaping learning support, e.g. in modeling tools, as well as
in shaping teaching didactics, e.g. in focussing on typical learning difficulties.
The present dissertation project is part of a long-term research program to study learning of
conceptual modeling, and is situated in a distance learning and teaching context involving
introductory courses on conceptual modeling with large cohorts (up to 1,500 students).
Primary objective of this dissertation project is to develop a user-friendly, intuitive and robust
browser-based modeling application, which allows for tracking modeler interactions to
obtain a more comprehensive understanding of how beginning modelers learn. The proposal
1 University of Hagen, Enterprise Modelling Research Group, Universitätsstr. 41, 58097 Hagen, benjamin.ternes@
fernuni-hagen.de
Web-based Modeling Platform 139
at hand reports on the current state of the dissertation project and, in particular, details
related work, outlines the current development stage of the web platform and describes
intended next steps in the research process.
2 Research Approach
The dissertation project follows a construction-oriented research method with a research
configuration tailored to the research artifact and cognitive interest [Fr06]. The main
knowledge contribution of this dissertation project is comprised of a software artifact, i.e.
the web-based modeling platform which features a novel approach to track the learning of
beginning modelers and for identifying and analyzing patterns of learning processes based
on tracking data. Selected steps of the research process include (1) critical analysis of the
literature to derive a set of requirements towards the artifact; (2) the design of the software
artifact, and (3) its evaluation in an initial experimental setting. To achieve these objectives,
a literature review is presently underway that addresses the theoretical foundations including
theories of language acquisition, learning and teaching conceptual modeling, improving the
process of process modeling, among others. In parallel, a prototypical implementation of
the web-based modeling platform has been developed including first approaches tracking
user interactions and analyzing tracking data.
3 Related Work
Early contributions to the discussion of improving the process of process modeling, e.g.
[PZW10, Pi14], considered to develop a modeling environment for the analysis of the
process of process modeling with a tool, named Cheetah Experimental Platform (CEP).
CEP is designed to support the workflow of controlled experiments—each step in the
experimental workflow uses components provided by CEP [PZW10, p. 15]. The Cheetah
Modeler, which is a part of CEP, enables the investigation of how process models are
constructed [PZW10, p. 16]. It further offers different analysis functionalities, e.g. Cheetah
Analyzer, which can record the process of creating process models with Cheetah Modeler
[PZW10, p. 17] and replay them with the integrated step-by-step execution. However, while
the step-by-step execution is a suitable approach to reconstruct the process of process
modeling, important thoughts and cognitive performances cannot be tracked, e.g. when
creating synchronizations/parallelizations. Another experimental environment to analyze
modeling behavior is JMermaid, see [SSD14]. This tool is very related to the presented
dissertation project and is based on a process mining analysis of conceptual modeling
behavior of novices. The authors focus on improving teaching practices in the field of
conceptual modeling by investigating the perspectives of process-oriented feedback. An
approach to visualize the generated results is introduced, e.g. [Cl13], allowing for further
analysis, e.g. for specific pattern reconstruction. Contrary to this approach, the dissertation
focusses on two specific modeling approaches: data modeling with a didactically prepared
variant of the Entity-Relationship Model [Ch76] and business process modeling with
140 Benjamin Ternes
MEMO OrgML [Fr11]. The rationale for preferring MEMO as learning language for
this dissertation project over Business Process Model and Notation (BPMN), ARIS or
comparable methods is based on two considerations: (1) MEMO provides an extensive
set of constructs for roles and units; (2) in contrast to other approaches, like ARIS, the
language specifications of the MEMO method are freely available and documented in several
publications. For future work, it could be significant to investigate the process of creating,
for example, organizational structure diagrams or other modeling approaches. In the context
of declarative process modeling and understandability, in [Ha13], an empirical investigation
is reported, asking subjects in an exploratory study to describe an illustrated process,
which was handed out before. Additionally, the authors applied a think-aloud approach
and recorded the subjects’ actions as audio- and video-files [Ha13, p. 7]. A framework for
assessing the perceptual properties of visual notations in software engineering is presented
in [Mo09]. A further approach for improving the process of process modeling is to use
predefined patterns. In [KR15], a set of Domain Process Patterns (DPP) for improving the
process of process modeling is introduced.
Eye tracking is a popular research method in psychology and for gathering information from
online web pages. Recording eye movements of participants viewing a web page can reveal
information about the page design and its strengths and weaknesses. To improve the insights
of the modeler behavior, [Pi13] presented an eye movement tracking analysis. However, eye
movement tracking only measures the participants’ attention and not which information
will be processed—looking at one place does not imply that a modeler recognizes or even
processes the information. About concerning the investigation of learning and teaching
conceptual modeling with eye movement tracking, biases resulting from different peripherals
have to be considered, e.g. the participants could unconsciously move the mouse to any
corner without losing their eye fixation points.
Additional related research fields, which have not yet been elaborated, are learning natural
languages and learning artificial languages. There might be similarities between learning
natural languages and learning artificial languages for conceptual modeling. Different
knowledge areas of further work remain unexplored, for example, cognitive science,
psychology and, especially the Cognitive Load Theory (CLT) [CS91], which is a widely
accepted theory and has been empirically validated in studies so far [Ba02] as well as
approaches and guidelines for learning, e.g. [Ki02], are intended to be worked on for the
dissertation project in the future. Based on previous research in this field, this project
differentiates at obtaining a more detailed understanding of how to track modeling beginners
by a web-based modeling platform and, also, to achieve more insights in the learning process
of modeling beginners. I assume such understanding to be valuable for the design of user
interface of modeling tools as well as for more accurate interpretations for implicit feedback
to support the learning and teaching process.
4 Status Quo
At present, a prototypical implementation of the web-based modeling platform has been
implemented exploring design and implementation strategies for efficient user tracking,
Web-based Modeling Platform 141
tracking data persistency and analytics. The prototype currently implements graphical
editors for a process modeling language MEMO OrgML [Fr11] and a variant of the Entity-
Relationship Model. Prior to the prototypical implementation, initial requirements have
been defined based on an analysis of literature on the process of process modeling, learning
conceptual modeling and tracking approaches as well as on learning visual notations.
These requirements are further extended and refined as the prototyping proceeds. Currently,
the theoretical background is further explored by further literature analysis with regard
to empirical and theoretical findings on learning, teaching, and language acquisition.
Subsequently, different tracking approaches will be integrated, such as history event tracking
and replay functionalities to gather a large amount of information over the user interaction
process with the graphical editors. Corresponding analytics of tracking data will be added
with first prototypical implementations already implemented.
5 Next Steps and Challenges
Next steps of the dissertation project include (1) a comprehensive literature review on
learning and teaching conceptual modeling and improving the process of process modeling;
(2) developing a final version of the web-based modeling platform; (3) implement further
functionalities to enrich the tracking of modeler interactions and to achieve insights into
learning processes of conceptual modeling. A fundamental assumption of the research is
that modelers’ learning process can be tracked by observing modelers’ interactions with
the modeling platform. For the dissertation project, it is an open question at present how
to enrich user interactions by additional means, e.g. by history tracking, video recording,
thinking out loud and other richer means of observations.
Many directions of further work have not yet been explored in the present dissertation
project. One major challenge is how to achieve insights into modelers’ learning process for
more accurate interpretations for implicit feedback on conceptual modeling and how to
track modelers’ interactions in an adequate fashion. Beyond the tracking of user interactions,
the influence of relevant other modeling aspects outside of the modeling tool has to be
considered. Which of these aspects could be captured how remains open. Achieving the
primary objective of this dissertation project implies to incorporate insights from a number
of other scientific disciplines, e.g., linguistics, semiotics and philosophy of language, and,
hence, suggests transdisciplinary collaboration—which I welcome and seek.
Acknowledgements
I would like to express my gratitude to my dissertation advisor, S. Strecker, for the discussions
and advice which have resulted from the joint work in preparation of this dissertation
proposal.
142 Benjamin Ternes
References
[Ba02] Bannert, M.: Managing cognitive load—recent trends in cognitive load theory. Learning
and Instruction, 12(1):139–146, 2002.
[Ch76] Chen, P. P.-S.: The Entity-Relationship Model—Toward a Unified View of Data. ACM
Transaction Database Systems, 1(1):9–36, 1976.
[Cl13] Claes, J.; Vanderfeesten, I.; Pinggera, J.; Reijers, H. A.; Weber, B.; Poels, G.: Visualizing
the process of process modeling with PPMCharts. In (La Rosa, M.; Soffer, P., eds): Business
Process Management Workshops. BPM 2012. LNBIP, volume 132, pp. 744–755. Springer,
Berlin, Heidelberg, 2013.
[CS91] Chandler, P.; Sweller, J.: Cognitive Load Theory and the Format of Instruction. Cognition
and Instruction, 8(4):293–332, 1991.
[Fr06] Frank, U.: Towards a pluralistic conception of research methods in information systems
research. Technical report, ICB-Research Report No. 7, Institute for Computer Science
and Business Information Systems (ICB), University Duisburg-Essen, 2006.
[Fr11] Frank, U.: MEMO Organisation Modelling Language (2): Focus on Business Processes.
Technical report, ICB-Research Report No. 49, Institute for Computer Science and Business
Information Systems (ICB), University Duisburg-Essen, 2011.
[Ha13] Haisjackl, C.; Zugal, S.; Soffer, P.; Hadar, I.; Reichert, M.; Pinggera, J.; Weber, B.: Making
sense of declarative process models: Common strategies and typical pitfalls. In (Nurcan,
S.; Proper, H. A.; Soffer, P.; Krogstie, J.; Schmidt, R. Halpin, T.; Bider, I., eds): Enterprise,
Business-Process and Information Systems Modeling. LNBIP, volume 147, pp. 2–17.
Springer, Berlin, Heidelberg, 2013.
[Ki02] Kirschner, P. A.: Cognitive load theory: implications of cognitive load theory on the design
of learning. Learning and Instruction, 12(1):1–10, 2002.
[KR15] Koschmider, A.; Reijers, H. A.: Improving the process of process modelling by the use of
domain process patterns. Enterprise Information Systems, 9(1):1–29, 2015.
[Mo09] Moody, D.: The ’Physics’ of notations: Toward a Scientific Basis for Constructing Visual
Notations in Software Engineering. IEEE Transactions on Software Engineering, 35(6):756–
779, 2009.
[Pi13] Pinggera, J.; Furtner, M.; Martini, M.; Sachse, P.; Reiter, K.; Zugal, S.; Weber, B.:
Investigating the process of process modeling with eye movement analysis. In (La Rosa,
M.; Soffer, P., eds): Business Process Management Workshops. BPM 2012. LNBIP, volume
132, pp. 438–450. Springer, Berlin, Heidelberg, 2013.
[Pi14] Pinggera, J.: The Process of Process Modeling. Dissertation, University of Innsbruck,
Department of Computer Science, 2014.
[PZW10] Pinggera, J.; Zugal, S.; Weber, B.: Investigating the Process of Process Modeling with
Cheetah Experimental Platform. In: Proc. ER-POIS’10, pp. 13–18. 2010.
[SSD14] Sedrakyan, G.; Snoeck, M.; De Weerdt, J.: Process mining analysis of conceptual modeling
behavior of novices–empirical study using JMermaid modeling and experimental logging
environment. Computers in Human Behavior, 41:486–503, 2014.