TOOL-A modeling tool and modeling observatory: An update on research and prototype development

Abstract

The research prototype TOOL integrates a modeling tool with a modeling observatory for studying modelers' reasoning, modelers' line of modeling arguments, and their modeling decisions while conceptual modeling. Studies using TOOL as a modeling observatory yield insights, for example, into modeling dificulties experienced and non-experienced modelers encounter. The TOOL modeling observatory combines multiple modes of observation and data collection including (1) tracking modeler-tool interactions on the modeling canvas, (2) recording verbal data protocols of modelers' thinking out loud, (3) mouse pointer on-screen tracking and full-screen capturing, and (4) surveying modelers before and after modeling-to account for the richness of the cognitive processes involved in conceptual modeling, and to contribute to a richer understanding of modeler reasoning and decision-making, to identify common modeling and learning dificulties, and, ultimately, to design tool support to mitigate dificulties and to improve assistance for (non-)experienced modelers. As a companion to the prototype demonstration, we summarize changes, improvements and new prototype features.

Keywords

• eol>Modeling tool
• Conceptual modeling
• Data modeling
• Business process modeling
• Design Science Research

1. Introduction and Context

What (non-)experienced modelers reason while conceptual modeling and how they arrive at modeling decisions, which modeling and learning dificulties they face and why, and how to overcome these dificulties by tailored modeling tool support are questions of relevance and importance to practicing modelers and, likewise, to conceptual modeling research. For the past nine years, we have been designing, developing, and evaluating TOOL, a modeling tool integrating a research observatory aimed at studying individual modeling processes online, in the field, and under laboratory conditions—to contribute to a richer understanding of modeler reasoning, modelers’ line of modeling arguments, and modelers’ decision-making—and to identify common modeling and learning dificulties, and modeling styles and related patterns of modeling processes. Studying progressively more and more individual modeling processes working on modeling tasks of diferent complexity promises to cumulatively contribute to the empirical foundation of conceptual modeling research, and, ultimately, to enable us to design targeted (tool) support for modelers at diferent stages of their learning and mastering of conceptual modeling [ 1 ].

2. TOOL prototype : Recent Updates

TOOL is designed and implemented as a web application with a JavaScript-driven front-end and a Java EE (Enterprise Edition)-based backend, and, as a design research artifact, also serves as a research laboratory for studying (web technology) software stacks and tool-chains (such as developing, shipping, and running applications). As a research subject by itself, we develop, study, iteratively refine, explore and evaluate TOOL, its software architecture, underlying technology stack as well as our development tooling and our (academic) software development process.

TOOL currently implements three graphical modeling editors, with the Entity-Relationship Model (ERM), and Business Process Model and Notation (BPMN) editors previously discussed in [ 2 ]. A major update pertains to the implementation of a graphical metamodeling editor using the MEMO Meta Modeling Language (MML) [ 3 ] which significantly simplifies the addition of further modeling languages (or of variants of already implemented languages), see Fig. 1. Metamodels created using the new metamodeling editor result in textual stencil set exports of explicit typing rules (abstract syntax) as a starting point for the implementation of further modeling languages in TOOL. Stencil sets provide explicit typing, connection rules, visual appearance (concrete syntax), and other features that diferentiate a model editor from generic vector-oriented drawing tools. 1–4

In addition, we address the ergonomics and usability of the graphical modeling editors: Modeler feedback from production use, observations during studies, and systematic testing were used to improve the usability and ergonomics of the graphical editors for both ERM and BPMN. Most of these improvements are subtle and detailed, but lead to an improved and more eficient modeling experience—one of our main requirements and research and development objectives, cf. [ 4 ]. For example, in the ERM editor, TOOL now provides auto-guided (quick) modeling features with targeted support for adhering to the abstract syntax of the modeling language, and an auto layout feature for aligning model elements on the canvas. Moreover, we enhanced the Automated Assistant presented in [ 4, 5 ]. The Automated Assistant is an NLP-based feedback assistant built into TOOL’s graphical ERM editor that suggests meaningful candidates of entity type, corresponding attribute, and relationship type identifiers based on a natural language description of a modeling task (see Fig. 2). Specifically, the Automated Assistant now uses Glove (Global Vectors for Word Representation, comparable with WordNet and Word2Vec approaches) to identify synonyms and homonyms, i. e., words are represented as vectors, named word vectors, to obtain real numbers for comparing words and their meaning in a context which improves its precision and recall [ 4, 5 ]. 3. Observation studies using TOOL as a modeling observatory The TOOL prototype has enabled us to conduct complementary observation studies on individual modeling processes leading to revised insights into modeling dificulties of more and less experienced data modelers [ 6 ] and into modeling styles of said data modelers [ 7 ]. From 2018 to 2021, TOOL prototypes served to collect close to 100 hours of verbal protocol data and modeler-tool interaction data etc. in studies run at the University of Hagen, the Universitat Politècnica de València, Spain and the Katholieke Universiteit Leuven, Belgium [ 1 ]. Findings from these studies feed back into research and development on TOOL aimed at improving targeted modeler tool support, e. g., by implementing modeler support for identifying sensible contextual labels for model elements via Natural Language Processing (NLP); a research path we intend to pursue further [ 5 ]. In January 2022, we have, for the first time, conducted a research study online accessing the TOOL prototype over the Internet. In a research setting similar to the earlier within-the-same-room observation studies, eight subjects were observed while performing a data modeling task (with Zoom employed for observing the subjects’ gestures and facial expressions as additional cues). Overall, the TOOL instance used in the study showed a reliable performance under varying network conditions at peak times of home ofice work in the COVID epidemic. In one exceptional case, we encountered data loss due to network failure at the subject’s site, so that the observation had to be restarted and run again from the beginning. Apart from this isolated case, no technical issues with TOOL were observed.

References

[1] S. Strecker , K. Rosenthal , B. Ternes , Studying conceptual modeling processes: A modeling tool, research observatory, and multimodal observation setup , in: H. Gimpel (Ed.), Market Engineering, Springer, 2021 , pp. 99 - 111 .

[2] B. Ternes , K. Rosenthal , S. Strecker , J. Bartels, TOOL-A Modeling observatory & tool for studying individual modeling processes , in: J. Michael , V. Torres (Eds.), Proceedings of the Demo Track co-located with the 39th International Conference on Conceptual Modeling, ER 2020 , Vienna, Austria, Nov 3- 6 , 2020 , pp. 178 - 182 .

[3] U. Frank , The MEMO Meta Modelling Language (MML) and Language Architecture , ICB Research Report 43 , University Duisburg-Essen, 2011 .

[4] B. Ternes , K. Rosenthal , S. Strecker, Automated Assistance for Data Modelers combining Natural Language Processing and Data Modeling Heuristics: A Prototype Demonstration , in: R. Lukyanenko , B. M. Samuel , A . Sturm (Eds.), Proceedings of the ER Demos and Posters, 40th International Conference on Conceptual Modeling (ER 2021 ), volume 2958 of CEUR Workshop Proceedings, CEUR-WS.org , 2021 , pp. 25 - 30 . URL: http://ceur-ws. org/ Vol- 2958 /paper5.pdf.

[5] B. Ternes , K. Rosenthal , S. Strecker, Automated assistance for data modelers: A heuristicsbased natural language processing approach , in: European Conference on Information Systems, ECIS 2021 , Research Papers, 148 , Association for Information Systems, 2021 . https://aisel.aisnet.org/ecis2021_rp/148.

[6] K. Rosenthal , S. Strecker , O. Pastor , Modeling dificulties in data modeling - similarities and diferences between experienced and non-experienced modelers , in: G. Dobbie, U. Frank, G. Kappel, S. W. Liddle , H. C. Mayr (Eds.), Conceptual Modeling - 39th International Conference, ER 2020 , Vienna, Austria, November 3- 6 , 2020 , Proceedings, volume 12400 of Lecture Notes in Computer Science, Springer, 2020 , pp. 501 - 511 .

[7] K. Rosenthal , J. Wagner , S. Strecker , Modeling styles in conceptual data modeling: Reflecting observations in a series of multimodal studies , in: European Conference on Information Systems, ECIS 2022 , Research Papers, 73 , Association for Information Systems, 2022 . https: //aisel.aisnet.org/ecis2022_rp/73.