Automated Assistance for Data Modelers combining Natural Language Processing and Data Modeling Heuristics: A Prototype Demonstration? Benjamin Ternes1 , Kristina Rosenthal1 , and Stefan Strecker1 Enterprise Modelling Research Group, University of Hagen, Hagen, Germany, {benjamin.ternes,kristina.rosenthal,stefan.strecker}@fernuni-hagen.de Abstract. Identifiers of model elements convey semantics of concep- tual models essential to interpretation by human viewers. Prior research shows that devising meaningful identifiers for model elements challenges data modelers from early learning stages to advanced levels of mod- eling expertise, constituting one of the most common difficulties data modelers face. We demonstrate the Automated Assistant, an integrated modeling tool support component combining natural language process- ing techniques and data modeling heuristics to provide data modelers with modeling-time feedback on identifying and signifying entity types, relationship types, and attributes with meaningful identifiers. Different from other approaches to automating assistance for data modelers, the Automated Assistant implementation does not rely on fixed reference solutions for modeling tasks as it processes (m)any natural language de- scriptions of modeling tasks. We report on the current state of prototype development, discuss the Automated Assistant implementation and out- line future work. Keywords: Conceptual Modeling · Data Modeling · Modeling Tool · Natural Language Processing · Process-oriented Feedback 1 Introduction Model element identifiers (labels), e. g., for entity types, relationship types and attributes in an Entity-Relationship (ER) diagram, carry and convey semantics important to sensible interpretation of conceptual data models by human view- ers, semantics transcending the formal semantics of model elements [12]. Devising meaningful, appropriate, and expedient identifiers is a prerequisite for compre- hensible and usable conceptual models [7]. Empirical research shows that data modelers oftentimes face difficulties devising meaningful identifiers for model elements from early learning stages to advanced levels of modeling expertise, de- scribed as one of the most common difficulties beginning learners face [1,11]. To ? Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 26 B. Ternes, K. Rosenthal, S. Strecker support learners of data modeling in devising meaningful identifiers for model elements, we design and implement the Automated Assistant, a research pro- totype that provides modeling-time feedback to data modelers on their choice of model element identifiers based on the natural language description of the respective modeling universe of discourse (cf. [14]). The primary use case of the Automated Assistant is a learning context in which learners of data modeling are asked to create an ER diagram based on a textual description of a modeling task—a common learning scenario in many settings. Research, design and prototype implementation of the Automated As- sistant combine and build on research on data modeling heuristics for identifier formulation from natural language descriptions [2,15] and on research on Natu- ral Language Processing (NLP) [3,4]. Different from prior work (e. g.[9,15,5,8]), the Automated Assistant does not require the time-consuming ex ante construc- tion of (multiple) reference solutions to a specific modeling task description but works on (m)any natural language descriptions of a modeling task (as long as they comply with English grammar). The Automated Assistant prototype described in the present work substan- tially extends an earlier prototype implementation (cf. [14]). Specifically, we substantially extend the number of implemented data modeling heuristics for better detection of compound nouns signifying entity types as well as for detect- ing associated attributes, and we enhance the earlier prototype implementation by an identification component for data types of attributes. Moreover, the pro- totype implementation now presents generated candidates for element identifiers color highlighted to the modeler. 2 The Automated Assistant Prototype Overview The Automated Assistant prototype implementation builds on TOOL, a model- ing tool web application [13]. TOOL provides a graphical data modeling editor which implements a variant of the Entity-Relationship Model [10] to create ER diagrams. The Automated Assistant hooks into the graphical editor to assist data modelers at modeling-time with suggestions on signifying element identi- fiers. The Automated Assistant implementation utilizes the Stanford CoreNLP toolkit [6] which provides a pipeline framework for deriving linguistic annota- tions from natural language text input, in combination with heuristics rules for data modeling that we adapt and refine to the in-browser ER modeling vari- ant. Moreover, the Automated Assistant implements widely accepted naming conventions for ER modeling (e. g., [2]). In a nutshell, the prototype implementation of the Automated Assistant es- sentially applies a two-step procedure: First, starting from a (1) (natural lan- guage) modeling task description, a list of potential candidates for meaningful identifiers for entity types, associated attributes, and relationship types is gener- ated. Then, based on the results of step (1), feedback (2) is provided to modelers at modeling-time in the graphical modeling editor on their choice of model el- ement identifiers and data types—potential candidates for meaningful entity Automated Assistance for Data Modelers 27 Execution flow Executing heuristic rules & naming conventions Rule head (if) Named Entity Recognition Part-of-Speech Tagging Morphological Analysis (e.g., gender, dcoref) Sentence Splitting Syntactic Parsing Other Annotators Tokenization (tokenize) (lemma) (parse) Heuristics rules (ssplit) (pos) (ner) & naming conventions 37 Heuristics Natural language Annotated text 3 Conventions Lists of description of a (e.g., amod, NN) candidates modeling task Stanford CoreNLP pipeline Rule tail (then) (adapted for our application context) Linguistic annotations generates Fig. 1. Overview of the adapted annotation pipeline to generate a list of candidates for entity types, associated attributes, and relationship types (exemplary linguistic annotations generated using the the Stanford CoreNLP toolkit at https://corenlp.run). type identifiers as well as corresponding attributes, and data types are color highlighted in the textual description displayed to the modelers. To generate a list of potential candidates based on natural language description (cf. step 1), the pipeline of the CoreNLP toolkit is adapted to the primary use case of the Automated Assistant (see Fig. 2), and enhanced in several design iterations by conscientiously revising the applied heuristic rules for classifying certain pat- terns of statements of the English language for identifying identifiers of model elements (e. g., [2]). The identification of potential candidates mainly relies on part-of-speech tagging and statistical dependency parsing (e. g., words which modify nominal phrases—adjectival modifiers) and to establish (grammatical) relations between words (e. g., by typed dependencies). The Automated Assis- tant utilizes multiple NLP techniques (built into the CoreNLP implementation) for the analysis of modeling task descriptions in a specific order: tokenization, sentence splitting, part-of-speech tagging, morphological analysis, and syntactic parsing (see Fig. 2). The subsequent identification of candidates for identifiers for entity types, associated attributes, and relationship types builds on linguis- tic annotations returned from CoreNLP as a semantic graph consisting of lists of tuples. Starting from these linguistic annotations, the revised prototype im- plementation applies 37 heuristic rules (13 on relationship types, 13 on entity types, and 11 on associated attributes) and 3 naming conventions for formulat- ing model element identifiers to match them with the syntactical functions of the annotation pipeline. The heuristic rules and naming conventions are recon- structed, adapted and refined from a comprehensive review of prior work (the adapted annotation pipeline is described in further detail in [14]). 28 B. Ternes, K. Rosenthal, S. Strecker From a modeler’s perspective, the Automated Assistant suggests feedback (cf. step 2), i. e., whenever a modeler devises identifiers of a model elements on the modeling canvas of the graphical modeling editor (see Fig. 2). The provided feedback aims at supporting modelers in developing an understanding of how to identify and signify model elements meaningfully, and, in particular, to encourage modelers to rethink their choices for model element identifiers based on sensible auto-generated suggestions of the Automated Assistant. Hence, the Automated Assistant provides feedback in three categories: (a) ‘Great’ for positive feedback, e. g., if a label matches an entity type of the generated candidate list (in green); (b) ‘Reconsider’ for a label that is not mentioned in the modeling task description or that is not identified as a possible candidate for an entity type, attribute, or relationship type (problem-oriented feedback; in orange); (c) ‘Convention’ if the input does not follow the implemented naming conventions, e. g., if an entity type label begins with a lowercase letter, or a spelling error is examined (neutral feedback; in yellow). As a major usability enhancement to an earlier version of the Automatic Assistant (cf. [14]), we have further extended the integrated modeling tool support to color highlight possible entity types and associated attributes in the textual description, e. g., when there is uncertainty or when the modeler wants to compare them to the chosen model element identifiers on the modeling canvas (see Fig. 2, right side). Compared to earlier prototype implementations, the Automated Assistant now provides more accurate feedback on data types of attributes based on named entity recognition. A short video demonstrator of the Automated Assistant is available at: https://video.fernuni-hagen.de/Play/896 Fig. 2. Feedback on the modeler’s choice of model element identifiers (‘consultant’, ‘ProgrammingLanguage’, etc.) provided by the Automated Assistant (highlighted in color below ER diagram and on the right side in the task description) based on the processing of the natural language description of the task shown on the right by the Automated Assistant. Automated Assistance for Data Modelers 29 3 Discussion and Outlook The Automated Assistant provides modeling-time feedback to modelers on their choice of model element identifiers based on natural language processing of mod- eling task descriptions to assist in overcoming one of the most common (learning) difficulties data modelers face, i. e., devising meaningful and expedient identi- fiers for model elements [1,10]. Evaluation of the feedback generated by the Automated Assistant from three modeling task descriptions of increasing length and complexity suggests that the generated feedback is similar to human (e. g., instructor’s) advice, and helpful to learner of data modeling. Compared to earlier prototype implementations (see [14]), precision and re- call for five demonstration cases, i. e., modeling tasks used in teaching concep- tual data modeling, improved in recognizing entity type and relationship type identifiers from the text descriptions by an average of 0.045 with the largest in- crease in attribute identification (entity types: precision 0.88 (from 0.84), recall 0.97 (from 0.94); attributes: precision 0.86 (from 0.79) 0.86, recall 0.79 (from 0.71); relationship types: precision 0.84 (from 0.84), recall 0.74 (from 0.69)). The newly added identification of data types achieves a precision of 1.00 and a recall of 0.66. In terms of precision and recall, the current prototype implemen- tation performs on par or better than related approaches (e. g., [5,9]), especially with respect to finding candidates for entity type identifiers, with prior research demonstrating values for recall from 0.92 [9] to 0.95 [8]. Closer inspection of precision and recall results show, as expected, the heuristic nature of the rules applied to recognize relationship types, entity types, attributes, and correspond- ing data types—and of the NLP techniques we use. Still, compound nouns with three (and more) words such as ’social security number’ are, for example, not always properly recognized—a limitation we have already addressed by revising the implemented data modeling heuristics for entity types and attributes but which needs further work. For example, further prototype evaluation exemplifies that the identification of inherent relationships of compound nouns should be improved posing a particular challenge as there are many ways to combine them (see [5]). Moreover, we are currently implementing heuristic rules for identifying generalization hierarchies in textual descriptions to provide data modelers with automated assistance in modeling generalization relationships. Future work on the Automated Assistant will also address the identification and recognition of synonyms in the textual description by utilizing word embedding approaches such as WordNet. We are currently planning for further systematic evaluation by empirical studies investigating how well the feedback provided by the Automated Assis- tant supports learners of data modeling in devising meaningful and appropriate identifiers for model elements. Building on prior mixed-methods research into data modeling processes [10,11,13], modelers will be observed from complemen- tary perspectives, e. g., based on the think-aloud method and surveying modelers on their perception of the feedback as provided by the Automated Assistant. Sub- sequently, we are planning to apply the Automated Assistant in an introductory course on data modeling with 200+ students per semester to collect feedback on the support provided by the Automated Assistant. 30 B. Ternes, K. Rosenthal, S. Strecker References 1. Batra, D., Hoffer, J.A., Bostrom, R.P.: Comparing representations with relational and EER models. Communications of the ACM 33(2), 126–139 (1990) 2. Chen, P.P.: English Sentence Structure and Entity-Relationship Diagrams. Infor- mation Sciences 29, 127–149 (1983) 3. Chowdhury, G.: Natural language processing. Annual review of Information science and technology 37, 51–89 (2003) 4. Liddy, E.D.: Enhanced Text Retrieval Using Natural Language Processing. Bulletin of the American Society for Information Science and Technology 24(4), 14–16 (2005) 5. Lucassen, G., Robeer, M., Dalpiaz, F., van der Werf, J.M.E., Brinkkemper, S.: Ex- tracting conceptual models from user stories with Visual Narrator. Requirements Engineering 22(3), 339–358 (2017) 6. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The Stanford CoreNLP Natural Language Processing Toolkit. In: 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. pp. 55–60. Baltimore, Maryland (2014) 7. Mendling, J., Reijers, H.A., Recker, J.: Activity labeling in process modeling: Em- pirical insights and recommendations. Information Systems 35(4), 467–482 (2010) 8. Omar, N., Hanna, P., Mc Kevitt, P.: Heuristics-based entity-relationship modelling through natural language processing. In: 15th Artificial Intelligence and Cognitive Science Conference. pp. 302–313. Dublin, Ireland (2004) 9. Omer, M., Wilson, D.: Implementing a Database from a Requirement Specification. International Journal of Computer and Information Engineering 9(1), 33–41 (2015) 10. Rosenthal, K., Strecker, S.: Toward a taxonomy of modeling difficulties: A multi- modal study on individual modeling processes. In: 40th International Conference on Information Systems, ICIS 2019, Munich, Germany, December 15–18, 2019 (2019) 11. Rosenthal, K., Strecker, S., Pastor, O.: Modeling difficulties in data modeling: Similarities and differences between experienced and non-experienced modelers. In: 39th International Conference on Conceptual Modeling, ER 2020, Vienna, Austria, November 3–6, 2020. LNCS, vol 12400. pp. 501–511 (2020) 12. Speaks, J.: Theories of Meaning. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, winter 2019 edn. (2019) 13. Strecker, S., Rosenthal, K., Ternes, B.: Studying Conceptual Modeling Processes: A Modeling Tool, Research Observatory, and Multimodal Observation Setup. In: Gimpel, H., Krämer, J., Neumann, D., Pfeiffer, J., Seifert, S., Teubner, T., Veit, D., Weidlich, A. (eds.) Market Engineering – Insights from Two Decades of Research, pp. 99–111. Springer, Cham (2021) 14. Ternes, B., Rosenthal, K., Strecker, S.: Automated Assistance for Data Modelers: A Heuristics-based Natural Language Processing Approach. In: 29th European Conference on Information Systems, ECIS 2021, Marrakech, Morocco, June 15–17, 2021 (2021) 15. Tjoa, A.M., Berger, L.: Transformation of requirement specifications expressed in natural language into an EER model. In: Elmasri, R., Kouramajian, V., Thal- heim, B. (eds.) Proceedings of the 12th International Conference on the Entity- Relationship Approach, ER 1993, Arlington, Texas, USA, December 15–17, 1993. LNCS, vol 823. pp. 206–217 (1994)