This research introduces the use of a Large Language Models (LLM) assistant into the modeling tool proposed for Tonto, a text-based language for ontologies based on the Unified Foundational Ontology (UFO). By integrating Tonto with LLMs, we aim to improve conceptual modeling by employing LLMs in assisting tasks such as summarizing models, inferring meta properties of classes and the ontological nature of their instances, and creating elements based on context. This project investigates how this integration can improve modeling eficiency, accuracy, and the overall user experience with an LLM-powered assistant.
Conceptual modeling is fundamental within Computer Science, impacting areas like AI and software
design. It ofers a way to formally represent real-world ideas, essential for building efective information
systems. Strong theoretical groundwork and tools are key to ensuring model quality and supporting
analysis. Recently, foundational ontologies have become the basis for conceptual modeling languages,
providing a structured framework for representing knowledge. The Unified Foundational Ontology
(UFO) [
UFO was employed in the design of the OntoUML profile for UML class diagrams [
Despite the various benefits of the language, it relies on a diagrammatic notation, which imposes
certain burdens for managing large models and coping with version management (among other tasks).
The utility of representing ontologies through a textual syntax has led to the creation of tools such as
Turtle [
LLMs have been proving to have good results when dealing with natural language tasks, and even
formal language tasks. A number of works are using Large Language Models to enhance modeling, for
example GPT4 with its code generating capabilities [
This work is based on the language for well-founded ontologies named Tonto (Textual Ontologies)1 and its set of tools in order to enable conceptual modeling based on Ontological foundations with UFO and OntoUML. The set of tools include a Command Line Interface (CLI)2, a Visual Studio Code Extension3, and a Package Manager to allow the modularization of models. Tonto syntax is created to be more concise and easier to read than OWL serializations, allowing fast understanding of concepts in a model and to better verify changes in version control tools. It also provides semantically-motivated syntactic verification of models to ensure ontological consistency.
Unlike OntoUML and Tonto, other existing textual modeling languages and the mentioned modeling approaches using LLMs lack the systematic grounding in a foundational ontology, limiting the benefits achievable from the rigorous distinctions and principles such a framework provides. Objectives. Tonto, as a textual language, has the potential for integration with LLM-based tools. Recent research highlights the remarkable zero-shot and few-shot capabilities of Large Language Models (LLMs), such as GPT-4 [20], in diverse Natural Language Processing (NLP) tasks. This demonstrates their potential as versatile tools for addressing NLP challenges [21].
Three key objectives are proposed: (i) Develop an integration Framework, creating a robust interface between Tonto and advanced LLMs (e.g., GPT [20]), which will enable bidirectional communication and processing of modeling language and natural language data, making it possible to insert and retrieve information from a model specified in Tonto with natural language and vice-versa; (ii) Explore the potential of LLMs to provide assistance in the following areas: Summarizing models, inference of stereotypes (using UFO theory) [22], and the creation of new elements for the model [15] for contextaware guidance; (iii) Assess the impact of LLM-powered features on modeling eficiency, accuracy, and user experience. Iterate on the integration framework for optimization.
Overall, the combination of UFO as a foundational ontology and LLMs have the potential to make conceptual modeling more eficient, accurate and with better UX.
The project will adopt a mixed-methods approach, combining both qualitative and quantitative research techniques. The first step is prototyping and experimenting, where the development of LLM-enhanced Tonto prototypes would be developed.
Prompt Engineering. According to [23], diferent techniques could be used with LLM in the prompt design to increase the quality of the results. One basic way is to describe the task and hope the LLM already knows how to do it, named zero-shot learning, however this is not always reliable. A better method is to provide a few examples of what you’re looking for, named few-shot learning, helping the LLM learn the pattern. For complex tasks, you can even show the LLM step-by-step how to reach the answer, making it easier for the model to follow your thinking. This method is called chain-of-thought. Evaluation. We propose doing controlled experiments comparing the use of traditional Tonto with the LLM-powered version, measuring task completion time, model quality metrics, and user satisfaction. Each analysis will compare the prompt engineering techniques to discover the best performing one. The second step is executing qualitative studies with domain experts to gather feedback on the utility and usability of LLM-augmented features, and in-depth interviews for identifying potential challenges and areas for improvement. 1The Tonto EBNF specification can be found at https://w3id.org/tonto/ebnf. 2https://w3id.org/tonto/cli 3The extension is available in the marketplace at https://w3id.org/tonto/extension.
The goal of this research is to answer three research questions: (i) Could an LLM-based assistant facilitate the development of higher-quality models with reduced efort? (ii) To what extent can LLMs improve the eficiency and accuracy of conceptual models created in Tonto? (iii) What are the key user perceptions and challenges regarding the integration of LLMs within a conceptual modeling environment?
This research holds the potential to improve the field of conceptual modeling by introducing AIpowered capabilities. Success could lead to more accessible and intuitive modeling environments and enhanced understanding of complex systems through clearer and better-structured models.
Many ideas for improving conceptual modeling using LLMs are emerging recently. For example, [17] proposes an approach to improve completion in domain modeling activities. [15] shows how given a textual description in its entirety, a complete domain model is derived without any human interaction. [16] enables domain experts to construct purposive models by operating at natural language level. [18] shows an automated approach for the extraction of UML class diagrams from natural language software specifications. Prior to LLMs, there were already some attempts for automated generation of Domain Models using NLP techniques [24, 25, 26] . None of these works take into account using a foundational ontology, like UFO, when using LLMs to build or complete ontologies, meaning that using LLMs with Tonto could make a contribution to the state of the art of the field. the 25th Intl Conf on Model Driven Engineering Languages and Systems: Companion Proceedings, 2022, pp. 257–266. [15] K. Chen, Y. Yang, B. Chen, J. A. Hernández López, G. Mussbacher, D. Varró, Automated Domain Modeling with Large Language Models: A Comparative Study, in: 2023 ACM/IEEE 26th Intl Conf on Model Driven Engineering Languages and Systems (MODELS), IEEE, Västerås, Sweden, 2023, pp. 162–172. doi:10.1109/MODELS58315.2023.00037. [16] V. Kulkarni, S. Reddy, S. Barat, J. Dutta, Toward a Symbiotic Approach Leveraging Generative AI for Model Driven Engineering, in: 2023 ACM/IEEE 26th Intl Conf on Model Driven Engineering Languages and Systems (MODELS), IEEE, Västerås, Sweden, 2023, pp. 184–193. URL: https:// ieeexplore.ieee.org/document/10343767/. doi:10.1109/MODELS58315.2023.00039. [17] M. B. Chaaben, L. Burgueño, H. Sahraoui, Towards using Few-Shot Prompt Learning for Automating Model Completion, in: 2023 IEEE/ACM 45th Intl Conf on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), IEEE, Melbourne, Australia, 2023, pp. 7–12. URL: https://ieeexplore.ieee.org/document/10173909/. doi:10.1109/ICSE-NIER58687.2023.00008. [18] S. Yang, H. Sahraoui, Towards automatically extracting UML class diagrams from natural language specifications, in: Proceedings of the 25th Intl Conf on Model Driven Engineering Languages and Systems: Companion Proceedings, MODELS ’22, Association for Computing Machinery, New York, NY, USA, 2022, pp. 396–403. URL: https://doi.org/10.1145/3550356.3561592. doi:10.1145/ 3550356.3561592. [19] R. Saini, G. Mussbacher, J. L. C. Guo, J. Kienzle, Machine learning-based incremental learning in interactive domain modelling, in: Proceedings of the 25th Intl Conf on Model Driven Engineering Languages and Systems, MODELS ’22, Association for Computing Machinery, New York, NY, USA, 2022, pp. 176–186. URL: https://doi.org/10.1145/3550355.3552421. doi:10.1145/3550355. 3552421. [20] OpenAI, GPT-4 Technical Report, 2024. URL: http://arxiv.org/abs/2303.08774. [21] C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, D. Yang, Is ChatGPT a General-Purpose Natural
Language Processing Task Solver?, 2023. URL: http://arxiv.org/abs/2302.06476. [22] P. P. F. Barcelos, T. P. Sales, E. Romanenko, J. P. A. Almeida, G. Engelberg, D. Klein, G. Guizzardi, Inferring Ontological Categories of OWL Classes Using Foundational Rules, in: Frontiers in Artificial Intelligence and Applications, IOS Press, 2023. URL: https://ebooks.iospress.nl/doi/10. 3233/FAIA231122. doi:10.3233/FAIA231122. [23] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, G. Neubig, Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing, 2021. URL: http://arxiv.org/abs/ 2107.13586. [24] M. Ibrahim, R. Ahmad, Class Diagram Extraction from Textual Requirements Using Natural Language Processing (NLP) Techniques, in: 2010 Second Intl Conf on Computer Research and Development, IEEE, Kuala Lumpur, Malaysia, 2010, pp. 200–204. URL: http://ieeexplore.ieee.org/ document/5489508/. doi:10.1109/ICCRD.2010.71. [25] M. S. Jyothilakshmi, P. Samuel, Domain ontology based class diagram generation from functional requirements, in: 2012 12th Intl Conf on Intelligent Systems Design and Applications (ISDA), IEEE, Kochi, India, 2012, pp. 380–385. URL: http://ieeexplore.ieee.org/document/6416568/. doi:10. 1109/ISDA.2012.6416568. [26] C. Arora, M. Sabetzadeh, L. Briand, F. Zimmer, Extracting domain models from natural-language requirements: approach and industrial evaluation, in: Proceedings of the ACM/IEEE 19th Intl Conf on Model Driven Engineering Languages and Systems, MODELS ’16, Association for Computing Machinery, New York, NY, USA, 2016, pp. 250–260. URL: https://doi.org/10.1145/2976767.2976769. doi:10.1145/2976767.2976769.