This article presents the advantages of formal approaches to terminology evaluation and validation. The focus will be solely on so-called "conceptual" terminologies, for which a term is a "designation that represents a general concept by linguistic means" [1] and a concept a "unit of knowledge created by a unique combination of characteristics" [ibidem]. Evaluation and validation will focus on the conceptual system represented in the form of an ontology in the sense of knowledge engineering. It will then be possible to formally verify many properties, such as the consistency of concept definitions, and metrics such as the structural richness of the notional system. Nevertheless, the price to be paid is not insignificant and calls for new solutions.
Terminology evaluation is an essential part of terminology building. The task is made all the more difficult by the large number of criteria involved: terminological coverage, clarity, precision and consistency of definitions, richness of the conceptual model, compliance with standards, and so on. The definition of these criteria varies according to whether the emphasis is placed on the linguistic or the conceptual dimension of terminologies, and according to the linguistic and conceptual theories chosen.
For the purposes of this article, we will restrict ourselves to so-called conceptual
terminologies2. That is, terminologies for which a term is a "designation that represents a general
concept by linguistic means" [
To this limitation, we add a second. We will focus solely on the conceptual dimension, bearing
in mind that it is not without consequences for the linguistic dimension, if we consider that the
definition of a term is a linguistic explanation of the formal definition of the concept denoted by
the term [
The advantage of a formal and computational representation of the conceptual system5 is that it allows verifying properties, such as the consistency of concept definitions. It would have been 3rd International Conference on “Multilingual digital terminology today. Design, representation formats and management systems” (MDTT) 2024, June 27-28, 2024, Granada, Spain.
roche.university@gmail.com (C. Roche); mpapado811@gmail.com (M. Papadopoulou) 0000-0002-0756-0559 (C. Roche); 0000-0001-6451-8712 (M. Papadopoulou) © 2024 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
2 In line with ISO standards TC 37 and the General Theory of Terminology (GTT) [19].
3 Stable knowledge at a given time for a community of practice. Clearly, domain models can be subject to
modifications (evolutions). Taking diachrony into account, both linguistically and conceptually, is a separate issue.
4 In referential lexical semantics, for instance, the concept is considered to be extralinguistic knowledge whose
definition does not depend on the discourses to which it may give rise, and it ought not to be confused with the
signified of a term, which is a question for linguistics.
5 A formal and computational representation of the conceptual system is an ontology in the sense of Knowledge
Engineering. [21] defines an ontology as an “explicit specification of a conceptualization”. [22] presents different
difficult to do if we had only used natural language "definitions". It also allows applying metrics
to the conceptual structure, such as "Inheritance Richness", which measures the distribution of
information on the different levels of the class hierarchy [
The article is structured in six chapters. It starts with an introduction. The second chapter is dedicated to issues of competency questions, which play an important role in the design, evaluation, and validation of ontologies. After a presentation of the field of application in section 3, we present the different concept theories used in terminology and the different formalisms and representation languages (part 4). The fifth chapter will be devoted to evaluation criteria and their implementation, and in particular to the translation of competency questions into a computer-readable language enabling the evaluation of the conceptual system. We shall thus see, as one might have suspected, the importance of concept theory and its representation in the evaluation of the conceptual system of terminology and the problem posed by the concept theory. The article concludes by proposing an alternative to the Description Logic approach currently dominant in terminology.
The article will be illustrated with an example in Digital Humanities about ancient Greek vases, kraters6, in particular, based on the Beazley pottery database.7
Competency questions are questions that the system will be expected to answer satisfactorily.
[20]. Posed at the start of a project, they are used in many areas of IT. When applied to ontology
building, they come into play in the specification and validation phases. In the specification phase,
they are used to identify the concepts involved, their structure, and their relationships. They are
also used in the validation phase, to check that the conceptual model does indeed answer the
questions posed, provided that these competency questions can be translated into a language that
allows the computer representation of the conceptual system to be queried. [
Competency questions are chosen so as to address, as far as possible, all aspects of the linguistic and conceptual dimensions of terminology: identification of terms and their equivalents as well as their definition, identification of concepts, and the relationships linking them, whether hierarchical or associative. "What are the different types of vases?", "What is the definition of a bell krater?", "What are the differences between a column krater8 and a bell krater9?" are three examples of competency questions that, applied to the field of ancient Greek vases, raise different problems and require different solutions. The first refers to the hierarchy of concepts and their designations, while the second raises the problem of defining a concept and its representation. The third question adds a further problem to the previous one. It requires a formal representation of the definition that can be subjected to computation.
In this article, we'll be focusing on ancient Greek vases, and more specifically on kraters as they appear in the Classical Art Research Centre's Beazley Archive Pottery Database (BAPD)10 openly accessible online. "The term 'krater' suggests a mixing-vessel (compare Greek kerannumi - to mix), and we know that the wine served at the symposium was mixed with water. On vases decorated with symposium-scenes, a large open container with a foot is often depicted, and the name krater is appropriate. Examples can be traced back to the large Geometric examples that were used as grave-markers, and this funerary connection continues to be important. Excavations of burial-sites have shown that they could be used in Greek settlements overseas as containers of definitions of ontology, which we can summarize as follows: an ontology is a shared set of concepts and relationships of a domain expressed in a formal and computer-readable language. 6 A krater is an ancient Greek vase for mixing wine and water for the use of participants at ancient Greek symposia (banquets or dinner parties). 7 Beazley Archive Pottery Database: https://www.carc.ox.ac.uk/carc/pottery 8 https://www.carc.ox.ac.uk/record/70930F54-F3E9-4CD8-9626-305CEF7FC47A. 9 https://www.carc.ox.ac.uk/record/0E901EB4-23AB-4549-AEB9-F8F4BA0DDBB6. 10 The Classical Art Research Centre (CARC) is one of the strategic research units of the Oxford University Faculty of Classics. The online Beazley Archive Pottery Database (BAPD) is at the heart of CARC's activities [Beazley]. ashes, and South Italian, especially Apulian, volute-kraters often carry explicit funerary iconography. In the Athenian repertoire, there are four main types identified today: column-, volute-, calyx- and bell-.”11
This field lends itself to a definition of concepts as a combination of essential characteristics12
as recommended by the ISO 1087 and 704 standards (see Figure 1). These characteristics are
reflected both in the terms used to designate the concepts and in the definitions themselves:13
"column-krater": « Named for its column-like handles, the column-krater is […] », “volute-krater”:
« The volute-krater is named after its handles », “calyx-krater”: « The handles of the calyx-krater
are placed low down on the body[…] » [
Insofar as the conceptual system is based on essential characteristics, both in terms of the definition of concepts and their organization, the problem posed is that of their representation. Examples of such characteristics include the function of kraters (mixing wine and water) versus amphorae (storage and transport), the type of mouth (in this case, open), the presence or absence of a neck or foot, the type of handles and their placement on the vase, and so on.
There are several theories of the concept15 depending on whether the focus is on the nature of
objects, on their structure, e.g., in terms of attributes, or on their relationships (for a detailed
11 https://www.carc.ox.ac.uk/carc/resources/Introduction-to-Greek-Pottery/Shapes/Kraters.
12 According to the ISO 1087 standard, the exact definition is as follows: « unit of knowledge created by a unique
combination of characteristics ». It also includes non-essential features in the sense that they are not essential to
understanding the concept. For the purposes of this article, we will only consider essential characteristics.
13 These are definitions of "thing", in the sense that they describe what the objects denoted by the term are.
14 https://www.carc.ox.ac.uk/carc/resources/Introduction-to-Greek-Pottery/Shapes/Kraters.
15 A concept is understood here as domain knowledge about a plurality of things that verify the same property. It
aims to organize the objects that populate reality.
presentation of these theories, see [
Depending on the field, terms are far from arbitrary but are valuable sources of information.
This is the case when objects are named according to their function, « the term 'krater' suggests
a mixing-vessel (compare Greek kerannumi - to mix) » [
Comparing objects to find out what sets them apart also helps to identify characteristics that are both essential and delimiting. Bell kraters and calyx kraters both have upward-curling handles, a characteristic that distinguishes them from column kraters and volute kraters. Bellkraters and calyx-kraters are differentiated by the positioning of the upward-curling handles, placed high on the body for the former and low on the body for the latter. Bell-kraters and calyxkraters are named after the shape of the body for the former and the handles for the latter (Figure 2).
Figure 3 below presents the Krater ontology in the form of a Porphyrian tree19. In order to
clearly distinguish between the linguistic and conceptual dimensions, as well as the different
16 ISO 1087 [
Thus, the name of a concept21 is not a term, since they do not belong to the same dimension22. While terms are given by discourse, concept names are built in such a way that, by reading them, we understand the nature of the objects subsumed by the concepts. Thus, the name of a species (specific concept) is built from the name of the genus (generic concept) plus the name of the specific difference (differentia specifica) (see Figure 3).
The same applies to term definitions written in natural language and concept definitions written in formal languages. A term definition is a definition of "thing", i.e. a linguistic explanation of the concept denoted by the term. The two definitions are therefore closely linked. If the chosen concept theory allows, it is possible to generate natural language definition patterns for the term from the formal definition of the concept denoted by the term. Thus, from the formal definition of the concept <Vessel for mixing wine with water with open mouth with foot with handles>, denoted by the term "krater" in English, constructed from the generic concept <Vessel>24: <Vessel for mixing wine with water with open mouth with foot with handles> ::= <Vessel>+/for mixing wine with water/+/with open mouth/+/with foot with handles/ it is possible to generate the following definition of the term "krater" in English: "Vessel for mixing wine with water, with open mouth, with foot with handles. The terminologist or expert can then edit the definition to improve the wording. 20 An ontoterminology is a terminology whose conceptual system is a formal ontology. [9]. 21 The use of the term "identifier" instead of "concept name" would be more appropriate, since the concept is extralinguistic. 22 If we consider that a concept is extralinguistic knowledge, in the sense that it is not linguistic knowledge, but domain knowledge. Concept representation requires artificial knowledge representation languages. 23 CmapTools is a concept map editor [CmapTools], a semi-formal representation language, that is an important aid for identifying and structuring concepts before formalizing them. 24 Written in the LOK language (Language for Ontological Knowledge) of the ontoterminology construction environment TEDI: http://ontoterminology.com/tedi.
Among the various possible representation formalisms, we should mention RDF (Resource Description Format) [10] and its extensions. These W3C standards make it possible to represent terminologies, both the conceptual system and the linguistic dimension, in the form of knowledge graphs for sharing and manipulation purposes. The diversity of these languages makes it possible to represent the different dimensions of terminology. These formalisms, at the heart of Linked and Open Data and the Semantic Web, have become an absolute must25. SKOS, an acronym for Simple Knowledge Organisation System [11], can be used for terminology purposes. Concepts are represented as skos:Concepts, and terms as labels attached to concepts using properties such as skos:prefLabel. Using the skos:definition property is a simple way of representing the concept definition. While this formalism answers the 1st competency question - the concept hierarchy is a hierarchy of SKOS concepts linked by the skos:broader and skos:narrower properties - it reduces the definition to a simple textual annotation (skos:definition) and is unable to answer the 3rd question. Nor does it address consistency issues relating to the definition of concepts.
Checking logical properties, and reasoning about the conceptual model, requires a formal representation of the conceptual system and, in particular, a formal representation of the definition of concepts. This is why, for the purposes of this article, we have chosen the dominant model of Description Logic [12] and their implementation in the ontology-building environment Protégé [13] [14]. Protégé is the most widely used free open-source environment, benefiting from a large community of users. The notion of concept is replaced by that of class, centered on the notion of individuals (objects26). The definition of classes is based not on what things "are", but on their relationships (properties). Thus, a class is formally defined by restrictions on the relationships between its instances. Class and concept are important notions that, far from being mutually exclusive, complement each other27. The notion of essential characteristic, on which the concept is based, does not exist in Description Logic. The essential characteristics must therefore be translated into the form of individuals linked by relationships. This requires a change of perspective on the object. From the object considered as a whole, we need to "break it down" into a multitude of parts linked by relationships. Certain essential characteristics lend themselves well to this analysis28. Thus, the essential characteristic /with column like handles/ will be expressed in the form of a restriction on the "has-part" property, requiring every column krater to be linked to at least one column-like handle29. The same applies to other parts of the vase, such as the neck and foot. But what about characteristics such as function? One solution is to create as many individuals as there are characteristics, but this raises problems of a different kind. Indeed, insofar as relationships only link individuals, classes must themselves be considered as individuals. Defining a class as an instance of itself (i.e., punning30), however open to criticism, is a possible solution.
Ontologies are exported in the form of RDF knowledge graphs31 whose formal dimension is represented using vocabularies32 specific to ontologies such as OWL (Ontology Web Language) 25 These formalisms are vocabularies for naming concepts and properties dedicated to modeling a domain. 26 Objects are generally referred to as individuals. The elements of a class are also called instances of the class. 27 Some notions lend themselves better to modeling in the form of classes than concepts. Thus, the class of Parisians, people living in Paris, is the set of individuals of type 'Person' linked to the individual 'Paris' by the relation 'lives-in' and is formally defined by: { x / Person (x) ∧ lives-in (x, Paris) }. 28 We could even argue the opposite, that certain essential characteristics simply reflect relationships between individuals. This is the case with the partitive relationship. 29 It is even possible to specify the exact number of handles. 30 https://www.w3.org/2007/OWL/wiki/Punning. 31 A knowledge graph is a knowledge base that stores knowledge in a machine-readable form. 32 Unfortunately, the W3C does not make a clear distinction between vocabularies, terminologies and ontologies when referring to the various RDF formalisms: "There is no clear division between what is referred to as "vocabularies" and "ontologies". https://www.w3.org/standards/semanticweb/ontology. [15]. Figure 4 is an illustration of the kraters example built using Protégé implementing the choices we have just described. Figure 5 shows the formal definition of the Bell krater class: - terms are represented as annotations using the rdfs:label property. Using language tags is a simple means to take into account multilingualism - term definitions use the skos:definition property - the different types of kraters are defined as sub-classes of the krater class - essential characteristics are represented either as object property restrictions on the ‘has-part’ relationship, or as individuals, for example ‘for_mixing_wine_and_water’ linked to the class by the hasFunction property - the use of a logical formula to express the absence of a part: not(hasPart some Neck) - vases are represented as individuals linked by the rdf:type property to the class corresponding to their type - the otv:attributedTo property (object property) is used to associate a vase with its author, whose name is represented by the foaf:name property, taken from the FOAF33 vocabulary dedicated to the representation of people and groups of people. 33 FOAF, acronym that stands for Friend Of a Friend, is dedicated to describing persons, their activities and their relations to other people and objects: http://xmlns.com/foaf/spec/.
As the conceptual system of terminologies is represented in the form of a formal ontology, it is
possible to profitably apply ontology evaluation methods. For example, there are metrics for
assessing the "quality" of an ontology in terms of structural richness, such as "Class Richness",
which measures the distribution of individuals across classes, to be correlated with "Inheritance
Richness" in relation to the depth and breadth of an ontology, "Relationship Richness" and
"Attribute Richness", which measure the expressive power of the ontology [
The formal ontology being represented as a RDF knowledge graph37, it can then be queried
using SPARQL [17], a language dedicated to querying RDF databases. Translating the competency
questions into SPARQL allows to check that the conceptual system answers them satisfactorily
[
Since terms are represented by labels attached to concepts (OWL classes) using the rdfs:label property, the first Competency Question, "What are the different types of vases?", will be translated by the following SPARQL query. It consists of iteratively traversing the class hierarchy (rfds:subClassOf):
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX owl: <http://www.w3.org/2002/07/owl#> SELECT ?term FROM <http://www.ontologia.fr/OTB/krater.rdf> WHERE { ?concept rdf:type owl:Class.
?concept rdfs:label "vase"@en. 34 https://ontometrics.informatik.uni-rostock.de/wiki/index.php/Schema_Metrics. 35 The notion of consistency is defined in terms of inconsistency. A class is said to be inconsistent if, according to its formal definition, it cannot contain any individuals. For example, a class that would have been defined as a subclass of two classes previously defined as disjoint. 36 These operations are performed using reasoners available in the Protégé environment. 37 The krater ontology is available at: http://www.ontologia.fr/OTB/krater.rdf.
The second Competency Question "What is the definition of a bell krater?" refers more to the definition of the term than to the formal definition of the concept. Using the SKOS vocabulary and the skos:definition property, this definition can be represented in a simple way. The corresponding SPARQL query is then as follows: PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX skos: <http://www.w3.org/2004/02/skos/core#> SELECT ?definition FROM <http://www.ontologia.fr/OTB/krater.rdf> WHERE { ?concept rdf:type owl:Class.
?concept skos:prefLabel "bell krater"@en. ?concept skos:definition ?definition.
FILTER (lang(?definition) = 'en') } This query returns the answer: "Krater without neck, with upward curling handles placed high on the body."@en
The third Competency Question, "What is/are the difference(s) between a column krater and a bell krater?" requires a formal representation of the definitions so that they can be subjected to calculation. We are therefore interested in the definitions of the concepts denoted by each of these terms.
RDF38 representation of classes as combinations of essential characteristics requires a specific vocabulary. OTV, for Onto Terminology Vocabulary39, is a RDF vocabulary that provides the classes and properties needed for such a representation. Essential characteristics are modeled using OWL individuals, instances of a specific class, the otv:Difference class (see lines 8 and 12 below in the SPARQL query). The otv:Concept class allows concepts to be considered as individuals, i.e. instances (rdf:type) of the otv:Concept class (see lines 6 and 10 below). A concept can then be linked to its essential characteristics via the otv:difference property (see lines 9 and 13 below). 38 RDF and RDFS for RDF Schema. RDFS is an extension of the basic RDF vocabulary [18]. 39 http://www.ontologia.fr/OTB/otv.
Kraters share the following essential characteristics: /for mixing wine with water/, /with foot/, /with handles/, /with open mouth/. They differ according to their specific characteristics, such as /with upward curling handles/.
The SPARQL query below shows how a bell krater differs from a column krater. These
characteristics are the values of the ?diff2Name variable returned by the SPARQL query.
[
List of essential characteristics of bell kraters: - "upward curling handles placed high on the body" - "with upward curling handles" - "without neck"
Terminology evaluation and validation are essential steps in any terminology building process. Beyond the indispensable validation by experts, certain criteria, such as the consistency of definitions, require a formal representation of the conceptual system. This is also the case if we want to "prove" that the terminology does indeed answer competency questions. Beyond the advantages of such a representation, which paves the way for the operationalization of terminologies, the price to be paid is not insignificant. The choice of concept theory and representation language has a strong impact on the design and construction of terminologies, imposing on experts and terminologists ways of thinking that are not always their own [24]. Consider for example the "clarity" of class definitions expressed in terms of property restrictions for people who do not necessarily have the necessary background in logic [25]. A promising alternative is the reverse approach, in which terminological principles are translated into a computer model [26]. The TEDI ontoterminology construction environment is a case in point. The kraters example has also been implemented using TEDI. Figure 5 shows the two dimensions, linguistic and conceptual, associated with the "bell krater" dictionary entry exported by TEDI. The definition of the term "bell krater" is generated from the formal definition of the concept denoted by the term, given that the generic concept <Vessel for mixing wine with water with handles with foot with open mouth> is denoted by the term "krater".The ontoterminology in RDF format exported by TEDI is available at http://ontologia.fr/OTB/krater.rdf. 40 otv, for OntoTerminology Vocabulary, is an RDF vocabulary dedicated to the construction of ontoterminologies.
We gratefully acknowledge the generous funding of the European Commission under Horizon ERA Chair project TALOS AI4SSH, Grant Agreement n° 101087269 (https://cordis.europa.eu/project/id/101087269). [9] C. Roche, Le terme et le concept : fondements d’une ontoterminologie, in : Proceedings of TOTh 2007, Terminologie & Ontologie : Théories et applications, Annecy (France), 2007, pp. 1-22. [10] RDF, Resource Description Framework, https://www.w3.org/RDF/. [11] SKOS Simple Knowledge Organization System, https://www.w3.org/2004/02/skos/. [12] F. Baader, D. Calvanese, D. McGuinness, D. Nardi, P. Patel-Schneider, The Description Logic
Handbook, Cambridge University Press, 2003. [13] Protégé, https://protege.stanford.edu/. [14] M. Horridge, A Practical Guide To Building OWL Ontologies Using Protégé 4 and CO-ODE
Tools, Edition 1.3, University of Manchester, 2011. [15] Web Ontology Language https://www.w3.org/OWL/. [16] H. Kondylakis, A. Nikolaos, P. Dimitra, K. Anastasios, K. Emmanouel, K. Kyriakos, S. Iraklis, K.
Stylianos, N. Papadakis, Delta: A Modular Ontology Evaluation System, in: Information 12, 301, 2021, https://doi.org/10.3390/info12080301. [17] SPARQL, https://www.w3.org/TR/sparql11-query/. [18] RDFS, https://www.w3.org/TR/rdf12-schema/. [19] D. Candel, D. Samain, D. Savatovsky, Eugen Wüster et la terminologie de l’école de Vienne,
SHESL, HEL Livres, 2, Paris, 2022. [20] Y. Ren, A. Parvizi, C. Mellish, J. Z. Pan, K. Van Deemter, R. Stevens, Towards competency question-driven ontology authoring, in: European Semantic Web Conference, 2014, pp. 752– 767. [21] T. Gruber, A Translation Approach to Portable Ontologies. Knowledge Acquisition, 5(2), 1993, pp. 199–220. [22] N. Guarino, D. Oberle D, S. Staab, What Is an Ontology?, in: Handbook on Ontologies. Berlin,
Springer, 2009, pp. 2-17. [23] CmapTools, https://cmap.ihmc.us/. [24] C. Roche, M. Papadopoulou, Mind the Gap: Ontology Authoring for Humanists, in: 1st International Workshop for Digital Humanities and their Social Analysis (WODHSA)September 23-25, Graz (Austria), 2019, https://ceur-ws.org/Vol-2518/paperWODHSA7.pdf [25] M. Musen, The Protégé project: A look back and a look forward. AI Matters, Association of Computing Machinery Specific Interest Group in Artificial Intelligence, 1(4), June 2015. doi:10.1145/2557001.25757003. [26] C. Roche C, La Théorie du Concept des Normes ISO à l’Ere Numérique, in : Digital Scholarship in the Humanities, Volume 38, Issue Supplement_1, June 2023, pp. 73–82. https://doi.org/10.1093/llc/fqad023. [27] ISO 704, Terminology work - Principles and methods, 2022.