A Comparative Study of Simple and Complex Art Interpretations in Linked Open Data Using ICON Ontology

Abstract

This paper focuses on exploring how art interpretation can be eficiently represented within the linked open data framework, according to two main scenarios. To do so, we first introduce ICON 2.0, a (lighter) extension of the iconographical and iconological artworks interpretations ontology (ICON) for the description of artwork subjects and meanings, and then we compare data encoded with the original (more complex) version with the extended version. ICON 2.0 introduces super-properties of chains to create shortcuts, enabling direct links between artworks and their respective first, second, and third level of interpretation subjects. By leveraging these super-properties, the extended ontology aims to streamline the process of connecting artworks with their associated subjects and cultural contexts. Throughout the study, we assess the benefits and shortcomings of using either ICON or ICON 2.0. We examine their potential impact on information retrieval, eficiency, and information loss, taking into consideration real-case scenarios of the integration of cultural heritage linked open data from current knowledge graphs. We show how the superproperty-based extension can be used to describe simple interpretations without information loss and avoiding redundant reifications, and how it can coexist with more complex descriptions by facilitating the retrieval of the subject matter. However, we also identify tradeofs regarding the use of shortcuts in the conceptualisation of contrasting interpretations. The study concludes by providing valuable insights into the practical implications of these approaches for enriching the accessibility and depth of art interpretation in the linked open data ecosystem.

Keywords

• eol>Iconographical and iconological interpretations
• Artworks
• Semantic Web
• Ontology

-

1. Introduction1 Researchers are giving more and more attention to the importance of providing a rich description of semantic assertions and their provenance [ 1 ]. In the Cultural Heritage domain (CH), this led to the creation of standards such as CIDOC-CRM2[ 2 ] allowing detailed descriptions of the main facts of the domain. Nevertheless, this good practice can not always be implemented due to the lack of available source data. On one hand, much information on online CH datasets was injected from catalogues realised when the sensibility for structured and reliable data was still not acquired, for instance in the case of the ArCo knowledge graph [ 3 ]. On the other hand, producing thorough descriptions in a structured format may raise problems of resources and time needed for maintenance and querying if applied on a larger scale. This led to the implementation of some solutions on an ontological level. One of them is the creation of shortcuts, which were adopted, among the others, by CIDOC-CRM and the Simulation Ontology [ 4 ]. This solution allows for preserving fully described data while querying them through shortcuts, improving performances.

In this paper, we propose an extension of the ICON ontology [ 5 ] to allow light descriptions of iconographical and iconological information (i.e. the represented subject matter and meanings) that may be observed in artworks, and we compare the simpler and more complex version of two case studies to highlight the benefits and shortcomings of both versions. Whereas the complex (original) version includes a thorough description of the interpreter, matching the need of providing authoritative and thoroughly described data, the new extended version will allow producing simple descriptions from catalogue entry texts and improve the performance of retrieval of datasets already using ICON. The extension consists of the introduction of three property chains relating the artwork to the subjects identified at the three levels of interpretations, namely pre-iconographical, iconographical, and iconological.3 The introduction of the properties allows the users to go beyond the subject identification with currently available ontologies (discussed in section 6) by introducing a subdivision into levels, which is an important aspect for iconographical inquiries, present in the major domain theorisations and cataloguing indications.4. The rest of the paper is divided as follows. Section 2 describes the structure of ICON ontology and how it conceptualises the diferent levels of interpretation. Section 3 contains an explanation of the problem statement that led to the creation of this work. In section 4, we describe our solution as an extension to the ICON ontology. Section 5 is about the comparison between two interpretations encoded in the complex and lighter version of ICON. In section 6 we briefly compare our extension with other properties fulfilling a similar purpose. Finally, section 7 concludes the paper with final remarks on using simpler and complex methods to model artistic interpretations.

2. Background

In this section, we briefly describe the main classes and properties of ICON 1.0 that are needed to understand the newly implemented shortcuts. For a complete description of ICON, in3We use Panofsky’s theory [ 6 ] as a theoretical support to describe the three levels of interpretations. At the pre-iconographical level, we recognize basic elements and motifs, like trees or animals. Moving to the iconographical level, we identify specific iconographic subjects, such as Thor or other mythological figures, or specific themes or narratives, such as biblical scenes, allegories of virtues, or historical events. Finally, at the iconological level, we delve into deeper cultural and philosophical meanings, connecting the artwork to abstract concepts or cultural phenomena, like love, power, or the human condition.

4For the theories using levels of interpretations, see [ 7 ]. Cataloguing standards such as CDWA and CCO [ 8 ] suggest providing a subject description following the three levels of interpretation. cluding alignment to foundational ontologies, direct and indirect reuse of other existing ontologies, and axiomatisation, we refer to [ 5 ]. ICON conceptualises the act of interpreting the subject matter and meanings represented in an artwork by an observer (also referred to as interpreter). Each interpretation is composed of recognitions of subjects, divided into three levels, grading from a more superficial to a more in-depth understanding of the artwork content. The diferent types of recognitions (subclasses of the icon:Recognition class) of a work of art (icon:Artwork class) are made by one or more interpreters. The type depends on the level of interpretation. For every level, a recognition is linked to a work of art via the property icon:aboutWorkOfArt (inverse icon:hasRecognition). On a pre-iconographical level, we link the icon:PreiconographicalRecognition class to the recognised artistic motif, which can refer to a natural element (such as a tree), an expressional quality (such as an emotion), or an action (such as eating). To do so, we conceptualized the class icon:ArtisticMotif and then linked it to the classes icon:Action, icon:NaturalElement, and icon:ExpressionalQuality, respectively with the properties icon:hasNaturalMeaning (for natural elements and actions) and icon:hasExpressionalMeaning (for expressional qualities). At this level, an interpreter might also recognise a composition (icon:Composition), intended as a group of artistic motifs. In this case, the pre-iconographical recognition is linked to a composition using the property icon:recognizedComposition, and a composition is linked to the artistic motifs that form part of it using the icon:hasPart property. On an iconographical level, we link the icon:IconographicalRecognition class to the image (class icon:Image) it recognises using the icon:recognizedImage property. The image is then linked to the iconographical subject (character, symbol, place, personification, named object, event) it refers to via diferent properties according to the type of subject. On this same level, an invenzione (class icon:Invenzione), which can be either a story or an allegory (classes icon:Story, Allegory), can also be recognised in a work of art, an iconographical recognition is linked to it via the property icon:recognizedInvenzione. Stories and Allegories are composed of the same iconographic subjects listed above, this relationship is expressed by the property icon:composedOf. Finally, on an iconological level, an iconological recognition (class icon:IconologicalRecognition is linked to the recognised instrinsic meaning (class icon:IntrinsicMeaning) via the property icon:recognizedIntrinsicMeaning. The intrinsic meaning can use any other recognized element from other levels of interpretation to support its claim, and then it is linked to an abstract concept (we reuse the Dolce ultralight dul:SocialObject to represent this) or a cultural phenomenon (class icon:CulturalPhenomenon) it represents via the properties icon:recognizedConcept and icon:recognizedCulturalPhenomenon.

3. Motivation

In this section, we provide a brief description of two motivating scenarios. For each motivation, a real example extracted from existing CH datasets will be presented. In section 5, the examples will be represented according to the extended ontology to evaluate the eficacy of the update.

3.1. Motivating scenario 1: free text subject description in artworks catalogue entries

Many cultural institutions release their data with poor structured iconographical descriptions [ 9 ]. Despite the main cataloguing standards recommending the use of the three levels of description approach, no further indications of providing a structured description are given. Consequently, many institutions provide iconographical information in free text. Furthermore, it is likely that complex information about one or multiple interpreters is not provided, nor that indications on how the subjects relate to each other are given. Therefore, in the perspective of using ICON for converting the subject matter catalogue description field expressed as a free text, the major part of the information required for a complete description using ICON ontology is lacking. In this scenario, the use of ICON would be against good practices, because reification (widely adopted in ICON 1.0) would be used when not strictly needed, which is an aspect negatively registered for established RDF quality evaluation criteria (e.g. the interoperability metric of avoiding blank nodes and RDF reification [ 10 ]).

Real example The real scenario for this motivation is extracted from Arco, the Italian Cultural Heritage KG [ 3 ], containing RDF descriptions of 820 thousand cultural entities from the catalogue of the Italian Ministry of Cultural Heritage and Activities. This source was chosen as a good representative of the Knowledge Graphs created by the Cultural Heritage institutions which provide a catalogue description according to shared standard practices.

The chosen example is the Hohenstein Adolf’s manifesto with allegorical figures 5 (figure 3.1). The artwork instance is linked, using the property dc:description, to a string containing iconographical keywords, belonging to diferent levels, separated by a semicolon:

Iconographic reading: male figures; female figures; allegorical figures; Work; Art;

Sculpture; Painting; Music; trumpet; standard; palm branch; laurel branch; hammer; palette; brushes; lira; work tools; heraldic crests; shield. Merchandise category/Type of event: cultural events; celebrations; the fiftieth anniversary of the statute. Names: 50th Anniversary of the Albertine Statute. Places: Turin6

From this description, no information about possible relations among the described subjects is provided, nor further details about their possible sources and about the interpreter.

3.2. Motivating scenario 2: art historians’ interpretations

The description of a significant number of resources with ICON, which includes reifications and a high level of detail, may lead to backward results, such as complex queries and slow results retrieval. Nevertheless, the use of many classes and relations is needed for a thorough description of resources. One scenario belonging to this case is the modelling of art historians’ interpretations, in which the artwork’s deep comprehension provides detailed descriptions of the represented subjects, of the sources motivating certain stylistic decisions and meanings.

5URI: https://w3id.org/arco/resource/HistoricOrArtisticProperty/0500671662 6English translation is provided by the authors. The original text is retrievable at the artwork URI

Real example The real example for this motivating scenario is extracted from the Iconology dataset[ 11 ]. The dataset contains an RDF description made according to ICON of a selection of iconographical-iconological interpretations by the art historian Erwin Panofsky about a total of circa 400 artworks. For this scenario, we focus on the Iconology dataset as it provides thoroughly described art historians’ interpretations, transposing in data the complexity embedded in art historical research.

The art historians Panofsky and Saxl in the article Classical Mythology in Medieval Art [ 12 ] investigate the reception of classical antiquity during the Middle Ages. A relief on the Saint Mark’s basilica in Venice represents an allegory of Salvation in which Christ saves human souls, symbolised by the deer he is carrying, and defeats the devil, symbolized by the dragon, by stepping on it. Since the visual arrangement of the figures is a direct reuse of another Roman relief placed on the external wall of the Basilica representing Hercules carrying the Erymanthian boar, the two scholars recognise this formal motif recovery as the attitude of the Medieval artists, who were incapable of retaining a classical form without destroying its classical subject. With ICON, it is possible to describe the complex relations between actions (e.g., the act of carrying), objects, and their symbolical meanings (e.g. deer as a symbol of the Christian soul), along with the punctual relations between the parts of the two artworks which were objects of visual pattern borrowings. Nevertheless, the complex structure of the ontology makes the retrieval of simple objects challenging. For example, in order to answer the question Which subjects are depicted in the artworks?, a union of three diferent SPARQL patterns, one for each level of interpretation, should be performed.

Table 1 resumes the requirements that the two motivating scenarios need. Both examples provide a subject matter description. None of them provide multiple interpretations. Nevertheless, the second example provides the description of relations between subjects (i.e. from which attributes and level 1 subjects is the iconography composed) and relation with external sources, in this case, the visually cited artworks. Both examples provide further details for the subjects (e.g. plurals, adjectives). Consequently, whereas the second example requires a model for expressing the complexity of its multiple characteristics, the first example provides relatively 4. ICON 2.0 plain iconographic information. We envision that both cases would benefit from a simplified information retrieval of at least the key elements, i.e. the subjects recognised at each level. To comply with the motivating scenarios presented in the previous section, here we present an extended version of ICON (ICON 2.0), encompassing simplified interpretation statements. In detail, we introduce three new properties, each corresponding to the subjects identified at diferent levels of interpretation. These properties establish connections between instances of the class icon:Artwork and their respective described subjects.

First, we directly connect an artwork to its pre-iconographical depiction(s) by defining the property icon:preiconographicallyDepicts. This property is declared as a super property of a chain connecting the icon:Artwork class to the classes icon:NaturalElement, icon:Action, or icon:Expression that are recognized in the first level of interpretation. These chains come into play when the elements of the first level of interpretation are identified either through the recognition of an artistic motif or as part of a recognised composition. As the ontology can be integrated with other subject types that may not suit the entities listed above, such as writings (crm:E33_Linguistic_Object), the property chain includes also a property path that involves the relation crm:P138_represents, as shown in figure 2. In this way, other types of recognized objects can be included.

Second, we introduce the icon:iconographicallyDepicts property to link an artwork and a second level of interpretation element that the artwork represents. This property, similar to the previous one, is made a super property of a chain that connects an artwork to instances of classes defining characters, symbols, personifications, places, events, named objects, stories, and allegories (including their stories and allegories components).

Third, we define the property icon:iconologicallyRepresent to link an artwork to the cultural phenomenon or abstract concept it represents, characterising the third level of interpretation. This property serves as the super property of a chain connecting an icon:Artwork to instances of either icon:CulturalPhenomenon or dul:SocialObject classes.

7Images were respectively retrieved from 1) ARCO; 2) Europeana; image license: CC BY-ND; rights owner: University of Bologna 3) www.canalgrandevenezia.it; image license: CC BY-NC-SA 3.0 IT

We present the chains as shown in Protégé in figure 2.

This extension to the ontology maintains the original structure intact and does not change its core design. Instead, it facilitates the inference of the newly introduced properties from the pre-existing (complex) version of the ontology. As a result, this extension ensures compatibility with version 1, allowing seamless integration between ICON 1.0 and ICON 2.0.

5. Comparison between two Case studies

In this section, we discuss the use of the extended ICON ontology in the case studies described in section 3.

In the first motivating scenario, the artwork presented a free text enumerating descriptive subjects, with no distinctive rule to discriminate their level. Possibly, a level distinction between subjects could be detected using automatic rules (e.g. all the names in the section "Places" could be converted as second-level subjects with type icon:Place). Once the levels and type of subjects are assigned, the iconographical content can be described with the relations, as shown in figure3.

In the second scenario, in which data is already thoroughly described with ICON ontology, the retrieval of recognised subjects can be simplified with the newly introduced properties, as shown in figure 3. By exploiting OWL (Ontology Web Language) logical backend, the triples containing the new properties can be automatically generated through inferences made by a reasoner. A thorough description of this case study and its ICON description is provided in the ontology documentation. 8

In order to retrieve the subjects depicted in an artwork, the SPARQL query that can be performed using the newly introduced properties is slightly shorter than the one querying the

8Available at https://w3id.org/icon/docs

more complex version9. For this comparison, we present both the previous and the current query, respectively shown in figures 4 and 5. 10

To summarise, we provide a brief comparison of what the two versions of the ontology can cover in both the examples in table 2. With this extension, the ICON ontology can now currently easily describe and retrieve the recognized subjects according to the levels of interpretation, 9The Iconology Dataset can be queried through the SPARQL endpoint available at https://projects.dharc.unibo. it/icondataset/sparql

10In both cases, the queries contain a UNION clause between the patterns for retrieving subjects of each level to include in the search those artworks without a description at all the levels. while maintaining means for a more detailed description when needed (scenario 2). We also acknowledge that these case studies do not contain contrasting interpretations of the same artwork. In such a case, the use of shortcuts could cause ambiguous information retrieval as contrasting depicted entities in the same painting would be linked to it without the provenance information (i.e, the interpreter), making the presence of both simple and complex version necessary as in the case of scenario 2.

6. Comparison with existing properties

In this paragraph, we will briefly discuss the diference between the newly introduced shortcuts and existing properties that are currently used to link artworks with their subject.

Notably, Dublin Core [ 13 ] utilizes dc:subject to establish a link between a resource and its subject, a practice common in knowledge graphs associating subjects with artworks. Similarly, the schema:about property in the general domain knowledge graph YAGO [ 14 ] and dbo:subjectTerm in DBpedia [15] serve a parallel purpose. For the same purpose, Wikidata’s [16] wdt:P180 links artworks to their depictions.

However, the novel shortcuts in the extended ICON ontology ofer distinct advantages. A key distinction is the possibility to explicitly state the levels of iconographical and iconological interpretations, unlike the broader scope of the aforementioned relationships, which lack similar expressiveness. Additionally, the interoperability of super property chains with the intricate iconological description in the ICON ontology can be used to align data seamlessly with other complex iconographical and iconological datasets. This is especially valuable for general knowledge graphs that contain artworks, which could be aligned with domain-specific art datasets like the ICON dataset. We do not mention CIDOC-crm here because the ICON ontology is already aligned to it [ 5 ].

7. Discussion and conclusions

The ICON ontology provides a mean for formally describing iconographical and iconological interpretations with a high level of granularity according to three levels of interpretation, grading from a basic to a profound artwork understanding. The subdivision, based on Erwin Panofsky’s theory, is included in the main CH cataloguing standard descriptions (CDWA, CCO). This characteristic is fundamental for expressing the complexity characterising domainspecific claims, as the literature by art historians demonstrates (scenario 2). Nevertheless, the lack of a means for describing iconographical content when scarce or incomplete data is provided could have drawbacks, forcing the user to describe simple information in a verbose way (scenario 1). The current update aimed at filling this gap by introducing three new properties aimed at directly identifying the depicted subjects according to the level of interpretation. For this purpose, we described and represented with the new properties two examples belonging to two diferent motivating scenarios, respectively 1) a museum catalogue entry providing iconographical information in free text, and 2) complex art historians’ interpretation of the artwork’s iconographical and iconological meanings. Whereas, in the first case, the description was suficient for representing the subject matter, in the second case, the three properties were added to a pre-existing thorough description with ICON. This demonstrates the eficacy of the updated ontology in describing diferent scenarios. The eficiency in information retrieval improved, too. The comparison among the SPARQL queries to retrieve the represented subjects in the two ontology versions shows that the update simplifies this core information retrieval in both the envisioned scenarios. Nevertheless, some trade-ofs emerged. Whereas the use of resources decreases in the first scenario, since limited triples are needed for the description, it increases in the second one. Following, the ontology cannot provide a light description of subjects qualities and relations (e.g. indicating the attributes of an iconography), a case quite common in free texts’ iconographical description.

In this diversified setting in which the granularity of the artwork descriptions and interpretations can vary significantly, it is important to introduce these shortcuts to improve the interoperability between knowledge graphs with minimal and detailed iconographic and iconological information. Moverover, as seen in section 6, the introduced properties, while simpler than the normal ICON ontology schema, are still more expressive than general properties that link artworks with their subject matter.

A possible shortcoming of the use of the newly introduced properties lies in the context of multiple, contrasting interpretations of the same artwork. In instances where a consensus or harmonious interpretation is achieved, the employment of the streamlined version of the ontology holds the promise of eficient and efective subject matter retrieval. This approach facilitates the seamless extraction of pertinent information without compromising the integrity of the intended interpretation.

However, the complexity of artistic expression often gives rise to divergent viewpoints, with diferent interpreters attributing disparate meanings to various elements within the same work of art. It is here that the limitations of the shortcut-enhanced ontology become evident. By connecting all depicted elements directly to the artwork without reification, the ontology erases the valuable context that distinguishes the individual interpretations. The depth and nuances of pre-iconographic, iconographic, and iconological layers of analysis risk being overshadowed by this generalized linkage, leading to the potential loss of the provenance of each interpreter’s distinct perspective.

This supports the statement that the intention behind ICON 2.0, featuring the aforementioned shortcuts, is not to entirely replace the original complex ontology, but rather to coexist alongside it within the same knowledge graph when complex information is present. This dual existence ensures that users seeking to delve into, and highlight, contrasting interpretations, have the means to query the intricate and comprehensive rendition of the ICON ontology. In conclusion, by preserving the availability of the original complex version when needed, and by presenting newly introduced shortcuts for simpler interpretations, the ontology caters to a diverse range of scholarly and analytical needs, accommodating both seamless retrieval and in-depth exploration of contrasting and complex viewpoints in the realm of linked open data of artistic interpretations. [15] S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, Z. Ives, Dbpedia: A nucleus for a web of open data, in: The semantic web, Springer, 2007, pp. 722–735. [16] D. Vrandečić, M. Krötzsch, Wikidata: A free collaborative knowledge base, Commun.

ACM 57 (2014) 78–85. doi:10.1145/2629489.

References

[1] M. Daquino , V. Pasqual , F. Tomasi , F. Vitali , Expressing without asserting in the arts , in: Proceedings of the Italian Research Conference on Digital Libraries , Padova, Italy, 2021 .

[2] C. Bekiari , G. Bruseker, M. Doerr , C.-E. Ore, S. Stead , A. Velios , Definition of the cidoc conceptual reference model v7.1.1 , 2021 . doi: 10 .26225/FDZH-X261.

[3] V. A. Carriero , A. Gangemi , M. L. Mancinelli , L. Marinucci , A. G. Nuzzolese , V. Presutti , C. Veninata , Arco: The italian cultural heritage knowledge graph , in: C. Ghidini , O. Hartig , M. Maleshkova , V. Svátek , I. Cruz , A. Hogan , J. Song , M. Lefrançois , F. Gandon (Eds.), The Semantic Web - ISWC 2019 , Springer International Publishing, Cham, 2019 , pp. 36 - 52 .

[4] B. Sartini , M. van Erp , A. Gangemi , Marriage is a peach and a chalice: Modelling cultural symbolism on the semantic web , in: Proceedings of the 11th on Knowledge Capture Conference, K-CAP '21 , Association for Computing Machinery, New York, NY, USA, 2021 , p. 201 - 208 . doi: 10 .1145/3460210.3493552.

[5] B. Sartini , S. Baroncini , M. van Erp , F. Tomasi , A. Gangemi , Icon: An ontology for comprehensive artistic interpretations , J. Comput. Cult. Herit . 16 ( 2023 ). doi: 10 .1145/ 3594724.

[6] E. Panofsky , Meaning in the visual art : papers in and on art history, Doubleday Anchor book , 59 , Anchor Books , New York, 1955 .

[7] S. Baroncini , M. Daquino , F. Tomasi , Modelling art interpretation and meaning. A data model for describing iconology and iconography , Aida Informazioni ( 2021 ) 39 - 62 .

[8] P. Harpring , E. Lanzi , L. McRae , et al., Cataloging cultural objects: A guide to describing cultural works and their images , American Library Association , 2006 .

[9] S. Baroncini , B. Sartini , M. van Erp , F. Tomasi , A. Gangemi , Is dc:subject enough? a landscape on iconography and iconology statements of knowledge graphs in the semantic web , Journal of Documentation 79 ( 2023 ) 115 - 136 . doi: 10 .1108/JD-09-2022-0207.

[10] M. Färber , F. Bartscherer , C. Menne , A. Rettinger , Linked data quality of dbpedia, freebase, opencyc, wikidata, and yago , Semantic Web 9 ( 2018 ) 77 - 129 .

[11] S. Baroncini , M. Daquino , F. Tomasi , Exploring iconographical and iconological content in semantic data through art historians' interpretations, in: La memoria digitale. Forme del testo e organizzazione della conoscenza . Atti del XII convegno annuale AIUCD , Siena, 2023 , pp. 9 - 14 .

[12] E. Panofsky , F. Saxl , Classical mythology in mediaeval art , Metrop. Mus. Stud . 4 ( 1933 ) 228 - 280 .

[13] S. L. Weibel , T. Koch , The dublin core metadata initiative, D-lib magazine 6 ( 2000 ) 1082 - 9873 .

[14] T. Rebele , F. Suchanek , J. Hofart , J. Biega , E. Kuzey, G. Weikum, Yago: A multilingual knowledge base from wikipedia, wordnet, and geonames , in: The Semantic Web-ISWC 2016 : 15th International Semantic Web Conference, Kobe, Japan, October 17-21 , 2016 , Proceedings, Part II 15 , Springer, 2016 , pp. 177 - 185 .