<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Empowering Digital Transformation in Tourism through Intelligent Methods for Representation and Exploitation of Cultural Heritage Knowledge</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Salvatore Carta</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Gianni Fenu</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alessandro Giuliani</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marco Manolo Manca</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mirko Marras</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Leonardo Piano</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alessandro Sebastian Podda</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Livio Pompianu</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Sandro Gabriele Tiddia</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of Cagliari, Department of Mathematics and Computer Science</institution>
          ,
          <addr-line>Cagliari</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Nowadays, the cultural heritage wealth held within museums, archives, libraries, and heritage sites is under continuous threat due to various environmental, geopolitical, and resource factors. The need for proper preservation of such invaluable cultural assets worsens the risk of their loss and hinders global access to their knowledge. This paper describes our first steps towards preserving cultural heritage knowledge through intelligent digital services. Our work is conducted as part of the PNRR e.INS “Ecosystem of Innovation for Next Generation Sardinia” project, Spoke 6, which aims to support digital transformation in various domains, including cultural heritage, in the Sardinian region, Italy. The main objectives of this project include the digitization of cultural heritage knowledge by employing innovative solutions for a proper knowledge representation, the exploitation of such digital knowledge for proposing contextual and personalized services, and suitable user-friendly interfaces and applications that enable both operators and the general public to access, explore, and engage with such knowledge.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Cultural Heritage</kwd>
        <kwd>Knowledge Graph</kwd>
        <kwd>Knowledge Extraction</kwd>
        <kwd>Personalization</kwd>
        <kwd>Ranking</kwd>
        <kwd>Recommendation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Background</title>
      <p>
        Cultural Heritage (CH) is the legacy of tangible and intangible assets representing human
culture, history, identity, and diversity. UNESCO defines it as “ the entire corpus of material signs
– either artistic or symbolic – handed on by the past to each culture and, therefore, to the whole of
humankind”1. It includes numerous varieties of entities, such as archaeological sites, monuments,
museums, artworks, music, or festivals. CH is a priceless educational, touristic, economic, and
social cohesion resource [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. However, it continuously faces challenges threatening its existence
and transmission to future generations. Such threats are related to manifold aspects. On the
one hand, some of them are often unpredictable or hard to handle, such as human-induced risks
(e.g., vandalism or looting) or natural phenomena (e.g., natural disasters or climate change). On
the other hand, a crucial weakness is the lack of adequate resources, strategies, or policies for
the conservation of CH. The preservation of CH is an essential task that requires appropriate
resources and policies to protect human culture’s diverse and incalculable expressions. In this
scenario, numerous CH sites and entities must address various threats to their preservation and
sustainability. Some of them include:
• The lack of funding or expertise. CH entities need regular maintenance and
documentation to avoid deterioration. These actions often require financial and human resources
not always available or afordable. For example, some institutions may need more funds,
equipment, materials, or experienced specialists to carry out adequate interventions.
• The scarcity of documentation. From the point of view of promotion and dissemination,
many CH sites or artefacts may not have accurate documents of their history, origin, or
general information to assign them the proper value and significance.
• The shortage of awareness and participation. Numerous CH sites depend on the
attention and participation of the general public and local communities to ensure their
appreciation. However, stakeholders often lack knowledge, motivation, or interest to
promote the respect and value of the CH the people encounter or to engage them in
collateral activities, leading to the scenario where people may be unaware of the significance
of particular CH entities or may have adverse or indiferent attitudes toward them.
In such a context, the CH domain must specialize and become more innovative and sustainable
to enhance the preservation and conservation of cultural identity. For this purpose, promoting
and strengthening the identity and thousand-year history of our territory and culture plays
a crucial role in facilitating the growth of awareness and participation of people and local
communities and in encouraging stakeholders and institutions to invest efort and resources
for CH preservation. Our work aims to propose innovative solutions for addressing the most
critical issues in this scenario, as described in the following Sections 2 and 3.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Motivation</title>
      <p>
        The lack of appropriate instruments has often led to losing valuable content for preserving and
promoting our cultural heritage. To overcome this issue, the typical solution is the integration
of specific digital technologies and services [
        <xref ref-type="bibr" rid="ref2 ref3 ref4 ref5">2, 3, 4, 5</xref>
        ] that (i) ensure stability and integrity over
time, (ii) transform content to be more accessible, usable, and interactive, and (iii) enable the
use of communication channels, such as the Web or mobile technologies, facilitating
dissemination on a global scale. However, the digitization is currently facing several issues [
        <xref ref-type="bibr" rid="ref6 ref7">6, 7</xref>
        ]. In
particular, digital content must be actionable, i.e., capable of stimulating concrete actions by
users. Therefore, defining or adopting strategies to engage audiences actively and creatively,
ofering immersive, personalized, and participatory experiences is crucial.
      </p>
      <p>
        In this context, an adequate knowledge representation is paramount for implementing digital
services [
        <xref ref-type="bibr" rid="ref8 ref9">8, 9</xref>
        ]. Indeed, cultural archives are often digitized by scanning documents and photos,
including metadata, pre-processing, and indexing. The digitized data are usually shared as static
open data archives to facilitate access, innovation, and collaboration, enabling the development
of virtual reality, mobile, and Web applications. Static archives highly limit the understanding of
CH because of the lack of dynamic and interconnected representations of information, and they
usually fail to capture the complex relationships between entities, thus limiting the analysis
of historical and cultural relationships. For example, let us consider an institutional site that
encapsulates information to help tourists discover a historic city in Italy. Static open data
archives could be used on a website page to list places of interest with essential details such as
names, addresses, and times. Suppose a tourist wants a deeper, more connected perspective.
In that case, the limitation of static archives emerges, e.g., they may want to understand the
relationships between historical places, such as which monument is related to a particular
historical event or which artist created a work of art. These interconnections are complex
to represent in a static archive. In contrast, Knowledge Graphs (KGs) [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], i.e., structured
representation of information that semantically links concepts, entities, and relationships in
a meaningful and easily interpretable way, could visualize connections between monuments,
historical events, and figures, providing a more comprehensive view. Unlike static archives,
which are isolated data collections, KGs highlight connections between data, allowing for a
deeper, more contextualized understanding. This would uncover interesting relationships,
linking cultural elements, local traditions, and contemporary contexts.
      </p>
      <p>
        However, creating KGs is a challenging task [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Defining semantic relationships requires
domain understanding and specific skills. The lack of intuitive and automated tools hinders
widespread KG creation, especially for those without technical skills. To this end, our work
focuses on defining and implementing a new no-code methodology for creating and exploiting
KGs, including two key elements. First, we aim to design an innovative automated artificial
intelligence-based approach that employs Large Language Models (LLMs) [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] to transform
unstructured and structured digitized information into coherent semantic triples (entity, relation,
entity). These triples will form the nodes and relationships of the KG. In particular, the devised
KGs can link and organize data from historical archives, museums, cultural institutions, tourist
guides, and other relevant sources, enabling an in-depth understanding of our culture, facilitating
the user experience, as well as creating contextual or personalized recommendations. Following
the previous example, suppose a tourist explores a Roman historic site or monument through its
website. He/she might receive ofers on related souvenirs, such as Roman temple miniatures or
books on history, guided tours of the site’s history, restaurants with Roman cuisine, scheduled
events, and personalized suggestions based on user’s past visits and preferences. The second
innovative contribution of the work focuses on artificial intelligence-based methods that use
LLMs for exploiting KGs in contextual and personalized targeting, thus providing accurate and
explainable suggestions. The following section describes our contribution in detail.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. Contribution</title>
      <p>This contribution describes our first steps toward promoting the preservation of CH knowledge
by devising and implementing innovative digital services. Our work is conducted as part of the
PNRR e.INS “Ecosystem of Innovation for Next Generation Sardinia” project, Spoke 6, aimed at
supporting digital transformation in various domains, including cultural heritage.</p>
      <sec id="sec-3-1">
        <title>3.1. Overview</title>
        <p>
          In the context of improved digital transformation in tourism, the main goal of our work is the
cooperation between technology partners and culture operators to develop a highly innovative
service for the automatic digitization and interconnection of cultural heritage through the
generation of proper KGs from structured and unstructured information and their exploitation
for devising specific novel services also tailored to individual visitors or specific groups. As
KGs can link and organize data from historical archives, museums, cultural institutions, tourist
guides, and other relevant sources, enabling an in-depth understanding of our cultural heritage
and facilitating the user experience, creating proper KGs and related digital services facilitates
the enhancement and promotion of our heritage. They also enable knowledge sharing among
various communities and make collaboration among cultural institutions possible. In detail,
appropriate KGs for cultural heritage through innovative artificial intelligence techniques that
exploit the potential of LLMs will be created, and algorithms and platforms that implement
specific contextual and personalized targeting systems will be developed. For the sake of
completeness, let us point out that nevertheless LLMs may produce wrong facts due to their
embeddings [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ] and training methodologies [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] or may generate biased content, especially
when used for personalization [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ], a proper prompting-based strategy may outperform classical
or human-based methodologies [16], also in proposing proper recommendations [17].
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Objectives and outcomes</title>
        <p>The devised tools and services aim to increase the competitiveness and attractiveness of the
Sardinian region. The goal is guided by a series of objectives, including:
• The digitization of cultural heritage by creating usable, comprehensive, robust, and
coherent knowledge graphs.
• The exploitation of the created KGs for innovative, efective, and eficient products and
services that take advantage of the potential of the developed knowledge base.
• The development of a technological infrastructure that provides the aforementioned
contextual and personalized functionalities.</p>
        <p>Our work will yield distinct products and outcomes, as follows:
• First, proper KGs for the cultural heritage domain, created in accordance with the
requirements analysis and use cases defined during the first step.
• The KGs will be fed into contextual and personalized targeting systems that provide
(groups of) visitors with suggestions based on their context, interests, and preferences.
• A properly accessible, usable, immersive, and customizable technological infrastructure
that provides the contextual and personalized features devised in the earlier stages.</p>
      </sec>
      <sec id="sec-3-3">
        <title>3.3. Architecture</title>
        <p>For the sake of clarity, in Figure 1, the high-level architecture of the general platform is depicted.
According to the primary outcomes described in Section 3.2, the platform is composed of
three main modules: knowledge graph generation, contextual and personalized targeting, and
infrastructure. Each module is briefly described in the following.</p>
        <sec id="sec-3-3-1">
          <title>3.3.1. Knowledge Graph Generation</title>
          <p>This module is aimed at generating the final KGs for CH. In particular, for each use case defined
during the project, relevant triples (in the form of &lt;subject, relation, object&gt;) are
extracted from structured or unstructured textual sources provided by stakeholders by employing
innovative algorithms based on a proper LLM-prompting strategy. A specific computer-aided
validation methodology will be used to investigate the correctness of the extracted triples. In
detail, innovative and eficient active learning algorithms will embrace an incremental approach
for determining the correctness of triples. This will be achieved through an iterative query
strategy in which, at each step, a human operator is asked to validate a limited set of triples,
pre-classified by the algorithm trained in the previous step, with the aim of incrementally
correcting the error, and, therefore, improving the efectiveness step by step. The output will be
a proper KG composed of the generated triples for a given use case.</p>
        </sec>
        <sec id="sec-3-3-2">
          <title>3.3.2. Contextual and personalized targeting</title>
          <p>This module aims to generate proper explainable recommendations by innovative LLM-based
algorithms that explore the paths of the generated KGs. For example, if we consider the path
[&lt;user_1&gt; visited &lt;Castle of Cagliari&gt; associated_to &lt;Ancient Sardinia&gt;
associated_to &lt;ExhibitionX&gt;], the resulting text to explain a recommendation for the
place ExhibitionX to user_1 could be “It is related to the Castle of Cagliari, which you visited
recently, as both belong to the Ancient Sardinia category”. This approach improves the user
experience and confidence. This module first analyzes the KGs by exploiting ad-hoc
methodologies to understand entity structures and relationships, identify afinity relationships between
entities, and adequately represent entities to overcome the limitations of existing methods.
Afterwards, the module generates recommendations with a twofold strategy. On the one hand,
specific targets, such as identifying similar entities or forming clusters of related entities, are
defined. Recommendations are generated based on clusters and relationships, validating their
quality (contextual targeting). On the other hand, personalized representations are created by
combining afinity information and specific user characteristics, including interaction history.
The personalized representations will allow us to provide recommendations tailored to users’
individual needs or specific contexts ( personalized targeting).</p>
        </sec>
        <sec id="sec-3-3-3">
          <title>3.3.3. Infrastructure</title>
          <p>The final technological infrastructures will integrate (i) the essential services for knowledge
ingestion, i.e., the information extraction and management from multiple sources of structured
and unstructured information; (ii) the KG creation and management services, including the
human-assisted triples validation; (iii) the targeting services (both contextual and personalized);
(iv) the application layer of end-user services with various management services (e.g., user
creation, profiling, ID management and authentication, travel experience historicization, or
sharing and social media interfaces). The integrated platform will be accessible to two main
categories of users: first, the entities involved in the knowledge construction phase (cultural
institutions, touristic operators, or other stakeholders), i.e., users designated for the data
ingestion and the iterative manual triples validation, and second, the end-users, such as a website
visitor, which will be the beneficiary of the targeting services.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Expected Impact</title>
      <p>The final integrated tool may enable (a) high quality and customization in the search, retrieval,
and fruition of knowledge in the cultural domain referring to a target territory; (b) active
support and guidance of the user in the construction of custom services such as route stages
and places/resources according to his or her interests; (c) proactive formulation of personalized
suggestions and recommendations regarding ideas, visits, experiences, and related products, (d)
support in sharing knowledge and experiences with other users and visitors connected to the
platform and its services to assist in creating and developing cultural communities.</p>
      <p>Our work is expected to positively afect the cultural communities from several perspectives.
In particular, we expect an increase in the quality and quantity of available information on the
CH and the related services of the involved territories, increased visibility and enhancement
of the cultural heritage at a national and international level, greater collaboration and sharing
among the diferent communities and cultural institutions involved in the project and augmented
competitiveness and innovation of tourism and culture operators in Sardinia.</p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <p>We acknowledge financial support under the National Recovery and Resilience Plan (NRRP),
Mission 4 Component 2 Investment 1.5 - Call for tender No.3277 published on December 30,
2021 by the Italian Ministry of University and Research (MUR) funded by the European Union
– NextGenerationEU. Project Code ECS0000038 – Project Title eINS Ecosystem of Innovation
for Next Generation Sardinia – CUP F53C22000430001- Grant Assignment Decree No. 1056
adopted on June 23, 2022 by the Italian Ministry of University and Research (MUR).
//doi.org/10.1145%2F3409256.3409834. doi:10.1145/3409256.3409834.
[16] J. Baek, A. F. Aji, A. Safari, Knowledge-augmented language model prompting for zero-shot
knowledge graph question answering, 2023. arXiv:2306.04136.
[17] R. M. Harrison, A. Dereventsov, A. Bibin, Zero-shot recommendations with pre-trained
large language models for multimodal nudging, 2023. arXiv:2309.01026.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>E.</given-names>
            <surname>Panzera</surname>
          </string-name>
          ,
          <source>The Socio-Economic Impact of Cultural Heritage: Setting the Scene</source>
          , Springer International Publishing, Cham,
          <year>2022</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>41</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>030</fpage>
          -94468-
          <issue>1</issue>
          _
          <fpage>1</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>F.</given-names>
            <surname>Maietti</surname>
          </string-name>
          ,
          <article-title>Heritage enhancement through digital tools for sustainable fruition-a conceptual framework</article-title>
          ,
          <source>Sustainability (Switzerland) 15</source>
          (
          <year>2023</year>
          ). doi:
          <volume>10</volume>
          .3390/su151511799.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>M.</given-names>
            <surname>Morandotti</surname>
          </string-name>
          , E. Doria,
          <article-title>Information system as tool for cultural heritage documentation and preservation</article-title>
          .
          <source>protocol structuring and testing on a case study</source>
          , volume
          <volume>48</volume>
          ,
          <year>2023</year>
          , p.
          <fpage>1081</fpage>
          -
          <lpage>1088</lpage>
          . doi:
          <volume>10</volume>
          .5194/isprs-Archives
          <string-name>
            <surname>-XLVIII-M-</surname>
          </string-name>
          2
          <string-name>
            <surname>-</surname>
          </string-name>
          2023-1081-
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>M.</given-names>
            <surname>Raykovska</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Jones</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Vasilev</surname>
          </string-name>
          ,
          <article-title>A photographic documentation worklfow for digitization of cultural heritage: The 14 century church of sv. nikola in kalotina, bulgaria</article-title>
          , volume
          <volume>48</volume>
          ,
          <year>2023</year>
          , p.
          <fpage>1287</fpage>
          -
          <lpage>1293</lpage>
          . doi:
          <volume>10</volume>
          .5194/ isprs-Archives
          <string-name>
            <surname>-XLVIII-M-</surname>
          </string-name>
          2
          <string-name>
            <surname>-</surname>
          </string-name>
          2023-1287-
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S. M.</given-names>
            <surname>Carta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Consoli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Giuliani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. S.</given-names>
            <surname>Podda</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. R.</given-names>
            <surname>Recupero</surname>
          </string-name>
          , Culturai:
          <article-title>Semantic enrichment of cultural data leveraging artificial intelligence</article-title>
          ,
          <source>IEEE Access 10</source>
          (
          <year>2022</year>
          )
          <fpage>127328</fpage>
          -
          <lpage>127344</lpage>
          . doi:
          <volume>10</volume>
          .1109/ACCESS.
          <year>2022</year>
          .
          <volume>3226070</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>L.</given-names>
            <surname>Baratin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Gasparetto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Tronconi</surname>
          </string-name>
          ,
          <article-title>What communication about heritage conservation? a challenge for digital documentation content</article-title>
          , volume
          <volume>48</volume>
          ,
          <year>2023</year>
          , p.
          <fpage>183</fpage>
          -
          <lpage>194</lpage>
          . doi:
          <volume>10</volume>
          .5194/ isprs-Archives
          <string-name>
            <surname>-XLVIII-M-</surname>
          </string-name>
          2
          <string-name>
            <surname>-</surname>
          </string-name>
          2023-183-
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.</given-names>
            <surname>Adane</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Chekole</surname>
          </string-name>
          , G. Gedamu,
          <article-title>Cultural heritage digitization: Challenges and opportunities</article-title>
          ,
          <source>International Journal of Computer Applications</source>
          <volume>178</volume>
          (
          <year>2019</year>
          )
          <fpage>1</fpage>
          -
          <lpage>5</lpage>
          . URL: http: //www.ijcaonline.org/archives/volume178/number33/
          <fpage>30748</fpage>
          -
          <lpage>2019919180</lpage>
          . doi:
          <volume>10</volume>
          .5120/ ijca2019919180.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>N.</given-names>
            <surname>Barbuti</surname>
          </string-name>
          ,
          <article-title>Thinking digital libraries for preservation as digital cultural heritage: By r to r4 facet of fair principles</article-title>
          ,
          <source>Int. J. Digit. Libr</source>
          .
          <volume>22</volume>
          (
          <year>2021</year>
          )
          <fpage>309</fpage>
          -
          <lpage>318</lpage>
          . URL: https://doi.org/10. 1007/s00799-020-00291-7. doi:
          <volume>10</volume>
          .1007/s00799-020-00291-7.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <article-title>Evaluating visitor experience of digital interpretation and presentation technologies at cultural heritage sites: a case study of the old town, zuoying</article-title>
          ,
          <source>Built Heritage</source>
          <volume>4</volume>
          (
          <year>2020</year>
          ).
          <source>doi:10.1186/s43238-020-00016-4.</source>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>L.</given-names>
            <surname>Ehrlinger</surname>
          </string-name>
          , W. Wöß,
          <article-title>Towards a definition of knowledge graphs</article-title>
          ,
          <source>in: International Conference on Semantic Systems</source>
          ,
          <year>2016</year>
          . URL: https://api.semanticscholar.org/CorpusID: 8536105.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>C.</given-names>
            <surname>Peng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Xia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Naseriparsa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Osborne</surname>
          </string-name>
          ,
          <article-title>Knowledge graphs: Opportunities and challenges</article-title>
          ,
          <source>Artificial Intelligence Review</source>
          (
          <year>2023</year>
          ).
          <source>doi: 10.1007/s10462-023-10465-9.</source>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>S. R.</given-names>
            <surname>Bowman</surname>
          </string-name>
          , Eight things to know
          <source>about large language models</source>
          ,
          <year>2023</year>
          . arXiv:
          <volume>2304</volume>
          .
          <fpage>00612</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>K.</given-names>
            <surname>Sun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y. E.</given-names>
            <surname>Xu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Zha</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X. L.</given-names>
            <surname>Dong</surname>
          </string-name>
          , Head-to-tail:
          <article-title>How knowledgeable are large language models (llm)? a.k.a. will llms replace knowledge graphs?</article-title>
          ,
          <year>2023</year>
          . arXiv:
          <volume>2308</volume>
          .
          <fpage>10168</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>L.</given-names>
            <surname>Berglund</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Tong</surname>
          </string-name>
          , M. Kaufmann,
          <string-name>
            <given-names>M.</given-names>
            <surname>Balesni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. C.</given-names>
            <surname>Stickland</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Korbak</surname>
          </string-name>
          ,
          <string-name>
            <surname>O. Evans,</surname>
          </string-name>
          <article-title>The reversal curse: Llms trained on "a is b" fail to learn "b is a"</article-title>
          ,
          <year>2023</year>
          . arXiv:
          <volume>2309</volume>
          .
          <fpage>12288</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>E. J.</given-names>
            <surname>Gerritse</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Hasibi</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. P. de Vries</surname>
          </string-name>
          ,
          <article-title>Bias in conversational search: The double-edged sword of the personalized knowledge graph</article-title>
          ,
          <source>in: Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval</source>
          , ACM,
          <year>2020</year>
          . URL: https:
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>