<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>October</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>On the Need for a Body of Knowledge on Recommender Systems</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Juri Di Rocco</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Davide Di Ruscio</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Claudio Di Sipio</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Phuong T. Nguyen</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Claudio Pomo</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>DISIM, Università degli studi dell'Aquila</institution>
          ,
          <addr-line>67100 L'Aquila</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>SisInf Lab, Politecnico di Bari</institution>
          ,
          <addr-line>70125 Bari</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2021</year>
      </pub-date>
      <volume>1</volume>
      <issue>2021</issue>
      <abstract>
        <p>Recommender systems (RSs) are becoming widespread in diferent application domains to provide personalized items to given service users. Because of such an increasing adoption of RSs, it is becoming urgent to define a precisely curated and organized core set of concepts and practices, i.e., a Body of Knowledge (BOK), as already done in other disciplines, including software engineering and model-driven engineering. The opportunities related to the availability of an RSBOK are manifold, and diferent stakeholders would benefit from it including, developers, teachers, and newcomers to the RS community. Further than motivating a BOK for recommender systems and discussing corresponding envisioned opportunities and challenges, we also propose a methodology that can be employed to support the definition of an RSBOK.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        was defined with diferent goals, including “ 1)
promoting a consistent view of software engineering worldwide;
Recommender systems (RSs) are complex software sys- 2) specifying the scope of, and clarify the place of
softtems that can provide users with relevant items of inter- ware engineering with respect to other disciplines [. . . ]; 3)
est for the particular application domains and contexts characterizing the contents of the software engineering
dis[
        <xref ref-type="bibr" rid="ref5 ref6">1, 2</xref>
        ]. Over the last decade, diferent types of recom- cipline; 4) providing a foundation for curriculum
developmendation technologies have been conceived by both ment and for individual certification and licensing material ”
industry and academia to improve the relevance of the [
        <xref ref-type="bibr" rid="ref7">3</xref>
        ]. Similarly, a Body of Knowledge for Software
Lanitems being recommended. RSs have become pervasive, guage Engineering has been recently promoted with the
and almost in any application domain, there is the avail- aim of assembling and organizing “artifacts, definitions,
ability of software systems in charge of supporting users methods, techniques, best practices, open challenges, case
in undertaking the particular tasks at hand (whether it be studies, teaching material, and other components that will
software developers who are working on some software afterwards help students, researchers, teachers, and
practicomponents, or users who want to select the next movie tioners to learn from, to better leverage, to better contribute
to watch). to, and to better disseminate the intellectual contributions
      </p>
      <p>
        While RSs are becoming ubiquitous, we believe that it and practical tools and techniques coming from the SLE
is necessary to ensure that the next generation of engi- field ” [
        <xref ref-type="bibr" rid="ref8">4</xref>
        ]. A Body of Knowledge for Model-Based
Softneers will have a clear understanding of the fundamental ware Engineering (MBSE) has been recently promoted
techniques and tools underpinning the development and [
        <xref ref-type="bibr" rid="ref9">5</xref>
        ] as an extension of SWEBOK to characterize the MBSE
usage of RSs. To this end, it is necessary to agree on the discipline in the context of Software Engineering. Many
core concepts, mechanisms, and practices related to the other BoKs have been defined over the last decade in
difdevelopment and use of RSs. Therefore, as done in other ferent application domains including data management,
software disciplines, we foster an RS Body of Knowl- enterprise architecture, business analysis, project
manedge (RSBOK) definition to formalize and characterize agement, and data management. All of them share the
the Recommender System discipline. goals of characterizing the contents of a particular
disci
      </p>
      <p>
        To the best of our knowledge, SWEBOK is the first pline and to support various activities including training
body of knowledge that was conceived for characteriz- and the development of new technologies and tools.
ing the software engineering discipline [
        <xref ref-type="bibr" rid="ref7">3</xref>
        ]. SWEBOK In this paper, we introduce some preliminary thoughts
of an RS Body of Knowledge (RSBOK) definition for the
Software Engineering domain. We promote a
comprehensive perception of the core concepts, mechanisms, and
practices related to the development and deployment of
RSs. The ultimate aim is to understand the fundamental
techniques and tools pertinent to the development and
usage of recommender systems in software engineering.
      </p>
      <p>Altogether, this is expected to come in handy for those
who work as recommender systems designers.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Knowledge-aware Recommender Systems</title>
      <p>– Languages to be adopted: To make the definition
and tools to widen and simplify the adoption of
RSs in any complex software systems. The
knowledge encoded in the envisioned RSBOK
framework can help developers, among others, to distill
the algorithms that should be made available to
ifnal users.
– Teachers: They are researchers, practitioners, and
educators in general who are in charge of
training new generation of professionals in the design,
development and operation of new RSs. To this
end, RSBOK can be used as a reference point, e.g.,
to link code examples, textual explanations,
experiences, RS usage and development best practices,
and code examples that demonstrate the usage of</p>
      <p>RS technologies.
– Contributors: These users correspond to RSs
developers, and adopters who are willing to extend
the knowledge formalized in RSBOK.</p>
      <p>
        Despite their enormous popularity and the remarkable
performance that recommender systems have achieved in
recent years, one of the long-standing problems afecting
the performance of these systems concerns the sparsity of
interactions between users and items. Over the past years,
recommender system designers have relied on additional
sources of information to overcome this issue. Modern
RSs combine collaborative information with metadata
(e.g., tags, reviews), social connections, image and audio
signal-derived features, and contextual data [
        <xref ref-type="bibr" rid="ref10">6</xref>
        ] to build
domain-dependent, cross-domain, or context-aware
recommendation models. Among the various sources, one
of the most relevant is Knowledge Graphs (). Thanks
to the heterogeneous fields covered by  and the
myriad of specific techniques that have been developed,
knowledge-based recommendation systems emerged as
a novel research field in the RecSys community. The field As previously mentioned, the wanted RSBOK
is generally known as knowledge-aware recommenda- paradigm should consist of a precise formalization of
tion systems (KaRS [
        <xref ref-type="bibr" rid="ref1">7</xref>
        ]) and blends the most advanced concepts and tools underpinning the development, usage
machine learning algorithms with cutting-edge knowl- and enhancement activities of any RSs. It is expected to
edge representation paradigms. This collective efort has provide an efective means to characterize and to support
resulted in several improvements in recommendation [
        <xref ref-type="bibr" rid="ref2">8</xref>
        ], diferent related activities including training and the
deknowledge completion [
        <xref ref-type="bibr" rid="ref3">9</xref>
        ], preference elicitation, and velopment of new technologies and tools. Recently, we
user modeling research, thus producing a vast literature. had already the need to investigate and formalize the RS
ifeld in Software Engineering, and we came up with a
3. Opportunities and Challenges model representing all the features typically supported
by RSs [
        <xref ref-type="bibr" rid="ref4">10</xref>
        ]. Figure 1 represents only the top-level
feaConceiving a Body of Knowledge for Recommender Sys- tures or recommendation systems, i.e., Data
Preprocesstems (RSBOK) would disclose several opportunities based ing, Capturing Context, Producing Recommendations, and
on the availability of a common and formally defined vo- Presenting Recommendations which are the main
funccabulary that would prescribe the usage and development tionalities typically implemented by recommendation
of recommender systems on clearly defined foundations, systems [11, 12].
instead of relying on some uncommon understanding. We performed such a conceptualization to underpin
      </p>
      <p>
        The RSBOK definition is indeed a community efort to the design and development of the diferent RSs
develformalize and share knowledge from diferent stakehold- oped in the context of the EU CROSSMINER project [
        <xref ref-type="bibr" rid="ref4">10</xref>
        ].
ers on conceptual and practical RS aspects. However, the We extracted all the shown components mainly from
investment would pay of because we foresee at least the existing studies [11] as well as from our development
following kinds of users that can take advantage of the experience under the needs of the CROSSMINER project.
RSBOK paradigm [
        <xref ref-type="bibr" rid="ref8">4</xref>
        ]: A similar conceptualization work has been done to
design and develop the Elliot framework [13], which aims
– Newcomers: They are perspective users and re- at supporting reproducible recommender systems
evalsearchers who do not have any knowledge about uation. The authors had to conceptualize diferent
recRSs, and are willing to understand them. They ommendation algorithms, splitting strategies, evaluation
can benefit from the results of the conceptualiza- protocols, metrics, and tasks to simplify the specification
tion eforts, e.g., to get an overview of the typ- and execution of experimental pipelines by processing
ical algorithms employed to develop RSs, or to simple configuration files as the one shown in Fig. 2
explore available linked textual explanations or Even though the opportunities related to the
availexamples about some typically used evaluation ability of an RSBOK are immense in our opinion, its
methodologies. realization can be hampered by a number of challenges
– Developers: These include advanced RSs develop- including the following ones:
      </p>
      <p>ers, who are interested in conceiving techniques</p>
      <p>and the usage of RSBOK homogenous, it is
necessary to decide the languages and tools that need
to be adopted. Feature diagrams, Ecore models,
and OWL ontologies are only examples of
possible notations that might be employed to formalize
the results of the conceptualization eforts.
– Realization process: By looking at the ways other</p>
      <p>BOKs have been developed, we believe that we
need a community efort, which has to be
performed by following precise protocols,
moderation mechanisms, quality check procedures, to
name a few. In this respect, it is of great
importance to support contributions that may come
from diferent stakeholders. Consequently, to
keep the quality of the resources under control,
it is necessary to define processes and setup tools
for moderating the diferent contributions and to
make sure that they are all homogenized.
– Engagement: Even though the RSs opportunities
can be convincing, they might not be enough to
engage people in concretely contributing with
the RSBOK definition and managing its whole
lifecycle.
To facilitate a clear understanding of the fundamental
techniques and tools underpinning the development and
usage of RSs, in this paper we envisaged the core
concepts, mechanisms, and practices related to the
development and use of RSs. We aim to foster RSBOK, an RS Body
of Knowledge definition to formalize and characterize
the Recommender System domain.</p>
      <p>For future work, we plan to tackle the challenges
mentioned in Section 3. In particular, it is necessary to put
into efect the conceived paradigm by realizing its
constituent components. Among others, we will investigate
and select suitable tools and languages, attempting to
make the definition and usage of RSBOK homogeneous.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>V. W.</given-names>
            <surname>Anelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Basile</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. G.</given-names>
            <surname>Bridge</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. D.</given-names>
            <surname>Noia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Lops</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Musto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Narducci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Zanker</surname>
          </string-name>
          ,
          <article-title>Knowledge-aware and conversational recommender systems</article-title>
          , in: S. Pera,
          <string-name>
            <given-names>M. D.</given-names>
            <surname>Ekstrand</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Amatriain</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. O'Donovan</surname>
          </string-name>
          (Eds.),
          <source>Proceedings of the 12th ACM Conference on Recommender Systems, RecSys</source>
          <year>2018</year>
          , Vancouver, BC, Canada, October 2-
          <issue>7</issue>
          ,
          <year>2018</year>
          , ACM,
          <year>2018</year>
          , pp.
          <fpage>521</fpage>
          -
          <lpage>522</lpage>
          . URL: https://doi.org/10.1145/3240323.3240338. doi:
          <volume>10</volume>
          .1145/3240323.3240338.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>V. W.</given-names>
            <surname>Anelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. D.</given-names>
            <surname>Noia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. D.</given-names>
            <surname>Sciascio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ragone</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Trotta</surname>
          </string-name>
          ,
          <article-title>How to make latent factors interpretable by feeding factorization machines with knowledge graphs</article-title>
          , in: C.
          <string-name>
            <surname>Ghidini</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          <string-name>
            <surname>Hartig</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maleshkova</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Svátek</surname>
            ,
            <given-names>I. F.</given-names>
          </string-name>
          <string-name>
            <surname>Cruz</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Hogan</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Song</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Lefrançois</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Gandon</surname>
          </string-name>
          (Eds.),
          <source>The Semantic Web - ISWC</source>
          <year>2019</year>
          , Proceedings,
          <string-name>
            <surname>Part</surname>
            <given-names>I</given-names>
          </string-name>
          , volume
          <volume>11778</volume>
          of Lecture Notes in Computer Science, Springer,
          <year>2019</year>
          , pp.
          <fpage>38</fpage>
          -
          <lpage>56</lpage>
          . URL: https://doi. org/10.1007/978-3-
          <fpage>030</fpage>
          -30793-
          <issue>6</issue>
          _3. doi:
          <volume>10</volume>
          .1007/ 978-3-
          <fpage>030</fpage>
          -30793-6\_3.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>G.</given-names>
            <surname>He</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. X.</given-names>
            <surname>Zhao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Wen</surname>
          </string-name>
          ,
          <article-title>Mining implicit entity preference from user-item interaction data for knowledge graph completion via adversarial learning</article-title>
          , in: Y.
          <string-name>
            <surname>Huang</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          <string-name>
            <surname>King</surname>
          </string-name>
          , T. Liu, M. van Steen (Eds.),
          <source>WWW '20: The Web Conference</source>
          <year>2020</year>
          , Taipei, Taiwan,
          <source>April 20-24</source>
          ,
          <year>2020</year>
          , ACM / IW3C2,
          <year>2020</year>
          , pp.
          <fpage>740</fpage>
          -
          <lpage>751</lpage>
          . URL: https://doi.org/ 10.1145/3366423.3380155. doi:
          <volume>10</volume>
          .1145/3366423. 3380155.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>J. Di</given-names>
            <surname>Rocco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. Di</given-names>
            <surname>Ruscio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Di Sipio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. T.</given-names>
            <surname>Nguyen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Rubei</surname>
          </string-name>
          ,
          <article-title>Development of recommendation systems for software engineering: the CROSSMINER experience 26 (</article-title>
          <year>2021</year>
          )
          <article-title>69</article-title>
          . URL: https://doi.org/10.1007/s10664-021-09963-7. doi:
          <volume>10</volume>
          . 1007/s10664-021-09963-7.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>M. P.</given-names>
            <surname>Robillard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Maalej</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. J.</given-names>
            <surname>Walker</surname>
          </string-name>
          , T. Zimmermann (Eds.),
          <source>Recommendation Systems in Software Engineering</source>
          , Springer Berlin Heidelberg, Berlin, Heidelberg,
          <year>2014</year>
          . doi:
          <volume>10</volume>
          .1007/ 978-3-
          <fpage>642</fpage>
          -45135-5.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Rokach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Shapira</surname>
          </string-name>
          ,
          <article-title>Introduction to recommender systems handbook</article-title>
          , in: F.
          <string-name>
            <surname>Ricci</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Rokach</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Shapira</surname>
          </string-name>
          , P. B.
          <string-name>
            <surname>Kantor</surname>
          </string-name>
          (Eds.),
          <source>Recommender systems handbook</source>
          , Springer US, Boston, MA,
          <year>2011</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>35</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>P.</given-names>
            <surname>Bourque</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. E.</given-names>
            <surname>Fairley</surname>
          </string-name>
          , IEEE Computer Society, Guide to the
          <source>Software Engineering Body of Knowledge</source>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>B.</given-names>
            <surname>Combemale</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Lämmel</surname>
          </string-name>
          ,
          <string-name>
            <surname>E. Van Wyk</surname>
          </string-name>
          ,
          <source>SLEBOK: The Software Language Engineering Body of Knowledge (Dagstuhl Seminar 17342)</source>
          (
          <year>2018</year>
          )
          <article-title>10 pages</article-title>
          .
          <source>doi:10.4230/DAGREP.7.8</source>
          .45.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>F.</given-names>
            <surname>Ciccozzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Famelis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Kappel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Lambers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Mosser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. F.</given-names>
            <surname>Paige</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Pierantonio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Rensink</surname>
          </string-name>
          , [11]
          <string-name>
            <given-names>J.</given-names>
            <surname>Bobadilla</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Ortega</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Hernando</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>GutiérR. Salay</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Taentzer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Vallecillo</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          <article-title>Wim- rez, Recommender systems survey, Knowl. mer, Towards a body of knowledge for model- Based Syst</article-title>
          .
          <volume>46</volume>
          (
          <year>2013</year>
          )
          <fpage>109</fpage>
          -
          <lpage>132</lpage>
          . URL: https://doi. based software engineering,
          <source>in: Proceedings of org/10</source>
          .1016/j.knosys.
          <year>2013</year>
          .
          <volume>03</volume>
          .012. doi:
          <volume>10</volume>
          .1016/j. the 21st ACM/IEEE International Conference on knosys.
          <year>2013</year>
          .
          <volume>03</volume>
          .012. Model Driven Engineering Languages and Sys- [12]
          <string-name>
            <given-names>A.</given-names>
            <surname>Bellogín</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Said</surname>
          </string-name>
          ,
          <article-title>Improving accountability tems: Companion Proceedings</article-title>
          , ACM, Copenhagen in recommender systems research through reproDenmark,
          <year>2018</year>
          , pp.
          <fpage>82</fpage>
          -
          <lpage>89</lpage>
          . doi:
          <volume>10</volume>
          .1145/3270112. ducibility,
          <source>CoRR abs/2102</source>
          .00482 (
          <year>2021</year>
          ). URL: https:
          <fpage>3270121</fpage>
          . //arxiv.org/abs/2102.00482. arXiv:
          <volume>2102</volume>
          .
          <fpage>00482</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>V. W.</given-names>
            <surname>Anelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Bellini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. D.</given-names>
            <surname>Noia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. L.</given-names>
            <surname>Bruna</surname>
          </string-name>
          , [13]
          <string-name>
            <given-names>V. W.</given-names>
            <surname>Anelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bellogín</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ferrara</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Malitesta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Tomeo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. D.</given-names>
            <surname>Sciascio</surname>
          </string-name>
          ,
          <article-title>On the role of time and F. A</article-title>
          .
          <string-name>
            <surname>Merra</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Pomo</surname>
            ,
            <given-names>F. M.</given-names>
          </string-name>
          <string-name>
            <surname>Donini</surname>
            ,
            <given-names>T. D.</given-names>
          </string-name>
          <string-name>
            <surname>Noia</surname>
          </string-name>
          ,
          <article-title>Elsessions in diversifying recommendation results, liot: A comprehensive and rigorous framework for in: F.</article-title>
          <string-name>
            <surname>Crestani</surname>
            ,
            <given-names>T. D.</given-names>
          </string-name>
          <string-name>
            <surname>Noia</surname>
          </string-name>
          , R. Perego (Eds.),
          <article-title>Pro- reproducible recommender systems evaluation, in: ceedings of the 8th Italian Information Retrieval F</article-title>
          . Diaz,
          <string-name>
            <given-names>C.</given-names>
            <surname>Shah</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Suel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Castells</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Jones</surname>
          </string-name>
          , T. Sakai Workshop, Lugano, Switzerland, June 05-07,
          <year>2017</year>
          , (Eds.),
          <source>SIGIR '21: The 44th International ACM SIGIR</source>
          volume
          <volume>1911</volume>
          <source>of CEUR Workshop Proceedings</source>
          , CEUR- Conference on Research and Development in InforWS.org,
          <year>2017</year>
          , pp.
          <fpage>92</fpage>
          -
          <lpage>96</lpage>
          . URL: http://ceur-ws.org/ mation Retrieval,
          <source>July 11-15</source>
          ,
          <year>2021</year>
          , ACM,
          <year>2021</year>
          , pp. Vol-
          <volume>1911</volume>
          /16.pdf.
          <volume>2405</volume>
          -
          <fpage>2414</fpage>
          . URL: https://doi.org/10.1145/3404835. 3463245. doi:
          <volume>10</volume>
          .1145/3404835.3463245.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>