<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Mechanisms for Diverse Personalized Bayesian Recom mendations for the Tourism Domain</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Errikos Streviniotis</string-name>
          <email>estreviniotis@intelligence.tuc.gr</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Georgios Chalkiadakis</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="editor">
          <string-name>Bayesian Recommender System, Personalized Recommendations, Social Choice Theory</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Conference on Recommender Systems</institution>
          ,
          <addr-line>Seattle, WA</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Technical University of Crete</institution>
          ,
          <addr-line>Chania</addr-line>
          ,
          <country country="GR">Greece</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>In this work, we employ several multiwinner voting rules from the social choice literature to the personalized recommendations problem. Specifically, we equip with such mechanisms a Bayesian recommender for the tourism domain, allowing for efective personalized recommendations while promoting diverse results with respect to travel-related features. Our system models both users and items-i.e., tourist points of interest (POIs)-as multivariate normal distributions. We employ a novel, lightweight preference elicitation process, during which the user is presented with and asked to rate a small number of POIs-related images. We then use these ratings to guide a Bayesian updating process of beliefs regarding the user's preferences. Moreover, we study the efectiveness of our approach when we equip our system with some prior knowledge regarding the (average) preferences of a specific tourists' type (i.e., tourists of a specific age group), given data collected via questionnaires from actual visitors of a popular tourist resort on a Greek island. Finally, we conduct a systematic experimental evaluation of our approach by applying it on a real-world dataset. Our results (i) highlight the ability of our system to successfully produce personalized recommendations that match the specific interests of a single user; (ii) confirm that the employment of prior knowledge regarding the preferences of tourists, based on their demographics, guides our recommender to avoid the cold-start problem; and (iii) demonstrate that the use of multiwinner mechanisms allows for diverse recommendations with respect to travel-related features, and increased system performance in the case of limited user-system interactions.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Recommender systems in tourism play the key role of digital guides for the various activities
that a tourist destination might provide to visitors based on their preferences [
        <xref ref-type="bibr" rid="ref1 ref2 ref3 ref4">1, 2, 3, 4</xref>
        ].
Tourism recommenders can be broadly categorized as hotel RSs, restaurant RSs, tourism RSs
that are associated with group recommendations, tour planning (or travel packages); and tourist
attraction RSs (i.e., points of interests, museums, etc.) [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. However, in the complex domain of
tourism, most of the times the user-items ratings are very sparse compared to other domains (e.g.,
the movies domain), and as such the employment of classic recommender system approaches,
e.g., collaborative filtering techniques, can be a complicated task [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Additionally, many tourists
RecSys Workshop on Recommenders in Tourism (RecTour 2022), September 22th, 2022, co-located with the 16th ACM
have limited time when they visit a travel destination, and as such a recommender system
should be able to provide eficient recommendations in order to maximize their satisfaction
with a light-weight user-system interaction. Finally, visitors commonly use their mobile phones
as a tool in order to exploit an unknown destination. As such, the development of recommender
systems that use computationally eficient algorithms that can run on the mobile devices of the
users is of utmost significance.
      </p>
      <p>
        Against this background, in this work we introduce a personalized Bayesian recommender
system for the tourism domain and evaluate it on a real-world dataset for a tourist destination
in Greece—specifically that of Agios Nikolaos, Crete. The dataset was created for the needs of a
real-world tour planning recommender system that is currently being developed in collaboration
with an e-commerce company and the corresponding municipality. Generally, Bayesian methods
are able to provide eficient recommendations and, most importantly, such techniques can be
applied for real-time mobile recommendation services [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. In more detail, real world data on
points of interests (POIs) and users’ preferences were collected exploiting: (i) information and data
provided by the Municipality of Agios Nikolas, Crete; (ii) online sources; and (iii) questionnaires
that were filled by tourists. We propose an image-based process to elicit user’s preferences,
by eficiently updating system’s beliefs about the user’s interests via a computational eficient
Bayesian updating procedure that exploits the priors’ conjugacy property. Furthermore, we
equip our system with prior knowledge, obtained via questionnaires from real-world tourists,
and experimentally study its efectiveness.
      </p>
      <p>
        Our main contribution in this paper, however, is putting forward a novel recommendation
mechanism that can be used to increase the diversity of the final personalized recommendations
(instead of simply “greedily” recommending the POIs so-far-perceived-as-best). Such diversity
is important, and contributes to the overall quality of the recommendations—especially when
these do not rely on ratings of other tourists/users, but are strictly personalized (in the sense
that they are produced given only a few interactions with the user in question, as is the case in
our system). Our mechanism is inspired by multiwinner election rules used in social choice
theory [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], and which come complete with theoretical guarantees regarding properties satisfied
by the set of the election winners (e.g., proportionality of the representation). Specifically, our
approach creates a “personalized election” in order to “elect” a set of  “winners” (corresponding
to the  final recommendations made to the targeted tourist-user). We study several voting
rules and show that such an approach can be useful since it provides diverse results with respect
to travel related features—e.g., Culture, History, Cuisine, and other characteristics that a tourist
attraction may provide. For instance, assume that a user has due to various reasons (e.g., could
have been pressed with time) rated highly only cultural POIs, when visiting a tourist destination.
A mechanism that produces diverse final recommendations (i.e., a set of “election winners”),
would present to the user POIs that are related to various categories, and not just Culture-related
ones—a fact that is in fact expected to enhance the tourist experience. As such, our approach
can be thought of as aiming to tackle the manifestation of the classic exploration vs exploitation
problem [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] in this domain. Our experiments confirm that using multiwinner elections leads
to improved system performance when the user-system interactions are limited, while the
advantage thus provided decreases or evaporates with increased user-system interactions. To
the best of our knowledge, this is the first time that such an approach is used in the recommender
systems literature.
      </p>
      <p>Finally, we conduct a systematic experimental evaluation of our recommender system by
applying it on a real-world dataset of the popular Greek island tourist resort in question.
Our results confirm the efectiveness of our approach, and highlight its ability to provide top
personalized recommendations with respect to user’s preferences and interests.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Background and Related Work</title>
      <p>In this section, we provide the necessary background for this work. Specifically, we begin
by briefly reviewing some recommender systems for individual users in the tourism domain.
Finally, we describe some well-known notions of the social choice theory and present several
rules for multiwinner-elections.</p>
      <sec id="sec-2-1">
        <title>2.1. Personalized Recommendations</title>
        <p>
          In general, tourism is a domain that contains various items of diferent types (e.g., leisure POIs,
cultural POIs etc.), while it is highly connected with users’ preferences and interests. Thus,
the development of eficient recommender system approaches is crucial for such a domain.
There is a plethora of tourism- or travel-oriented recommenders, potentially classified in
diferent categories, as listed in [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. Most of those systems recommend POIs that correspond to
touristic attractions (e.g., restaurants, hotels, historical sites or museums), that are ideally highly
connected with each individual tourist’s preferences. Here we brief-review a few representative
such systems.
        </p>
        <p>
          An extensive overview of tourism recommender system algorithms is provided by Borràs et al.
[
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. The authors discuss alternative user interfaces, recommendation techniques and additional
services that such system may provide. Sánchez and Bellogín [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] presented a survey of
recommender systems that employ Location-Based Social Networks for POIs recommendations,
while discussing open challenges for such approaches. A restaurant recommender system
for mobile devices introduced by Zeng et al. [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. Specifically, their approach adopts a user
preference model based on the restaurants that the user has already visited in the past, and
produce the final recommendations by exploiting the exact location of the user. Kbaier et al.
[
          <xref ref-type="bibr" rid="ref13">13</xref>
          ] develop a Hybrid recommender system that generates personalized recommendation by
combining three diferent techniques, i.e., collaborative filtering, content-based filtering and the
demographic filtering. A personalized tour planner, called eCOMPASS, was proposed by Gavalas
et al. [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. In their work, the system recommended personalized tours, i.e., a route that contains
several POIs, by also considering: (i) the weather forecast, and (ii) the possibility that the visitor
can use the public transit to move among the POIs. PersTour [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ] is another recommendation
system that was developed for recommending several POIs to the visitors by considering (i)
the popularity of a specific POI; and (ii) the preferences of a specific tourist. Additionally,
PersTour modifies the tourist’s visiting time for a specific POI, based on how relevant this
POI is to her interests. Mishra et al. [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] introduced a recommender system that employs the
Sentiment Intensity Analyzer (SIA) in order to find the most appropriate POIs for a specific
user by exploring the available reviews of other visitors. In [16] authors introduced a
picturebased recommender approach which suggests tourist destinations for a specific individual.
Specifically, the user selects any set of pictures which is then used as input to computer vision
models that generate a profile which describes the preferences of the tourist. Massimo and Ricci
[
          <xref ref-type="bibr" rid="ref1">1, 17</xref>
          ] studied the problem of next POIs recommendations, i.e., given a set of POIs that a specific
tourist has already visit provide recommendations for the not-yet-visited POIs. Specifically,
Inverse Reinforcement Learning (IRL) techniques were employed in order to learn the reward
function and the optimal POI selection policy. Finally, Gavalas et al. [18] provide a review of
the state-of-the-art techniques for mobile recommender systems in the tourism domain.
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Social Choice Theory and Multiwinner Election Rules</title>
        <p>
          Social choice theory is a theoretical framework that applies to various domains such as
economics, political science, computer science, etc. In general, social choice theory studies
aggregation mechanisms of individual preferences or interests in order to reach a collective
choice or decision [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. The core scenario of collective decision making explored in this field
is the election of a single “winner” based on the preferences of several voters over a number
of alternatives. Many single-winner voting rules have been proposed in the literature with
most known plurality, Borda count, Copeland, etc. [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. On another point of view, a diferent
kind of election is that of selecting a  -sized group of candidates, referred to as committee,
instead of a single winner, i.e., a single alternative. This type of elections are also known as
multiwinner elections [19] and can be categorized as: (i) Shortlisting, (ii) Diverse Committee
and (iii) Proportional Representation mechanisms based on their type and their properties [20].
Intuitively, a Shortlisting mechanisms elects a committee consisting of the alternatives that
have the best quality with respect to some feature(s), while a Diverse Committee mechanism
elects a “committee” consisting of alternatives that are diverse based on some feature(s). In
our, novel for multiwinner elections, application domain, assume users that prefer to go for
shopping when they visit a tourist destination. A mechanism that produces a diverse committee
as the final recommendations, i.e., the winners, would present to the user POIs that are related
to various categories of shopping, i.e., a jewelry shop, a shop for local products, a clothing shop
etc. By contrast, the applications of a Shortlisting mechanism may produce recommendations
that focus on a specific category of shopping, e.g., only souvenir shops. Finally, a Proportional
Representation mechanism selects a committee that captures all the diferent preferences of the
voters proportionally.
        </p>
        <p>We now focus on approval-based rules, in which each voter lists the candidates she approves,
i.e., the alternatives that she likes. The most common approval-based mechanism is the
wellknown Approval Voting. In more detail, Approval Voting assigns one point to each approval
for an alternative and then chooses the alternative with the highest score. As such, in
multiwinner elections, Approval Voting elects a committee consisting of  alternatives that the
voters approved most frequently [19]. In the case of a single winner, Approval Voting has a
number of desirable properties, however it fails to achieve proportional representation in the
multiwinner scenario [21]. Notably, an approval-based rule which satisfy strong axioms for
election proportionality is the Proportional Approval voting (PAV) mechanism. Specifically, under
the PAV rule the contribution of each voter to the final score of the committee is determined by
how many candidates from the voter’s approval set have been elected [21]. However, Skowron
et al. [22] proved that finding winners according to PAV mechanism is NP-hard. As such, the
Reweighted Approval Voting (RAV), a greedy variant of PAV has been introduced and can be
considered as a good approximation algorithm for PAV [19]. Formally, RAV is defined as:
Definition 2.1. [19] Consider an election with  voters where the  -th voter approves candidates
in the set   . RAV starts with an empty committee  and executes  rounds. In each round it adds
to  a candidate  with the maximal value of ∑∶∈  |∩ 1 |+1 .</p>
        <p>
          Another approval-based rule is the Bloc rule; according to Bloc, each voter selects her  favourite
alternatives and the mechanism elects a committee consisting of the alternatives that were
mentioned more frequently [23]. Finally, a well-known non-approval-based rule, the  -Borda
selects the  alternatives with the highest Borda score [
          <xref ref-type="bibr" rid="ref8">23, 8</xref>
          ].
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Our Approach</title>
      <p>In this work we introduce a system that enhances the visitors’ experience of a tourist destination
by recommending POIs that “match” their interests and preferences. Inspired by the work
of Babas et al. [24], we designed a Bayesian recommender system which performs Bayesian
updating in order to learn user’s (which corresponds to tourist’s) preferences and provide her
with eficient recommendations. Specifically, we model the users and the items (i.e., the POIs of
a tourist destination), using a common representation—i.e., multivariate normal distributions
over ranges of values, describing the degree that each feature describes a specific user or item.
We note that, our system is unaware of every user’s real distribution, while it is fully informed
about each item’s distribution. The goal of our agent, i.e., recommender system, is to “predict”
the distribution describing a specific user in order to recommend the POIs most appropriate for
her, thus increasing her satisfaction.</p>
      <p>To this purpose, our system performs a series of “questions” in order to determine user’s
preferences by exploiting the data derived by this procedure. Specifically, we employ an
image-based user preference elicitation approach, by presenting to the user a set of generic
travel-related pictures and noting her “likes” in order to build a user model. Moreover, we equip
our system with some prior knowledge regarding the general preferences of a specific type
of visitors, i.e., tourists that belong to the same age group. Finally, we design a social choice
inspired mechanism that produces diverse personalized recommendations to the user with
respect to travel-related features. Specifically, given a specific user we create a “personalized
election” based on her inferred model and employ various multiwinner election rules in order
to produce diverse personalized recommendations.</p>
      <sec id="sec-3-1">
        <title>3.1. Bayesian Inference</title>
        <p>When a new user enters our system, commonly no (prior) information regarding her preferences
is provided to our recommender system,1 i.e., regarding her multivariate normal distribution.
As such, we employ the Normal-Inverse Wishart (NIW) [25] conjugate priors to model the
system’s beliefs regarding the user’s underlying parameters. In general, the NIW distribution
is a multivariate four-parameter family of continuous probability distributions that has the
1We study later the case in which our system is able to exploit prior information regarding a user based on
demographics information derived by real tourists via questionnaires.</p>
        <p>=</p>
        <p>0
 0 +</p>
        <p>0 +</p>
        <p>⋅  ̄
⋅  0 +
  =  0 + 

 =  0 + 
 0 ⋅ 
 0 + 
Ψ = Ψ0 +  +
⋅ ( −̄  0) ⋅ ( −̄  0
)


=1
 =
∑(  − ) ̄ ⋅ (  − ) ̄ 
(1)
(2)
(3)
(4)
(5)
(6)
(7)
where  ̄ is the sample mean,  is the number of the samples,   are the samples drawn from the
data,  is a scatter matrix. Finally, we can use an Inverse Wishart and a Normal distribution to
derive the covariance matrix Σ and the mean  given the updated beliefs, as follows:
Σ ∼ ℐ  (Ψ  ,   )
|Σ ∼  (
 , Σ/  )
desired property of conjugacy of a multivariate normal distribution with unknown mean and
covariance matrix. The use of conjugate priors ofers a closed-form for the computation of the
posterior distribution, resulting in a computationally eficient Bayesian updating procedure [ 24].
Specifically, we can update the prior hyperparameters—  0,  0 (the mean vector),  0 (degrees of
freedom), and Ψ0 (the precision matrix)—using samples drawn from the data to get the posterior
ones, as follows:</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Recommendation Process</title>
        <p>
          First, when a user registers in the system the preference elicitation procedure begins. In more
detail, the preference elicitation procedure is an iterative process where in each iteration our
system presents  alternative generic images to the user. We use the term “question” to refer
to such a presentation of a set of generic images. These generic images, similar to POIs, are
represented as multivariate Gaussians. Each generic image corresponds a specific type of POIs,
i.e., a restaurant, a monument, a beach etc. We select this picture-based approach to elicit
user’s preferences, since there is a great complexity regarding the tourism product [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ], and such
approaches have been eficiently applied in order to elicit travel-related preferences [ 26, 27].
We highlight that the set of the generic images and the set of POIs do not share any common
element (i.e., there are no items that belong to both lists).
        </p>
        <p>Note that our agent selects which generic images to present to the user based on the available
information (represented by the multivariate distribution), regarding her preferences (see
Section 3.3). However, when a new user enters the system, most of the times we have no
available information about her, so we have to randomly pick some generic images in the first
iteration, unless our system has some prior knowledge at its disposal (e.g., a prior knowledge
regarding the preferences of a specific age group, as we discuss later in Section
4.1.2). In such a
case, our recommender is able to exploit the extra knowledge in order to pick the images that
will present to the user in the first iteration. Once our algorithm selects the generic images that
will present to the user, the user picks the image that is most “attractive” to her, with respect to
her interests, and provides a rating on a 5-level Likert scale, where 5 implies that this image fits
her preferences perfectly.</p>
        <p>In our approach, similarly to that of [24], we use the Kullback-Leibler (KL) Divergence criterion
in order to compute the similarity between any item (generic image or POI) and any user based
on her preferences and interests. In particular, knowing that both users and items share a
common representation—since both are modelled as multivariate normal distributions— we
can employ the KL-divergence criterion in order to find “how similar” their distributions are.
Formally, the KL-divergence between a Gaussian  and a Gaussian  , of dimension  each, is
given by:
(8)
(9)
 ( ||) =
1
2
|
1
2
−1  | +
 (( 
−1  )−1) −
+ (  −   )  −1(  −   )</p>
        <p>2
1
2
where   ,   ,   and   are the distributions’ parameters, and  (⋅) is the trace of the
corresponding matrix [28]. In principle, a small KL-divergence between a Gaussian  and a Gaussian 
means that they are similar, while a large KL-divergence means the distributions are not similar.
Thus, in our work we make the natural assumption that the more similar the distributions of
a user  and an item  are, the higher the rating (of user  for item  ) would be. As such, the
(predicted) rating of a user (represented as a Gaussian  ) for an item (represented as a Gaussian
 ) can be defined as:
 , =  −
 (||)

where  is the maximum rating the user can give to an item, i.e.,  = 5 .</p>
        <p>Then, depending on the provided rating our system draws an appropriate number of samples
from the distribution of the selected generic image. Specifically, we use the logistic function [29]
and the rating of the user in order to compute the exact number of samples that will be drawn
from the generic image’s distribution. Intuitively, the form of this function fits to our purposes,
since a high rating (signifying the user likes a generic image), means that the distributions of
the user and the image are similar, and as such a suficiently large number of samples from the
image’s distribution will contribute to construct a good model regarding user’s preferences;
while a small rating means that the user does not feel that this image describes her preferences
well. Thus, a small number of samples should be drawn since they are not representatives of
user’s interests. Once the user enters her rating, our approach performs a Bayesian updating in
order to produce an updated type of user (i.e., distribution) combining prior knowledge and
the new data of this iteration, which corresponds to an user-system interaction. Note that the
posterior derived in iteration  will be the prior for iteration  + 1 , with which our system will
choose which generic images to present to the user next (see Section 3.3). That is, the generic
images at each iteration are chosen based on the beliefs updated on the previous iteration. This
procedure terminates after  iterations. As such, our system estimates the parameters of user’s
(, Σ ) based on the hyperparameters of the NIW distribution (see Section 3.1).</p>
        <p>Finally, our greedy version of our system utilizes the estimated parameters of the user in
order to produce its final recommendations, by applying the KL divergence—i.e., it greedily
recommends the  POIs that are more similar to the tourist’s inferred model. We note that,
more user-system interactions, i.e., more  iterations, can provide our system better indications
regarding the feature preferences of a specific user resulting a more representative user model.</p>
      </sec>
      <sec id="sec-3-3">
        <title>3.3. Generic Images Selection</title>
        <p>One crucial aspect of our recommendation system is the selection of an efective method which
decides the items that will be shown to the user at each iteration. The item selection method
should handle in an eficient way the</p>
        <p>exploration vs exploitation dilemma in order to increase the
performance of our system. In our work we studied three alternative item selection methods,
namely: a greedy mechanism based on the KL-divergence, the VPI exploration [24], and the
Boltzmann selection. The Boltzmann selection mechanism performed better and as such is the
method we detail here. Notably, in the experimental evaluation the results corresponds only to
this mechanism.</p>
        <p>Intuitively, Boltzmann exploration [30] tells us to pick an action with a probability that is
proportional to its average reward. As such, actions with greater average rewards are picked
with higher probability. Formally, at each time step  , our agent assigns a selection probability
to each item  using the following formula:
  () =</p>
        <p>, /
∑

=1   , /
(10)
where  =  ⋅   , with  be a constant value,  , is the quantity computed in Equation 9, and
 &lt; 1 . In this method, the exploration vs exploitation trade-of is controlled via a temperature
parameter  that decreases over time to progressively reduce exploration. Specifically, for very
small values of  the action with the highest average reward (in our case, the highest predicted
 , rating) is more likely to be selected. On the other hand, in initial stages where the value of 
is large, the Boltzmann method efectively corresponds to a random policy.</p>
      </sec>
      <sec id="sec-3-4">
        <title>3.4. Generating Personalized Recommendations using Multiwinner Election</title>
      </sec>
      <sec id="sec-3-5">
        <title>Mechanisms</title>
        <p>As described in Section 3.1, our system learns the preferences of a user via a Bayesian inference
procedure that “builds” a distribution that describes the user with respect to some travel-related
features. Moreover, our recommender greedily recommends the  items that are more similar
to the tourist’s inferred distribution based on the KL divergence criterion. However, such an
approach can lead to several undesirable properties regarding the final recommendations that
the user will receive. For example, let us assume that a tourist during the preference elicitation
process chose to give a high rating only to restaurants.In this case, a “greedy” approach will
produce recommendations which will only be related to restaurants. Such a property might
be desirable in other domains such as movies (e.g., action movies, etc.) but in the tourism
domain, a tourist would like to have a variety of diferent POIs that she could visit in a travel
destination, e.g. a restaurant, a monument, a bar, and so on. In other words, in order for the
recommender algorithm to be able to provide a multitude of choices to the tourist, it should
guarantee that there will be some diversity between its final  recommendations. Moreover, we
remind the reader that the computation of the recommendations’ quality is based on the user
model constructed so far. However, such a model is constructed using only limited user-system
interactions resulting to a rather naive estimation of the actual user model; thus, we conjecture
that providing diversity in the final recommendations is helpful. Thus, it is natural to apply a
multiwinner voting rule from the social choice theory in order to ensure diverse results with
respect to any subset of features. Figure 2 depicts our proposed approach.</p>
        <p>In general, given a set of users (or voters),  , and their preferences, multiwinner election
mechanisms select a  -sized set of alternatives (i.e., “a committee”). Such mechanisms satisfy
specific properties based on their type (see Section 2.2). Given this, we now detail our approach
to provide diverse recommendations to a tourist with respect to some travel-related features,
instead of greedily selecting the POIs that are more similar to user’s inferred model.</p>
        <p>To this end, given a specific tourist that is described by a multivariate Gaussian distribution
over a set of features, we create a “personalized election” in order to produce our final
recommendations. In more detail, given a user  , we exploit her mean vector,   , in order to create a
set of voters based on  ’s values over the selected travel-related features, i.e., the values on the
  for each (selected) feature. As such, we generate a set of voters,  , that provides proportional
representation of user’s preferences over the features, i.e., a feature that has a high score will
be represented by more voters than a feature that has a low value for  . Specifically, for any
travel-related feature  , with value   , we generate ⌈  ⋅ 10⌉ voters. Moreover, we make the
natural assumptions that any voter that has been generated from feature  : (i) approves an item
 , i.e., a POI, that has a value that is greater than 3 on feature  , i.e.,   ≥ 3; and (ii) prefers an
item  over an item  if and only if  has a greater value than  on feature  , i.e.,  , &gt;  , ,2. On the
other hand the set of alternatives,  , and the set of POIs are identical. Therefore, we can apply
2We note that if  , =  , we randomly select which item is preferred ( ≻  or  ≻  ) for each voter independently.
any3 multiwinner rule on this election in order to be able to pick several “personalized” winners
that also satisfy specific properties (e.g., proportionality of the preferences’ representation)
guaranteed by the rule in question, based on the travel-related features and the preferences
of the tourist. As such, by not simply “greedily” recommending the so-far-perceived-as-best
POIs, the proposed approach provides diverse recommendations. This fact is evident in our
experimental results, where we see that using multiwinner elections leads to improved system
performance when the user-system interactions are limited—with this advantage decreasing or
evaporating with increased user-system interactions.</p>
        <p>We also point out that such an approach allows us to provide diverse recommendations
with respect to any selected subset of travel-related features. In more detail, our approach
can maintain such property by creating a “personalized election”, where the voters have been
produced solely from the selected subset of features following the procedure that has been
already described in this section. (Of course, the set of the alternatives remains the same—i.e., it
comprises all the available POIs.)</p>
        <p>3In our experiments below, we test several multiwinner election rules—specifically, AV, RAV,  -Borda and Bloc
(see Section 2.2).</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Experimental Evaluation</title>
      <p>In this section we present a series of experiments to evaluate: (i) the ability of our approach to
learn the preferences of a user, and provide high quality recommendation wrt her preferences;
and (ii) the performance of several multiwinner election mechanisms with respect to the quality
of the final recommendations. We use a real-world dataset including 430 POIs of a popular
tourist resort on a Greek island. Alongside the available POIs, we used 90 generic images for
the preference elicitation process. Note that for this series of experiments we assume that each
user, POI, and generic image can be described as a multivariate normal distribution, where
 = 12 . Specifically, we consider the following travel-related features: Culture, Sun &amp; Sea,
History/Archaeology, Adventure/Sports, Afordable Prices, Family-friendly activities &amp; facilities,
Rural Tourism, Luxury Accommodation, Nightlife, Gastronomy/Cuisine, General Shopping and
Shopping Local Products. We note that the Boltzmann exploration parameters for all series of
experiments were set to:  = 1,  = 0.5 and  ≤ 3 , with  0 = 0 and  +1 =   + 1.</p>
      <sec id="sec-4-1">
        <title>4.1. Personalized Recommendations</title>
        <p>First we present a series of experiments performed to evaluate the greedy version of our approach
for personalized recommendations, using synthetic tourists. In more detail, using
preferencerelated data collected from actual tourists via questionnaires, we generate 500 synthetic tourists
for the age groups of 18-25, 26-35, 36-45, 46-55, 56-67 and 67 plus respectively, i.e., 3000
synthetic users in total. The average of the answers (i.e., values in the scale of 1...5 recorded
on the questionnaires) of each age group for the 12 features provided by the questionnaires is
calculated for each corresponding feature. In order to generate the synthetic users for each age
group, we take 500 samples from a multivariate normal distribution, whose mean vector is the
average values of the 12 features; the covariance matrix is diagonal and each diagonal element
takes a value equal to 1. We run a series of user-system simulations with varying slate size—i.e.,
number of generic images  = {1, 2, 3, 4, 5} that are presented to a user in each question asked,
and number of questions  = {1, 2, 3, 4, 5} asked. Finally, for each ⟨, ⟩ combination we ran
1000 simulations: in every simulation, we randomly pick a tourist out of our 3000 synthetic
users. As an evaluation metric, we compare the real distribution that represents user’s actual
preferences with her inferred one—i.e., the distribution that our algorithm constructed via the
preference elicitation method (see Section 3.2)—by using Equation 9. In what follows, we denote
this resulting “predicted rating” metric by  for short. Notice that had the inferred distribution
been identical to the real one, this metric’s value would be equal to 5, i.e., the maximum rating
that an item can receive by a user.</p>
        <sec id="sec-4-1-1">
          <title>4.1.1. No prior knowledge employed</title>
          <p>Table 1 illustrates the results of the “greedy” approach on a first set of experiments that does not
exploit any prior knowledge regarding the user—i.e., the system does not have any information
regarding the user’s age group, but uses an uninformative prior (i.e., one with a mean vector
that contains in each dimension a value of 1, and a diagonal covariance matrix where each
diagonal element takes a value equal to 2). Note that the presented results are average values
over 1000 simulations for each ⟨, ⟩ combination setting tested. We can see that for  fixed
across diferent settings, the  metric achieved by our algorithm increases as the number of
questions, i.e.,  , increases. Such a result is expected, since when the system makes more
questions to a user, more information regarding her preferences is revealed, and as such our
approach builds a better model with respect to the user’s preferences. Also, for a fixed number
of  questions, we observe that as  (i.e., the number of alternative pictures shown per question)
increases, the  achieved increases as well. This is due to the fact that when the system provides
more options to a user, then a picture that best captures her preferences is easier to be found.
Thus, by increasing the number of options  , our system is able to build a better user model,
since it exploits better quality information regarding user interests.</p>
          <p>Following that, we used another metric that captures the efectiveness of our approach based
on the quality of the recommended POIs. Specifically, for each individual that interacts with the
system, we use her “real” distribution in order to create the list   , which contains the top-N
POIs for this specific user (i.e., given a user  , the  POIs from the dataset that score the highest
 , scores). At the same time, we use the inferred model for this user (i.e., the distribution that
our system created via the preference elicitation procedure), in order to create the list   , which
contains the top-20 POIs for the inferred user. Thus, we can measure the similarity between the
lists   and   by finding the common elements, i.e., POIs, that these two lists share: that is, we
count how many of the (real) user’s top-N items coincide with ones in the recommended, given
the inferred user model, top-20 list of items. For this set of experiments we set  = {20, 43, 86} .
Note that, the “top-20” corresponds to only the 4.5% of our dataset. Similarly, the “top-43”
constitutes only the 10% of our dataset, while “top-86” the 20%.</p>
          <p>Table 2 shows the average of 1000 simulations for each combination of ⟨, ⟩ . As seen, for a
given  (or a given  ), the percentage of common elements that lists   and   share, is rising
as  (or  ) rises. This is natural, since for larger  (or  ) our system collects more information
regarding user’s preferences and as such is able to provide better recommendations. Indicatively,
in settings with n=5, m=5 our agent recommends POIs with 38.82% of them being among the
best 20 POIs of each user. Accordingly, 57.81% of them being among the best 43 POIs and
75.19% of them being among the best 86 POIs of each user. Thus, after only a small number of
interactions with each user, our approach is able to provide recommendations that match the
user interests to a large extent.</p>
        </sec>
        <sec id="sec-4-1-2">
          <title>4.1.2. Employing prior knowledge</title>
          <p>Here, we study the scenario in which our system has at its disposal some prior knowledge
regarding general user types preferences. Specifically, we construct some age-related priors—i.e.,
prior distributions regarding the general preferences of tourists that belong to the same age
group, by exploiting data collected via questionnaires from actual visitors. These priors are
constructed in the exact same way as the ones from which we draw the synthetic users from, with
the only diference being that we insert higher uncertainty, i.e. the diagonal covariance matrix
elements have a value equal to 2. In this scenario, we study the performance of our recommender
for each age group and we compare it with the case where no such extra information is available
to us. For each age group, we generated 1000 users via the process already described in the
beginning of this section, along with their corresponding priors.</p>
          <p>Tables 3, 4, 5 and 6 depicts the average results over 1000 simulations for the age group of
1825, 26-35, 36-45, 46-55, 56-67 and 67 plus. In more detail, Tables 3 and 5 capture the performance
of our approach for each age group when no prior information is available. Again, we use the 
metric in order to evaluate the inference of our system and the number of recommended POIs
that belongs to the “top-20” of the user. Tables 4 and 6 present our corresponding results when
our system knows the user’s age group and can thus exploit its prior knowledge regarding
this age group’s preferences. First, we notice that with such prior knowledge in its disposal,
our recommender significantly and consistently outperforms the version that has no prior
knowledge available.</p>
          <p>Moreover, the system is then able to provide high-scoring recommendations with only limited
interaction among the user and the system, i.e. when the values of  and  are small, vastly
outperforming the uninformed version. However, as the  and  values increase, the margins
between the performance of the two versions decrease. Such a result is expected since the prior
knowledge gives insights to our system regarding the user’s preferences.</p>
          <p>Of course, given the way the generic age-related priors are constructed and the fact that the
corresponding synthetic users’ distributions are closely related to the priors, these results have
to be taken with a grain of salt. However, they do show that high-quality priors can indeed be
exploited by our system. Moreover, the experiment provides further insights to the method’s
behaviour, given the following interesting observation: Notice that, with prior knowledge
available, there are cases where the performance of our approach in terms of recommendations
made using KL-divergence, drops with more questions asked or images displayed (cf. Table 6).
(Similarity of   and   for “top-20”).</p>
          <p>(Similarity of   and   for “top-20”).</p>
          <p>We attribute this behaviour to the fact that apparently our age-related prior is of a very high
quality (indeed, it is very similar to the synthetic user’s distribution), and is thus able to capture
the preferences of a user to a very large extent—i.e., it describes her interests for every
travelrelated feature that we have used in our approach. By contrast, when the preference elicitation
process kicks in, we present generic images that cannot but represent a very specific type of
POIs (e.g., a beach, a restaurant etc.). Thus, when the visitor selects and rates a picture, the
samples that will be drawn in order to perform the Bayesian updating of the user’s model will
help improve beliefs only for specific features, causing a small drop in recommendations’ quality.
Notice that this is despite the fact that performance wrt. the  -metric consistently increases
with questions asked and images displayed.</p>
        </sec>
        <sec id="sec-4-1-3">
          <title>4.1.3. Multiwinner elections for personalized recommendations</title>
          <p>Here we evaluate our social choice-inspired mechanism for generating our final
recommendations to a tourist. To this end, we created 1000 synthetic tourists following the exact same
procedure already described earlier (see Section 4.1), and applied the following well-known
aggregation strategies: (i) Approval Voting (AV), (ii) Reweighted Approval Voting (RAV), (iii)
Bloc, and (iv)  -Borda. We chose to evaluate our approach by measuring the similarity between
the lists   and   for the top-20 POIs of each user.</p>
          <p>Table 7 illustrates the results of our approach on this set of experiments. Note that the
presented results are the average values over 1000 simulations of experiments for each ⟨, ⟩
combination. First, we observe that when the values of  and  are small (i.e. 1 or 2), the greedy
approach usually ranks lower in terms of similarity score among most multiwinner election
mechanisms methods—while for  = 1 and  = 2 all multiwinner election mechanisms perform
better than “greedy” with respect to our metric. These performance results are natural, since for
smaller  and  we collect limited information regarding the interests of the user. As a result,
our inferred model does not describe the user preferences very accurately. Thus, the selection of
the POIs based on the model inferred so far—given only the very limited information provided
by the user—is not the optimal policy. On the other hand, the employment of multiwinner
election mechanisms allows us to move beyond this suboptimal greedy approach, while it still
exploits our inferred model (see Section 3.4). However, as  and  increase, and our system
is able to collect more information regarding the interests of the tourist, we notice that the
greedy performs better than most of the voting rules, but its advantage is in most cases not
significant—in contrast to what was the case for low  and  values, where the advantage of
the social choice-inspired methods was broad. Additionally, the employment of the voting
rules results to a diverse “elected committee”, corresponding to diverse final recommendations
with respect to the travel-related features. Thus, we see that such mechanisms can help us (i)
achieve better performance regarding the quality of our recommendations when the user-system
interaction is limited; and (ii) produce various POIs of diferent types, by simply paying a small
quality penalty compared to the performance of the greedy approach.</p>
          <p>Finally, we notice that when  = 1 the RAV rule performs better with respect to our evaluation
metric, showing that a proportional representation of the travel-related features (our “voters”),
is a rational choice when limited information is in our disposal. As the user-system interaction
increase, and more information regarding her interests is available to our system, the AV
approach achieves the highest performance.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions and Future Work</title>
      <p>In this paper, we designed a personalized Bayesian recommender system for the tourism domain.
Our system employs several well-known multiwinner voting rules on a “personalized election”
generated by the user’s inferred model in order to produce the final recommendations. It models
both tourists and POIs as multivariate normal distributions and learns the preferences via a
lightweight picture-based elicitation process that guides a Bayesian updating process. Moreover,
we equipped our system with prior knowledge exploited via questionnaires from real tourists,
and studied the efectiveness of our approach. Finally, we conducted a systematic experimental
evaluation of our approach using a real-world dataset of a tourist destination. Our results (i)
confirm that our approach is able to accurately model the users and thus to provide efective
personalized recommendations; and (ii) demonstrate that the use of multiwinner mechanisms for
personalized recommendations allows for diverse recommendations in terms of travel-related
attributes, which are nevertheless of high quality and clearly outperform the simple “greedy”
Bayesian approach when information provided by the users is limited (and thus our confidence
in the inferred user model is low).</p>
      <p>In future work, we intend to further evaluate our approach in scenarios in which diferent
types of prior knowledge is available—i.e., when we have and can exploit information regarding
the general preferences of a type of visitors not only based on the age group that they belong
to, but also their cultural background, their gender, etc. We are currently working on the
extension of our proposed system in order to provide group recommendations, since usually
people choose to travel with company. Specifically, in ongoing work we study the properties of
several multiwinner election mechanisms for the group recommendation problem with respect
to well known fairness metrics (i.e., the m-PROPORTIONALITY and the m-ENVY-FREENESS
fairness metrics [31] ones). Finally, we intend to test our approach with actual tourists, via
incorporating our recommendation techniques in diferent versions of a real-world mobile
application for tours planning which is currently under development.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgments</title>
      <p>This research has been co-financed by the European Union and Greek national funds through
the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call
RESEARCH-CREATE-INNOVATE B cycle (project code: T2EDK-03135). E. Streviniotis was also
supported by the Onassis Foundation - Scholarship ID: G ZR 012-1/2021-2022.
in: 2020 Seventh International Conference on Information Technology Trends (ITT), 2020,
pp. 174–179. doi:10.1109/ITT51279.2020.9320885.
[16] M. Sertkan, J. Neidhardt, H. Werthner, Pictoure - a picture-based tourism recommender,
in: Fourteenth ACM Conference on Recommender Systems, RecSys ’20, Association for
Computing Machinery, New York, NY, USA, 2020, p. 597–599. URL: https://doi.org/10.1145/
3383313.3411526. doi:10.1145/3383313.3411526.
[17] D. Massimo, F. Ricci, Next-poi recommendations matching user’s visit behaviour, in:
W. Wörndl, C. Koo, J. L. Stienmetz (Eds.), Information and Communication Technologies
in Tourism 2021, Springer International Publishing, Cham, 2021, pp. 45–57.
[18] D. Gavalas, C. Konstantopoulos, K. Mastakas, G. Pantziou, Mobile recommender systems
in tourism, Journal of Network and Computer Applications 39 (2014) 319–333.
[19] P. Faliszewski, A. M. Slinko, N. Talmon, Multiwinner voting: A new challenge for social
choice theory, 2017.
[20] E. Elkind, P. Faliszewski, P. Skowron, A. Slinko, Properties of multiwinner voting rules,</p>
      <p>Social Choice and Welfare 48 (2017) 599–632.
[21] H. Aziz, M. Brill, V. Conitzer, E. Elkind, R. Freeman, T. Walsh, Justified representation in
approval-based committee voting, Social Choice and Welfare 48 (2017) 461–485.
[22] P. Skowron, P. Faliszewski, J. Lang, Finding a collective set of items: From proportional
multirepresentation to group recommendation, Artificial Intelligence 241 (2016) 191–216.
[23] E. Elkind, P. Faliszewski, J.-F. Laslier, P. Skowron, A. Slinko, N. Talmon, What do
multiwinner voting rules do? an experiment over the two-dimensional euclidean domain,
in: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17,
AAAI Press, 2017, p. 494–501.
[24] K. Babas, G. Chalkiadakis, E. Tripolitakis, You are what you consume: A bayesian method
for personalized recommendations, in: Proc. of the 7th ACM Conf. on Recommender
Systems, RecSys ’13, ACM, NY, USA, 2013, p. 221–228.
[25] J. Schmee, T. Anderson, An introduction to multivariate statistical analysis, Technometrics
28 (1986) 180.
[26] J. Neidhardt, R. Schuster, L. Seyfang, H. Werthner, Eliciting the users’ unknown preferences,
in: Proc. of the 8th ACM Conf. on Recommender Systems, RecSys ’14, ACM, NY, USA,
2014, p. 309–312.
[27] J. Neidhardt, L. Seyfang, R. Schuster, H. Werthner, A picture-based approach to
recommender systems, Information Technology &amp; Tourism 15 (2015) 49–69.
[28] F. Nielsen, R. Nock, Clustering Multivariate Normal Distributions, Springer Berlin
Heidelberg, Berlin, Heidelberg, 2009, pp. 164–174.
[29] D. R. Cox, The regression analysis of binary sequences, Journal of the Royal Statistical</p>
      <p>Society: Series B (Methodological) 20 (1958) 215–232.
[30] D. Carmel, S. Markovitch, Exploration strategies for model-based learning in multiagent
systems, Aut. Agents and Multi-agent Systems 2 (1999) 141–172.
[31] D. Serbos, S. Qi, N. Mamoulis, E. Pitoura, P. Tsaparas, Fairness in package-to-group
recommendations, WWW ’17, 2017, p. 371–379.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>D.</given-names>
            <surname>Massimo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricci</surname>
          </string-name>
          ,
          <article-title>Harnessing a generalised user behaviour model for next-poi recommendation</article-title>
          ,
          <source>in: Proceedings of the 12th ACM Conference on Recommender Systems</source>
          , RecSys '18,
          <string-name>
            <surname>Association</surname>
          </string-name>
          for Computing Machinery, New York, NY, USA,
          <year>2018</year>
          , p.
          <fpage>402</fpage>
          -
          <lpage>406</lpage>
          . URL: https://doi.org/10.1145/3240323.3240392. doi:
          <volume>10</volume>
          .1145/3240323.3240392.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>B.</given-names>
            <surname>De Carolis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Novielli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. L.</given-names>
            <surname>Plantamura</surname>
          </string-name>
          , E. Gentile,
          <article-title>Generating comparative descriptions of places of interest in the tourism domain</article-title>
          ,
          <source>in: Proceedings of the Third ACM Conference on Recommender Systems</source>
          , RecSys '09,
          <string-name>
            <surname>Association</surname>
          </string-name>
          for Computing Machinery, New York, NY, USA,
          <year>2009</year>
          , p.
          <fpage>277</fpage>
          -
          <lpage>280</lpage>
          . URL: https://doi.org/10.1145/1639714.1639767. doi:
          <volume>10</volume>
          .1145/ 1639714.1639767.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A.</given-names>
            <surname>Dadoun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Troncy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Defoin-Platel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. A.</given-names>
            <surname>Solano</surname>
          </string-name>
          ,
          <article-title>Predicting your next trip: A knowledge graph-based multi-task learning approach for travel destination recommendation</article-title>
          , 2021 Workshop on Recommenders in Tourism, RecTour
          <year>2021</year>
          2974 (
          <year>2021</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>K. H.</given-names>
            <surname>Lim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Chan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Leckie</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Karunasekera</surname>
          </string-name>
          ,
          <article-title>Personalized tour recommendation based on user interests and points of interest visit durations</article-title>
          ,
          <source>in: Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI'15</source>
          , AAAI Press,
          <year>2015</year>
          , p.
          <fpage>1778</fpage>
          -
          <lpage>1784</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>K.</given-names>
            <surname>Chaudhari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Thakkar</surname>
          </string-name>
          ,
          <article-title>A comprehensive survey on travel recommender systems</article-title>
          ,
          <source>Archives of Computational Methods in Engineering</source>
          <volume>27</volume>
          (
          <year>2020</year>
          )
          <fpage>1545</fpage>
          -
          <lpage>1571</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A.</given-names>
            <surname>Delic</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Neidhardt</surname>
          </string-name>
          ,
          <article-title>A comprehensive approach to group recommendations in the travel and tourism domain</article-title>
          ,
          <source>in: Proc. of the 25th Conf. on User Modeling, Adaptation and Personalization</source>
          ,
          <source>UMAP '17</source>
          , ACM, NY, USA,
          <year>2017</year>
          , p.
          <fpage>11</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>Y.-Y.</given-names>
            <surname>Chen</surname>
          </string-name>
          , A.-J. Cheng, W. H. Hsu,
          <article-title>Travel recommendation by mining people attributes and travel group types from community-contributed photos</article-title>
          ,
          <source>IEEE Transactions on Multimedia</source>
          <volume>15</volume>
          (
          <year>2013</year>
          )
          <fpage>1283</fpage>
          -
          <lpage>1295</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>F.</given-names>
            <surname>Brandt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Conitzer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Endriss</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Lang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. D.</given-names>
            <surname>Procaccia</surname>
          </string-name>
          , Handbook of Computational Social Choice, 1st ed., Cambridge University Press, USA,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Sutton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. G.</given-names>
            <surname>Barto</surname>
          </string-name>
          ,
          <source>Reinforcement Learning: An Introduction</source>
          , MIT Press, Cambridge, MA, USA,
          <year>1998</year>
          . URL: http://www.cs.ualberta.ca/%7Esutton/book/ebook/the-book.html.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>J.</given-names>
            <surname>Borràs</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Moreno</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Valls</surname>
          </string-name>
          ,
          <article-title>Intelligent tourism recommender systems: A survey</article-title>
          ,
          <source>Expert Systems with Applications</source>
          <volume>41</volume>
          (
          <year>2014</year>
          )
          <fpage>7370</fpage>
          -
          <lpage>7389</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>P.</given-names>
            <surname>Sánchez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bellogín</surname>
          </string-name>
          ,
          <article-title>Point-of-interest recommender systems based on location-based social networks: A survey from an experimental perspective</article-title>
          ,
          <source>ACM Comput. Surv</source>
          . (
          <year>2022</year>
          ). URL: https://doi.org/10.1145/3510409. doi:
          <volume>10</volume>
          .1145/3510409, just Accepted.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>J.</given-names>
            <surname>Zeng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Wen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Hirokawa</surname>
          </string-name>
          ,
          <article-title>A restaurant recommender system based on user preference and location in mobile environment</article-title>
          ,
          <source>in: 2016 5th IIAI International Congress on Advanced Applied Informatics</source>
          , IEEE,
          <year>2016</year>
          , pp.
          <fpage>55</fpage>
          -
          <lpage>60</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>M. E. B. H. Kbaier</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Masri</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Krichen</surname>
          </string-name>
          ,
          <article-title>A personalized hybrid tourism recommender system</article-title>
          ,
          <source>in: 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA)</source>
          ,
          <year>2017</year>
          , pp.
          <fpage>244</fpage>
          -
          <lpage>250</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>D.</given-names>
            <surname>Gavalas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Kasapakis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Konstantopoulos</surname>
          </string-name>
          , G. Pantziou,
          <string-name>
            <given-names>N.</given-names>
            <surname>Vathis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Zaroliagis</surname>
          </string-name>
          ,
          <article-title>The ecompass multimodal tourist tour planner</article-title>
          ,
          <source>Expert systems with Applications</source>
          <volume>42</volume>
          (
          <year>2015</year>
          )
          <fpage>7303</fpage>
          -
          <lpage>7316</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>R. K.</given-names>
            <surname>Mishra</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Urolagin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. A. A.</given-names>
            <surname>Jothi</surname>
          </string-name>
          ,
          <article-title>Sentiment analysis for poi recommender systems,</article-title>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>