<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Prediction of User-Brand Associations Based on Sentiment Analysis</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Mariella Bonomo</string-name>
          <email>mariella.bonomo@unipa.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Simona E. Rombo</string-name>
          <email>simona.rombo@unipa.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Filippo Rotolo</string-name>
          <email>iflippo.rotolo@community.unipa.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>DataPlat'23: 2nd International Workshop on Data Platform Design</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Mathematics and Computer Science, University of Palermo</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Workshop Proce dings</institution>
        </aff>
      </contrib-group>
      <abstract>
        <p>Finding the right users to be chosen as targets for advertising campaigns is not a trivial task, and it may allow important commercial advantages. A novel approach is presented here for the recommendation of new possible consumers to brands interested in distributing advertising campaigns, ranked according to the “compatibility” between users and brands. A database containing both descriptions associated with diferent brands, and textual information about users' opinions on diferent topics, is required in input. Then, sentiment analysis techniques are applied to measure to what extent the users match with the brands, based on the texts associated with their opinions. The approach has been tested on both synthetic and real datasets, and with two diferent formulations, showing promising results in all the considered experiments.</p>
      </abstract>
      <kwd-group>
        <kwd>social advertising</kwd>
        <kwd>social networks</kwd>
        <kwd>sentiment analysis</kwd>
        <kwd>user-brand associations</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        An important issue in the context of digital advertising is
how to optimize the efects of marketing communication,
trying to involve in advertising campaigns those potential
ing the distribution of advertisements to uninterested
users. Automatic systems able to suggest a set of target
users for advertising campaigns, possibly ranked
according to their potential approval rating, provide three main
benefits: (i) minimization of costs for the dissemination
of the advertising campaign through digital media, which
is often very expensive; (ii) improvement of the user
experience, since only the possibly interested customers are
contacted with advertisements which could be useful for
them; (iii) avoid the spread of useful information through
the social and other digital channels.
dation of a list of possible consumers to be suggested
as target for a specific advertisement campaign, ranked
moting the campaign. In particular, we assume that a
database containing a number of descriptions associated
with diferent brands is available, and a number of
potential customers together with their opinions on diferent
topics as well. Suitable “tag” may be generated to
summarize the brand descriptions, and the proposed approach is
based on the adoption of sentiment analysis techniques
[
        <xref ref-type="bibr" rid="ref1 ref2 ref3 ref4">1, 2, 3, 4</xref>
        ] to understand if users are compatible or not
(F. Rotolo)
CEUR
htp:/ceur-ws.org
ISN1613-073
© 2023 Copyright for this paper by its authors. Use permitted under Creative
      </p>
      <p>CEUR</p>
      <p>Workshop Proceedings (CEUR-WS.org)
with the brands, based on the texts associated to their
opinions. The proposed approach is the core of a more
general framework, implementing a big data analytics
platform.</p>
      <p>
        The presented methodology has been validated first
Characteristic (ROC) analysis [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] which has shown an
Area Under the Curve (AUC) equal to 0.975 in ideal case,
and to 0.84 when a small percentage of noise is injected
in the dataset. Then, a database of brands and possible
customer features has been built, using web pages of
some real brands and data retrieved from the Twitter
social network [
        <xref ref-type="bibr" rid="ref6 ref7">6, 7</xref>
        ]. In particular, two diferent sets of
experiments have been performed on this latter database.
First, the accuracy of the method in correctly associating
users that are “followers” of some brands with those
brands, based only on their tweets and on the brand
approach has been able to correctly associate the 93.7% of
the considered followers. Then, the ability of the method
by applying the K-Nearest Neighbors approach, which
has returned the best result of 85.71% correctly classified
users for  = 6 .
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Related</title>
    </sec>
    <sec id="sec-3">
      <title>Work</title>
      <p>
        The authors of [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] use Diferential Language Analysis
(DLA) in order to find language features across millions
of Facebook messages that distinguish demographic and
psychological attributes. They show that their approach
can yield additional insights (correlations between
personality and behavior as manifest through language) and
more information (as measured through predictive
accuracy) than traditional a priori word-category approaches.
      </p>
      <p>Here we propose a novel approach for the recommen- tags, have been evaluated, showing that the proposed
according to their “compatibility” with the brand pro- in classifying potential new customers has been tested
Let  = { 1,  2, … ,   } be a set of  users, representing
possible customers of brands in the set  = { 1,  2, … ,   }.</p>
      <p>The main aim here is to return a set  = { 1,  2, … ,   }
of  ranks, one for each brand, each containing the list
of users in  , ranked according to their potential “match”
with the corresponding brand. This allows managers
of advertising campaigns to select as targets only those
users who may be potentially the most interested ones.</p>
      <p>In order to understand to what extent each user matches
with each brand, we assume that suitable tags may be
associated with the brands. As an example, keywords may
be extracted from textual exploration of their websites;
also, such tags could be explicitly proposed by the
campaign managers, according to the specific products object
of the campaign. Let  1,  2, … ,   be the lists of tags
associated to  1, … ,   , respectively. It is worth pointing
out that diferent brands may share some common tags.</p>
      <p>Let   ∈  ( = 1, … ,  ) be a user and   ∈  ( = 1, … ,  )
be a brand. The Compatibility Index between   and   is
defined as:</p>
    </sec>
    <sec id="sec-4">
      <title>3. Proposed Approach</title>
      <p>In this section the approach proposed here is described
in detail. In particular, it can be considered as the core
of a more general platform, based on big data analytics
as illustrated in Figure 1. The module Sentiment Analysis
corresponds to the methodology described here.</p>
      <p>Two main aims can be identified: (1) given a set of
potential users, finding the best targets for an advertising
campaign, and (2) given a new user in input, returning
the best brand for which it can be a potential new
customer. Sections 3.1 and 3.2 show these two diferent
formulations of the presented approach.
where |  | is the number of tags associated to   , and
  (, ) is the match, intended as a sort of liking rate, of
the user   for the  -th tag of   ( = 1, … , |  |).</p>
      <p>
        In particular, the match   (, ) is obtained by
sentiment analysis techniques [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. The key aspect of
sentiment analysis is to analyze the body of a text for
understanding the opinion expressed inside it on some topics.
Such an opinion, positive, negative or neutral, is usually
referred to as polarity [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
      </p>
      <p>Polarity can be expressed as a numerical rating,
representing a sort of sentiment score. There are diferent
approaches to identify the sentiments expressed on topics.
The main algorithms proposed for polarity computation
can be distinguished in two main categories:
 , where each rank is a set of triplets ⟨  ,   ,   (, )⟩ , sorted
according to   (, ) . A toy example is illustrated below.</p>
      <sec id="sec-4-1">
        <title>User</title>
        <sec id="sec-4-1-1">
          <title>Karmen</title>
        </sec>
        <sec id="sec-4-1-2">
          <title>Pedro</title>
        </sec>
        <sec id="sec-4-1-3">
          <title>William</title>
        </sec>
        <sec id="sec-4-1-4">
          <title>Natan</title>
        </sec>
        <sec id="sec-4-1-5">
          <title>Andrea</title>
        </sec>
        <sec id="sec-4-1-6">
          <title>Sonia</title>
        </sec>
        <sec id="sec-4-1-7">
          <title>Janny</title>
        </sec>
        <sec id="sec-4-1-8">
          <title>Marian</title>
          <p>I
C
0.83
0.83
0.83
0.67
0.33
0.33
0.17
0.00</p>
        </sec>
      </sec>
      <sec id="sec-4-2">
        <title>User</title>
        <sec id="sec-4-2-1">
          <title>Marian</title>
        </sec>
        <sec id="sec-4-2-2">
          <title>Andrea</title>
        </sec>
        <sec id="sec-4-2-3">
          <title>Karmen</title>
        </sec>
        <sec id="sec-4-2-4">
          <title>Natan</title>
        </sec>
        <sec id="sec-4-2-5">
          <title>Sonia</title>
        </sec>
        <sec id="sec-4-2-6">
          <title>William</title>
        </sec>
        <sec id="sec-4-2-7">
          <title>Pedro</title>
        </sec>
        <sec id="sec-4-2-8">
          <title>Janny</title>
        </sec>
      </sec>
      <sec id="sec-4-3">
        <title>User</title>
        <sec id="sec-4-3-1">
          <title>Sonia</title>
        </sec>
        <sec id="sec-4-3-2">
          <title>Pedro</title>
        </sec>
        <sec id="sec-4-3-3">
          <title>Janny</title>
        </sec>
        <sec id="sec-4-3-4">
          <title>Karmen</title>
        </sec>
        <sec id="sec-4-3-5">
          <title>Natan</title>
        </sec>
        <sec id="sec-4-3-6">
          <title>William</title>
        </sec>
        <sec id="sec-4-3-7">
          <title>Andrea</title>
        </sec>
        <sec id="sec-4-3-8">
          <title>Marian</title>
          <p>I
C
1.00
0.87
0.87
0.62
0.62
0.50
0.37
0.37
Example 1
Let  = {  ,   ,  ,   ,   ,   ,
  , }
has  
has  
= {ℎ ,    ,</p>
          <p>}
= {ℎ,
 , , }
and  = {    ,  ,   }
users and brands stored in the input database,
respectively. Ferrari has tags   = { ,  ,  }
be the
, Oracle
, and Walmart
the features are the polarity values retrieved from the
texts associated to the users in the input database, for
the set of all tags associated to all brands. The K-Nearest
Neighbors (KNN) classical approach can be then applied
to predict the class label for each new potential customer,
represented analogously by a features vector, by
computing the (e.g., Euclidean) distance between this vector
and those in the classes, and choosing as class label the
most represented in the top  neighbors. The following
, respectively. example shows the KNN for the dataset in Example 1.
• Supervised machine learning algorithms, trained
sentiment, using for example support vector ma- tween each user and brand.</p>
        </sec>
      </sec>
      <sec id="sec-4-4">
        <title>Ferrari</title>
      </sec>
      <sec id="sec-4-5">
        <title>Oracle</title>
      </sec>
      <sec id="sec-4-6">
        <title>Walmart</title>
        <p>textual descriptions stored for users in the database (e.g., Example 2
Walmart, respectively.
and  
their tweets), while Table 2 shows the three ranks   ,</p>
        <p>obtained for the three brands Ferrari, Oracle and</p>
        <p>As the result of this toy example, Karmen, Pedro and
William are the best targets for Ferrari; Marian and
Andrea for Oracle; Sonia, Pedro, Janny, Karmen and Natan
for Walmart.
3.2. Prediction Based on K-Nearest</p>
        <p>Neighbors
Let  1,  2, … ,   be  classes, each associated to a brand
in  , respectively. Objects in the classes are vectors of
features, such that each vector represents an user and</p>
        <p>Let Ferrari = {Karmen, Pedro}, Oracle = {Marian, Andrea}
and Walmart = {Sonia, Janny} be three classes, such that
the users are represented by the corresponding rows in
for which the class labels are to be predicted, and  = 3 .
For Natan, the closest users are Pedro, Sonia and Janny,
therefore it will be put in the class Walmart. William has
Karmen, Pedro and Sonia as closest neighbors, therefore
its predicted class label is Ferrari.</p>
        <p>Three classes have been built by choosing three tags for
each, and by generating 15 users who like the
corresponding brand and 15 who do not like it. The users have been
obtained by generating sentences explicitly expressing
positive (negative, respectively) opinions on that brand.
Then, pairs of users and brands such that the user likes
the brand, have been used as true positives, while pairs</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>4. Results</title>
      <p>4.1. Benchmarking of Sentiment Analysis</p>
      <p>
        Libraries
An important problem in the considered context is that
it is often very dificult to find “true positive” and “true Here we have referred to the unsupervised lexicon-based
negative” samples in order to validate the proposed ap- approaches for the polarity computation, due to the fact
proaches. In particular, we have referred to the Twitter that they best fit with the considered context. In
parsocial network for the real data considered here, as ex- ticular, Figure 2 shows the number of negative, neutral
plained in detail in Section 4.3. If one wants to validate and positive polarity values returned by the considered
the proposed approach, that is based on textual infor- algorithms (implemented by the corresponding Python
mation stored in a database, by using independent in- libraries). It is evident that the results may be also very
formation not related to the available textual one used diferent for the evaluated sentiment analysis techniques,
for the classification, a simple method is to search for that use diferent dictionaries and approaches to calculate
those users who are also “followers” of the considered the polarity . For instance, the ’Flair’ library tends to rate
brands. Indeed, to follow a brand, the user has made an the tweets only positively or negatively, diferently than
action, and this can be considered as an explicit declara- the other ones.
tion of compatibility with the brand, that is not related In order to provide a comparison of the considered
to the user’s textual information. Therefore, followers algorithms, a small set of 20 tweets for each tag has been
can be used as true positives. However, the same cannot manually verified to understand which approach has
rebe done for the true negative samples, due to the fact turned the right polarity value. Cumulatively, the best
that negative relationships independent from the texts performing algorithm is Vader [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], which is also in
accor(tweets, in the considered case) are not available. For this dance with the literature, since it uses a lexicon approach
reason, we have built a synthetic dataset, as explained in which has shown to be successful in the social media
Section 4.2. context.
      </p>
      <p>Before going through the description of results, we
present in Section 4.1 a benchmarking analysis we have 4.2. Synthetic Dataset
carried out, in order to decide the specific sentiment
analysis algorithm to adopt for our experiments.
of the considered brands. Most tweets contain text and
embed URLs, pictures, usernames, and emoticons.
Therefore a pre-processing of tweets has been performed, such
that tweets are filtered, and incomplete/inconsistent data
eliminated. In more detail, each tweet has been suitably
cleaned, by removing:
• URLs;
• tagged users names;
• special characters.</p>
      <p>Moreover, all emoji symbols have been translated into
text, and language contractions into their extended forms.</p>
      <p>Figure 3: ROC curve for the synthetic dataset. Hashtags have been extended in sentences. In order
to optimize the sentiment analysis process, the word
lemmatization with part of speech has been applied for
all extracted tweets. It is worth to point out that, in
the pre-processing described above, stop words have not
been removed, due to the fact that they can be important
to establish the word polarity. Indeed, the sentiment
analysis software libraries usually take them into account
for this reason.</p>
      <p>
        Tweets have been preprocessed also to eliminate
duplicate tweets and retweets from the data, which led to a
ifnal sample of 12, 585 tweets and 498 users that can be
considered significative (i.e., they have a suficient
number of tweets associated to perform the analysis).
SenFigure 4: ROC curve for the synthetic dataset with noise. timent analysis has been carried out using the Python
VADER library. The VADER Sentiment Analyzer [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]
combines qualitative and quantitative methods to
produce a gold-standard sentiment lexicon and uses it to
with the user who does not like the brand as true neg- determine the polarity of tweets and to classify them
atives, respectively. Receiver Operating Characteristic according to multiclass sentiment analysis. This library
(ROC) analysis has been used to verify the efectiveness uses the classification of the preprocessed tweets such
of the approach, and Figure 3 shows the ROC curve ob- that the polarity is considered:
tained for this “perfect” dataset, for which the Area Under
Curve (AUC) has resulted to be equal to 0.975. • positive, with score’s value 1;
      </p>
      <p>Then, in order to verify also the robustness of the • negative, with score’s value 0;
method, a small percentage of noise has been introduced • neutral, with score’s value −1.
by exchanging the 20% of sentences between the two
types of users in the dataset. Figure 4 shows the ROC
curve obtained in this case, with an AUC equal to 0.84.
4.3. Real Data Validation
The dataset coming from real data associated with input
brands, and related targets, used for the validation is
shown in Table 3. In particular, tags have been obtained
from the keywords of the brands web pages.</p>
      <p>
        As for the users’ opinions, they have been generated as
follows. The data of tweets for each user have been
downloaded from Twitter by the Twitter APIs [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. The Twitter
APIs extract the data from Twitter accounts (in a certain
date, time, number of followers or following, etc.). For
the experiments described here, 129,274 tweets have been
extracted from 1,046 users, chosen among the “followers”
      </p>
      <p>
        For the purposes of our research, the values returned by
VADER have been normalized in the range [
        <xref ref-type="bibr" rid="ref1">0, 1</xref>
        ].
      </p>
      <p>Results obtained on real data shows the ability of the
proposed approach in finding the best targets for
advertising campaigns. Indeed, the 93.7% of efective followers
have been correctly put in the first positions of the ranks,
cumulatively. This shows that users who have provided
preferences for some brands, as testified by the fact that
they follow them, have scored high values of the
Compatibility Index, with references to such brands, as expected.</p>
      <p>The K-NN analysis has been performed by choosing
three brands for which the tags are diferent and the
resulting classes are thus well separated. The chosen
brands are Adidas, Barilla and Microsoft. Only the
followers of such brands have been put into the three classes.</p>
      <p>The user-vectors have been built from the polarities
obtained, for each follower in the class, for the tags of the
corresponding brand. Then, the K-NN has been applied
by leaving out the 20% of users for each class (test set)
and keeping inside only the 80% of them (training set).</p>
      <p>The best resulting accuracy performed by the proposed
approach is of 85.71% test users correctly classified,
obtained for  = 6 .</p>
    </sec>
    <sec id="sec-6">
      <title>5. Conclusion</title>
      <p>We have presented a novel approach for the
recommendation of new possible customers for existing brands. The
approach is based on the assumption that textual
information is stored in a database on both users and brands.
Sentiment analysis techniques have been adopted to
measure to what extent an user matches a brand, according
to the retrieved users’ opinions on diferent topics. Two
diferent formulations of the proposed approach have
been described, and the preliminary results obtained on
both synthetic and real datasets are promising, looking
at the high values of accuracy reached in both cases.</p>
      <p>In the future, we plan to finalize the implementation of
all modules of the general platform illustrated in Figure
1. Moreover, further work will regard the consideration
of reciproque compatibility measures, in order to predict,
for example, if users with high matches with a specific
brand may also likely have high compatibility with other
brands.</p>
    </sec>
    <sec id="sec-7">
      <title>Acknowledgments</title>
      <p>This research has been partially supported by the projects:
“AMABILE - Amarelli BIg data and bLockchain
Enterprise platform” (CUP: B76G20000880005) funded by the
Italian Ministry of Economic Development, and “Big
knowledge graphs modelling and analysis for problem
solving in the web and biological contexts” (2022, CUP:
E55F22000270001), funded by INDAM GNCS.
1–7</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>R.</given-names>
            <surname>Feldman</surname>
          </string-name>
          ,
          <article-title>Techniques and applications for sentiment analysis</article-title>
          ,
          <source>Communications of the ACM</source>
          <volume>56</volume>
          (
          <year>2013</year>
          )
          <fpage>82</fpage>
          -
          <lpage>89</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>S.</given-names>
            <surname>Bhatnagar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Choubey</surname>
          </string-name>
          ,
          <article-title>Making sense of tweets using sentiment analysis on closely related topics</article-title>
          ,
          <source>Social Network Analysis and Mining</source>
          <volume>11</volume>
          (
          <year>2021</year>
          )
          <fpage>44</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A.</given-names>
            <surname>Mee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Homapour</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Chiclana</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Engel</surname>
          </string-name>
          ,
          <article-title>Sentiment analysis using TF-IDF weighting of UK mps' tweets on brexit</article-title>
          ,
          <source>Knowledge Based Systems</source>
          <volume>228</volume>
          (
          <year>2021</year>
          )
          <fpage>107238</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>S.</given-names>
            <surname>Barreto</surname>
          </string-name>
          ,
          <string-name>
            <surname>R.</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Carvalho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Paes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Plastino</surname>
          </string-name>
          ,
          <article-title>Sentiment analysis in tweets: an assessment study from classical to modern word representation models</article-title>
          ,
          <source>Data Min. and Knowl. Disc</source>
          .
          <volume>37</volume>
          (
          <year>2023</year>
          )
          <fpage>318</fpage>
          -
          <lpage>380</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Hanley</surname>
          </string-name>
          , et al.,
          <article-title>Receiver operating characteristic (roc) methodology: the state of the art</article-title>
          ,
          <source>Crit Rev Diagn Imaging</source>
          <volume>29</volume>
          (
          <year>1989</year>
          )
          <fpage>307</fpage>
          -
          <lpage>335</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>S.</given-names>
            <surname>Elbagir</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <article-title>Analysis using natural language toolkit and vader sentiment</article-title>
          ,
          <source>in: Proceedings of the International MultiConference of Engineers and Computer Scientists</source>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>H.</given-names>
            <surname>Kwak</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Park</surname>
          </string-name>
          , S. Moon,
          <article-title>What is twitter, a social network or a news media?</article-title>
          ,
          <source>in: Proceedings of the 19th international conference on World wide web</source>
          ,
          <year>2010</year>
          , pp.
          <fpage>591</fpage>
          -
          <lpage>600</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>H.</given-names>
            <surname>Schwartz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Eichstaedt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Kern</surname>
          </string-name>
          , et al.,
          <article-title>Personality, gender, and age in the language of social media: The open-vocabulary approach</article-title>
          ,
          <source>PLoS ONE 8</source>
          (
          <year>2013</year>
          )
          <article-title>e73791</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>J.</given-names>
            <surname>Lin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Sugiyama</surname>
          </string-name>
          , M.-
          <string-name>
            <given-names>Y.</given-names>
            <surname>Kan</surname>
          </string-name>
          , T.-S. Chua,
          <article-title>New and improved: Modeling versions to improve app recommendation</article-title>
          ,
          <source>in: Proceedings of the 37th International ACM SIGIR Conference on Research &amp;#38; Development in Information Retrieval, SIGIR '14</source>
          ,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          ,
          <year>2014</year>
          , pp.
          <fpage>647</fpage>
          -
          <lpage>656</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>S.</given-names>
            <surname>Liang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Zhang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Ren</surname>
          </string-name>
          , E. Kanoulas,
          <article-title>Dynamic embeddings for user profiling in twitter</article-title>
          ,
          <source>in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &amp; Data Mining, KDD</source>
          <year>2018</year>
          , London, UK,
          <year>August</year>
          19-
          <issue>23</issue>
          ,
          <year>2018</year>
          ,
          <year>2018</year>
          , pp.
          <fpage>1764</fpage>
          -
          <lpage>1773</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>M.</given-names>
            <surname>Bonomo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Ciaccio</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. De Salve</surname>
            ,
            <given-names>S. E.</given-names>
          </string-name>
          <string-name>
            <surname>Rombo</surname>
          </string-name>
          ,
          <article-title>Customer recommendation based on profile matching and customized campaigns in on-line social networks</article-title>
          ,
          <source>in: ASONAM'19: International Conference on Advances in Social Networks Analysis and Mining</source>
          , Vancouver, British Columbia, Canada,
          <fpage>27</fpage>
          -
          <issue>30</issue>
          <year>August</year>
          ,
          <year>2019</year>
          ,
          <year>2019</year>
          , pp.
          <fpage>1155</fpage>
          -
          <lpage>1159</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>M.</given-names>
            <surname>Bonomo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. La</given-names>
            <surname>Placa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. E.</given-names>
            <surname>Rombo</surname>
          </string-name>
          ,
          <article-title>Identifying the k best targets for an advertisement campaign via online social networks</article-title>
          ,
          <source>in: Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management</source>
          ,Volume
          <volume>1</volume>
          : KDIR, Budapest, Hungary,
          <source>Nov. 2-4</source>
          ,
          <year>2020</year>
          ,
          <year>2020</year>
          , pp.
          <fpage>193</fpage>
          -
          <lpage>201</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>R.</given-names>
            <surname>Wagh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Punde</surname>
          </string-name>
          ,
          <article-title>Survey on sentiment analysis using twitter dataset</article-title>
          , in: Second International Conference on Electronics,
          <source>Communication and Aerospace Technology (ICECA)</source>
          ,
          <year>2018</year>
          , pp.
          <fpage>208</fpage>
          -
          <lpage>211</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Nanli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Ping</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Weiguo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Meng</surname>
          </string-name>
          ,
          <article-title>Sentiment analysis: A literature review</article-title>
          , in: 2012
          <source>International Symposium on Management of Technology (ISMOT)</source>
          , IEEE,
          <year>2012</year>
          , pp.
          <fpage>572</fpage>
          -
          <lpage>576</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>T.</given-names>
            <surname>Joachims</surname>
          </string-name>
          ,
          <article-title>Making large-scale svm learning</article-title>
          ,
          <source>Practical Advances in Kernel Methods-Support Vector Learning</source>
          (
          <year>1999</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>S.</given-names>
            <surname>Baccianella</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Esuli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Sebastiani</surname>
          </string-name>
          ,
          <article-title>Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining</article-title>
          ,
          <source>in: Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>A.</given-names>
            <surname>Akbik</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Bergmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Blythe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Rasul</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Schweter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Vollgraf</surname>
          </string-name>
          ,
          <string-name>
            <surname>Flair:</surname>
          </string-name>
          <article-title>An easy-to-use framework for state-of-the-art nlp, in: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics (demonstrations</article-title>
          ),
          <year>2019</year>
          , pp.
          <fpage>54</fpage>
          -
          <lpage>59</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>J. P.</given-names>
            <surname>Gujjar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. P.</given-names>
            <surname>Kumar</surname>
          </string-name>
          ,
          <article-title>Sentiment analysis: Textblob for decision making</article-title>
          ,
          <source>Int. J. Sci. Res. Eng. Trends</source>
          <volume>7</volume>
          (
          <year>2021</year>
          )
          <fpage>1097</fpage>
          -
          <lpage>1099</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>C.</given-names>
            <surname>Fellbaum</surname>
          </string-name>
          , Wordnet, in: Theory and applications of ontology: computer applications, Springer,
          <year>2010</year>
          , pp.
          <fpage>231</fpage>
          -
          <lpage>243</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>A. K.</given-names>
            <surname>Singh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Verma</surname>
          </string-name>
          ,
          <article-title>An eficient method for aspect based sentiment analysis using spacy and vader</article-title>
          ,
          <source>in: 2021 10th IEEE International Conference on Communication Systems and Network Technologies (CSNT)</source>
          , IEEE,
          <year>2021</year>
          , pp.
          <fpage>130</fpage>
          -
          <lpage>135</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>F.</given-names>
            <surname>Å. Nielsen</surname>
          </string-name>
          ,
          <article-title>A new anew: Evaluation of a word list for sentiment analysis in microblogs</article-title>
          ,
          <source>arXiv preprint arXiv:1103.2903</source>
          (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>M. R.</given-names>
            <surname>Islam</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. F.</given-names>
            <surname>Zibran</surname>
          </string-name>
          ,
          <article-title>Sentistrength-se: Exploiting domain specificity for improved sentiment analysis in software engineering text</article-title>
          ,
          <source>Journal of Systems and Software</source>
          <volume>145</volume>
          (
          <year>2018</year>
          )
          <fpage>125</fpage>
          -
          <lpage>146</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>A.</given-names>
            <surname>Amin</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Hossain</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Akther</surname>
          </string-name>
          ,
          <string-name>
            <surname>K. M. Alam</surname>
          </string-name>
          ,
          <article-title>Bengali vader: A sentiment analysis approach using modified vader</article-title>
          , in: 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE),
          <year>2019</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>