In the last decade, new ways of shopping online have increased the possibility of buying products and services more easily and faster than ever. In this new context, personality is a key determinant in the decision making of the consumer when shopping. A person's buying choices are influenced by psychological factors like impulsiveness; indeed some consumers may be more susceptible to making impulse purchases than others. Since affective metadata are more closely related to the user's experience than generic parameters, accurate predictions reveal important aspects of user's attitudes, social life, including attitude of others and social identity. This work proposes a highly innovative research that uses a personality perspective to determine the unique associations among the consumer's buying tendency and advert recommendations. In fact, the lack of a publicly available benchmark for computational advertising do not allow both the exploration of this intriguing research direction and the evaluation of recent algorithms. We present the ADS Dataset, a publicly available benchmark consisting of 300 real advertisements (i.e., Rich Media Ads, Image Ads, Text Ads) rated by 120 unacquainted individuals, enriched with Big-Five users' personality factors and 1,200 personal users' pictures.
Nowadays, online shopping plays an increasingly significant role
in our daily lives [
Past studies on recommender systems take into account
information like user preferences (e.g., user’s past behavior, ratings, etc.),
or demographic information (e.g., gender, age, etc.), or item
characteristics (e.g., price, category, etc.). For example, collaborative
filtering approaches first build a model from a user’s past
behavior (e.g., items previously purchased and/or ratings given to those
items), then use that model to predict items (or ratings for items)
that the user may have an interest in by considering the opinions
of other like-minded users. Other information (e.g., contexts, tags
and social information) have also taken into account in the design
of recommender systems [
The impact of personality factors on advertisements has been
studied at the level of social sciences and microeconomics [
To the best of our knowledge, the impact of personality factors
on advertisements has been largely neglected at the level of
advert recommendation. There is a high potential that
incorporating users’ characteristics into recommender systems could enhance
recommendation quality and user experience. For example, given a
user’s preference for some items, it is possible to compute the
probability that they are of the same personality type as other users, and,
in turn, the probability that they will like new items [
Moreover, personality has shown to play an important role also
in other aspects of recommender systems, such as implicit
feedback, contextual information [
The ADS Dataset is a highly innovative collection of 300 real
advertisements rated by 120 participants and enriched with the users’
five broad personality dimensions, which have been shown to
capture most individual differences [
In this work we carry out prediction experiments performing two
different tasks: ad rating prediction or ad click prediction, with
the goal in mind to analyze the effect of using personality data
for recommending ads. Therefore, we propose Logistic
Regression (LR) [
Summarizing, the contribution of this work is two-fold: Dataset: we collect and introduce a representative benchmark for computational advertising enriched with affective-like metadata such as personality factors. The benchmark allows to (i) explore the relationship between consumer characteristics, attitude toward online shopping and advert recommendation, (ii) identify the underlying dimensions of consumer shopping motivations and attitudes toward online in-store conversions, and (iii) have a reference benchmark for comparison of state-of-the-art advertisement recommender systems (ARSs). To the best of our knowledge, the ADS dataset is the first attempt at providing a set of advertisements scored by the users according to their interest into the content.
Code library: we present two broad classes of prediction accuracy measures, depending on the task the recommender system is performing: “ad rating prediction” or “ad click prediction”, and provide a code library, integrating the evaluation metrics with uniform input and output formats to facilitate large scale performance evaluation. The code library and the annotated dataset are available on the project page1.
The rest of the paper is organized as follows: in Section 2 we present and describe the ADS Dataset. We perform a corpus analysis investigating on the linkages between buying habits, recommendations, and personality. In Section 3, we survey a large set of evaluation metrics in the context of the property that ARSs evaluate. In Section 4 we conduct experiments for each scenario taken into account in this work, investigating on the strengths and weakness of using personality data as features for recommendation. Finally, in Section 5 conclusions are given, and future perspectives are envisaged.
The corpus includes 300 advertisements voted by unacquainted individuals (120 subjects in total. Note, the data collection process is still running). Adverts equally cover three display formats: Rich Media Ads, Image Ads, Text Ads (i.e., 100 ads for each for1http://giorgioroffo.it/?ADSdataset mat). Participants rated (from 1-star to 5-stars) each recommended advertisement according to if they would or would not click on it (some examples are shown in the Fig.1). We labeled adverts as “clicked” (rating greater or equal to four), otherwise “not clicked” (rating less than four). The distribution of the ratings across the adverts that were scored by the users turns out to be unbalanced (4,841 clicked vs 31,159 unclicked).
Advert content is categorized in terms of 20 main product/service categories. For each one of the categories 15 real adverts are provided. Table 1 reports the full list of the categories used with the associated class annotations and the percentage of clicks received. At the category level, the distribution of the ratings results to be balanced (1,229 clicked vs 1,171 unclicked), where a category is considered to be clicked whenever it contains at least one clicked advert.
Inspired from recent findings which investigate the effects of
personality traits on online impulse buying [
Recent soft-biometric approaches have shown the ability to
unobtrusive acquire these traits from social media [
As a result, the proposed benchmark includes 1,200 spontaneously uploaded images that hold a lot of meaning for the participants and their related annotations: positive/negative (see Table 2 for further details). The images are personal (i.e., family, friends etc.) or just images participants really like/dislike. The motivations for labeling a picture as favorite are multiple and include social and affective aspects like, e.g., positive memories related to the content and bonds with the people that have posted the picture. Moreover, they are provided with a set of TAGS describing the content of each of them.
Finally, many other users’ preference information are provided. Table 2 lists the raw data provided with the dataset, such as users’ past behavior selected from a pre-defined list (e.g., watches movies, listen songs, read books, travel destinations, etc.), demographic information (like age, nationality, gender, etc.). Note, all data is anonymized (i.e., name, surname, private email, etc.), ensuring the privacy of all participants.
For further analyses related to the adverts’ quality, this benchmark also provides the entire set of 300 rated advertisements (500 x 500 pixels) in PNG format. 2.1
The subjects involved in the data collection, performed all the steps of the following protocol:
- Step 1: All participants have filled in a form providing, anonymously, several information about their preferences (e.g., demographic information, personal preferences).
- Step 2: All participants have filled the Big Five Inventory-10 to
measure personality traits [
- Step 3: The participants voted each advert according with if they would or not click on the recommended ad. Ads have been displayed in the same order to all the participants.
- Step 4: The participants submitted some images that they like (Positives) and some others that disgust or repulse them (Negatives). Once they have uploaded their images, they also added some TAGS that describe the content of each image. 2.2
This corpus involves 120 English native speakers between 18 and 68. The median of the participants age is 28 ( =31.7, =12.1). Most of the participants have a university education. In terms of gender, 77 are females and 43 males. The percentage distribution of household income within the sample is: 23% less or equal to 11K USD per year, 48% from 11K to 50K USD, 21% from 50K to 85K USD, and 8% more than 85K USD. The median income is between 11K and 50K USD.
In analyzing this complex data, one can observe that users’ preferences are not independent of each other, they are likely to be co-expressed. Hence, it is of great significance to study groups of preferences rather than to perform a single analysis. This fact is A A N 0 E 0 E O
C C A A 5 3.8 2.5 1.3 5 3.8 2.5 1.3 O
C
C
5
3.8
2.5
1.3
5
3.8
2.5
1.3
(Cluster 1 - M: 50%, F:50%)
(Cluster 2 - M: 35.7%, F:64.3%)
(Cluster 3 - M: 100%, F:0%)
(Cluster 4 - M: 0%, F:100%)
(Cluster 5 - M: 0%, F:100%)
(Cluster 6 - M: 11.1%, F:88.9%)
(Cluster 7 - M: 75%, F:25%)
(Cluster 8 - M: 0%, F:100%)
also true for personality factors, analyzing subsets of data yields
crucial information about patterns inside the data. Thus, clustering
users’ preferences can provide insights into personality of
individuals which share the same preferences. We performed a statistical
analysis of personality and users’ preferences, linking the 5
personality factors and the most favorite users’ product categories (i.e.,
most clicked) by means of the affinity propagation (AP) clustering
algorithm [
AP is an algorithm that takes as input measures of similarity between pairs of data points and simultaneously considers all data points as potential exemplars. We calculated a similarity input matrix between each individual ui considering as feature vectors vi a binary sequence of click/no-click (i.e., vi is 1 300). AP exchanges real-valued messages between data points until a high-quality set of exemplars and corresponding clusters gradually emerges. Hence, the number of clusters is automatically detected, and when applied on ADS data, AP grouped the data into 8 different clusters.
Figure 2 illustrates 8 spider-diagrams, one for each cluster. Each diagram shows the average of the big-five factors regarding the subjects within the group (reported in figure as O-C-E-A-N).
Then, we ranked the most clicked categories according with samples within the group in order to compare these two variables by means of correlation obtaining interesting clues.
For instance, let us consider the cluster number 6 where 88.9% of the members are females, and 11.1% males and the average of the group members age is 28. The first 5 most clicked categories are Baby, followed by Consumer Electronics, Stationery & Office Supplies, Home, and Jewelery & Watches. This group is characterized by high neuroticism (see the diagram in Figure 2.(Cluster 6)), those who score high in neuroticism are often emotionally reactive and vulnerable to stress, high neuroticism causes a reactive and excitable personality, often very dynamic individuals. This group also share the highest levels of extroversion, high extraversion is often perceived as attention-seeking, and domineering.
Cluster 5 shows a subset of individuals which scores low for all the types (see the plot in Figure 2.(Cluster 5)). For instance, those with low openness seek to gain fulfillment through perseverance, and are characterized as pragmatic sometimes even perceived to be dogmatic. Some disagreement remains about how to interpret and contextualize the openness factor. The first 5 most clicked categories are Clothing & Shoes, Health & Beauty, Jewelery & Watches, Outdoor Living, and then Consumer Electronics. In this case the average of the group members age is 68, and the cluster contains 100% females.
Cluster 7 is characterized by good levels of conscientiousness that is the tendency to be organized and dependable, aim for achievement, and prefer planned rather than spontaneous behavior. This cluster scores low in agreeableness, which is related to personalities often competitive or challenging people. The openness factor (>2.5) reflects the degree of intellectual curiosity, creativity and a preference for novelty and variety a person has. Interestingly, among the most preferred categories there are Console & Video Games, Consumer Electronics, Grocery and Computer Software.
Research in the ARS field requires quality measures and evaluation metrics to know the quality of the techniques, methods, and algorithms for predictions and recommendations. In this section we review the process of evaluating an ARS on two main tasks: (i) measuring the accuracy of rating predictions, and (ii) measuring the accuracy of click predictions.
In most online advertising platforms the allocation of ads is dynamic, tailored to user interests based on their observed feedback. In this first scenario, we want to predict the feedback a user would give to an advert (e.g. 1-star through 5-stars). In such a case, we want to measure the accuracy of the system’s predicted ratings.
lar metric used in evaluating the accuracy of predicted ratings. The system generates predicted ratings r^u;a for a test set T of useradvert pairs (u,a) for which the true ratings ru;a are known. The RMSE between the predicted and actual ratings is given by:
Mean square error (MSE) is an alternative version of RMSE, the main difference between these two estimators is that RMSE penalizes more large errors, and MSE has the same units of measurement as the square of the quantity being estimated, while RMSE has the same units as the quantity being estimated. Therefore, MSE is given by
Mean Absolute Error (MAE) is a popular alternative, given by M SE = M AE = 1
X jT j (u;a)2T s 1
X jT j (u;a)2T (r^u;a
ru;a)2: jr^u;a ru;aj: As the name suggests, the MAE is an average of the absolute errors erru;a = jr^u;a ru;aj, where r^u;a is the prediction and ru;a the true value. The MAE is on same scale of data being measured. 3.2
In many applications the recommendation system tries to recommend adverts to users in which they may be interested. For example, when items are added to the queue, Amazon suggests a set of adverts on which the user would most probably click. In this case, we are not interested in whether the system properly predicts the ratings of these adverts but rather whether the system properly predicts that the user will click on them (e.g. they perform a conversion). Therefore, we then have four possible outcomes for a recommended advertisement, as shown in Table 4.
Clicked Not clicked
Recommended True-Positive (tp) False-Positive (fp)
Not recommended False-Negative (fn) True-Negative (tn)
We can count the number of examples that fall into each cell in the table and compute the following quantities: where true negative rate is also called Specificity. We can expect a trade-off between these quantities; while allowing longer recommendation lists typically improves recall, it is also likely to reduce the precision. We can compute curves comparing precision to recall, or true positive rate to false positive rate. Curves of the former type are known simply as precision-recall curves, while those of the latter type are known as a Receiver Operating Characteristic or ROC curves. A widely used measurement that summarizes the
is useful for comparing algorithms independently of application.
When evaluating precision-recall (or ROC curves) for multiple
test users, a number of strategies can be employed in aggregating
the results, depending on the application at hand. The usual
manner in which precision-recall curves are computed in the
information retrieval community [
In this section we show results obtained for the two types of scenarios introduced in Sec. 3. We conduct prediction experiments to explore the strengths and weakness of using personality traits as features for recommendation. 4.1
Since a prediction engine lies at the basis of the most
recommender systems, we selected some of the most widely used
techniques for recommendations and predictions [
Let us say X = fx1; :::; xN g is the set of observations, where the vectors xi correspond to features coming only from the group “users’ preferences” as described in Table 2 and N = 120 stands for the number of users involved in the experiment.
A feature is the user’s selection from a pre-defined list of choices, hence, for each feature vector one element is 1 and the others are 0. Then, each column vector xi is obtained by stacking the features on top of one another.
Regression is performed over the 20 product categories. The prediction problem is solved using LR, L2-SVR, and SVR-rbf, while feeding them with and without features coming from “personality traits”. All experiments were performed using a k-fold approach (k = 10). In k-fold cross-validation, X is randomly partitioned into k’s equal sized subsamples (the folds are the maintained the same for each algorithm in comparison). Of the k subsamples, a single subsample is retained as the validation data for testing the model, and the remaining k - 1 subsamples are used as training data. The cross-validation process is then repeated k times, with each of the k subsamples used only once as the validation data. The k results from the folds can then be averaged to produce a single estimation.
Our experimental protocol includes feature selection, which
represents an important pre-processing step given the sparse nature of
the input data. It allows to remove many redundant features by
reducing the dimensionality of the problem at hand. Hence, the
representation above serves as a basis for the feature ranking and
selection strategy. Ranking features allow us to detect a subset of
cues which is not redundant. Accordingly, we use the training data
obtained after the split as input of the infinite feature selection
(InfFS) [
Finally, the Inf-FS assigns a score to each feature of the initial set; where the score is related to how much the given feature is a good candidate regarding the regression task. Therefore, ranking the outcome of the Inf-FS in descendant order allows us to perform the subset feature selection throughout a model selection stage. In this way, we reduce the amount of features, by selecting 75% of the total. The selected features are: the number of favorite websites, T.V. programmes, sports, past times, the most watched movies and most visited websites, where we add the big-five personality traits. 4.3
In this section we report results for rating prediction showing
that traces of user’s personality can improve the prediction
performance of the evaluated methods significantly. Statistical evaluation
of experimental results has been considered an essential part of
val1
0.9
0.8
idation of machine learning methods. Given the user ui, labels are
assigned to each category by averaging the votes they gave to the
category items such as ui = fy1; :::; y20g; y 2 [
Figure 3 illustrates prediction results in term of RMSE, MSE and MAE plots across the categories. This first analysis shows how personality traits affect prediction performance. In order to assess if the difference in performance is statistically significant, t-tests have been used for comparing the accuracies. This statistical test is used to determine if the accuracies obtained with and without B5 are significantly different from each other (whereas both the distribution of values were normal). The test for assessing whether the data come from normal distributions with unknown, but equal, variances is the Lilliefors test.
Results show a statistical significant effect of personality traits while using L2-SVR (p-value < 0.05, Lilliefors Test H=0) and LR (p-value < 0.01, Lilliefors Test H=0).
As for the SVR-rbf, even if improvements in terms of prediction are not significant (B5 against no-B5), it is still interesting to notice the performance loss on categories 6 and 8, where errors go high significantly. In such a case, the B5 features do not seem to have much predictive power, however, they seem to play the role of a reliable stabilizer, but also that of an independent mediator and supporter of the regression process. 4.4
This section shows an offline evaluation of click prediction. Along the lines of the previous experiment, a k-fold cross-validation is used. The experiment is performed at the category level, in order to
In this paper, we conducted a within-subject user study to investigate on the relations between users’ personality related to their buying behavior and preferred item categories. A deeper analysis may involve the use of bi-clustering methods. Comparing to traditional clustering methods biclustering is not a blackbox technique. Comprehensibility is one of its main advantages, i.e. it is possible to understand why objects ended up in the same cluster.
It is worth noting that the goal of these experiments is to show
how personality traits affect the prediction. In order to improve
prediction accuracy, specific feature designing processes are needed so
as to represent personality data and to standardize their definitions
to be used as input recommender data towards to improve
recommendations. In our experiments, we used a set of sparse binary
features converted from the original categorical features.
Moreover, many other algorithms may be used for this tasks, like the one
proposed in [
For instance, the personality diagnosis [
Most of these recommender systems use to split each test user profile into sets of observed items and hidden items. The former is used as input for each recommender, the latter for performance evaluation. In our experiments, we did not use any information about the previous users’ clicks, which turns out to be a more difficult task. We decided on this solution to move the focus of attention on personality data and not on other features like previous clicked ads. 5.
In this paper, we presented the ADS Dataset, a collection of 300 real advertisements rated by 120 unacquainted participants. We conducted a within-subject user study to investigate potential user issues of the personality on their buying behavior and preferred item categories.
The corpus has been collected with the main goal of studying the possible achievable benefits of employing personality traits in modern recommender systems. To obtain stronger and more relevant results for this community, appropriate and high-level features needed to be designed that carry important information for inference. In this paper, we only use raw data as sparse binary features converted from the original categorical features. We used standard techniques for recommending ads in order to show how personality traits affect the prediction, and, at the same time, set a baseline for future work.
We then reviewed a large set of properties, and explain how to evaluate systems given relevant properties. We discuss how to compare ARS based on a set of properties that are relevant for the application. Therefore, we review two main types of experiments in an offline setting, where recommendation approaches are compared with different selections of features (i.e., with and without personality traits) accordingly with our goal. We also discuss how to draw trustworthy conclusions from the conducted experiments.
Future work includes, but is not necessarily limited to, (1) feature engineering and designing for ARSs, represent personality data and standardize their definitions to be used as input recommender data towards to improve recommendations; (2) inference of personality traits and novel approaches for mapping pictures tagged as favorite into personality traits; and (3) identification of the underlying dimensions of consumer shopping motivations and personality factors.
We hope that this work motivates researchers to take into account the use of personality factors as an integral part of their future work, since there is a high potential that incorporating these kind of users’ characteristics into ARS could enhance recommendation quality and user experience. 6.