In this paper, we describe the participation of the Language Technologies Lab of INAOE at PAN 2015. According to the Author Profiling (AP) literature. In this paper we take such discriminative and descriptive information into a new higher level exploiting a combination of discriminative and descriptive representations. For this we use dimensionality reduction techniques on the top of typical discriminative and descriptive textual features for AP task. The main idea is that each representation, using the full feature space, automatically highlights the different stylistic and thematic properties in the documents. Specifically, we propose the joint use of Second Order Attributes (SOA) and Latent Semantic Analysis (LSA) techniques to highlight discriminative and descriptive properties respectively. In order to evaluate our approach, we compare our proposal against a standard Bag-of-Words (BOW), SOA and LSA representations using the PAN 2015 corpus for AP. Experimental results in AP show that the combination of SOA and LSA outperforms the BOW and each individual representation, which gives evidence of its usefulness to predict gender, age and personality profiles. More importantly, according to the PAN 2015 evaluation, the proposed approach are in the top 3 positions in every dataset.
The Author Profiling (AP) task consists in knowing as much as possible about an
unknown author, just by analysing a given text [
Broadly speaking, in text classification tasks there are three general-key procedures;
i) the extraction of textual features, ii) the representation of the documents, and iii) the
application of a learning learning algorithm. In the context of the AP tasks, for the first
step, specific lexical (e.g., simple words, function words) [
According to the AP task literature, most of the work has been devoted to the first step: to identify the most useful-interesting textual features for the target profiles. In spite of the usefulness of previous interesting textual features and the good results achieved by the configuration BoW-SVM, the research community has put little effort to deepen in the second and third steps: alternative representations and learning algorithms for the AP task. The main shortcomings of the BoW-SVM approach are well known from other text mining task.
To overcome the latter shortcomings, in this paper we focus in the second step, i.e.
the representation of the documents, in order to improve the representation of tweets.
The main goal of our approach is to compute high quality discriminative and
descriptive features built on the top of the state-of-the-art typical textual features (e.g.,
content words, function words, punctuation marks, etc.). For this, we propose to combine
two state-of-the-art dimensionality reduction techniques that best contribute to
automatically stress the contribution of the discriminative and descriptive textual features.
According to the literature the most frequent textual features (e.g., function words,
stopwords, punctuation marks) provide important clues about the discrimination of the
authors. For this we need a representation highly based in term frequencies, that stresses
the contribution of such discriminative attributes and produces highly discriminative
document representations. To capture this information contained among textual
features we use Second Order Attributes (SOA) computed as in [
The rest of this paper is organized as follows: Section 2 introduces the proposed representation, in Section 3 some characteristics of the corpus PAN15 are explained briefly, Section 4 explains how we performed the experiments and the results we obtained, finally Section 5 shows our conclusions. 2
Along this section we briefly describe each representation and the proposed strategy to compute the final representation of documents. In Section 2.1 we explain the SOA representation to get the discriminative features. In Section 2.2 we explain the LSA algorithm with which we intend to get descriptive features. Finally, in Section 2.3 we explain how we join these representations for the AP task. 2.1
The stylistic textual features have proven to be useful for AP task [
The representation as described in [
dk =
X tfkj t
tj∈Dk lenght(dk) j
where Dk is the set of terms that belongs to document dk. For more details please
refer to [
Besides the usefulness of stylistic features, thematic information has proven to be an
important aspect for the AP Task [
LSA is a method to extract and represent the meaning of the words and documents.
LSA is built from a matrix M where mij is typically represented by the TFIDF [
M = UΣVT
Where The Σ values are called the singular values and U and V are the left and right
singular vectors respectively. U and V contains a reduced dimensional representation
of words and documents respectively. U and V emphasizes the strongest relationships
and throws away the noise [
The idea is to use the representations built under the whole feature space to automatically highlight the discriminative and descriptive properties in documents. The intuitive idea is to take advantage of both approaches in a representation using early fusion. Let xj be the j − th training instance-profile under LSA representation with k dimensions and yj be the same instance-profile under the SOA representation with m dimensions, the final representation is show in Expression 3.
zj = hxj1, . . . , xjk, yj1, . . . , yjmi
Z = [ hzj, cji dj∈D (3) (4)
Where cj is the class of the j − th training instance-profile. 3
We have approached the PAN 2015 AP task as a classification problem. PAN 2015 corpora is composed by 4 datasets in different languages (Spanish, English, Italian and Dutch). Each dataset has labels of gender (male, female), age 1 (18-24, 25-34, 35-49, 50-xx) and five personality traits values (extroverted, stable, agreeable, conscientious, open) between -0.5 and 0.5. In Table 1 we show the number of Author-Profiles per language.
For personality identification Table 2 shows the relevant information (in terms of classes). For each language it shows the range and the number of the classes for each trait2. For personality we consider each trait value in the train corpus as a class. For example, if only two values (e.g., 0.2 and 0.3) are observed in the training corpus, then we built a two class classifier (e.g., 0.2 and 0.3) 3. 4 4.1
We use for each experiment the following configuration: i) for terms we use words, contractions, words with hyphens, punctuation marks and a set of common emoticons, 1 Age data for Italian and Dutch languages are not available. 2 The ranges with asterisk indicate that a value between the range is missing. For example, in
Spanish (extroverted and conscientious) the -0.1 is missing.
3 For each personality trait in each language the number of the classes are variables between
them, see Table 2
ii) we consider the terms with at least 5 occurrences in the corpus, iii) the number of
concepts for LSA is set to k = 100. We perform an stratified 10 cross fold validation
(CFV) using the training PAN15 corpus and a LibLINEAR classifier [
The aim of this first experiment is to analyse the performance of LSA, SOA and the BOW approach in the AP tasks. We experiment with LSA and SOA separately and finally with the two approaches together. We are interested in observing the contribution of discriminative-stylistic (captured by SOA) and descriptive-thematic (captured by LSA) information in the AP task. For gender prediction, in Table 3 we can see that considering the individual representations, LSA obtains the best results, which outperforms the BOW approach in every language. When LSA and SOA are together the result only improves in English, which is an important remark since the English language is the bigger-robust collection (see Table 1). The following conclusions can be outlined from Table 3: – The descriptive information captured by LSA is the most relevant information for gender prediction in PAN 2015 AP dataset. This is because LSA obtained the best average individual performance. – The pure discriminative information captured by SOA only outperforms BOW in Dutch documents. But the combination of LSA and SOA obtained an improvement of around 4% in accuracy for English gender detection. We think, SOA could improve the results if more documents are available 4.
For age prediction, Table 4 shows the experimental results. Recall that the age data is available only for English and Spanish languages. As in the last experiment LSA obtains the best individual performance, but in this experiment the combination of LSA and SOA obtains an improvement in both collections. It is worth noting that despite of the small datasets, for age prediction SOA could contribute to improve the classification5.
Finally for personality prediction Table 5 shows the performance of BOW and LSA
plus SOA performance by language in the personality detection task. For this
experiment, although the results seems promising they should be taken with caution. This is
due to the lack of data and the number of classes that we consider (one class for each
observed value) one correct/wrong predicted instance is enough to change the results
considerably. For this specific experiment in personality, we built a representation on
the entire dataset, then we evaluate using a 10CFV. In general, the results suggest that
the combination of LSA plus SOA gets similar or better results than the typical BOW
approach. Given evidence of the usefulness of the discriminative features and the
descriptive features.
4 SOA has proven outstanding results in recent years in the PAN AP tracks [
Acknowledgment This work was partially funded by the program SEP-CONACYT-ANUIES-ECOS Nord under the project M11-H04. Álvarez-Carmona thanks for doctoral scholarship CONACyT-Mexico 401887. Also, López-Monroy thanks for doctoral scholarship CONACyT-Mexico 243957.