This paper describes the methods we have employed to solve the author profiling task at PAN-2015. The proposed system is based on simple content based features to identify an author's age, gender and other personality traits. The problem of author profiling was treated as a supervised machine learning task. First content based features were extracted from the text and then different machine learning algorithms were applied to train the models. Results showed that content based features approach can be very useful in predicting the author's traits from his/her text.
Authorship attribution concerns with the classification of documents into the classes to be predicted based on the writing style of their authors. In the case of author verification and author identification tasks, the style of individual authors is examined. Whereas author profiling mean to distinguish between classes of authors studying their sociolect aspect, that is, how language is shared between people. This helps in predicting profiling aspects such as age, gender or personality type. Author profiling is a problem of increasing importance in several applications like forensics, security and marketing. E.g., from a forensic linguistics prospect, the linguistic profile of the sender of a harassing SMS message can be identified. Similarly, from a marketing perceptive, companies would like to know the demographics of the people that like or dislike their products on the basis of the text analysis of online product reviews and blogs.
In recent years, automatic detection of an author’s profile from his/her text has
become an emerging and popular research area
A significant amount of research in automatic classification of texts into the classes to be
predicted, has already been done by different researchers and linguists using several
different machine learning techniques
Several approaches have been implemented and experiments conducted for
selecting the best possible features set for the most accurate classification. Houvardas and
Stamatatos (2006) showed the usefulness of n-gram, whereas Koppel et al. (2009) shown
the effect of gender and age in blogging sites by considering different word classes and
showing the relation of the word classes with the author’s age and gender. Koppel et al.
(2009); Estival et al. (2007) have identified that the Part-of-speech is also an
commendable linguistic feature and
The data used in our experiments is the training dataset of PAN-2015 1. The corpus consists tweets on different topics, grouped by author and labeled with his/her language, gender, age group and 5 personality traits ( extroverted (Ex), stable (St), agreeable (Ag), conscientious (Co) and open (Op)). The documents are categorized as in languages (English, Dutch, Italian and Spanish), two genders (male and female), and four groups ( 18-24, 25-34, 35-49 and 50-XX). With regard to personality traits, for each trait the scores lies between -0.5 and 0.5. Documents in the corpus consist of a collection of posts made by a single user.
The corpus was balanced gender wise within each age group but imbalanced in terms of age representation and the five personality traits scores distribution (-0.5 to 0.5). The proportion of languages, gender and each age group in the corpus within the training dataset is presented in Table 1, whereas the personality traits distribution in table 2.
Prior to any model training or testing, we apply some pre processing steps to all documents. We eliminated all the data contents that were not determined to be the text written from the user like XML tags, as our primary source of features is the text written
Class
Agreeable Conscientious Extroverted
Open Stable by an author. Now as all the user posts lie within the unparsed data tags of the source’s xml file, we disregard any text not within these tags and HTML tags also. 4
As male and females like to write about different topics, they use different words
accordingly. This leads to the fact that content based features can be an important tool
to distinguish between texts of males and females
We calculated the frequencies of different unigrams in the texts written by a particular profile. Then, for every unigram, we calculated the ratio of its frequencies in the tweets of different classes like male and female, different age groups and different personality traits. Finally we selected the features on the basis of two combined factors. First the unigrams with the highest frequencies in the corpus, and second the difference of frequencies in different classes which are to be classify from one another. The frequencies of some of the most frequently used unigrams for English language in the corpus and their frequency comparison along gender and age groups is given in table 3. Similarly this routine has carried out for all the four languages individually and four sets of content based features selected for each language each. Then for each language, two different sets of features has been used, one for gender and age group prediction and one for the personality traits classification. 5
For each of the four languages, we trained different models. Then for each language, we trained two models, one for age and gender and one for the personality traits. So there were total training models build all on the content based features. We ran the experiments on four machine learning classifiers: J48, Random Forest, Support Vector Machines (SMO), and Naive Bayes. The evaluation measures used as instructed by PAN 2015 2, accuracy for age and gender and root mean squared error for the five personality traits. 6
Language Gender Age
English 0.914 0.967 Italian 1.000 NA Spanish 0.940 0.990 Dutch 1 NA
Both Ex St Ag 0.894 0.076 0.093 0.088 NA 0.028 0.051 0.048 0.930 0.085 0.110 0.084 NA 0 0 0 Table 4. Results on training data.
Based on the performance of the four classifiers (see section 5) on the training data, we choose only one single classifier (SVM) for all the classes of age, gender and personality in our final software that we have submitted in the competition. We are repoting the results that we have achieved on the training data in table 4 and finally on the testing data in table 5. Here the combined accuracy for age and gender (open), the combined root mean squared error for all five personality traits (RMSE) have been reported as well. Again here the age group results for Italian and Dutch are missing in both training and testing data suggesting there is only 1 class.
In comparison of performance in different languages, our system performed better for English and Dutch as compared to Spanish and Italian. Traits wise the performance
is reasonably good for Spanish gender, English age and Dutch personality. Overall, our system couldn’t perform as well on the testing data as it performed on training data. The possible reasons for this are the variation of language in training and testing data and the nature of the content based approach. 7
In this paper, we have presented a content based technique for the automatic classification of the author’s gender, age and personality from their writing. This work has a number of potential applications like marketing, forensics, and security. We have performed our experiments on the training data provided by the PAN-2015 organizers. We applied some simple content based techniques and the results we achieved are highly motivational showing the usefulness of content based features in predicting the author’s profile from text. In future work, the results can be further improved by incorporating and finding more suitable features.