Author profiling is the task to infer some information about an author by analyzing her/his writing style. It's application in forensics, business intelligence and psychology makes this topic interesting for researching. In this notebook, we present our baseline approach using SVM and Linear Discriminant Analysis (LDA) classifiers. We analyze features obtained from LIWC dictionaries, these are frequencies of use words by categories, which gives a general view about how the author writes and what he/she is talking about. According the experimental results, those are significant features to differentiate gender, age-group and personality. Although they are relatively few (not more than 100), they allow to discriminate with an acceptable accuracy.
Studies have demonstrated evidence of differences in the writing style according the
gender and age of the authors. These differences are detected with the use of
functionwords and content-words. The function-words define how the person use the grammar
and build sentences. On the other hand, content-words indicate what the person is
talking about. For example, Pennebaker [
In the case of age, Pennebaker [
More recently studies expanded this analysis to determinate weather the personality
influence the writing style too. For example Yarkoni [
Based on this evidence, we believe that the word categories are important features to determine the profile of an author of a text, and we will try to measure how much they can tell us about the authors of tweets. Therefore, in this notebook, we present a baseline author profile classifier based on statistical learning over word category features. First, we present the feature pre-processing, extraction and selection. Then, the classification models to identify gender, age group, and personality. And finally, we present the description of the experiments, the results and the conclusions. 2 2.1
The studies mentioned previously used the Linguistic Inquired and Word Count (LIWC)
[
The LIWC categories are grouped in linguistic dimensions (e.g., part-of-speech); content dimensions(e.g., emotions and activities); or spoken dimensions (e.g. fillers and no fluencies markers). LIWC has dictionaries in a variety of languages, for this experiment English, Italian, Spanish and Dutch dictionaries are used. Each dictionary contains the list of predefined categories and the words associated with them, for example for the category positive emotions some associated words are “fun", “nice", “succes", etc. They were created over reiterative process of human judgment and were tested in several times in different studies to assure their validity. 2.2
Additionally, we include other two groups of categories: punctuation marks and tweet. We include seven categories in punctuation marks: question mark, exclamation, period, comma, colon, semi-colon and all punctuation. The last one groups any punctuation mark including the mentioned before. The tweet categories are added for the nature of the corpus data and because they are frequently employed by tweet users: emoticons, hyper-link, hashtag and references to other users. In Table 1 we can see a summary of the total categories analyzed in this experiment. The tweets were given in XML files. Each XML file was processed to extract only the text information given as scale. First, the text was tokenized using white space (including tab, change of line among others) and punctuation marks as separation characters. We consider the apostrophe as part of the word, for example “she’s" is consider a single word. This choose is related to the LIWC dictionaries that consider them in this way. Once the tokens are obtained, they were counted in the corresponding categories. As mentioned before, in the dictionary each category has a set of words, so, if the token is part of the set of word of a category then it sums one to that category. One token can appear in many categories (e.g. “I" as pronoun and function word).
The granularity of the models is set by user and not by tweet, so, there is a vector of categories per each user. Once we complete the counting per user, we divide each count by the total number of words by user in other to obtain the relative frequencies. Finally, we keep the frequencies in a matrix format, where the columns are the word categories and the rows are the users. Thus, each row represent the distribution over LIWC categories for the given user.
One additional step is performed before the feature selection, the relative word frequencies x are scaled by calculating the z score respect to each category according to the following formula: z = x : (1) where, and are the mean and standard deviation of the frequencies in each category.
The frequency of use of words in not uniform in a language. Some of them are highly used (e.g., function words) and some others have low frequency of use (e.g., topic related words), the relative frequencies are scaled because we need to compare them obtaining their use related to each particular category and not to the general use of language. 2.4
Even we have a reduce set of features, we need to ignore noisy and irrelevant features
before applying a classification scheme. Additionally, it derives in an easier linguistic
explanation about the key features to discriminate among the different classes.
Gender and Age Group Fourth feature selection methods were evaluated to
determine the more suitable for the data: Manual selection, Information Gain, Odd ratio
and Support Vector Machine Recursive Feature Elimination (SVM RFE). The manual
selection was based on [
Personality In the case of personality, the number of classes to discriminate is larger than the previous models and it is more difficult to associate specific categories to each class. Consequently, the methods mentioned before did not have significant improvement in comparison of using the full set of features. So for this case, we applied Forward-Backward Feature Selection, trying to improve the Root Mean Squared Error (RMSE). 3 3.1
These classes were defined as categorical. We have two classes for gender: “Male"
and “Female", and fourth classes for age: “18-24", “25-34", “35-49", and “50-xx". The
classification was made with -SVM [
In the case of personality, we define one model per each personality. Our first approach was to define the classes as categorical without taking into account the order of the score of the personality, so we have 11 classes from “-0.5" to “0.5" with one decimal of difference. We choose two classifiers: -SVM with set to 0.01 and with radial kernel, and Linear Discriminant Analysis (LDA). The implementation was done in R with the libraries “e1071" and “MASS". 4 4.1
The corpus to develop the models is a training set of tweets in English, Spanish, Italian
and Dutch given by PAN 2015 Author Profiling task [
In the feature selection, the experiment shows that there are some categories to discriminate between gender which are independent of language while other are different for each language, and the same patter for the other models. Tables 5, 6, and 7 contain the common categories that were found in one or more languages.
The training and testing are implemented separately. The outputs are the models
and the vectors of means and standard deviation calculated with the training data. These
vectors are used to calculate the z score of the testing data. This step corresponds to
Software 1 of TIRA [
The global ranking of our solution for English was 7th over 22, Dutch 5th over 20, Italian 9th over 19, and Spanish 8th over 21. are measured by accuracy in %,“BOTH" is the accuracy when gender and age were both well classified. The personality traits were measure with RMSE rounded to two decimal points, “RMSE" is the average of all personalities traits.
English Performance Best Our solution Mean Worse Dutch Performance Best Our solution Mean Worse Italian Performance Best Our solution Mean Worse
The present approach using LIWC Categories has demonstrated being a good solution regarding the limitation of having a few quantity of features compared with other solutions. According to the testing results, it had better performance than the average state of the art. Moreover, it is simple and efficient. But the most important point is that we can linguistically justify the classification decision because we can know which are the key features for the decision process. Deeper analysis is needed to extract the explanation of correct and incorrect assignment of classes; and to compare the differences in the results using SVM or LDA classifier. We think that it can be improved in future with a finer analysis of features and selection methods, and with a more appropriate definition of the classes and modeling for age-group and personality traits.