Author profiling task considers different profile dimensions of the author of the text. This year shared task [12] [11] is focusing on gender and language variety. Previous competitions explored properties like gender, age group [13] and personal traits [8]. This task is interesting from both industrial and scientific points of view. Applications like accurate advertising targeting, security and forensic fields make this task highly relevant for practice. Also, the task can be considered as a tool for filling missing information about a person in some political or demographic research. Research community also pays attention to the task special track of PAN [7] shared task is held since 2013. Each year contributed a new language or new profile dimension to classify. The common part of all years was gender identification. The first task was on blog data in Spanish and English [10]. Competition in 2014 concentrated on different sources like reviews, tweets etc. [9]. The task of 2015 extended by additional languages and realvalued personal traits [8]. The main characteristic of the most recent shared task was cross-genre. The target was to develop a model such that it will be robust to the domain of data [13]. Since gender identification was presented in all previous competitions, there were many tested approaches. The main features were n-grams and various text statistics [4]. Language variety task was first to appear at PAN 2017, but there were language variety detection competitions like Discriminating between similar languages and national language varieties (DSL) 2016 [1]. Winning approach of this contest used char n-grams in wide range (1-7) with a linear classifier [3]. We used this method not only for language variety task but also for gender classification. A new feature of the current shared task is language variety. Each language has several variants. For instance, we have two several Portuguese: Brazil variant and European one. The task is to distinguish one from another. Languages and their varieties can be found in Table 1. Our approach tries to automatically extract features for each of variant Portuguese, English, Spanish and Arabic without any linguistic knowledge. We use char n-grams as features and logistic regression as a classifier. Evaluation metric is accuracy for both subtasks.
This section is about our approach to current PAN Author Profiling task. First, we briefly discuss preprocessing steps. Then, we describe how we construct the feature space. Finally, we explain our choice of logistic regression as our classifier.
We did not perform any preprocessing like removing hashtags, HTML tags and urls, because we considered it as potentially informative features.
Our main assumption was to consider all short texts written by a single author as an object in machine learning task formulation. We formulated the problem as classification task with two or more classes depending on language (Table 1). If language has more than two varieties we used "one versus other" scheme. Let dataset
to be consisted of pairs "object-class", x i ∈ R n . Each object x i has one of Z class labels y i ∈ Y = {1, . . . , Z}. We have to find mapping f ∈ F : R d → Y, which minimizes empirical risk on dataset D:
where F -family of models. Feature space was constructed such that for each language corpus we performed counting of character level n-gram in some range. This counts were used as features. The number of authors and features for different tasks can be founded in Table 2. One can see that the data is quite sparse. Density distribution of non-zero n-grams for Portuguese is shown in Figure 1. We did not used higher-order n-grams because of RAM restric- 2. Languages and Varieties tions, although [3] reported quality to increase up to 7 char n-gram level. We performed classification by means of logistic regression model with regularization parameter C = 1. Our choice was justified by the fact that logistic regression has high bias and low variance.
In this section we describe our results during cross-validation and on the test set. Next we present embedding of the data in low-dimensional space. Finally, we discuss about feature importances of our classifier.
Evaluation metric this task is accuracy:
We evaluated quality of gender and language variety subtasks separately by using crossvalidation scheme with five folds. Results can be found in Table 3. It was interesting to see how data is located in a feature space. To do so we exploited modern dimensionality reduction and data visualization techniques. Our choice of algorithm was t-SNE [2] since it reported to be fast when the number of objects is small and tends to efficiently preserve local structure of the data. Also, Python scikit-learn [5] implementation of the algorithm supports sparse matrices as an input. Example for Portuguese authors is at Figure 3. Unfortunately, axes of this algorithm have no clear interpretation.
We investigated absolute values of coefficients of our model for Portuguese language variety. This values can be considered as feature importances (Figure 4). Axis x means position in array of linear regression coefficients sorted in descending order. Axis y is absolute value of the coefficient. One can see that on the one hand feature coefficients have pretty low magnitude, but on the other hand there is group of features with relatively high importance.
We explored a simple and robust method for gender and language variety classification for PAN17 Author Profiling task. It turned out that high-order char n-grams are good features that are easy to generate with no need of handcrafting or expert linguistics knowledge. The main disadvantage of such features is that this is almost impossible to perform error analysis. We trained logistic regression classifier for both subtasks and evaluated accuracy measure. We will explore effects on quality measure due to adding even more n-grams.