Author Profiling is one of the most important tasks in authorship analysis. In PAN 2019 shared tasks, the gender identification of the author is the main focus. Compared to the previous year the author profiling task is expended by having documents written by bots. In order to tackle this new challenge we propose a two phase approach. In the first phase we exploit the TF-IDF features of the documents to train a model that learns to detect documents generated by bots. Next, we train three models on character-level and word-level representations of the documents and aggregate their results using majority voting. Finally, we empirically show the effectiveness of our proposed approach on the PAN 2019 development dataset for author profiling.
As computational power grows and artificial intelligence techniques evolve, new
challenges, that were out of reach of machine learning in few years ago, emerge. One such
a problem is author profiling. Different from the traditional authorship identification,
in which a closed set of possible authors is known, author profiling aims to determine
what are the characteristics of the authors: their age, gender, native language among
others[
In the past years, multiple shared tasks have been organized on the topic of author
profiling[
The rest of the paper is organized as follows. Section 2 provides background and presents some related works on author profiling in general. Section 3 introduce our approach. Results are covered in Section 4. In Section 5 we draw a conclusion. 2
The author attribution task is presented for the first time in PAN 2013 [
In a similar work to our work, author of [
In this section, we first describe how we pre-process the data, then we present the features we extract from the text. Finally, we describe the supervised models we use for classification. 3.1
The first step in the proposed algorithm is to clean the input documents. In this step,
we combine each user tweets and then removed the Emojis, URLs, Mentions, Hashtags,
Smileys, Numbers, and Reserved words(RT, FAV) using tweet-preprocessor1 python
library. Next, using NLTK 3.0 Toolkit[
3.2
Term Frequency-Inverse Document Frequency (TF-IDF) was a common feature
transformation technique for detecting author’s style. First, we build a vocabulary using
preprocessed train-set for each language with frequency term 5. Next, using the scikit-learn
[
Module
Parameters Possible values Because the first part of the task is to identify the type of users and in case of human we have to identify the gender of users. If in the first part of the task we predict authors wrongly as a bot so we have to be precise to build an accurate system to compensation the wrong predictions of the first part of the task. Since ensemble methods usually produce more accurate solutions than a single model we used the ensemble method for gender profiling which uses majority voting for making a final prediction for the test instances.
Since we have two class here(male and female) so we use three different models for the
ensemble method, TF-IDF[
TF-IDF for TF-IDF model we use the same way that we used for type modeling in the previous section with a linear SVM classifier.
N-gram To consider character level features we used character 4-gram frequencies in combination with a supervised method called a linear SVM method. The parameters which we used for 4-gram are in table 2. These are obtained by grid search tools from scikit-learn library. All runs were performed with 5-fold cross-validation, while optimizing the accuracy target.
Module
Parameters
Possible values
Doc2Vec Document embeddings is a Paragraph Vector, an unsupervised algorithm that
learns fixed-length feature representations from variable-length pieces of texts, such
as sentences, paragraphs, and documents. With regard to this idea and based on work
[
Emoji Regard by our hypothesize emojis play an important role in identifying gender so ignoring emojis may lead to loss of valuable information about the author’s gender. As a result, we used emoji in varies ways. Namely, we calculated the number of occurrences of emoji in a given document. In the other case, we used emoji as sentiment analysis features to distinguishing between male and female. However, using emoji as a feature caused to collapse our results thus we decided to ignore them.
Contracted Wordforms We use is based on the discrepancies in spelling for words that allow contracted forms, e.g.,I will (I’LL), are not (aren’t). People typically favor one of the alternatives, and thus we use forms based on contracted apostrophes as discriminative features for detecting the gender of each author.
Quotation Marks Some authors may prefer either single or double quotation marks. We use the difference between the number of single and double quotes in a given document to see which group(male, female) prefer to use single or double quotation.
Sentence Length we noticed that some authors prefer to write long sentences where they use more conjunction in their text whereas some of them use short sentences as a result we consider length as a feature also we calculate the number of conjunctions in a given document and treat them as a feature.
Negations Another feature we use is based on the negation form e.g., impossible(not possible) to identifying the similarity of authors. We calculated the number of negations for each given document and added them to other features but we didn’t get any notable result. 3.5
We submitted our final run, a linear SVM system for type identification(human and bots) which uses TF-IDF and in case of human we use ensemble system for gender identification(male and female) which uses TF-IDF , 4-gram with a linear SVM system, and Doc2Vec with a LogisticRegression system. This process visualized in the figure 1.
In Table 3, the performance of three models for gender identification wich described in section 3 are presented. It is clear that the ensemble of all three models is the best model in average.
Language
EN SP
TF-IDF
Furthermore, we present the detailed results of the author profiling on development dataset of PAN 2019 in Table 4. As you can see, the average score obtained by this approach for English is 0.8330 and for Spanish is 0.7358 .
Models Type Model Gender Model
TP Test Size TF-IDF+N-gram+Doc2Vec In this paper, we proposed an algorithm for Author Profiling task in PAN 2019. We present our supervised approach, which uses a TF-IDF representation of the documents to distinguish between human and bots. Whereas, for detecting gender we present our ensemble approach which use a TF-IDF, N-gram, and Doc2Vec as feature extractions which makes predictions using an ensemble of diverse models including LinearSVC and Logistic Regression. Next, we uses dynamically determined parameters from an ad-hoc grid search approach to find the optimal values. Our evaluation shows that our approach is very capable of distinguishing between human and bot. However, identifying gender needs further work. The proposed algorithm implemented in Python and published on GitHub2.