Author profiling, being an important problem in forensics, security, marketing, and literary research, needs to be accurate. With massive amounts of online text readily available on which we might need to perform author profiling, building a fast system is as important as building an accurate system, but this can be challenging. However, the use of distributive computing techniques like MapReduce can significantly lower processing time by distributing tasks across multiple machines. Our system uses MapReduce programming paradigm for most parts of the training process, which makes our system fast. Our system uses word n-grams including stopwords, punctuations and emoticons as features and TF-IDF (term frequency inverse document frequency) as the weighing scheme. These are fed to the logistic regression classifier that predicts the age and gender of the authors. We were able to obtain a competitive accuracy in most categories and even obtained winning accuracy for two of the categories each in both test corpus 1 and test corpus 2.
The process of identifying age-group, gender, native language, personality and other aspects that constitute the profile of an author, by analyzing his/her writings is called author profiling. Since most of the text is now online and written behind a curtain of anonymity, author profiling has become a very important problem. In fields like forensic, security, literary research and marketing, finding out an author’s demographic and personality information has proven to be very helpful. In literary research, author profiling helps to resolve disputed authorship for unknown documents. In forensics, author profiling can be used to shortlist potential suspects, given a piece of writing from them. In marketing, ad campaigns can be directed towards the demographic of users that review certain products the most.
The PAN’14 [
Prior work has tackled the task of author profiling by employing a range of lexical,
stylistic, syntactic and readability measures. Schwartz et al. [
In this paper, we have experimented with word and character n-grams as features.
We also analysed the use of five different classification algorithms viz Naive Bayes,
Cosine Similarity, Weighted Cosine Similarity, LIBLINEAR [
Most of our training takes place in Hadoop and uses MapReduce. For our training process, we started by randomly dividing the available data into training and cross validation dataset in a ratio of 70:30. The training data was then preprocessed to filter out all the HTML and XML tags. The plain text files thus obtained were then combined into sequence files because MapReduce jobs run faster when we use a small number of large files rather than when using a large number of small files. We then used MapReduce jobs to compute feature vectors. 2.1
Since most of the training data was from some sort of social media, we used
ArkTweet-NLP tokenizer [
After obtaining the n-grams, we compute the inverse document frequency (IDF) count for each token. Based on these counts, we filter out the tokens that have not even been used by at least two authors. We have an idf MapReduce job to compute the idf. The mapper computes the partial count of each individual ngram and passes them to the reducer, which sums these partial counts to produce the final idf counts. The filter job takes the idf counts and a threshold and filters out all the tokens whose idf score is less than the threshold. Also, this job creates a dictionary file that contains all the unique tokens mapped to integer ids. After the tokens are filtered out, our TF-IDF vector creation job takes in the idf counts and dictionary file to compute the tf-idf scores for the remaining tokens. This map only job outputs a tf-idf vector for each document. 2.2
For training, we tried five different classification algorithms. For Naive Bayes, cosine similarity, and weighted cosine similarity, we created another MapReduce job that computes all the statistics needed for classification. The weighted cosine similarity algorithm multiplies the cosine similarity score by prior probability score for each class. The mapper emits class label as key and tf-idf vector as value. Instead of building separate models for age groups and gender, we considered the problem as a 10-class problem by combining the age and gender classes. The class labels were extracted from the document names and were mapped to unique integer ids. The mapper also emits a special key -1 and a vector that contains partial counts of number of documents with that class label. We used VectorSumReducer provided by Mahout as reducer, which groups vectors by their class id and sums them. For both logistic regression and SVM, we first transformed the tf-idf score vectors into a format as expected by LIBLINEAR. Then we trained on these feature vectors by using the LIBLINEAR command utility. 2.3
The PAN’14 shared task organizers provided us with a Virtual Machine(VM) with 4GB of memory. We could have deployed Hadoop in pseudo distributive mode and ran the MapReduce version of our application for testing. However, it would not have given us any advantage, as the process would run on just that machine, which did not even have multiple cores. Running the tests on Hadoop would thus only add overhead and would make our system slower. So, for testing, we created a normal java application (not MapReduce), that would read the trained models and predict class labels for the test documents. For testing, we need to create test vectors similar to trained vectors. Hence, we applied the same steps as we did for the training data. After the test document was preprocessed to removed HTML tags and tokenized using Ark-Tweet-NLP tokenizer, we generated word or char n-grams by using Lucene 1. 3
We setup a local Hadoop 2 cluster with a master node and 7 slave nodes, each node having 16 cores and 12GB memory. We are running Hadoop version 1.0.4 and Mahout 1 http://lucene.apache.org/ 2 http://hadoop.apache.org/ version 0.7. Since the data is large, MapReduce is ideal for feature extraction from this data. We were able to finish training in a short amount of time even though the data is large. We were also able to train five different models because we did not need to spend a lot of time for feature extraction. In order to find the best model, we tried different classification algorithms and also compared the use of word vs character n-grams. We also performed experiments to find out if building separate models for different categories: blogs, social media, twitter and reviews produce better results than building a single, combined model for all categories. The test was performed in cross validation dataset, which was obtained by randomly separating 30% of the training data. 3.1
Since we already figured out that word n-grams are better at predicting the profile of
an author, we decided to use them as features rather than character n-grams. The next
decision we needed to make was either to build a separate model for each category
or to build a single, combined model for all the categories. Since we had already run
experiments for separate models, we built a single combined model next. But we still
built two separate models for English and Spanish. Table 3 tabulates the accuracy
obtained by running combined model and separate model for the cross validation set. It is
clear from the table that having a single model is better than using separate ones. This
might be because since all the data has been obtained from some sort of social media,
authors might use similar writing styles across all of them. So, having a model trained
on all categories or genres performs better than when separate models are trained on
each category. We also found logistic regression to be the best classification algorithms
for this task. So, our final model uses word n-grams trained on data from all categories
as features and logistic regression as the classification algorithm.
spanish-socialmedia and spanish-twitter for test corpus 1 and in english-socialmedia
and spanish-twitter for test corpus 2. We were ranked in the top three for most of the
other categories as well. This might even indicate that some problems are better suited
by simple solutions. For english-socialmedia, which has largest number of test
documents, we ranked second in runtime for both test corpora. Our system took 26 minutes
to complete the classification. Whereas, some of the other participants took from around
30 to 69 hours to complete the testing. Although even for other test corpora, our runtime
performances are quite fast, we could not obtain equally high ranking because we have
a huge model that takes a lot of time to load.
Since PAN’13 [
We want to thank PAN’14 organizers and committee members for organizing the author profiling task. This research was partially funded by The Office of Naval Research under grant N00014-12-1-0217 and National Science Foundation under grant 1350360.