Presently natural language processing for social media, in particular in the field of sentiment analysis and topic modeling, is gaining momentum for Russian texts. However, Slavic languages including Russian are still insufficiently explored in terms of computational sociolinguistics and authorship profiling (i.e. automatic identification of latent demographic features of online users such as gender, age, personality based on their texts). Being able to predict these features with a high degree of accuracy would certainly benefit marketing, psychological studies and security. In this paper we are attempting to build classifiers to predict gender of the author in Russian Twitter and Facebook texts and explore the effect of the cross-genre evaluation. We used the most common lemmas, a set of morphological and syntactic parameters as well as the part-ofspeech (POS) trigrams as features and multiple classifiers to train and test models. Twitter corpus was used for training, Facebook and test set of Twitter corpus were used for testing. The best models for Twitter were ExtraTreesClassifier and RandomForestClassifier with accuracy 0.72 and linearSVM for Facebook (0.71). The obtained results are comparable with stateof-the art results for Russian language for the texts of different genres.
The rapid growth of social media in recent years, exemplified by Facebook and Twitter, has led to a massive volume of user-generated informal text. This in turn has sparked a great deal of research interest in aspects of social media, including automatically identifying latent demographic features of online users. Many latent features have been explored, but gender and age have generated great interest. Accurate prediction of these features would be useful for marketing and personalization concerns, as well as for security. The majority of recent work in this area has focused on Twitter users [2; 3; 10]. Gender inference accuracy has been reported between 80% and 85%.
Gender prediction is one of the tasks which PAN (a series of scientific events and
shared tasks on digital text forensics) participants solve1 [9; 10]. However advances in
latent attribute inference on social media have been largely confined to English
content. Gender profiling for social media texts has not been sufficiently researched using
Slavic languages except a recent study of the Slovenian language [
Russian written texts were investigated in the context of author profiling, including
gender prediction. Using RusPersonality corpus which consists of the texts of
different genres (e.g. description of a picture, essays on different topics, etc.) labeled with
information on their authors (gender, age, results of psychological tests, and so on)
authors of [
The objective of the current paper is to build classifiers for identifying gender of
authors of Twitter and Facebook texts in Russian using different algorithms and
compare their accuracies. Following PAN 2016 framework [
Twitter. Annotating social media texts is what makes designing such corpora
particularly challenging. Some researchers automatically built Twitter corpora while others
have solved this problem by using labor-intensive methods. For example, Rao et al.
[
Facebook. 228 users of different age groups (20+, 30+, 40+) from different cities and occupations were randomly chosen (to get minimum mutual friendships) with no less than 1000 words per user.
Corpus statistics is presented in Table 1.
The general principles of processing the text corpora are as follows: 1. Non-Russian texts were removed; 2. Citations were removed; 3. Accounts of public people were not used as they might have someone else writing for them; 4. http references were removed. 5. All hashtags (marked with #) were replaced with the «hashtag» tag; 6. Named entities (marked with @) are replaced with the NER tag. Source Мale users Female users Twitter 543 543
Facebook 114 114
Given that the most frequent terms tend to select the most discriminative features
when applied to stylistic studies [
1) top 250 lemmas. We used 250 most frequent lemmas in the corpus. Note that we
chose to employ lemmas as Russian is a morphologically rich language where gender
is expressed explicitly in a range of grammatical structures. However, grammatical
gender markings are easily falsified. As we have in mind as a general goal to build a
system for gender prediction efficient, even in the case of gender imitation (valuable
for security reasons), we made a decision to avoid using the token-based approach.
For lemmatization we use Mystem [
3) we have chosen top 15 frequent POS trigrams which averaged values are different in males and female texts. At the first stage we chose POS trigrams which occurred in 75 % of the documents of the class and then calculated the difference between the average values of the frequencies of POS trigrams in texts by males and females and as a result, 15 POS trigrams with the largest difference in the average values were selected.
The values of all the properties were normalized either by the number of words in the documents or that of POS trigrams.
The majority of prior work in gender inference (and latent inference in general) has used support vector machine (SVM). We explored multiple classifiers from the scikitlearn Python library to train and test our models. The training was conducted using cross-validation for 30 folds by means of StratifiedKFold strategies. The Twitter corpus was divided into the training (90 %) and test (10 %) sets. Twitter was used for training; Twitter test set and the whole Facebook corpus were used for testing. 3
The classification results are presented in Table 2. The results listed in the tables are
average values of each “training-testing” cycle.
ing with Twitter where people share their comments without censorship, in a
spontaneous way, and where researchers can obtain a high number of texts per author, could
be a good manner to improve the performance of author profiling tasks in other
genres (such as blogs) for which it is more difficult to obtain sufficient training data”
[
The analysis of the frequencies of linguistic parameters enabled us to make some interesting observations (cf. [5; 8]) (t-test was used, p 0.05). More frequent character flooding (greaaattt (“отлиичноо”), oomph (“уффффф”), hurray (“урраааа”), theyyy “онииии”), diminutives (referring to men and food), named entity mentions (marked with @) are typical for female Twitter texts. As for Facebook, woman tend to use more specific geographic and proper names, as well as mention clothes, family and religious more often than males. In male Facebook texts there are more military, drinking, computer, food, car vocabulary.
For Twitter and Facebook texts the following are common: females tend to use the
conjunctions “and” («и»/«а»), negations “not” («не»), significantly more pronouns
“I” («я»), “my” («мой»), “own” («свой») as well as relative pronouns “the
whole”(«весь»), “this” («этот»), “such” («такой»), preposition “near” («у»); slightly
more – “we” («мы»), “you” («вы»), “she” («она»), adverb “very” («очень»). Males
prefer the prepositions “in” («в»), “on” («на»), “around” («по»), “about” («о»),
“from” («из»), “for” («для»), conjunction “but” («но»), pronoun “they” («они»). The
fact that pronouns are more common in female texts and prepositions in those by
males is in agreement with the observations made for other languages (for different
text genres), as well as for Russian texts of different genres (see [
The results for prediction of Twitter user gender using most common lemmas and
morphosyntactical parameters as features are comparable to those for Russian offline
texts from the corpus RusPersonality [
Note however that we consciously excluded the gender-marked parameters, which made the task more challenging. In future we are planning to expand our feature sets with special attention on content-independent parameters including readability measures, etc. which are more useful in gender detection designed for real-world application where users can imitate their writing pretending to be a people of opposite sex (for example, a pedophile can imitate the writing style of a young girl, to groom a child, etc.) Acknowledgment. This research is financially supported by the Russian Science Foundation, project No 16-18-10050 “Identifying the Gender and Age of Online Chatters Using Formal Parameters of their Texts”.
The work of the last author was in the framework of the SomEMBED TIN201571147-C2-1-P MINECO research project.