Author profiling is the identification of an author's traits by examining the text written by an author. Author profiling has many useful applications in security, criminal, marketing, identification of false accounts on shared communication websites, etc. We submitted our system to the FIRE'18-MAPonSMS (Multi-lingual Author Profiling on SMS), a shared task to classify the attributes of an author like gender and age group from multilingual text specifically English +Roman Urdu. Roman Urdu is common language specifically in SMS messaging, Facebook posts/comments and chats blog of games etc. Our presented system is based on 29 different stylistic features. On the training dataset, we have achieved best Accuracy = 73.714, for gender while using all 14-language-independent features together and Accuracy = 58.571 for age group by using all 29 features together. We obtained Accuracy = 0.55 and 0.37 on testing data for both gender and age respectively.
Author profiling (AP) is the task of determining the writer’s traits, like age, gender,
profession, personality types and mother language by analyzing the written document.
Due to the heap of information on social networks, it became essential to classify user’s
characteristics. This chore has diverse applications in forensics, security, and in
marketing fields [
Recent tendencies in the domain include Multi-lingual AP [
Following the emerging field, the FIRE’2018 shared task on Age and Gender identification in SMS based author profiles (MAPonSMS), provided the training and testing corpora that were composed of multi-lingual (English +Roman Urdu) SMS based documents.
Since the provided training data was in multi-lingual, one of our primary objectives was to determine how our proposed technique is performing in the multi-lingual text. The author profiling is supervised document classification task. We carried out different experiments, i.e. (i) using all 29 stylistic features, (ii) using all 14-language independent stylistic features and (iii) using individual language independent stylistic features.
In recent period, deep-learning methods [
The document is prepared as follows. Section 2 discusses related work has been done in this domain. Section 3 details about our approach for the FIRE’2018 shared the task. Results and their analysis in section 4. The final section is five which concludes the paper. 2
The International PAN Competitions made remarkable progress in author profiling,
especially for the gender and age identification tasks [
As researchers are attracted towards the multi-lingual settings, however, there is only
one research study found in the literature that considering the same genre author
profiling task for multi-lingual text. [
There are three comprehensive categories have been used for automatic identification of an author’s traits: (1) content-based methods – that aim to detect characteristic of a writer by using content of the text, (2) stylistic-based methods – which try to predict a writer’s demographics traits by analyzing writing style of the writer, and (3) topicbased methods – are applied to classify characteristics of an author by using debated topics in the text.
For the FIRE’18-MAPonSMS Author Profiling competition, our system1 is based on different statistical features. As this year training data of FIRE’18-MAPonSMS is based on the multi-lingual SMS messages, i.e. Roman Urdu and English. This systematic investigation purposes to detect some language independent stylistic features, which are likely to perform in multi-lingual text. List of language-independent as follows: Avg. Word Length, Avg. Sentence Length, %age of Words with Six and More Letters, %age of Words with Two and Three Letters, %age of Question Sentences, %age of Semicolons, %age of Punctuations, %age of Comma, %age of Short Sentences, %age of Long Sentences, %age of Capitals, %age of Colons, %age of Digits, and %age of Full Stop. However, further are not language independent: Avg. Syllables Per Word, %age of Pronouns, %age of Prepositions, %age of Coordinating Conjunctions, %age of Articles, %age of Words with One Syllable, %age of Words with Three Plus Syllables, %age of Adjectives, %age of Determiners, %age of Interjections, %age of Modals, %age of Nouns, %age of Personal Pronouns, %age of Verbs, and %age of Adverbs.
We can observe that the list mentioned above of features purposes to catch different stylistic facts from a multi-lingual written text, which can be useful to uncover the age group and gender of an unknown author. 4
The focus of the FIRE’18-MAPonSMS shared task 2018 is on two author attributes (1) age and (2) gender identification on the multi-lingual text. The organizers provided us training dataset composed of multi-lingual (Roman Urdu and English) SMS text messages. There are two classes for gender (male, female) and three classes for the age group (15-19, 20-24 and 25-xx). 4.1
In the training dataset, for the gender classification, we have 210 male while 140 female labeled text documents. On the other hand for age group classification 108 text documents are in the 15-19 age group, 176 are in the 20-24 category, and 66 are in 25xx age group. However, 150 files were provided for the testing phase. 1 The implementation (source code) details of our approach is provided in a repository at https://github.com/abdulsittar/Multilingual-Author-Profiling
Author profiling classification problem is handled as a supervised ML problem. For detection of the age group, there are multi–groups problem and objective is to classify the age amongst 3-groups: (1) 15-19, (2) 20–24 and (3) 25–xx. For the gender categorization, there are binary-groups and objective is to differentiate between 2 groups: (1) female as well as the (2) male.
10–fold cross-validation was used in experiments to evaluate the performance of our model. We conducted our experiments by using four different ML algorithms named Naive Bayes, J48, Random Forest and Logistic Regression. Implementation of WEKA was used for these algorithms. The scores produced using the stylistic features are manipulated as input to above-stated ML algorithms. 4.3
As suggested by the organizing team of the FIRE’18-MAPonSMS shared the task, the performance of the submitted automatic system for the age and gender was measured using accuracy. Accuracy is described as the proportion of the correctly classified predictions out of all the predictions made.
= 5
For all the tables shown in this results and analysis section, next mentioned terminologies are used. The “Classifier” indicates the ML algorithm which we have applied to generate the numeric scores (NB → Naive Bayes, RF → Random Forest, LR → Logistic Regression). Best results are highlighted in bold typeface. We performed three sets of experiments: (1) performance on all 29 features, (2) performance on all 14 language independent features (see section 3.1) and (3) performance on individual language independent features (by using every single feature).Results on Training Dataset
Table 1 shows the scores using all 29 stylistic features for both groups, i.e., age and gender. Using all features together we obtained best results by using Random Forest, Accuracy = 73.714 for the gender and 53.142 for the age group classification. Table 2 depicts the results for all 14-language independent stylistic features collectively. We achieved the best results for gender (Accuracy = 72.000) using Random Forest classifier and for the age (Accuracy = 58.571) by using Logistic Regression.
NB RF LR J48
NB RF LR J48
Table 3 displays the results using 14-language independent stylistic features individually (for every single feature). RF was performing better on both age and gender identification; therefore we are only reporting the results on single language-independent features by using RF.
The best results are obtained when “%age of Punctuations” single stylistic feature is used for gender (Accuracy = 64.857) and “%age of Long Sentences” for age (Accuracy = 50.285). This indicates that in SMS multi-lingual messages, one of the gender prefers Punctuations than the other, while one of the age groups prefers longer messages than others.
Overall concerning algorithms, for age group identification, the best scores are obtained using two classification algorithms named as RF and LG. For gender estimation, best sores are achieved by RF algorithm. This shows that the RF algorithm is suitable if we give a collection of attributes as an input to the algorithm in the classification problem. 5.1
We obtained Accuracy = 0.55 and 0.37 on testing data for both gender and age respectively, which is below the baseline (baseline for gender = 0.60 and age = 0.51). Joint estimation of our model for both age and gender is 0.23. 6
Correct profiling of an unknown author is getting a reputation for security point of view, investigation of criminal activities and the market research opinion. In this paper, we have participated in our approach in the FIRE'18-MAPonSMS author profiling shared task on age and gender identification in multi-lingual text. We have shown how the stylistic features and machine learning techniques enable an automatic system to determine different characteristics of an unknown author efficiently.
We have considered the stylistic features to uncover the traits of an author on the multi-lingual corpus. We implemented 29 stylistic features and performed three different set of experiments, i.e., compared the results by using all 29 features, analyzed the scores for 14-language independent features altogether and at the end using single language-independent features. We observed that best results are achieved when we used all 29 features together for gender (Accuracy = 73.714) identification by Random Forest and for the age (Accuracy = 58.571) group when used all 14-language independent features by Logistic Regression classifier.