PAN 2017 Author Profiling task include two target predictions, one is to predict the gender of text authors and second is to predict the language variety. The presented approach analyzed trends and topics followed in training dataset e.g. Authors discussing Politics, Tech, Religion, Nature etc. in their respective tweets. Along with that single words and word pair frequencies were also taken into account. A cross-lingual, general, simple and flexible approach was created that could be applied over all languages without any changes according to each task language.
Social media nowadays has become an important indicator for trend based analysis and become a reflection of not only our personality but also a powerful way of expression. We express our feelings about different events or persons through text in the form of short expressive sentences that carry information about a person’s likes, dislikes, beliefs and interests.
The benefits of Author profiling tasks include the right idea of author’s gender,
age, regions and traits, and this possibility of finding people’s traits is of growing
importance [
Also the task of Author profiling deals with the aspects of forensics where the
gender and specific regions of countries are important, and this task of author
profiling task of PAN@CLEF 2017 [
By identifying general topics discussed in social networks can provide us better
understanding of collective interests [
Five different trends including Politics, Technology, Nature, Travel and Religion
were analyzed. Frequent words appearing in texts of different languages were also
analyzed, because people use social media, like twitter to follow different topics and
issues that are trending in their respective geographical areas and of their specific
interests. They also use specific expressive words that are common or more frequent
in their regional areas, so term frequencies and word pair frequencies were also taken
into account because both are an important indicator for text based analysis [
Like previous year, the training dataset of PAN CLEF 2017 for author profiling
task contained xml files for each language representation with an exception that
demographic traits in current dataset are processed side by side [
In order to fulfill the task of Author Profiling at PAN CLEF 17, a general system was built so that it may be able to perform equally irrespective of language shifts from one to another or having to jump from one basic approach to another. By performance it’s meant that the system would predict both gender and language variety for each author in Twitter corpus without using complex linguistic analysis methods like Part of Speech (POS) analysis or stylistic approaches from variety to variety and for gender prediction as well.
A simple scoring approach is used where each document is assigned scores in order to classify it as one of the classes that are created as preprocessing of training dataset. Each class represents exactly one variety in each language representation with two subclasses under each variety representing gender i.e. male and female.
The experimental setup is divided into following steps
Trend word-lists Preparation Dataset Preprocessing Language Variety Classification
Gender Classification
A total of five general topics were chosen including Politics, Technology, Religion, Nature and Romance to create lists of 500 words in each language i.e. Arabic, English, Portuguese and Spanish and for each chosen topic. These word lists are referred as trend list in presented system, where L is the main language variety and i is variable representing specific trend. Different sources were used in this regard.
The system performs following steps in order to preprocess data for creation of language variety classes and gender subclasses.
All xml based twitter text was extracted from each language variety training dataset discarding Hash-tags, html/xml tags, html/web links and extra white spaces in order to get plain text from each file and all text was combined as a single document
representing a language variety. Where v is the represented language variety in document. 2.
All extracted text was combined in order to find top 100 most frequent words and word pairs
each, where t is the id for specific term (1) calculated. 3. Each common term occurring in both trend list and document is scored according to term occurrence and Trend score for each document is 4. Steps 2 and 3 are repeated in order to find gender based term frequency, word pair frequency and Trend scores calculation under each language variety v. 5. Steps 1 to 4 are repeated for each Language L. 6.
Hence we have created different language variety classes and gender subclasses
, where g represents the gender i.e. male or female as shown in 7. The System loops through each language folder of dataset and every document is processed in the same manner as a separate class with unknown variety label u and Author Identity id having its own , and
Following Steps are taken by the system in order to assign the class language variety label while looping through each variety class . with a 1. For each single word and word pair common in each is increased for class as shown in equation 4. and score ∑ ∑ (4) (5) 2. For each trend score in and in between both is calculated as shown in equation 5. absolute difference
The least trend difference score is added to class variety score and in this way a class with unknown language variety is assigned a language variety v having highest score .
Once the Language variety of Document class has been decided, the system repeats similar steps as shown in section 3.3 to find out the gender for class but this time the system has to decide among only two gender subclasses of variety class v. The gender subclass having highest score is assigned to class .
A 10-fold cross validation criteria was used during the initial validation of the
system and no different approach was used for prediction of language varieties among
main languages or gender prediction as shown in above section. Once all parameters
were adjusted during system validation over training datasets, the system was ready to
run over full training and test datasets provided at TIRA [
Following are the evaluator results shown in table 3.
The results consistency distributed over different languages in above table for both training and test datasets clearly indicate the generalization of used system; it’s because of the designed approach that denied over-fitting while training stages. The performance of model is good where it has to predict among smaller group of variables either it be gender or language variety, a typical example of which is the results of Portuguese language where the system’s prediction accuracy is highest for both gender and variety because each has only 2 values to predict from. As the number of values in language variety increases, the performance of system decreases and hence the overall results are affected.
In this paper a new and generalized approach is presented which may be able to perform with same attributes over all given languages in dataset without changing its modes or functionality with language change. The results however indicate a major flaw in the system over which more work is required. This flaw is the decrease in prediction when there is an increase in number of language varieties. This can however be improved either by increasing the quantity of single words and word pairs while the creation of variety classes or by increasing the trends.
The future work will certainly emphasize to improve the prediction eminence of the system by following the suggestions discussed above.