In digital documents analysis for forensic applications, when anonymous documents are presented and it is not possible with the available tools to determine the true author of the document, there are of vital importance methods that identify the characteristics of the Author Profile (Gender, Age, Personality, etc.). We propose to use a simple method of classification based on the similarity between objects, considering different features for documents representation: (a document corresponds to a set of tweets of a user), the terms used in the tweets, as well as characteristics of opinion and subjectivity presented in them. Our goal will be to classify, based on the content of the tweets, the Gender and language variety of an author from an unknown set of tweets corresponding to him. In the experiments we observed good results in Gender classification, but low values in language variety classification. We processed only the English dataset.
The PAN Profiling task for this edition is as follows: "Gender and language variety identification in Twitter. Demographics traits such as gender and language have so far investigated separately. In this task we will have participants with a corpus annotated with authors' gender and their specific variation of their native language: Although we suggest to participate in both subtasks (gender and language identification) and in all languages, it is possible participating only in one of them and in some of the languages.”
The proposal to identify these demographic traits in tweets implies that the natural language processing tools widely used for long documents analysis must be adapted to the features of the textual genre and the writing characteristics presented in tweets. We must emphasize that the complexity lies in the fact that for this genre there are no linguistic rules or writing standards. Language is informal, usually direct and full of emotions.
In past tasks of demographic traits identification on PAN evaluation framework [
For the classification process, decision tree-based approximations have been used, as well as SVM by a large number of competitors and a few others have used distancebased approximations to predict the closest class [14][15].
We are interested in implementing a distance-based classification strategy and with
this, use previous results presented in the Author Identification edition of 2015 [
We used two classification strategies, considering two documents representation variants. An instance based representation of the documents, where the set of tweets of an author (for each author it is available her/his gender and language variety) represents a document and with this idea, for each class (female class, male class) we have a set of documents. The second variant is a prototype-based representation, where a single document is formed for each class, and this document is constructed with all the tweets of each of the sample authors per class.
Figure 1 shows graphically the architecture of our proposal with the instance-based strategy.
The first step correspond to build the documents that will be used as objects for the similarity calculation in the classification method. For each author, we receive the set of tweets that she/he wrote, and with the concatenation of these tweets is formed a document for this author. Remember that, of each author, what we have is the gender and the language variety. We perform a pre-processing of the document in two stages. In a first stage, we segment the tweets with a tokenizer offered in FreeLing [13] [http://nlp.lsi.upc.edu/freeling/], specialized for the processing of tweets. Subsequently we proceed to the expansion of short terms used and contractions, and characteristics traits that are used in tweets such as the Hashtags, URLs, mentions, are replaced by certain fixed patterns, those traits we consider the content does not contribute to differentiate between tweets of different profiles. After these transformations, we have normalized the tweets a bit and next proceed to perform a syntactic analysis with the traditional POS-Tagging tools for English and Spanish according to the language of the tweets.
For the representation we use the classic Bag of Words and in this we integrate:
The lexical terms, the lemmas of these and the grammatical category.
Characteristics features of the tweets.
Features of subjectivity and opinion mining analysis [
With the lexical terms and lemmas, we hope to differentiate the documents of each class, because some of this features are proper of their class. For example, for language variety, some terms are used by Colombians unlike the rest, and thus similarly for each variant of Spanish. Considering the frequency of use of grammatical categories, would allow us to differentiate between tweets written by the male gender and those written by the female gender. For example, in [17] it is exposed several differences in the use of words and different Part of Speech analyzing women and men writing style.
The characteristic features of the tweets we extract correspond to hashtags, the mention of author, the mark of retweet, the use of URL, the use of intensifications (capital letters, deformation of words by repetition of characters, use of admiration signs), use of laughter expressions, use of emoticons and the use of informal language. For each of these traits we consider the position in which they are used, that is, the number of times used at the beginning of the tweet, at the end or elsewhere.
Additionally, we include the analysis of the frequency of features with subjective
information, for example, the number of positive or negative emoticons; the words used
were categorized as Positive (P), High Positive (HP), Negative (N) and High Negative
(HN), using the frequencies of this categories. We used a word polarity resource in
Spanish and English taken from [12], resources of emotion in Spanish [
For the classification of the set of tweets of an author in the Demographic traits of gender and language variety, we tried with two strategies. A strategy in which each document (set of author tweets) is used as an instance of the class to which it belongs and for the second strategy we construct a prototype of each class using the extracted features of the set of documents belonging to the class. Each of these strategies were evaluated with the tweets collections of the training set and was selected for the final evaluation, the one that showed more stable results in different executions.
In the instance-based strategy, it is calculated the similarity of the new document
with each sample document of the class, and then is computed the average similarity
obtained with the class. This analysis is done with each class of a Demographic Trait
and the object is going to belong to the class with which it obtains greater average
similarity. In the prototype-based strategy, the similarity of the new document is
calculated with the class prototype. This analysis is done for each class and the object is
going to belong to the class in which the similarity obtained was the highest (1-NN
[
The classification is done independently for each Author Demographic Trait, Gender classes (2 classes) and language variety (for English 6 classes and for Spanish 7 classes). Finally, the result is the combination of these two classifications. 3
The initial experiments were performed with the training collection released for this
year's 2017 task. We evaluated the accuracy obtained by performing a 2-cross fold
validation. In addition, we considered the training collection of the 2015 edition for the
Gender and Age classes. The description of these collections can be reviewed in [18]
[
A representation that considers the terms used in tweets, is able to differentiate to a large extent the sets of tweets written by authors of different genres. The proposed subjectivity and opinion features allow improvements in classification, but they are not substantial improvements. In the evaluation we made with the collections of 2015, we verified that each of the sets of features separately allows good identifications of the genre and that their combination increases the values obtained. The classification in language variety maintains low results and to a great extent this is due to the little difference that is observed between some of these classes and that many terms used by the authors are of universal character and are standardized in the community.
We achieved the lowest values of all the team and considering that a baseline method using the 1000 most frequent terms in a Bag of Word representation got better results, then we need to do an exhaustive evaluation of our method.
We must work on features selection strategies and the analysis of representative objects to each of the classes. We propose to evaluate a classification with rejection or abstention for those users whose tweets do not contain characteristic features with their class, for example for the idea of language and not penalize so much the possible bad classifications. 5 9. Hernández, L., López-Lopez, A., & Medina-Pagola, J. E. (2009). Recognizing Polarity and Attitude of Words in Text. In In Proc. F 14th Portuguese Conference on Artificial Intelligence, (EPIA’2009) (pp. 525–536). Aveiro, Portugal. 10. Hernández, L., López-Lopez, A., & Pagola, J. E. M. (2011). Classification of Attitude Words for Opinions Mining. International Journal of Computational Linguistics and Applications, 2(1–2), 267–283. 11. Ismael Díaz Rangel, Grigori Sidorov, Sergio Suárez-Guerra. Creación y evaluación de un diccionario marcado con emociones y ponderado para el español. Onomazein , 29, 23 p., 2014, DOI 10.7764/onomazein.29.5 12. Jose Manuel Yero Moreno, Reynier Ortega Bueno. Método no supervisado para la clasificación de polaridad en Twitter. VII Conferencia Internacional de Ingeniería Eléctrica. . pp. 1 - 4. Jun, 2014. ISBN: 978-959-207-529-0. 13. Lluís Padró, Evgeny Stanilovsky. FreeLing 3.0: Towards Wider Multilinguality Proceedings of the Language Resources and Evaluation Conference (LREC 2012) ELRA. Istanbul, Turkey. May, 2012. 14. Mirco Kocher, Jacques Savoy: UniNE at CLEF 2016: Author Profiling. CLEF (Working
Notes) 2016: 903-911 15. Maria José Garciarena Ucelay, Maria Paula Villegas, Dario G. Funez, Leticia C. Cagnina, Marcelo Luis Errecalde, Gabriela Ramírez-de-la-Rosa, Esaú Villatoro-Tello: Profile-based Approach for Age and Gender Identification. CLEF (Working Notes) 2016: 864-873 16. Patrick Juola. Authorship Attribution. In Foundations and Trends in Information Retrieval,
Volume 1, Issue 3, March 2008. 17. Pennebaker, J.W., Francis, M.E., Booth, R.J.: Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71 (2001) 18. Rangel, F., Rosso, P., Potthast, M., Stein, B.: Overview of the 5th Author Profiling Task at PAN 2017: Gender and Language Variety Identification in Twitter. In: Working Notes Papers of the CLEF 2017 Evaluation Labs. CEUR Workshop Proceedings, CLEF and CEURWS.org (Sep 2017)