This working note describes the system we used while participating in RusProfiling PAN 2017 shared task. The objective of the task is to identify the gender trait of the author from the author's text written in the Russian Language. Taking this as a binary text classification problem, we have experimented to develop a representation scheme for target classes (called class vectors) from the texts belonging to the corresponding target classes. These class vectors are computed from the traditional representation methods available in Vector Space Models and Vector Space Models of Semantics. Followed by the representation, Support Vector Machine with a linear kernel is used to perform the final classification. For this task, genre independent corpus is provided by the RusProfiling PAN 2017 shared task organizers. This proposed model attains almost equal performance across all the genre available in the test corpus.
• Computing methodologies → Natural language processing; Feature selection;
Prediction of author’s traits (gender, age, native language and
personality traits) from their texts is known as author profiling and its
applications in targeted internet advertising, forensic science and
consumer behaviour analysis induces the researchers1, 2, 3 and
the industries4 to develop a reliable author profiling systems. The
growth of digital text shared in social media (Facebook, Twitter)
feeds the researchers with the required corpus to develop the author
profiling systems and its related shared tasks to build the state of
art systems [
Unlike other text classification problems which identify the
context, here more than identifying the context shared by the author,
identifying the style used to share the content by the author is more
relevant [
By considering the above discussed points, we have experimented to develop a representation scheme for target classes, which is termed in this paper as class vectors. On successive representation of texts using the methods available in Vector Space Models (Document-Term Matrix and Term Frequency - Inverse Document Frequency) and Vector Space Models of Semantics (Document-Term Matrix with Singular Value Decomposition and Term FrequencyInverse Document Matrix with Singular Value Decomposition), we summed the text vectors in the respective Target classes to form the class vectors. Later, the variation between the class vectors and the text vectors are computed through the distance and correlation measures. These measures are taken as the features and fed to the Support Vector Machine with a linear kernel to make the final prediction. The experimented model s given in Figure 1.
The objective of the task is detailed in Section 2; statistics about the corpus are given in Section 3; underlying components for computing class vectors, feature learning, and classification method are explained in Section 4; cross-validation reports, results reported by the shared task organizers and observations about the results are detailed in Section 5.
Given the training corpus which consists of author’s text tagged with the author’s gender information, the objective is to predict the gender information for the author’s text available in the test corpus5. In the below given equation taд ϵ {male, f emale } and n represents the total number of author’s text in the corpus. train_corpus = {(text1, taд1), (text2, taд1), ..., (textn , taдn )} (1) test _corpus = {text1, text2, ..., text _n} (2) 3
For this task, the corpus has been provided by the RusProfiling
PAN 2017 shared task organizers [
Text representation is the task of transforming the unstructured texts into its equivalent numerical representation. On successive representation, further mathematical computation will be applied to it. 4.1
4.1.1 Vector Space Models. Document-Term Matrix (DTM) is
a basic representation method, which accounts the count of the
unique word’s present in the document [
Dnd×tmm = dtm(train_corpus | test _corpus)
Dntf×imd f = t f id f (train_corpus | test _corpus) (4) In the above equation, n represents the number of author’s text and m represents the number of unique words in the vocabulary. Unique words from the train and test corpus are used to build the common vocabulary.
4.1.2 Vector Space Models of Semantics. Singular Value
Decomposition applied to the matrix from VSM to get the semantic
representation of the author’s text. Applying matrix factorization on top
of the matrix from the VSM is known as Vector Space Models of
Semantics (VSMs) [
Und×tmn Σn×mVmT ×m = svd Dnd×tmm
Unt ×find f Σn×mVmT ×m = svd Dntf×imd f (6) In the above equation, U represents the basis vector representation of the author’s text and it is used while computing class vectors, 5http://en.rusprofilinglab.ru/rusprofiling-at-pan/ σ represents the singular values (significance of topics) in the descending order and V represents the basis vector representation of the words in the vocabulary. In this work, we have not performed the dimensionality reduction. 4.2
On the successive computation of Ddtm , Dntf×imd f , Und×tmn and Unt ×find f , n×m the class vectors are computed by summing the respective vectors of the author’s text belongs to the classes male and female. It is given as follows for computing class vectors from VSM, i=1
n C1m×aml e = Õ vsm_representation [i, :] i f taдi = male i=1 n C1f×emmal e = Õ vsm_representation [i, :] i f taдi = f emale vsm_representation ϵ nDnd×tmm , Dt f id f o n×m Similarly, class vectors from VSMs are computed as follows, n C1m×anl e = Õ vsms_representation [i, :] i f taдi = male i=1 n C1f×enmal e = Õ vsms_representation [i, :] i f taдi = f emale (11) i=1 vsms_representation ϵ nUnd×tmn , Unt ×find f o 4.3
The variation between the classes and the author’s texts are computed by measuring the distance and correlation between the class vectors (male and female) and the vector representation of the author’s text. We have considered correlation, cosine distance and Euclidean distance for measuring variation. This is given as follows, (3) (5) (7) (8) (9) (10) (12) (13) (14) (15) (16) Fn×6 = f eature_learn representation, Cmal e , Cf emal e representation ϵ nDndt×mm , Dntf×imd f , Und×tmn , Unt ×find f o
1×m 5
From the given corpus, the author’s texts are represented as the matrix as shown in Section 4.1. For DTM and TF-IDF representation, the CountVectorizer6 and Tfidf Vectorizer 7 modules from scikit-learn python library are used. SVD applied on this computed matrices and the basis matrices (U ) alone are kept for further processing. SVD performed by using the numpy python library8.
As given in Section 4.2, the class vectors are computed from the author’s text vectors. As given in Section 4.3, the feature matrix 6http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text. CountVectorizer.html 7http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text. TfidfVectorizer.html 8https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg. svd.html
Run3 Run4 Run5
600 370 228 400 776 94
computed by measuring the variation between the author’s text vectors and class vectors. The scipy python library used to compute the cosine distance, Euclidean distance, and correlation measures.
The computed feature matrix along with the target classes fed to SVM with a linear kernel to perform the final classification. SVM from the scikit-learn python library9 used for classification. In order to observe the training performance, we have computed the 10-fold cross validation score and given in Table 2 (Train CV). This is given as follows,
Accuracy = correctly predicted short texts total # short texts
(17) Íi1=01 Accuracyi T rain CV = (18) 10 Similar to the training corpus, the feature matrix computed for the ifve test corpus given by the shared task organizers and prediction for the author’s texts are found by using the model built in the training period. We have totally submitted the five runs and those details are given below, • Run1: TFIDF -> Class Vectors -> Feature Learning -> Classiifcation • Run2: DTM -> Class Vectors -> Feature Learning -> Classification • Run3: TFIDF -> SVD -> Class Vectors -> Feature Learning ->
Classification • Run4: DTM -> SVD -> Class Vectors -> Feature Learning ->
Classification • Run5: DTM -> Classification The results reported by the shared task organizers for the submitted ifve runs are given in Table 2. Out of five runs, Run2 performed better and attained 51.45% as the accuracy measure for the concatenated test corpus. Run5 attained 47.06% as accuracy measure for the concatenated test corpus and this ensures that class embedding enhanced the accuracy by 4%.
The given train corpus and the test corpus are represented as the author’s text vectors by using methods available in Vector Space
Model and Vectors Space Models of Semantics. Class vectors are computed from the author’s text vectors and variation between the class vector and the author’s text vectors are taken as the features to perform the Support Vector Machine based classification. The preliminary results showed that class vectors based classification improve the accuracy by nearly 4% for the final concatenated test corpus. There has been a large performance variation between the cross-validation score and the score against test corpus. Hence our future work will be focused more on reducing this margin and computing the class vectors though distributed representation methods.