Authorship attribution, being an important problem in many areas including information retrieval, computational linguistics, law and journalism etc., has been identified as a subject of increasingly research interest in the recent years. In case of Author Identification task in PAN at CLEF 2015, the main focus was given on cross-genre and cross-topic author verification tasks. We have used several word-based and style-based features to identify the differences between the known and unknown problems of one given set and label the unknown ones accordingly using a Random Forest based classifier.
Author identification has a long history that includes some famous disputed
authorship cases as well as various forensic applications. The task is used in order to
determine the author who wrote a chapter or passage of a book (e.g., the holy Bible, being
the most famous example). In general, the researches on author identification make
use of the structure of the text and the words used to describe the contexts. In
stylometry research, it is generally accepted that the authors have unconscious writing
habits that can be evident from their use of words and grammar or punctuation etc. as
such indicators could be reliable to identify an author [
In the paper, we have presented our approach of automatic authorship verification from a dataset containing cross-genre and cross-topic text snippets available in four different languages- Dutch, English, Greek and Spanish. In each of the problem sets, a group of known documents and one unknown document were provided. The goal of the present task is to predict if the author of an unknown document is same as that of the known document set by analyzing the similarity among the known documents and their differences (or similarities) to the unknown one. We have used 17 types of word based and style based features in total to find out the underlying similarities and differences of a set of documents. The Random Forest classifier that use a decision tree based approach and available with Weka1 tool was employed in our system to choose the important features as we were not sure about the importance of a specific feature out of the main features viz. punctuation, sentence length, vocabulary,
The rest of the paper is as follows. Section 2 consists of a brief discussion on the related work available till date. Section 3 describes the feature selection for implementing the Random Forest algorithm whereas Section 4 gives the details of the system architecture. In Section 5, the experiments with detailed analysis of results are presented. Finally, the conclusions and future scopes of the experiments are presented in Section 6. 2
It is observed that the two main factors that characterize a text are its content and style, and both can be used as means of authorship categorization. Generally, there are two kinds of problems in author identification, the first one being comparatively easier than the second one. In the first kind, a finite set of documents with known authors is given and the task is to comment on the authorship of the unknown documents. In the second kind of task, a set of documents of a particular author is given along with an unknown document and the task is to predict whether the unknown document is written by the same author or not.
of the author classification task. Some of the aspects such as the language of texts
used, length of the text snippets, number of authors and texts, types of features etc. are
considered as important for the classification task. It is observed that the number of
features is most often varied to determine the influence of certain types of features.
Corney et al. indicates that the most successful features are the function words and
character n-grams whereas McCombe performed the tests using word unigrams as
classification feature, for which the results were promising [
So far, all the researches we described used only the English data sets. In
contrast, author identification tasks were also performed on messages of other
languages. In order to identify authors of Greek texts published in a newspaper,
Stamatatos et al. used a combination of lexical measures and style markers and achieved an
accuracy of 87% in texts classification [
In literary authorship, the stylometric features are commonly used, for
example- Letter frequencies, N-gram frequencies, Function word usage, Vocabulary
richness, Lexical richness, Distribution of syllables per word, Word frequencies, Hapax
legomena, Hapax dislegomena, Word length distribution, Word collocations,
Sentence length, Preferred word positions, Prepositional phrase structure, Distribution
parts of speech, Phrasal composition grammar etc. [
3
author identification task. Thus, the number and types of features are often varied in author identification research to determine the influence of certain types of features.
For example: “We are participating in PAN, a part of CLEF 2015. Our main focus will be on Authorship Verification task- a subset of Author Identification.” Here, this particular feature value is: (number of punctuation/number of words) = 4/24 = 0.16666 We will calculate differences between unknown text and each known text of a problem set and take the average of those differences.
For example: “We are participating in PAN, a part of CLEF 2015. Our main focus will be on Authorship Verification task- a subset of Author Identification.”
Here, the feature values will be: StopCount= (number of stop/total number of punctuation) = 2/4 = 0.5 CommaCount= (number of stop/total number of punctuation) = 1/4 = 0.25 DashCount= (number of stop/total number of punctuation) = 1/4 = 0.25 ColonCount= (number of stop/total number of punctuation) = 0/4 = 0.0 ExclamationCount= (number of stop/total number of punctuation) = 0/4 = 0.0 QuestionCount= (number of stop/total number of punctuation) = 0/4 = 0.0 SlashCount= (number of stop/total number of punctuation) = 0/4 = 0.0 SemicolonCount= (number of stop/total number of punctuation) = 0/4 = 0.0 We will calculate differences between unknown text and each known text of a problem set and take the average of those differences in a similar manner like the previous one.
short (length<6) sentences to the total number of sentences is represented by this feature.
For example: “We are participating in PAN, a part of CLEF 2015. Our main focus will be on Authorship Verification task- a subset of Author Identification.” Here, first sentence has length of 10 and second sentence has length of 14. So we have 0 short sentences and 1 long sentence.
LongSentenceRatio=1/2=0.5 ShortSentenceRatio=0/2=0.0 Then again, we take the differences in a similar manner.
For example: Known-“We are participating in PAN, a part of CLEF 2015.” Unknown- “We are participating mainly in an Authorship Verification taska subset of Author Identification task.” Here, in Known, total number of words= 10 and unique words= 10. In Unknown, total number of words= 15 and unique words= 13 (without stemming and stopword removal) So, VocabularyStrength in Known=10/10=1.0.
VocabularyStrength in Unknown=13/15=0.8666.
So, the feature value will be: 0.1333
For example: Known-“We are participating in PAN, a part of CLEF 2015.” Unknown- “We are participating mainly in Authorship Verification task- a subset of Author Identification.” Here, some of the bigrams are- „We are‟, „are participating‟, „participating in‟ etc. and some trigrams are- „We are participating‟, „are participating in‟ etc. (without stemming and stopword removal) BigramOverlap=2 [„We are‟, „are participating‟] TrigramOverlap=1 [„We are participating‟], for this particular example.
For example: Known-“We are participating in PAN, a part of CLEF 2015.” Unknown- “We are participating mainly in Authorship Verification task- a subset of Author Identification.” Here, the POS tags are: We/Pronoun are/AuxVerb participating/Verb in/Preposition PAN/ProperNoun, a/Det part/Noun of/Preposition CLEF/ProperNoun 2015. Our/Pronoun main/Adjective focus/Noun will/AuxVerb be/Verb on/Preposition Authorship/Noun Verification/Noun task/Noun- a/Det subset/Noun of/Preposition Author/Noun Identification/Noun.
Feature value: 6
In the above example, we have sequence overlaps: „AuxVerb Verb‟, „Verb Preposition‟, „Det Noun‟, and „Noun Preposition‟. So, feature value is: 4
the beginning of sentences according to their frequency and then compare them among the known and unknown documents to find a lexical pattern.
In the above example, both known and unknown text starts with the POS tag Pronoun. So, feature value is 1.
As it is impossible to decide manually which of these features are most important or relevant to our problem structure, we decided to go for Decision Tree based classifier. Such classifiers are fast to train and easy to evaluate and interrupt and moreover non-parametric and for the very reason, we don't have to worry about the outliers or whether the data is linearly separable or not. The main disadvantage is that they easily overfit, but that's where ensemble methods like Random Forest come in. The main advantages of such approaches are: - Almost always have lower classification error and better F-scores than decision trees. - Almost always perform as well as or better than SVMs, but are easier to understand. - Deal really well with uneven data sets that have missing variables. - Give a good idea of which features in the data set are most important. - Generally train faster than SVMs. 4
5
Table 1 shows the detailed results of our system in PAN 15 authorship verification task as provided by PAN organizing committee. Our present approach achieved the third best result in Dutch language with an AUC score of 0.75874 and C@1 score of 0.68283. We observe that our system didn’t perform well in other languages like English, Greek and Spanish and obtained AUC scores 0.60174, 0.6126, 0.6096 and C@1 scores 0.57732, 0.5824 and 0.5768, respectively. One possible reason behind this fluctuation in result might be because of the variable number and size of the ‘known’documents. Cross-topic and cross-genre texts appear to be more common in English and Spanish, which produced a visible decline in the performance. To deal with multigenre, we trained our system to analyze multiple corpuses/genre-specific training data and build the trainer as a whole. However, we had to modify our code as PAN committee decided to append time stamps automatically to the output folder and therefore the merging of the training sets was not possible.
In this paper, we have presented our approach to build software for automatic authorship verification using methods of Text Analytics. It uses same features for all the language datasets and a Random Forest classifier to classify the unknown documents based on the features extracted. In the recent years, the practical applications for authorship attribution have grown in areas such as intelligence (linking intercepted messages to each other and to known terrorists), criminal law, civil law and computer security (tracking authors of computer virus source code). This activity is part of computer science for identifying technologies, including biometrics, cryptographic signatures, intrusion detection systems and others.