This paper deals with the sub-task of PAN 2017 Author Identification, which is to detect style breaches for unknown number of authors within a single document in English. The presented model is an unsupervised approach that will detect style breaches and mark text boundaries on the basis of different stylistic features. This model will use some classical stylistic features like POS analysis and sentence lexical analysis. Also some new features naming common English word frequencies within sentence text, sentence expression and sentence attitude have been proposed. The new features may not be directly linked to author's style of writing but to the subject/topic of sentence under analysis. Moreover the model uses sentence window for style detection. The sentence window may be extended to neighboring sentences during its unsupervised analysis.
Stylometry is an important tool in the field of digital text forensics, especially in cases where we have unidentified or dubious text documents [1] written by one or more authors. These documents do not have an external link, tool or repository to prove that which text passage relates to which author. In other words, we use stylometric approaches when we may have to ascertain if the acclaimed authorship of text document actually exists in circumstances where we do not have any external verification resources.
Stylometric approaches generally achieve higher accuracy for long documents [2] because longer documents contain more text to reveal stylistic features of authors like in the field of Intrinsic Plagiarism detection problem solving [3,4]. But in cases of short documents or texts e.g. in cases of social media like twitter where there may be fewer sentences by each author, Stylometric approaches my not get more accurate results. Although much work has been done in cases of scam emails [5], cyber-crimes [6] and fake service provision reviews [7] using Stylometric models.
One way of using stylometric approach in case of author attribution and author profiling is by training the computer applications over specific writing style of some specific author in a number of documents. But as discussed above the task of detecting style breaches within a document without knowing in advance about the exact number of authors is difficult task and also an objective for ongoing research. Detection of style breach is related to text segmentation where text boundaries are marked with detection in change of topics [8].
The presented model uses unsupervised classification approach to detect and mark passage boundaries in given documents on the basis of style breaches. A combination of well-known stylometric features like Syntactic, Lexical and content specific features [9] are used with features like ordinary words frequency, sentence expression and sentence attitude that may be related to textual topic specification and may not be directly related to author's style. But this approach may be very handy in cases where we want to relate one sentence to its neighboring sentences and thus detect exact passage boundaries within a given document.
Also this model is a good example of how a text as small as a sentence within a document may be helpful in finding its related sentences on the basis of stylometric and other parameters to help us figure-out the passage boundaries by unknown number of authors.
The training dataset of PAN at CLEF 2017 [8] for the task of style breach detection under main task of author identification. The dataset contained about 187 English text documents of different lengths and sizes over different topics like biography, politics, travel, hotels etc. Along with each text document a truth file was provided which contained exact character positions indicating style breach occurrences within that document, topic of document however remains unchanged.
The presented model uses different types of classical stylometric methods along with some new methods in order to find text borders where style breach is identified. The system used sentences as text segmentation unit. The sentence window keeps extending over its neighboring sentences until style breach is detected. Following are the methodology steps used by the system in order to find out style breaches.
Different types of lists of words were prepared from different internet sources [10,11,12,13] that express specific moods or human feelings. Seven expression lists of words were used including anger, confusion, curiosity, urgency, satisfaction, inspiration and happiness; where all lists comprised of about 200 words each. One reason for choosing only these seven expressions was the availability of proper expressive words over internet sources for these expressions. The second reason was to use limited set of expressions that may express human feelings while writing some text. More expressions may be included for future research. Two words additional words lists of about 500 words each of which reflecting positive or negative attitudes [14, 15] were included. An example of these expressive and attitude lists is shown in table 1 and table 2
These lists became the part of model and will be used for labeling of sentences in next methodology steps.
Each individual document D in the repository was segmented into sentences , , , ,…. . A simple algorithm was used to break a document into array of sentences. It first traverse through each character of document D from start until the any of the two characters '.' or '?' are encountered, which indicates sentence endings. The sentence is extracted and the algorithm continues from next character as start of next sentence.
Where i is the starting index of each sentence and n is the number of total sentences in D. The first three sentences of any document D will be the starting window (j = 1) for initializing point that may or may not extend and merge with next adjacent sentence windows (two at a time) depending on further analysis, also the adjacent sentence windows will also share boundary sentence as shown in equation 2 and 3.
The sentence is common boundary sentence in first and second windows and . This common sentence among two adjacent windows will increase the similarity index when comparing both windows for a possible merger/extension.
As discussed above n is the total number of sentences in any document and each sentence window W can have only three sentences in start (as shown in equations 2 and 3); hence the maximum number of text windows in any document will be as shown in equation 4.
Let's consider for an example j = 1, so first two sentence windows and are chosen for further analysis. The next steps performed by model are as follows.
windows is converted to its respective part of speech (POS) tags for each word present in texts as shown in table 4.
At this step, the model performs a lexical analysis for both text windows. In this analysis following features are extracted:
in the text window is extracted e.g.
= "e" in both text windows in shown table 4.
Most frequent non-alphanumeric and non-space character ( in the text window is extracted e.g. , .
="," in both text windows and .
in the text window is extracted where i in equation below is the index of word w e.g.
= "in" and = "of" in both text windows respectively as mentioned in table 4. The frequency of each word is calculated as shown in equation 5.
Word Frequency (
Character to Space Ratio is calculated for each text window as shown in equation 6.
Character to Space Ratio ( ) = (6)
of each window is calculated using the list L of 5000 common words. Let be a common word existing in both L and any text window where i specifies the index (i = 1… 5000) in L in eq. 7.
√ ∑ (7) Where k is the total number of coexisting words in both L and , and be the frequency of in , is the frequency of in list L (as shown in table 3) and l is the total number of words in .
Next two steps can be considered as sub-steps of Content based analysis.
The model will label each window with a specific feeling or human mood expression . Let i is the index (i = 1… 7) of expression list as shown in table 1, Let be a coexisting word in both and text window where m specifies the index in . Expression score is measured on the basis of following equation.
Where k is the total number of coexisting words in both and , and be the frequency of in . After calculating all seven expression scores the model will calculate e through following equation. (9) In cases where two or more expression scores are equal, or all expression scores are zero, the model will assign a "neutral" expression for window .
The model will label each window with a specific attitude or human behavior . Let i is the index (i = 1… 2) of attitude list as shown in table 2, Let be a coexisting word in both and text window where m specifies the index in . Attitude score is measured on the basis of following equation.
Where k is the total number of coexisting words in both and , and be the frequency of in . After calculating both positive and negative attitude scores the model will calculate a through following equation.
(11)
In case both scores are equal or zero, the model will assign a neutral attitude for e.g. both and have neutral attitude.
The system will now measure stylistic similarity score as shown in following equations (16) Where, for each x in equation 15, the similarity score is incremented accordingly. and are treated separately as matrices because these two contains decimal values. A matrix subtraction is applied to and
If cr and ci lie within a threshold range described in next section, then similarity score is incremented accordingly. Finally, it's time to decide whether or not to merge and on the basis of value of lies within a threshold range described in next section. At this point two cases will emerge:
Case-1: lies within a threshold range In this case and are considered merged, and a new resultant window will be created where r is the index of resultant window. The model will continue from step 1 of methodology for sentence and .
will keep expanding until case-1 keeps occurring and this resultant window will reflect a single style for all sentences contained within.
In this case the coexisting sentence in both adjacent windows will stay either in window or in e.g. let's assume in equations 2 and 3. 1.
will become a separate single sentence window .
After the style breach detection among first two consecutive sentence windows, new windows and will be compared starting from step 1 of methodology.
In the end we have a set of resultant windows known as R = where m is the maximum number of sentence windows and each in R is considered a breach detection.
A number of experiments were carried out in order to adjust the threshold values and for which the final F-Measure score was highest. Once the values were adjusted over the training dataset, the system was ready to run for test dataset provided at TIRA [17] in order to detect style breaches.
Following are the evaluator results shown in table 5. The results were improved for the final test dataset, however the model precision remained low from recall and that affected the final F-Measure score, which shows that more experiments over different data sources for adjusting threshold values may be required.
In this paper an unsupervised model for the detection of style breach is presented, this research field is rather new and more difficult to implement because non availability of any external resources for reference and also we only have to rely on stylistic attributes of unknown number authors that may or may not have contributed in the creation of text document under inquiry, hence this model presents new directions or ways i.e. Expression and Attitude labeling of textual windows in order to find style breach within sentences without the pre-assumption of authors style of writing and relying more on text content. In future the results can be improved with discovery of more text labels or with the addition of more expression lists and reduction of conventional stylistic approaches, this model can hence be applied to other languages as well.