This paper deals with the sub-task of PAN 2017 Author Identification, which is to detect style breaches for unknown number of authors within a single document in English. The presented model is an unsupervised approach that will detect style breaches and mark text boundaries on the basis of different stylistic features. This model will use some classical stylistic features like POS analysis and sentence lexical analysis. Also some new features naming common English word frequencies within sentence text, sentence expression and sentence attitude have been proposed. The new features may not be directly linked to author's style of writing but to the subject/topic of sentence under analysis. Moreover the model uses sentence window for style detection. The sentence window may be extended to neighboring sentences during its unsupervised analysis.
Stylometry is an important tool in the field of digital text forensics, especially in
cases where we have unidentified or dubious text documents [
Stylometric approaches generally achieve higher accuracy for long documents [
One way of using stylometric approach in case of author attribution and author
profiling is by training the computer applications over specific writing style of some
specific author in a number of documents. But as discussed above the task of
detecting style breaches within a document without knowing in advance about the
exact number of authors is difficult task and also an objective for ongoing research.
Detection of style breach is related to text segmentation where text boundaries are
marked with detection in change of topics [
The presented model uses unsupervised classification approach to detect and mark
passage boundaries in given documents on the basis of style breaches. A combination
of well-known stylometric features like Syntactic, Lexical and content specific
features [
Also this model is a good example of how a text as small as a sentence within a document may be helpful in finding its related sentences on the basis of stylometric and other parameters to help us figure-out the passage boundaries by unknown number of authors.
The training dataset of PAN at CLEF 2017 [
The presented model uses different types of classical stylometric methods along with some new methods in order to find text borders where style breach is identified. The system used sentences as text segmentation unit. The sentence window keeps extending over its neighboring sentences until style breach is detected. Following are the methodology steps used by the system in order to find out style breaches.
Words lists preparation Text segmentation into sentences Sentence window based syntactic analysis Sentence window based lexical analysis Content based analysis of sentence window Sentence window expression labeling Sentence window attitude labeling
Style breach calculation
Different types of lists of words were prepared from different internet sources [
These lists became the part of model and will be used for labeling of sentences in next methodology steps.
Each individual document D in the repository was segmented into sentences , , , ,…. . A simple algorithm was used to break a document into array of sentences. It first traverse through each character of document D from start until the any of the two characters ‘.’ or ‘?’ are encountered, which indicates sentence endings. The sentence is extracted and the algorithm continues from next character as start of next sentence.
D = + + + + …. + (1)
Where i is the starting index of each sentence and n is the number of total sentences in D. The first three sentences of any document D will be the starting window (j = 1) for initializing point that may or may not extend and merge with next adjacent sentence windows (two at a time) depending on further analysis, also the adjacent sentence windows will also share boundary sentence as shown in equation 2 and 3.
The sentence is common boundary sentence in first and second windows and . This common sentence among two adjacent windows will increase the similarity index when comparing both windows for a possible merger/extension.
As discussed above n is the total number of sentences in any document and each sentence window W can have only three sentences in start (as shown in equations 2 and 3); hence the maximum number of text windows in any document will be as shown in equation 4. (2) (3) (4)
Let’s consider for an example j = 1, so first two sentence windows and chosen for further analysis. The next steps performed by model are as follows. are = = + +
+
Max. Windows (m) =
windows is converted to its respective part of speech (POS) tags for each word present in texts as shown in table 4.
a lexical analysis for both text windows. In this analysis following features are extracted: window is extracted e.g.
in the text = „e‟ in both text windows in shown table 4.
text window is extracted e.g. , . =„,‟ in both text windows . in the and
Most frequent word in the text window is extracted where i in equation below is the index of word w e.g. = “in” and = “of” in both text windows respectively as mentioned in table 4. The frequency of each word is calculated as shown in equation 5.
∑ (5)
shown in equation 6. is calculated for each text window as
) =
of each window is calculated using the list L of 5000 common words. Let be a common word existing in both L and any text window where i specifies the index (i = 1… 5000) in L in eq. 7.
Where k is the total number of coexisting words in both L and , and be the frequency of in , is the frequency of in list L (as shown in table 3) and l is the total number of words in .
Next two steps can be considered as sub-steps of Content based analysis. 4. Sentence Window Expression Labeling: The model will label each window with a specific feeling or human mood expression . Let i is the index (i = 1… 7) of expression list as shown in table 1, Let be a coexisting word in both and text window where m specifies the index in . Expression score is measured on the basis of following equation.
Where k is the total number of coexisting words in both and , and be the frequency of in . After calculating all seven expression scores the model will calculate e through following equation. √ ∑
In cases where two or more expression scores are equal, or all expression scores are zero, the model will assign a “neutral” expression for window . 5. Sentence Window Attitude Labeling: The model will label each window with a specific attitude or human behavior . Let i is the index (i = 1… 2) of attitude list as shown in table 2, Let be a coexisting word in both and text window where m specifies the index in . Attitude score is measured on the basis of following equation.
Where k is the total number of coexisting words in both and , and be the frequency of in . After calculating both positive and negative attitude scores the model will calculate a through following equation. for
In case both scores are equal or zero, the model will assign a neutral attitude e.g. both and have neutral attitude. 6. Style Breach Calculation: After computing above mentioned stylistic and other attributes we get two result sets naming , and two matrices and for text windows and respectively { { [ [ ] ] (11) } } (14) (15) (16) (12) (13)
The system will now measure stylistic similarity score equations as shown in following
Where, for each x in equation 15, the similarity score is incremented accordingly. and are treated separately as matrices because these two contains decimal values. A matrix subtraction is applied to and (17)
If cr and ci lie within a threshold range described in next section, then similarity score is incremented accordingly. Finally, it’s time to decide whether or not to merge and on the basis of value of lies within a threshold range described in next section. At this point two cases will emerge:
In this case and are considered merged, and a new resultant window will be created where r is the index of resultant window. The model will continue from step 1 of methodology for sentence and .
= + + + + (18) will keep expanding until case-1 keeps occurring and this resultant window will reflect a single style for all sentences contained within.
Case-2:
does not lie within a threshold range
In this case the coexisting sentence in both adjacent windows will stay either in window or in e.g. let’s assume in equations 2 and 3.
1. will become a separate single sentence window . 2. Stylistic score is calculated for following same methodology steps and its distance from both and is calculated. 3. may remain in either of the two sentence windows depending on the distance value calculated. 4. If remains in then will be restructured for next consecutive sentences as shown below. 5. If remains in then
will be restructured as shown below. =
+ = + (19) (20)
After the style breach detection among first two consecutive sentence windows, new windows and will be compared starting from step 1 of methodology.
In the end we have a set of resultant windows known as R = where m is the maximum number of sentence windows and each in R is considered a breach detection.
A number of experiments were carried out in order to adjust the threshold values
and for which the final F-Measure score was highest. Once the values were
adjusted over the training dataset, the system was ready to run for test dataset
provided at TIRA [
Following are the evaluator results shown in table 5.
The results were improved for the final test dataset, however the model precision remained low from recall and that affected the final F-Measure score, which shows that more experiments over different data sources for adjusting threshold values may be required.
In this paper an unsupervised model for the detection of style breach is presented, this research field is rather new and more difficult to implement because non availability of any external resources for reference and also we only have to rely on stylistic attributes of unknown number authors that may or may not have contributed in the creation of text document under inquiry, hence this model presents new directions or ways i.e. Expression and Attitude labeling of textual windows in order to find style breach within sentences without the pre-assumption of authors style of writing and relying more on text content. In future the results can be improved with discovery of more text labels or with the addition of more expression lists and reduction of conventional stylistic approaches, this model can hence be applied to other languages as well.