English. The inclusion of semantic features in the stylometric analysis of literary texts appears to be poorly investigated. In this work, we experiment with the application of Distributional Semantics to a corpus of Italian literature to test if words distribution can convey stylistic cues. To verify our hypothesis, we have set up an Authorship Attribution experiment. Indeed, the results we have obtained suggest that the style of an author can reveal itself through words distribution too.
Stylometry, that is the application of the study of
linguistic style, offers a means of capturing the
elusive character of an author’s style by
quantifying some of its features. The basic stylometric
assumption is that each writer has certain
stylistic idiosyncrasies (a “human stylome”
One of the less investigated stylistic feature is the way in which authors use words from a semantic point of view, e.g. if they tend to use more, when dealing with polysemous words, a certain sense over the others, or senses that differ (even slightly) from the one that’s more commonly used (as it happens, typically, in poetry).
A possible approach to the analysis of this
characteristic is to consider the textual contexts in
which certain words appear. According to
Distributional Semantics (DS), certain aspects of the
meaning of lexical expressions depend on the
distributional properties of such expressions, or
better, on the contexts in which they are observed
In this work we would like to investigate if the analysis of the distribution of words in a text can be exploited to provide a stylistic cue. In order to inspect that, we have experimented with the application of DS to the stylometric analysis of literary texts belonging to a corpus constituted by texts pertaining to the work of six Italian writers of the late nineteenth century.
In the following, Section 2 gives a short insight on the state of the art of computational stylistic analysis, Section 3 describes the approach together with the corpus used to conduct our investigation and Section 4 discuss about results. Finally, Section 5 draws some conclusions and outlines some possible future works. 2
The very first attempts to analyze the style of an
author were based on simple lexical features such
as sentence length counts and word length counts,
since they can be applied to any language and
any corpus with no additional requirements
More complicated tasks such as full syntactic
parsing, semantic analysis, or pragmatic
analysis cannot yet be handled adequately by current
NLP technologies for unrestricted text. As a
result, very few attempts have been made to exploit
high-level features for stylometric purposes.
Perhaps the most important method of exploiting
semantic information so far was described in
The previously described features are
application independent since they can be extracted from
any textual data. Beyond that, one can define
application-specific measures in order to better
represent the nuances of style in a given text
domain (such as e-mail messages, or online forum
messages)
To the best of our knowledge, the application of
DS to the analysis of literary texts has been
documented in a rather small number of works
First, we want to specify that it is not our purpose to propose new ways to improve state-of-the-art AA algorithms. Indeed, our aim is just to verify the hypothesis that the distribution of words can provide an indication of a distributional stylistic fingerprint of an author. To do this, we have set up a simple classification task. Subsection 3.1 briefly depicts the data set we used, while Section 3.2 describes the steps implemented in our experiment. 3.1
In order to build the reference and test corpora, we started from texts pertaining to the work of six Italian writers working at the turn of the 20th century, namely, Luigi Capuana, Federico De Roberto, Luigi Pirandello, Italo Svevo, Federigo Tozzi and Giovanni Verga. We chose contiguous authors in chronological sense, whose texts are available in digital format (in fact we could not do a similar survey on the narrative of the 90s because it is still under copyrights). Indeed, we used texts freely available for download from the digital library of the Manunzio project, via the LiberLiber website1. Since they were encoded in various formats, such as .epub, .odt and .txt, our pre-processing consisted in converting them all in .txt format and getting rid of all xml tags, together with footnotes and editors’ notes and comments. 3.2
According to Rudman (1997), a striking problem in stylometry is due to the lack of homogeneity of the examined corpora, in particular to the improper selection or fragmentation of the texts, that might cause alterations in the writers’ style. In order to create balanced reference corpora, i.e. covering all the authors’ different stylistic and thematic phases, for each author, as shown in Figure 1, we built a reference corpus as the composition of the 70% of each single work (usually a novel). The same technique was used to create the 1http://www.liberliber.it/ test corpus by using the remaining 30% of each work. Typical AA approaches consist in analyzing known authors and assigning authorship to previously unseen text on the basis of various features. Train and test sets should then contain different texts. Contrary to the classical AA task, our train and test sets contain different parts of the same texts. Indeed, with this experiment, we wanted to understand if the semantics that an author bestows to a word, is peculiar to his writing. To prove this, we wanted to cover all the different stylistic and thematic phases an author can go through during his activity, hence the partition of all his texts in a reference and a test portion.
We then analyzed each reference and test
corpora with a Part-of-Speech (PoS) tagger and a
lemmatizer for Italian
3We performed different empiric setup of the window’s size and chose the one that showed more suitable results, according to what is stated by Kruszewski and Baroni (2014). tween the vectors representing the two words of each pair. This allowed us to evaluate the semantic relatedness between the words by assessing their proximity in the distributional space as represented by the cosine value: the more this value tends to 1, the more the two words of the pair are considered to be related. We then obtained two related word pair (RWP) lists for each author A: RWPrefA and RWPtestA. Figure 1 shows the process described above.
Since we wanted to focus on the analysis of the semantic distribution of words, we decided to exclude any possible “lexical bias”. For this reason, we restricted the analysis on a common vocabulary, i.e. a vocabulary constituted by the intersection of the six authors’ vocabularies. In this way, we prevent our classifier to exploit, as a feature, the presence of words used by some (but not all) of the authors. Moreover, we removed from the RWPtest lists all those pairs of words occurring frequently together in the same context, since they might constitute a multiword expression that, once again, could be pertaining with the signature lexicon of each author. To remove them, we computed the number of times (#co-occ in Table 1) they appeared together in the context window, as well as their total number of occurrences (#occa and #occb) and we excluded from the analysis those pairs for which the ratio between the number of co-occurrences and the total occurrences of the less frequent word was higher than the empirically set threshold of 0.5. The first two pairs of Table 1 would be removed as probable multiword (PM column in Table 1): “scoppio” (burst) and “risa” (laughter) could mostly co-occur in “scoppio di risa” (meaning “burst of laughter”) and the words “man” and “mano” (both meaning “hand”) could mostly co-occur in “man mano” (meaning “little by little”, or “progressively”).
Wa Wb scoppio–s risa–s man–n mano–n nausea–n disgusto–n piccolo–a grande–a #occa #occb #co-occ ratio PM 19 9 7 0.78 yes 50 1325 47 0.94 yes 27 26 0 0 no 248 237 14 0.06 no
Finally, we reduced the size of the six RWPref and RWPtest lists by sorting them in decreasing order of the cosine value and then by keeping the pairs with the highest cosine, selected using a percentage parameter as threshold4. We chose to introduce the parameter for two reasons: i) to avoid the classification algorithm to be disturbed by noisy (i.e. not significative) pairs which would not hold any relevant stylistic cue, and ii) to ease a literary scholar in the interpretation of the results by having to analyze just a limited selection of (potentially) semantically related word pairs.
For the last phase of our experiment we defined a classification algorithm to test the effective presence of stylistic cues inside the obtained RWPtest lists. We defined a classifier using a nearest-cosine method to attribute each test list to an author. The method consisted in searching for a pair of words contained in the test list inside each reference list and incrementing by 1 the score of the author whose reference list included the pair with the more similar cosine value (i.e. having the minimum difference): the chosen author was the one with the highest score. Table 2 shows the classification results for = 5%.
Capuana
Pirandello Svevo Tozzi Verga De Roberto De Roberto De Roberto De Roberto De Roberto Pirandello
Pirandello
Pirandello
Pirandello
Pirandello Tozzi/Verga Tozzi Svevo Verga
Svevo Verga As summarized in Table 3, a correct classification of all RWPs in RWPtest lists has been obtained with a value of 5%.
To help in interpreting the failure of the algorithm in classifying Tozzi’s test list for values lower than 5% (as shown in Table 3) we calculated the cardinality of the RWPtest lists for each author with the change in value (Tables 4).
It is possible to observe how the choice of influences the correct classification of Tozzi’s test list. Indeed, the use of a value below 5% has the effect of remarkably reducing an already small 4At the following url we have uploaded an archive containing all the data we have used and processed for our experiment: https://goo.gl/nrTqWh Svevo Verga
Verga #RWPtestCapuana #RWPtestDe Roberto #RWPtestPirandello #RWPtestSvevo #RWPtestTozzi #RWPtestVerga
Svevo Verga Verga 488 425 246 test list (RWPtextTozzi) as shown in Table 4. It is apparent that increasing the value of and consequently the number of significant RW pairs that are analysed, the system is able to correctly classify RWPtestTozzi (see the values in Tozzi’s row of Table 3). 5
In this paper we investigated the possibility that an analysis of the semantic distribution of words in a text can be potentially exploited to get cues about the style of an author. In order to validate our hypothesis, we conducted a first experiment on six different Italian authors. The results seem to suggest that the way words are distributed across a text, can provide a valid stylistic cue to distinguish an author’s work. Of course, it is not our intent, with this paper, to define new methods for enhancing state-of-the-art authorship attribution algorithms. Our research will focus, in the next steps, in detecting and providing useful indications about the style of an author. This can be done by highlighting, for example, atypical distributions of words (e.g. with contrastive methods) or by analysing their distributional variability. Furthermore, it could be interesting to use a different distributional measure, than the cosine, to test our hypothesis.