Drama is a story told through the live actions of characters. The Drammar ontology [LDP18,DLPar] identifies the core dramatic qualities of a dramatic text, namely Actions, Agents, Units/Scenes, and Conflicts, implicitly evoked by the dramatic text, as claimed by the scientific literature on drama analysis.
Drama relies on an internal coherence and a rich set of eventualities, related to the interactions among characters and the insurgence and resolution of conflicts. Dramas are very well structured. As a running example, we address the incipit of Anton Chekhov's "The Cherry Orchard ", in its English translation [Che17]:
A room which is still called the nursery. One of the doors leads into ANYA'S room. It is close on sunrise. ... DUNYASHA comes in with a candle, and LOPAKHIN with a book in his hand.
DUNYASHA. It will soon be two. [Blows out candle] It is light already.
• individual utterances are denoted by the correspondent acting characters;
• some editorial notes (in italics) interleave with the spoken parts where the authors suggest the environment changes or specific happenings;
• a strict separation between spoken and editorial fragments is imposed.
Our current research objective aims at supporting the automatic annotation of a drama, able to outline the evolution of the dramatic elements above through discrete events.
Following the observer point of view, [SZR07] propose the following definitions for the events:
• An Event is "a segment of time at a given location, that is conceived by an observer to have a beginning and an end; granularity of events can go from a second or less to tens of minutes".
• An Event model is "an actively maintained representation of the current event, which is updated at perceptual event boundaries".
• The Event segmentation is "the perceptual and cognitive process by which a continuous activity is segmented into meaningful events". From the psychological literature, we have that readers structure a narrative text into a series of events in order to understand and remember the text (cf. the experiments of [ZT01, ZS07, ZSSM10]). Events are coded at clause level. Relevant information for the narrative coding includes, e.g. [SZR07]:
• Time and Space information (as the presence of spatial changes, e.g., moving from one room to another inside a house can be meaningful);
• Objects, given the interaction of characters with elements of a scene;
• change of Character, revealed by the changes of the subject of a clause;
• Causes (causal relationship over activities) and Goals (new goal-directed activities), to be coded as core dimensions of Events.
Usually, clauses are also coded for terminal punctuation (e.g., periods and question marks) and non terminal punctuation (e.g., commas and semicolons). As the annotation of such dramatic aspects is time consuming, we aim at automatizing it, relying upon the lexical, grammatical, and editorial information, expressed by individual clauses. In this way, events can be recognized and properly segmented along the dramatic text. We will refer hereafter this process as the event segmentation.
Event segmentation is a task traditionally tackled in NLP according to sentence boundary detection methods (e.g. [Hea94], [SDDK11]) or cohesion-based clustering models (e.g. [Cho00]). Text segmentation methods usually search for the set of segments in a text that optimize some form of coherence of the content. Word usage is modeled in TextTiling [Hea94] for each sentence in a sequence and the two sides of a potential boundary are selected when large lexical difference is found. Prosodic features and lexical features are taken into account to model discourse as in the Hidden Markov Model segmentation proposed in [YXX + 16]. The lexical connectivity strength between two adjacent fragments of a text is used as hint in DivSeg ( [SDDK11]). Unsupervised approaches are based on probabilistic models (e.g. [Hea94], C99 [Cho00] or the DotPlotting algorithm [Rey94]) or agglomerative clustering [Yaa99]). In the former group, terms frequencies are used to identify topical segments (dense dot clouds on the graphic). e.g. DotPlotting [Rey94]. In the latter group, dendrograms are induced over paragraphs and transformed into a hierarchical segmentation [Yaa99]. Lexical chains methods are applied in an unsupervised manner as they exploit semantic lexicons to model word associations and semantic relations. In these methods, a chain links multiple occurrences of a term in the document: it is considered broken when there are too many sentences between two occurrences of a term. The Segmenter system ([KKM98]) detects such broken points across a document according to possibly multiple chains. Some of the methods use lexical resources or forms of ontological similarity to model similarity metrics between text blocks (sentences or paragraphs), based on semantic information (e.g. recognized named entity in the text). Wordnet or Wikipedia-based methods have been proposed to define semantic similarity metrics between text units. Recently, deep learning methods have been applied to Text Segmentation, specifically to the Topic-based segmentation task. In particular [LSJ18] presents an end-to-end segmentation model: first, a bidirectional recurrent neural network is used to encode input text sequences, and then, another recurrent neural network is used together with a pointer network to select text boundaries in the input sequence. Although very appealing, since it does not require hand-crafted features definition, this method requires a significant amount of training material, made of several hundreds of annotated documents.
In line with most of the above event segmentation approaches, we will rely on a machine learning perspective by assuming a set of textual resources as the triggering observations:
• L is a set of fully annotated drama fragments, whose segments are completely known (e.g., the example units 4 to 6 in Checkov's The Cherry Orchard reported in the paper Appendix or the nunnery scene from Shakespeare's Hamlet [LPD16], respectively);
• O L is a very small scale corpus, made of fragments from the possibly partially annotated opera (e.g., the complete annotated dramas "The Cherry Orchard" or "Hamlet", respectively, though they are usually neither segmented, nor annotated);
• O A (L) is a large scale corpus of unannotated texts of the same author (e.g., all of Checkov's plays or Shakespeare's plays, respectively);
) is a comprehensive corpus of the drama works of the same epoch (e.g., Contemporary play or Elizabethan theatre plays, respectively).
So, we rely on the chain
We propose an integration of unsupervised and supervised learning processes acting: our first attempt is to use the comprehensive O E (A(L)) to generate a lexical resource focused on the work and author style: according to unsupervised methods, such as [MCCD13], we can rely on word embedding for a large scale dictionary of lexical items: these generalize lexical semantics within the underlying targeted text genre. The proposal is to inject this information into the supervised steps that address the labeled material L, in order to fully label the entire work O L in an accurate manner. Annotated examples in L are the basic source of information for the segmentation stage.
Hereafter, we concentrate on the variety of lexical, grammatical and aspectual features (e.g. the mode and transitivity of a number of verbs involved in the dramatic action), suitably exploited for training a sequence labeling component over O L . We propose a structured learning paradigm based on independent kernels for training SVMs over L ( [STC04]) and apply them within a Markovian modeling, isomorphic to HMM. The major steps are thus:
to acquire lexical information in the form of a neural language model (in line with [MCCD13]), expressing general semantic properties of individual words. A specific treatment of some classes of words is here applied. For example, character names (e.g. Dunyasha and Lopakhin in The Cherry Orchard or Hamlet and Ophelia in Hamlet). This is a standard a-priori information for a drama that is mapped into the category label Character, in order to minimize sparsity.
• (Feature Modeling and Extraction) Feature extraction is applied to derive textual, editorial and narrative features, as discussed in Section 2.2
• (Model Optimization) Then, a structured Machine Learning model is applied to achieve segmentation as an IOB-like sentence labeling process, in order to organize sentences in units and hierarchies of scenes. The adopted algorithm is known as SVM-HMM ([TJHA05], adopted in [CB11, BCV + 16]).
The aim of a Markovian formulation of SVM is to make the classification of a input example x i ∈ R n (belonging to a sequence of examples) dependent on the labels assigned to the previous elements in a history of length m, i.e., x i−m , . . . , x i−1 . In our classification task, a drama is a sequence of utterances x = (x 1 , . . . , x s ), each of them representing the example x i , i.e., the specific i-th paragraph. Given the corresponding sequence of expected labels y = (y 1 , . . . , y s ), a sequence of m step-specific labels (from a dictionary of d symbols) can be retrieved, in the form y i−m , . . . , y i−1 . In our machine learning setting, labels are related to the Segmentation task: we will thus adopt the IOB notation so that each element in the drama will be associated to the label B if it is at the Beginning of a Unit,
In order to make the classification of x i also dependent on the previous decisions, we augment the feature vector of x i by introducing a projection function ψ m (x i ) ∈ R md that associates each example with a md−dimensional feature vector where each dimension set to 1 corresponds to the presence of one of the d possible labels observed in a history of length m, i.e. m steps before the target element x i .
In order to apply a SVM, a projection function φ m (•) can be defined to consider both the observations x i and the transitions ψ m (x i ) by concatenating the two representations as follows:
Notice that the symbol || here denotes the vector concatenation, so that ψ m (x i ) does not interfere with the original feature space, where x i lies. Kernel-based methods can be applied in order to model meaningful representation spaces, encoding both the feature representing individual examples together with the information about the transitions. According to kernel-based learning [STC04], we can define a kernel function K m (x i , z j ) between a generic item of a sequence x i and another generic item z j from the same or a different sequence, parametric in the history length m. It surrogates the dot product between φ m (•) such that:
We define a kernel that is the linear combination of two further kernels: K obs operating over the individual examples x i and a K tr operating over the feature vectors encoding the involved transitions. It is worth noticing that K obs neither depends on the position nor on the context of individual examples, in line with Markov assumption that characterizes a large class of these generative models, e.g. HMM. For simplicity, we define K tr as a linear kernel between input instances, i.e. a dot-product in the space generated by ψ m (•):
At training time, we use the kernel-based SVM in a One-Vs-All schema over the feature space derived by K m (•, •). The learning process provides a family of classification functions f (x i ; m) ⊂ R n+md × R d , which associate each x i to a distribution of scores with respect to the different d labels, depending on the context size m. At classification time, all possible sequences y ∈ Y + should be considered in order to determine the best labeling ŷ, where m is the size of the history used to enrich x i , that is: ŷ = arg max y∈Y + { i=1...m f (x i ; m)}
In order to reduce the computational cost, a Viterbi-like decoding algorithm is adopted1 as described in Fig. 1. The next section defines the kernel function K obs applied to specific turns in the drama.
Three types of kernels are applied for different types of features. Lexical features include sentence embeddings as linear combinations of individual word embeddings, grammatical patterns, such as verb-objects or subject-verb pairs, POS n-grams (n=3) and, finally, sentence properties such as length and complexity (e.g. number of different active mode verbs). Narrative features are strictly dependent on the narrative structure and express possible Characters and Actions in a turn. Named Entity Recognition is first run on the individual utterances, to capture character mentions. A narrative vector including the acting character (e.g. LOPAKHIN in line 0036 or 0038 in the Appendix) as well as all the other recently mentioned characters (e.g. LOPAKHIN and DUNYASHA in the editorial note at line 0042). Individual features modeling the number of mentioned or recently mentioned characters for each turn will be adopted. An aging mechanism defines lower scores for no longer mentioned characters. Finally, narrative features denoting the Actions mentioned in a turn will be adopted in order to account for the interaction (and possible conflicts) in an explicit way. Examples are motion verbs such as to come, to go, social verbs, such as to meet (see LOPAKHIN in unit 0040) or even emotional verbs (e.g. or to faint as in unit 0041. Specific dictionaries of English verbs and their nominalizations will be used here to denote narratively interesting Actions. Editorial features will depend on the material that includes the author's suggestions in the environment (see, for example, the sentence "A room which is still called the nursery" in the incipit). In this case, a representation that is similar to the one for the lexical features for individual acting turns is adopted, but the editorial material will be expressed through a separated vector, in order to play an independent role. Table 1: Performance scores and ablation analysis for the segmentation based on different lexical features. Tokenbased Accuracy figures are Strict when applied only to B-labeled paragraphs in the oracle, and Greedy when also all the consistently aligned I-labeled paragraphs are considered as correct.
In the current experimental stage, we applied ablation analysis to the set of Lexical Features as described in the previous section. The lexical model is tested via the HMM SVM framework (implemented within KeLP, [FCM + 18]) on the annotated version of The Cherry Orchard, in its English translation: the Appendix reports a short excerpt. The work includes 4 acts made of about 904 paragraphs, segmented into 67 units. The different tokens in our labeled corpus L = O L are thus about 22,800. Every paragraph has been considered part of a sequence of length k=5 that corresponds to the local input to the tagger. Every paragraph in the sequence is represented via feature vectors and several lexical representations have been adopted:
• (Simple Lexical ) A bag-of-word feature vector including lemmas, bi-grams and POS tags occurring in the target paragraph are represented.
• (Baseline) As a baseline, a set of simple heuristics from narrative features is used to simulate a blind typographic approach. The synthetic vector encodes only a label with the guessed editorial role of the paragraph. In this way, individual utterance and editorial notes are just kept separated.
• (Contextualized lexical ) Similar to the Lex feature vector, but extended with the vector of the preceding paragraph, in order to contextualize the model.
• (Word Embeddings) A real-valued vector that corresponds to the sentence embedding of the target paragraph is adopted. Each training paragraph is labeled according to the IOB notation, i.e. as B or I. A macro n-fold cross validation is applied with one fold per act, i.e n = 4. In one evaluation step, one act is removed from the dataset: training on the three remaining acts is carried out by leaving a 10% of paragraphs as development data (i.e. for tuning of the SVM parameters): the automatic tagging over the left-out act allows to measure and average the labeling accuracy. Each time, the training set is randomly split 90/10 to derive a development set used to tune the SVM parameters.
Measures of performance are class-based precision and recall, while accuracy is the percentage of paragraphs that are correctly re-labeled with respect to the original IOB label. Micro-average across the four different 4 folds is applied. Notice that for the unbalanced presence of the I tag (i.e. 92.6% of the paragraphs), the simple baseline model achieved a 93.5% of accuracy across all paragraphs. For this reason, in Table 1, we just report precision, recall and F1 for the two separated classes. Moreover, we report the strict accuracy, just computing the accuracy restricted to the B gold labeled paragraph. Notice that this class is defined by only 67 positive examples in the training dataset. Finally, the accuracy measured on only the aligned B and I paragraph is reported: it considers correct a paragraph labeled as "inner" by the system only when this does not violate any boundary B in the oracle annotation. As Table 1 shows more complex lexical features brings more information as they increase performance for each measure. Moreover, (last column in last line) the token based accuracy suggests that the current model correctly annotate about 70% of the paragraphs of the work, thus representing a large advantage against manual annotation.
Examples of mistaken segmentations are hereafter reported where the gold and automatic labels are shown after the row number for the different paragraphs, respectively: ... According to the gold labels, the B-labeled paragraphs in lines 803 and 903 are all wrong, while 801,804 and 901,902,905 are correctly aligned I-labeled paragraphs: these latter are retained in the greedy version of the Token-based Accuracy scores. Notice how the mistakes are mainly due to mismatches in the way editorial material is used by the human annotators. In the first example, lines 802, 803, the beginning is annotated to the sitting act of YASHA (line 802). In the second, the first speech of LOPAKHIN is used to start a new segment (line 904). On the contrary, in both cases the system has focused on the entrance of the new character to suggest the start (i.e. B labels in line 803 and 903).
These mild errors suggest how the generalization of the system at this current stage of development is already acceptable in several cases. Accuracy rates are thus expected to grow when more complex features (for example the narrative features that will better express the ontological information) will be adopted. This will be part of future work.