Moving from the assumption that formal, rather than content features, can be used to detect differences and similarities among textual genres and registers, this paper presents a new approach to the linguistic profiling methodology, which focuses on the internal parts of a text. A case study is presented showing that it is possible to model the degree of variance within texts representative of four traditional genres and two levels of complexity for each.1
The combined use of corpus-based and
computational linguistics methods to investigate language
variation has become an established line of
research. The heart of this research is the so-called
‘linguistic profiling’, a technique in which a large
number of counts of linguistic features
automatically extracted from parsed corpora are used as
a text profile and can then be compared to
average profiles for groups of texts
This paper moves in this framework but
presents a new approach of linguistic profiling,
in which the unit of analysis is not the document
as a whole entity, but the internal parts in which
it is articulated. In this respect, our perspective
is similar to the one proposed by
Our investigation was carried out on four genres:
Journalism, Educational writing, Scientific prose
and Narrative. For each genre, we selected the
two corpora described in Brunato and Dell’Orletta
(2017), which represent a ‘complex’ and a
‘simple’ language variety for that genre, where the
level of complexity was established according to
the expected reader. Specifically, the journalistic
genre comprises a corpus of articles published
between 2000 and 2005 on the general newspaper
La Repubblica and a corpus of easy-to-read
articles from Due Parole, a monthly magazine
written in a controlled language for readers with
basic literacy skills or mild intellectual disabilities
All corpora were automatically tagged by the
part-of-speech tagger described in Dell’Orletta
(2009) and dependency parsed by the DeSR parser
As a preliminary step for the analyses, all
documents were split into a fixed number of
sections, where each section is composed by a
certain number of paragraphs, roughly corresponding
to the three main parts of the rhetorical structure
of a text (i.e. introductory, body and
concluding paragraphs). According to the literature, for
some genres, such as academic writing, the
distinction into paragraphs is quite rigid and follows
the so-called ‘five-paragraphs’ format
In order to understand whether and to what extent the different parts of a text represent distinctive varieties with a peculiar linguistic structure, we carried out two statistical analyses. First, we assessed whether the difference of the feature values in each section was statistically significant. Specifically, we performed a pairwise comparison between each section and the following one (i.e. 1/2, 2/3, 3/4 etc), as well as between the first and the last section (i.e. 1/6); the latter was deliberately aimed at verifying whether our set of features alone is able to distinguish between the introductory and the closing part of a document, the two more distant sections of a text which are supposed to have a more codified structure. Secondly, we verified whether there is a correlation between the values of features in the two sections under comparison. For both analyses, all data were calculated across and within genre. The cross-genre analysis was focused on genre only, thus considering the two corpora representative of the complex and simple variety as a unique one for each genre. In the second scenario, the two corpora were kept distinct to investigate if there is an effect of genre that is preserved despite language complexity changes. 3
The set of features extracted from previously identified sections are distinguished into three different categories, according to the level of annotation from which they derive.
Raw Text Features: they include the average word and sentence length (char tok and n tokens in Table 2), calculated as the number of characters per token and of tokens per sentence, respectively.
of unigrams of part-of-speech distinct into 14 coarse-grained pos tags (cpos ) and the 37 finegrained tags (pos ) according to the ISST-TANL annotation.
Syntactic Features: these features model grammatical phenomena of different types, i.e: - the probability of syntactic dependency types e.g. subject (dep subj), direct object (dep dobj), modifiers, calculated as the distribution of each type out of the total dependency types according to the ISST-TANL dependency tagset; - the length of dependency links, i.e. the average length of all dependency links (each one calculated as the number of words occurring between the syntactic head and the dependent) (avg links l) and of the maximum dependency link (max links l); - the order of constituents with respect to the syntactic head: as a proxy of canonicity effects, it is calculated the relative position of the subject, object and adverb with respect to the verbal head and the position of the adjective with respect to the nominal head; - the parse tree structure, in terms of features calculating: the depth of the whole parse tree (sent depth) (in terms of the longest path from the root of the dependency tree to some leaf); the width of the parse tree (sent width), measured as the highest number of nodes placed on the same level; the average number of dependents for all verbal and nominal heads (avg dependent); - subordination features: within the group of syntactic features, a in–depth analysis was devoted to model subordination phenomena by measuring: the average distribution of subordinate clauses for sentence (avg sub clause), the percentage of subordinate clauses with respect to the main clause (% sub main) and the percentage of embedded subordinate clauses, i.e. subordinate clauses dependent on other embedded subordinate clauses (% sub minor); for each type, it is also calculated the average depth (subord depth) and weight (subord width) of the parse tree generated by the subordinate clauses and their relative order with respect to the clause on which they depend. 4
Table 2 illustrates the main findings we obtained.
Specifically, it shows all features which turned out to have a statistically significant variation in at least one of the six pairwise comparisons, or a correlation score > 0.3 according to the Spearman’s correlation coefficient. A first clear result is that the higher number of features varying in a statistically significant way occurs in the journalistic and scientific genre, both considered as whole (i.e. row g for each feature) and with respect to the language complexity variety (rows s and c). The opposite trend is reported for educational texts, which is probably due to the heterogeneous nature of this genre that includes documents of different textual typologies (course books, pieces of literature etc.).
If journalism and scientific prose are the two
genres with the highest internal variance, the
comparison between sections allows us to get a better
understanding of this data. Specifically, for both
genres, the majority of significant variations are
observed between the first and the second section
and between the first and the last one. This
suggests that the introduction is a stylistic unit with
a peculiar linguistic structure with respect to the
body and the conclusion. It is characterized e.g.
by shorter sentences (Figure 1), likely due to the
presence of the title in both newspaper and
scientific articles, and by a distinctive distribution
of Parts–of–speech (Figure 2). With this respect,
this data are consistent with other studies in the
literature, e.g.
Leaving aside the similar tendencies characterizing the introduction, Journalistic and Scientific prose show a different behavior when we focus on the internal structure of text. While in this case much fewer features vary in a significant way, the majority occurs in the journalistic genre only, especially between the second and the third section. Again, they concern a different distribution of morpho-syntactic categories but also some syntactic features related to subordination. According to these data, we can conclude that the journalistic genre has a more rigorous structure and that it is possible to capture the boundaries between different parts by using linguistic features that are not related to the content of the article. features
Rawtextfeatures g XX X - - - XX XX - - - - XX - - - - - X - - - - - ntokens s XX - - - - - XX - - - - XX - - - - - X - - - - X c XX - - - - XX - - - - - - - - - - - X - - - - - g - - - - - X X - - - - X - - - - - - - - - - - chartok s - - - - - XX X - - - - XX - - - - - - X - - - - c - - - - - - - X - - - - - - - - - - - - - - -
In this paper we have presented a novel approach to the study of language variation, which relies on the prerequisites of the linguistic profiling methodology but with the specific purpose of modeling the stylistic form of the different parts within a text. A cross-genre investigation on four traditional genres in Italian, and two levels of complexity for each, showed that morpho-syntactic and syntactic features are differently distributed across subsections of texts belonging to a specific genre and language variety. This approach has important implications for research on genre variation since it suggests that the characterization of texts and texts varieties should benefit by inspecting corpora from this fine-grained perspective. A better understanding of linguistic phenomena characterizing the introductory, middle and conclusive parts of a text is also highly relevant not only to enhance automatic genre classification but also for other natural language processing applications devoted to modeling style: e.g. in education, as a component of intelligent tutoring systems able to provide detailed feedback to students in writing courses or for the automatic generation of texts with the stylistic properties of a specific genre and level of complexity.
This work was partially supported by the 2year project ADA, Automatic Data and documents Analysis to enhance human-based processes, funded by Regione Toscana (BANDO POR FESR 2014-2020).