The Italian Sign Language (LIS) is the natural language used by the Italian Deaf community. This paper discusses the application of the Universal Dependencies (UD) format to the syntactic annotation of a LIS corpus. This investigation aims in particular at contributing to sign language research by addressing the challenges that the visual-manual modality of LIS creates generally in linguistic annotation and specifically in segmentation and syntactic analysis. We addressed two case studies from the storytelling domain first segmented on the ELAN platform, and second syntactically annotated using CoNLLU format.
The Italian Sign Language (LIS) is the natural
language used by the Italian Deaf community. Signed
languages have been extensively studied in the last
years
Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). this work is the Universal Dependencies formalism1 (UD), a de facto standard for syntactic annotation.
The main goals of this research are three. The
first goal is theoretical. We want to investigate the
expressiveness of UD tags and its relations with
LIS. Signed languages have peculiar forms of
lexicons and syntax and we want to experimentally
verify the expressive power of the UD formalism
in representing this richness. The second goal is
theoretical as well. We want to determine the
extent of the similarities and differences in syntactic
constructions between Italian (as reported in the
Italian-UD
For building the corpus, we selected two case studies in the storytelling domain: all of the sentences of two LIS videos, namely the fairy tale Cappuccetto Rosso (Little Red Riding Hood) and
the story I tre fratelli (The three brothers, written by the Italian writer Grazia Deledda), were collected in the novel treebank that, at the moment, is comprised of 257 dependency trees. While in the Cappuccetto Rosso story the signer signs the story without an Italian reference text, in the I tre fratelli story the signer is translating from a well defined written Italian text. By using the full original version of these stories, we had to face the challenge represented by unrestricted real data. For instance, very long and complex sentences were annotated and translated into LIS.
Considering the intrinsic complexity of the task
and the novelty of the project, in the preliminary
release of data described in this paper we only
addressed some of the features of LIS. For instance,
the location in space of a sign is only annotated
in the portion of analysis carried out on ELAN but
was not transferred in the CoNLL-U files.
Furthermore, non-manual elements – which are one of the
fundamental means used by LIS signers to convey
meaning
The paper is organized as follows. In the Section 2, the data collection and the morphological and syntactic annotation processes will be described. In Section 3, language-specific morphosyntactic phenomena are discussed mainly focusing on pointing signs as Highly Iconic Structures (henceforth HIS). The strategies used to annotate signs and their dependency relations are justified. Section 4 concludes the paper by providing some issue on the future development of the project. 2
In this Section, we describe the main steps for the realization of the annotation of the UD-LIS, the pre-processing, which consists in the analysis with ELAN, and the application of the tags and relations of the UD format for the generation of the CoNLL-U format of each LIS sentence. 2.1
Following the same annotation procedure applied
for the SSL treebank
Defining a strategy to gloss the signs included in
the Segno (sign) tier was a challenging task since
the final aim was that of providing unambiguous
and easily retrievable glosses. The rule that is
generally followed when writing down a sign gloss
is to write the translation of the sign as it would
be normally written in the vocal language, but in
capital letters. This writing strategy might cause
ambiguity since different variations of a sign can
be translated with the same gloss. At this stage of
the annotation process, glossing strategies adopted
for LIS and for the SSL treebank were compared,
in order to minimise ambiguity and create a
system that refers to both languages. The
developers of the SSL treebank started from the SSL
corpus that, in turn, referred to the SSL Dictionary
PRON
tionary: the Dizionario bilingue elementare della
Lingua dei Segni Italiana LIS
Luogo (location) defines where the sign is
articulated in the signing space and is based on the 16
sign locations identified by Radutzky
UD POS Tag includes the UD Part of Speech Tag4 associated to the sign. A peculiarity of this tier is that the POS Tag might not correspond to the form of the sign gloss. For the annotation of this tier, priority is given to the function of the sign within the sentence, rather than its gloss. In fact, in some cases the gloss associated to a sign will not correspond to the role of the sign in that specific syntactic context. For instance, the gloss of the sign POESIA (poesia-STS) suggests that the sign is a NOUN. Yet, in the I tre fratelli video, POESIA plays the role of an adjective. In the sign sequence POESIA TEMPO, the previously mentioned sign has the meaning of “tempo poetico” (poetic time). In this case, the UD POS Tag for the sign POESIA will be ADJ and not NOUN. This is because priority was given to the function of the sign within the sentence, rather than its gloss.
Traduzione (translation) provides a translation of the LIS sentence into spoken Italian. If the signs are a direct translation from Italian, as in the I tre fratelli video, the information included in this tier will be a word-for-word transcription of the spoken or written text. If the signer is not translating, as in the Cappuccetto Rosso video, the information included in the tier will be a translation into Italian that imitates the structure of the sentence in LIS as closely as possible. This tier was included to facilitate the understanding of a signed sentence given that the sequence of sign glosses will look 3https://www.spreadthesign.com/it.it/ search/ 4https://universaldependencies.org/u/ pos/ fragmented. By providing a linear translation into spoken Italian of each sentence, a non-signer will be able to have a general understanding of the sentence. Furthermore, as mentioned in the previous section, POS Tags might not correspond to sign glosses. Therefore, by providing this translation, any discrepancies between sign glosses and POS Tags will be justified. 2.2
The information encoded in ELAN was transferred in CONLL-U files and split into its ten columns, except for the LEMMA, DEPS and MISC columns, where no information is included.
Sign glosses and UPOS Tags were included
respectively in the FORM and UPOS columns,
and coarse-grained tags taken from the Tanl POS
Tagetset5 were included in the XPOSTAG
column. Language-specific annotation strategies can
be found in the FEATS column where specific
symbols and labels taken from different sources
were used to provide more information on the
peculiarities of a sign or of its production. Based
on SSLC tags
The annotation of the HEAD and DEPREL columns of the CoNLL-U format is at the very core of the research. As for the definition of a root node, the UD standard indicates that, in Italian, the root is usually a verbal predicate or a noun. If the verbal predicate is not present due to ellipsis, the root is moved to the leftmost dependent of the verbal predicate6.
The strategy for finding the syntactic structure of each sentence consists in looking at Italian and SSL treebanks, choosing the most appropriate annotation with respect to the specific phe
Tanl\_POS\_Tagset
6https://universaldependencies.org/it/ dep/root.html nomenon to be annotated or the context where it occurs. For instance, for the annotation of particular signs we were inspired by the solution adopted in SSLC (see 2.2.1). Nevertheless, if no annotation is deemed to be adequate, a novel independent solution that seems most fitting is applied. If a sentence presented a unique combination of signs and no corresponding or similar dependency relations were found in these other treebanks, the construction of the dependency tree was based on an independent solution that is compliant with the general criteria adopted for LIS in the novel treebank.
In the following Section, a small selection of phenomena encountered in the treebank are described. 3
In this section we discuss two LIS phenomena, i.e. pointing signs and repetitions, which we addressed in the development of the novel resource. 3.1
In LIS, pointing signs are HIS realized using an
extended index finger and carrying out several
functions. For example, a pointing sign can
function as a pronoun, as a determiner or as an adverb
Additionally, pointing signs can also be used by
the signer to refer to himself or herself during role
shift, that is, while impersonating a character, as
in Figure 2. A sub-type of pronominal or
determiner pointing signs are demonstrative pointing
signs
The repetition of non-pointing signs by means of
reduplication can be used to convey that an event
is still ongoing or that is happening several times
Another form of repetition of non-pointing
signs occurs in the Cappuccetto Rosso fairy tale.
In this case, the repetition cannot be classified as
reduplication since the sign is not repeated through
a circular movement
The paper describes the development of the novel resource LIS-UD, namely a treebank for LIS in the UD format8. Provided the preliminary stage of the project, the main aim of this work is to discuss the issues raised by the annotation in UD of a sign language. This issue has previously been only partially addressed in one other single project for the development of the SSL treebank, and so the novel resource can be the opportunity for better investigating it. Several future directions can be drawn for the development of our project. In particular, among these directions, we are planning first of all to draft a more detailed document about the annotation guidelines. This can help us in checking the material annotated until now, but also in driving the work of novel annotators.
8The current version of LIS-UD is available at: https: //github.com/alexmazzei/LIS-UD