Word syllabification and syllable analysis are two related issues of great importance in the study of language (written or spoken). These topics have attracted a large category of researchers, from pure linguists, in phonetics, to psycholinguists, computer scientists, speech therapists, etc. Thus, the syllable plays an important role in language learning and acquisition, speech recognition, speech production [1,2], language similarity [3], in text comprehensibility (Kincaid-Flesch formula [4]), in speech therapy, in poetry analysis [5,6], etc. Each language has its own way of grouping sounds into syllables and its own rules for dividing words into syllables. Linguistically, the syllable represents "the smallest phonetic trance likely to receive an accent and only one" [7], and the syllabic cut is seen by De Saussure [8] on the border between the implosion and the explosion of the spoken sound: "If in a chain of sounds one goes from implosion to explosion, one obtains a particular effect which is the indication of the boundary of the syllable".
The analysis of the words' syllabic structure also plays an important part in historical linguistics [9], not only in diachronic phonetics and phonology, but also in lexicology. Romance comparative linguistics, in particular, still needs a detailed overview of this aspect, as syllable, segmentation and prosody can give strong account on phonetic changes that haven't been explained yet. The "prosodic revolution" [10] from Latin to the Romance languages -including syncope (the loss of an intermediate syllable) and apocope (the loss of the final syllable) at a large scale -has led to major changes, but their weight is different from one idiom to another: while the Western Romance languages manifest highly evident differences from the Latin phonological and prosodic system, and the Eastern languages are considered to be most conservative from this point of view, Italian seems to be in between [10]. On the other hand, in Latin, the relation between stress and quantity grew stronger, thus short stressed vowels progressively gained length. It is noteworthy that this situation is best preserved in Italian, and not in the Eastern Romance idioms: thus, in Italian stress cannot skip a heavy penultimate syllable, and stress cannot fall further back than the antepenultimate syllable, a twofold characteristic feature of the Latin prosodic system. This is why we are taking Italian as a starting point for a largerscale study, oriented towards all Romance languages. The main difference between Latin and its modern descendants is that Latin stress was quantity-sensitive, leading thus to the following rule: in polysyllabic words, stress fell on a heavy penultimate (meaning, containing a long vowel), otherwise on the antepenultimate. Due to the collapse of vowel quantity as a distinctive feature in the vocalic system, no Romance language has retained the Latin stress rule as such [10]. As, from a statistic point of view, the greatest part of the Romance lexicon is represented by penultimate stressed words, a basic automatic mechanism would assign penultimate stress by default, whereas for both final and antepenultimate stress, the machine (as well as, not in a few cases, non-native speakers) would need further specification. As a consequence of the loss of Latin vowel quantity, Romance stress has ceased to be completely predictable. That is, partially, why in the majority of the traditional Romance compara-tive or historical grammars, there is no specific section devoted to syllabification [11], or, if there is, it focuses either on general prosodic features [12], or on the vowel evolution depending on its presence in an open or closed syllable [13]. The lack of a section dedicated to syllabification is also common in the historical grammars of Italian [14,11,15]. We will focus in this research only on written form of words, so we will investigate only the graphical syllabification and stress. By focusing on the graphical syllabification and stress in Italian, we aim to take a step forward towards the complete evaluation of the prosodic changes that took place in the transition from Latin to the Romance languages, and their influence on the Romance phonetics and phonology. A machinelearning model, capable of automatically inferring graphical syllabification and stress assignment, along with the purpose of creating a data-base containing the quantitative and qualitative description of syllabification and stress in the Romance languages, could be the first important task in the greater challenge of tracing the similarities and differences between the Romance languages and, more important, between Romance and Latin. From a typological point of view, the study of syllabification and stress can shed a new light on the universal features that, by defining our phonoarticulatory and phonoacoustic apparatus, have guided the languages' development and change. Given the promising results of this analysis, the present study can establish the basis of a research of the syllable in other languages, either linguistically or typologically related to Italian.
One of the studies that address automatic syllabification in Italian belongs to Bigi and Petrone [16], who proposed a tool that performs rule-based automatic segmentation. Adsett and Marchand [17] and Adsett et al. [18] investigated whether data-driven approaches outperform rule-based approaches for a language with a low syllabic complexity, such as Italian. The authors reached the conclusion that even in this case data-driven systems are the more appropriate approach. In terms of machine learning, the tasks of automatically inferring syllable boundaries and predicting stress assignment can be naturally framed as sequence labeling problems. While automatic syllabification has received more attention recently [19,20,21,22,23,24], stress placement has not been investigated as much [25].
Given the complexity of syllable applications and word syllabification, the presence of electronic resources dedicated to them becomes a necessity. While native speakers of a language generally do not have great difficulty in spelling words, the same cannot be said of those who learn a foreign language who often tend to apply their own rules to foreign words, and problems arise in automatic syllabification. This is because the rules of syllabification are linguistic rules, and they cannot always be easily modeled by the computer when there are no other linguistic factors that those rules take into account. For example, a rule that is present in many languages distinguishes between a vowel and a semivowel, but the computer is not able to easily recognize when the same sign has the value of a vowel and when it is a semivowel. Because of this, rule-based adaptations of syllabification systems [26] generally have higher errors, and many languages do not have an automatic syllabification system yet (for example, in the Python library, only a few languages have syllabification). The last few decades have brought the first data-driven syllabification systems.
However, in order to build such a system, training data is needed, and there are many cases in which the available data do not cover the whole language, and thus the systems have different results when the test corpus is changed.
Starting with these remarks, our main contributions are:
• We propose ItGraSyll (Italian graphical syllables), a dataset of 114, 503 Italian words, in orthographic form, containing annotations for their orthographic syllabification and stress placement1 • We perform quantitative and qualitative analyses of the previously built dataset.
• We analyze stress placement in the context of the Italian syllables. • We propose an automatic system of syllabification for Italian words.
In this section we perform various measurements regarding the syllables and stress placement of Italian written words and analyze the results. We perform, on Italian, an investigation similar to a previous investigations conducted on Romanian by Dinu and Dinu [27], Dinu and Dinu [28].
We build a dataset of Italian words starting from the online version of Dizionario italiano De Mauro, 2 which provides information regarding graphical syllabification and stress placement for the Italian vocabulary. Stressed syllables are also shown by having accents on the dominant vowel. Going further, this dataset will be referred to as ItGraSyll. We performed several pre-processing steps. We cleaned the resulted dataset by removing duplicates, prefixes and suffixes in order to remain with the base word; abbreviations and unwanted punctuation marks such as dots, commas, apostrophes and dashes were also excluded so we can correctly process each word and its syllable division. Finally, the dataset consists of 114, 503 words in orthographic form having between one and eleven syllables. The distribution of words per number of syllables is represented in Table 1
We In order to characterize the average length of a syllable measured in letters, we investigated two cases: a) the average length of the token syllables measured in letters is: 𝐿𝑆𝑦𝑙 𝑡𝑜𝑘𝑒𝑛 = 1,133,515/483,931 = 2.342 b) the type syllables are formed of #𝑇 𝑦𝑝𝑒𝑆𝑦𝑙 𝑙𝑒𝑡 = 13,576 letters. Thus, the average length of a type syllable measured in letters is 𝐿𝑆𝑦𝑙𝑡𝑦𝑝𝑒 = 13,576/3,730 = 3.639.
These statistics are computed for the words extracted from the dictionary, which were considered to be equally weighted. This excludes any information relating to the frequency of the words with respect to writing or speech. For future research, large corpora of Italian texts can be leveraged in order to recompute these values and include frequency-based weights.
A list of the most frequent 20 syllables is included in Table 2.
We identified a total of 67 different consonant-vowel structures. The most frequent 7 structures cover almost 97% of the total. Depending on the type-token ratio, the most frequent consonant-vowel structures are the following: a) for the type syllables: cvc (25%), ccvc (20.9%), cvvc (7.79%). b) for the token syllables: cv (58%), cvc (15%), ccv (7%), cvv (4.74%) and v (4.32%). Moreover, we observe that the cv structure corresponds to 40 out of the most frequent 50 syllables from the dataset.
We identified a total of 2,883 stressed syllables (type syllables). So, 847 syllables are never stressed. The most frequent 20 stressed syllables are represented in Table 3.
We observe that the most frequent stressed syllable (men) has a very high stress ratio (90%) when we compare the stressed occurrences with all its occurrences (stressed and unstressed) in our database. While in the top 20 of all syllables, men is the only syllable of length 3 (on the 14th position), for stressed syllables there are a couple of other syllables with a length greater than 2 (zio on position 6 with 34% stress ratio, gia on position 19 with 65% stress ratio). We investigate stress placement with regard to syllable structure and we provide in Table 4 the percentages of words having the stress placed on different positions (for top 5), counting syllables from the beginning and from the end of the words as well. We observe that in most cases the stress is placed on the second to last syllable.
Stress placement for Italian.
The syllables have a less intuitive behaviour, usually a small number of syllables cover a large part from a language. This is valuable for a large category of natural languages, including English, Dutch, Romanian [28], Korean, Chinese, etc. We investigate here if this empirical law is also applicable to Italian. We made this investigation both on stressed and general syllables.
The most frequent 30 Italian syllables (when stress placement is disregarded) cover almost 50% of #𝑇 𝑜𝑘𝑒𝑛 𝑠𝑦𝑙 , the most frequent 50 syllables cover 61%, the most frequent 100 cover 74% and the most frequent 150 syllables (i.e. 4% of #𝑇 𝑦𝑝𝑒 𝑠𝑦𝑙 ) cover 80% of #𝑇 𝑜𝑘𝑒𝑛 𝑠𝑦𝑙 . Over this number, the percentage of coverage rises slowly. 2,281 (61%) syllables of type syllables occur less then 10 times, and 1,174 syllables occur only once (hapax legomena).
A similar trend can be observed also for the stressed syllables. Further, we notice that the most frequent syllables cover a wide ratio of the total syllable frequency. For example, the 10 most frequent stressed syllable represent 31% of the total of stressed syllables, the top 50 syllables, 60% and the top 200 syllables, 81% of the token syllables.
The values are plotted in Figure 1, for all syllables and for stressed syllables. This results proves that the law is true for Italian too, a very small number of syllables cover a large part from Italian language (there are necessary only 150 syllables to cover 80% from language).
In this section we discuss two minimum effort laws that have been previously investigated for other languages and verify whether they apply for Italian as well.
Denoting by 𝐹 (𝑛) the frequency of a word having n syllables and by 𝑖 = ∑︀ 𝑛𝐹 (𝑛)/ ∑︀ 𝐹 (𝑛) the average length (measured in syllables) of the words, Chebanow [29] proposed the following law between the average 𝑖 and the probability of occurrences 𝑃 (𝑛) of the words having n syllables:
For Italian, 𝑖 = 4.226.
Hyphen Acc. Hyphen F1 Word Acc. GRU for syllabification w/o stress markers 99.74% 99.69% 97.61% GRU for syllabification w/ stress markers 99.82% 99.79% 98.41% GRU for stress prediction --94.45%
Performance metrics computed for the automatic syllabification and stress prediction on the test set. We computed accuracy and F1 scores on the sequence labelling predictions for syllabification, in order to assess how well the model predicts the positions where the syllables split. Word level metrics were computed for both syllabification and stress prediction; this kind of metrics are more strict since any misplaced hyphen in the syllabification makes the entire prediction wrong.
In Figures 2a and 2b we plot the probability distribution of the length of words (in syllables) -the practical and theoretical representations.
We observe that the two curves have comparable shapes, with a more prominent peak for the probability distribution in Figure 2a; this peak can be influenced by the fact that it is determined based on all the words in the dictionary, where many 4-syllable words are present.
Menzerath's law -later generalized by the Menzerath-Altmann law [30] -states that the bigger the number of syllables in a word, the lesser the number of phonemes composing these syllables. In other words, Menzerath's law expresses a negative correlation between the length of a word in syllables and the lengths in phonemes of its constitutive syllables. In cognitive economy terms, this means that the more complex a linguistic construct, the smaller its constituents. The law is expressed as follows:
where 𝑦 is the syllable length (the size of the constituent), 𝑥 is the number of syllables per word (the size of the linguistic construct), and 𝛼, 𝛽, 𝛾 are empirical parameters. Figure 2c shows that the law is satisfied for Italian.
We further investigate how a deep-learning model can automatically infer the syllabification and stress assignment of Italian words, given their orthographic representation.
Both tasks can be defined in terms of a sequence labelling problem, strategy which was previously successful used for Romanian [31,32]. Let us consider, for example, the word medaglione (the Italian translation of the word "locket"). For syllabification we can label each letter from the word either with the label 1, denoting that a syllable starts from that letter, or with the label 0, meaning the respective letter is not the first letter in its syllable. Similarly, for identifying the stressed vowel, we can label its position with a 1 and all other letters are assigned the label 0. We thus obtain for our example the sequence 1010100010 for syllabification and the sequence 0000000100 for stress prediction (i.e. me-dagliò-ne, the o vowel is stressed). With these definitions, we can now construct machine learning models for labelling the character sequences. The model we propose is a recurrent neural network based on Gated Recurrent Units (GRU) [33]. The model architecture is comprised from the following components:
• a character embedding layer, producing 64dimensional vectors for each unique character • a stacked bidirectional GRU, with 3 layers and a 128-dimensional hidden state; a 0.2-rate dropout applied after each of the first two layers • 0.5-rate dropout, after the last GRU layer, along with one-dimensional batch normalization • a time-distributed fully-connected layer with 256 output nodes and ReLU activation • a linear layer that projects the 256-dimensional vector into a single number, on which sigmoid activation is applied to infer the binary labels.
For training the models for both tasks, the dataset of words is split into 50% training examples and 50% test examples, unseen during training.
The loss function computed for the prediction made for a word, regardless of the task on which the model is trained, is the average of two terms: the first one is the average character-wise binary cross-entropy, while the second one is the root mean squared error computed between the vector of predicted labels and the groundtruth vector. The model is optimized using the Adam optimizer [34], with a learning rate of 0.0003, no weight decay, bath size of 32, and a LR scheduler that halves it every 5 epochs. The models are trained for 10-15 epochs.
For the task of automatic syllabification, we wanted to check if the presence of the stress markers affects the performance of the model. Because of that, we trained two models: the first one was trained using the spelling of the words with the stress markers removed, while the second one was trained with them included.
Predicted bàlano balanò fèmore femòre dòlmen dolmèn tùtolo tutòlo pudìco pùdico corsìa còrsia
True Predicted mu-o-ne muo-ne bion-da bi-on-da cli-en-te clien-te co-di-a-to co-dia-to ma-nu-brio ma-nu-bri-o spa-tria-to spa-tri-a-to
Examples of erroneous test predictions provided by the deeplearning models.
Table 5 contains the metrics computed on the test set, using the models trained for syllabification (both with and without stress markers) and the model trained for predicting the stressed vowel. We obtained a remarkable hyphen accuracy of 99.74% for syllabification without the stress markers, and, when we add the stress markers, we obtained an increasing accuracy, obtaining 99.82%.
Including the stress markers into the data used for syllabification improved the metrics across the board, most notably with a ∼ 1% increase in word-level accuracy, which considering the large amount of data, and the high accuracy scores is a significant improvement (460 fewer syllabification mistakes as opposed to the approach that excludes stress markers). Regarding the stress prediction, we obtained an accuracy of 94.45%. Table 6 showcases a series of wrong predictions generated by the models on the tests sets for stress assignment and syllabification.
We also look into the accuracy scores computed for the test set, when it is bucketed based on the real number of syllables of the test words. These results are shown in Figure 3 and Table 7. For stress assignment, accuracy decreases to a global minimum for disyllabic words, then starts to increase again with the number of syllables. For the syllabification task, including the stress markers seems to outperform excluding them in most scenarios, while both accuracies achieve a peak around the 5 syllables mark. This result seems to align with the distribution of syllables in the dataset, i.e. obtaining higher scores for the number of syllables with more examples. For stress assignment errors, we also investigate the placement of the predicted stressed syllable in relation with the true one (see Table 8). 95.6% of the errors misplaced the stressed syllable at most one position to the left, or to the right, while almost two thirds of the erroneous predictions placed the stress on the first syllable to the right of the correct one. Starting from the incorrect predictions for stress assignment, we compute how far the assigned stress is from the actual one, in numbers of syllables (delta). A delta of −2 means that the predicted stressed syllable is the second one to the left of the correct stressed syllable. A delta of 0 in this situation means that the algorithm predicted the stressed vowel incorrectly, but the prediction sits inside the correct stressed syllable.
In this paper we have investigated graphical syllabification and graphical stress assignment for Italian words.
We have started by building ItGraSyll, a dataset of Italian graphical syllabified words, with stress annotations as well, on which we have performed several quantitative and qualitative analyses, including the verification of two minimum effort laws for the case of Italian. Finally, we have proposed a recurrent neural network machine learning model for automatic syllabification and stress assignment for Italian written words. For stress prediction we have obtained 94.45% word-level accuracy, and for syllabification we have obtained 98.41% word-level accuracy and 99.82% hyphen-level accuracy. In future work we intend to extend the analysis from dictionary level to corpus level and to investigate other languages as well.
We want to thank the reviewers for their useful suggestions. Research supported by the Ministry of Research, Innovation and Digitization, CNCS/CCCDI UEFISCDI, SiRoLa project, number PN-IV-P1-PCE-2023-1701, Romania.