Plugin for automatisation of phonetic-phonological analysis and obtaining analytical feedback for Spanish learners Plugin para la automatización del análisis fonético-fonológico y la obtención de retroalimentación analítica para estudiantes de español Tamara Couto-Fernández1, Albina Sarymsakova2, Nelly Condori-Fernández3 and Patricia Martín-Rodilla4 134 University of A Coruña, Faculty of Computer Science, Camiño do Lagar de Castro, 6, A Coruña, 15008, Spain 2 University of A Coruña, Faculty of Philology, Campus da Zapateira, A Coruña, 15008, Spain Abstract We present in this article the Plugin for phonetic-phonological analysis in Spanish (PAFe), which consists of a series of scripts (a code written with a programming language (Python) that, implement three different intonation comparison algorithms of an ELE (Spanish as a foreign language) student and a native speaker of Spanish), allowing, in turn, three different types of analysis: global, tonal tendency and intersyllabic. In addition, PAFe has a database to keep a history of different types of data (user profile, pronunciation exercises and audios) and a graphical interface to include reports on pronunciation evolution in Praat, a tool for acoustic analysis. PAFe is a software solution that offers new functionalities of Praat and allows the following: (i) to perform a comparative analysis between the intonational patterns of an ELE student and a native speaker; (ii) to report the evolution of the acquisition of such patterns in Spanish thanks to the history of the stored data. In this way, automated feedback is provided to both students and teachers. Keywords 1 Praat, intonation analysis, ICT, Python. 1. Introduction Nonetheless, no tool provides both facilities at the same time, nor offers to monitor the evolution of the students. The present work is framed in the area of For this reason, we have decided to develop a natural language processing, specifically, in the system that complements language teaching, in comparative-contrastive analysis of intonation for particular, one that can be used remotely or in the didactic purposes provided by our original tool hybrid modalities. PAFe. Despite the existence of some tools, such Our tool offers the functionality to perform an as the Oplustil and Toledo [11] proposal, or the instant comparative analysis of a student's study by Strik, Truong, Wet and Cucchiarini [8], pronunciation, taking as a reference the speech of which offer results of phonetic-phonological a native speaker, and observing the evolution of similarity or detect errors made in pronunciation. this through data stored in history. SEPLN-PD 2022. Annual Conference of the Spanish Association for Natural Language Processing 2022: Projects and Demonstrations, September 21-23, 2022, A Coruña, Spain EMAIL: albina.sarymsakova@udc.es (A. Sarymsakova); tamara.cfernandez@udc.es (T. Couto-Fernández); n.condori.fernandez@udc.es (N. Condori-Fernández); patricia.martin.rodilla@udc.es (P. Martín-Rodilla) ORCID: 0000-0003-0381-0239 (A. Sarymsakova); 0000-0002- 1044-3871 (N. Condori-Fernández); 0000-0002-1540-883X (P. Martín-Rodilla) ©️ 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 83 For the development of our plugin, several with calls to Praat and generates information technologies have been used to support the work windows directly from Python code files. The done, such as Praat, Python and PostgreSQL. intermediary between Praat and the data managed in the database is Python. 2. Methodology We employ natural language processing and audio processing techniques in our tool, taking as our main source the human voice recordings of We start designing our work based on the native speakers and students. Praat allows us to following essential principles of intonation extract quantitative information at the prosodic analysis: level from the audios. 1. We annotate the syllables of each speech Subsequently, the native/student comparative act in a Praat textgrid (Boersma and Weenink algorithms in terms of prosodic aspects that are [1]; we identify pitch values of all vowels in presented and implemented by the tool can offer the syllables (voiced or voiced consonants are comparative information between two measured as well), using the Praat Script native/student audios to provide feedback in developed by Mateo Ruiz [9, 10], which Spanish language learning. These algorithms are extracts the absolute values in Hz, relativises an original contribution implemented in the tool them and draws the standardised melody since there was no algorithmic proposal of this graph; type for Spanish until now. 2. we discriminate relevant frequency We have developed the PAFe Plugin following values between tonal segments from irrelevant an iterative and incremental methodology based values; according to Cantero Serena [2, 3], on agile technologies and scrum development Font-Rotchés and Cantero Serena [6, 7], less methodology, based on the work of Schwaber and than 10% difference between segments is Sutherland [12]. Figure 2: Example of User considered imperceptible. Interface visualising the new functionalities added Once we have obtained the relevant data from in Praat the intonation analysis, we move on to the PAFe In the following, we describe the development of architecture. our tool. Our project develops an extension to an existing desktop application for acoustic speech analysis: Praat. Therefore, we start from a developed architecture to which a new module (PAFe) is coupled (Figure 1) consisting of Praat scripts, Python code and a PostgreSQL database. Figure 1: Overall architecture of PAFe Praat, through its scripting, allows command line calls to other systems, as described by Dragos-PaulPop [5], thus making it possible to extend the application through the use of other languages and technologies, external to Praat. This new module (PAFe) communicates with the original system by employing new Praat scripts that are associated with the application's menu Figure 2: Example of User Interface visualising items (see Figure 2), from which these files are the new functionalities added in Praat executed. Sometimes, the new module dispenses In the following, we describe the development of our tool. 84 3. Solution: PAFe Plugin syllable, the difference in pronunciation concerning the reference audio is indicated, as well as the percentage of similarity of tone and Our PAFe tool, in its final version, allows the average difference between two audios is comparative analysis by providing similarity obtained. According to the results obtained results and intonation graphs based on pitch through this last type of analysis, both the values2 and tonal tendency in each defined similarity and the difference between the segment and, finally, visualisation of a student's reference audio and the learners' audio are progress over time. We highlight the following shown more accurately. Finally, we can see the operations made possible by our plugin: evolution of our students' results through the 1. The application allows the creation of option to view the history. different profiles to facilitate the process of Finally, we show a flowchart (Figure 3) that managing the data uploaded by users. provides information about the behaviour of our a) First of all, the teacher is registered. plugin, exposing the functionalities and their b) A pupil is then assigned to the teacher interrelation, as well as presenting the operators previously registered. This step avoids that interact with the application and their confusion if there is more than one user of the restrictions. same computer or laptop. c) Finally, the profile of a native Spanish speaker is recorded to upload the data that will serve as a reference for the programme; 2. PAFe enables the management of WAV and TextGrid files3: our programme includes both storage and deletion of audio files and annotations; 3. It also allows for different types of acoustic analysis (global analysis, tonal tendency analysis and intersyllabic analysis): the algorithm that performs the global analysis consists of dividing the previously saved audios of learners and native speakers of Spanish into about 1000 intervals (discarding silences) to obtain very precise comparative values. However, this type of analysis does not provide feedback about possible deviations in tone but provides generic data on the percentage similarity of the native speaker's Figure 3: Use case diagram (PAFe functionalities and learner's audio. As far as tonal tendency and main actors) analysis is concerned, the programme works with .TextGrid annotations and the previously saved .WAV audio files. In this case, the 4. Illustrative example of intersyllabic utterances are divided by words and, to obtain analysis the similarity locally, it is indicated whether the pitch of each word has been reproduced In this section, we show how one type of correctly or not and, in case it has not been comparative analysis is carried out. To perform reproduced correctly, the percentage of the intersyllabic analysis, it is necessary to fill in deviation is indicated; the percentage of pitch a form (Figure 4) with the data that characterise similarity and the average difference between the audio of the learner we want to compare. two audios are also obtained. Finally, the intersyllabic acoustic analysis is a comparative analysis, syllable by syllable, of the similarity between the tone realisation of a learner and that of a native speaker; in this case, for each 2 3 Tone frequency in Hz File with tags segmenting associated audio 85 5. Conclusions In conclusion, we highlight the following key issues that we have addressed in this paper: 1. The PAFe tool allows different types of comparative-contrastive analysis of the intonation (global, tonal tendencies and intersyllabic) of EFL learners and native speakers of Spanish; Among them, we Figures 4: Form for conducting an intersyllabic consider the intersyllabic as the most accurate analysis since the results of tonal difference appear The audios of that student that meet these syllable by syllable and show the tonal properties are then filtered out and display a deviations of the students, and the global as the window with a drop-down menu for the selection most efficient in terms of response time since of the audio to be analysed. Once the audio is it does not require the uploading of TextGrids, selected, the corresponding. TextGrid file is and the segmentation is done in an automated selected in the same way. way, as shown by the empirical data of the Each type of analysis returns different results. Couto Fernández [4] work. For the intersyllabic analysis, we show a 2. This application has several functions; apart similarity result per syllable and the average from performing the intonational analysis, it percentage difference (Figure 5). Finally, we allows to store the audios, the . TextGrid files obtain a graph with the tonal differentiation and the results of the analysis (the history) of curves in each syllable for each audio (Figure 6). each utterance according to the profile of the speaker (student or native speaker of Spanish). 3. PAFe has been developed to achieve the following didactic objectives: to facilitate the work of teachers with regard to the identification and correction of intonation deviations (we have carried out an empirical Figures 5: Intersyllabic analysis information analysis with teachers of Spanish as a foreign language, where we measured the degree of satisfaction with PAFe, with positive results, as indicated in the work Couto Fernández [4]; to store the results of the analyses carried out for future improvement; to serve as a self- evaluation and self-correction tool for ELE students, given that the tool itself allows them to upload .WAV and . TextGrid files, run the analyses and obtain the results without constant help from teachers. As a future line of research, we highlight the need to measure this degree of feedback to students empirically. As far as we know, it is the only existing solution both under Praat and outside Praat that allows this type of analysis and offers feedback to the student in the Spanish language. We highlight Figure 6: Graph showing the tonal curves of each that as feedback and self-evaluation, our tool audio for each syllable (the X-axis represents the offers the percentage of similarity and difference syllable division of an utterance and the Y-axis of pitch values so that the student can correct his the pitch values). pronunciation. Also, as future lines of work, we plan to improve the graphical environment of the plugin and open to the student, as an end user, the possibility of its use via the web. 86 References [1] P. Boersma, D. Weenink, Praat: doing phonetics by computer, 2019. URL: http://www.praat.org/. [2] F. J. Cantero Serena, Teoría y análisis de la entonación, volume 54, 2002. [3] F. J. Cantero Serena, Análisis prosódico del habla: más allá de la melodía, Comunicación Social: Lingüística, Medios Masivos, Arte, Etnología, Folclor y otras ciencias afines 2 (2019) 485-498. [4] T. Couto Fernández, Una herramienta de análisis del habla de audio para proporcionar retroalimentación automática a los estudiantes en la pronunciación en español. UDC. A Coruña. [5] Dragos-Paul Pop, Adam Altar, Designing an MVC Model for Rapid Web Application Development, Procedia Engineering 69 (2014) 1172-1179. DOI: 10.1016/j.proeng.2014.03.106 [6] D. Font Rotchés, F. J. Cantero Serena, La melodía del habla: acento, ritmo y entonación, Eufonía: didáctica de la música (2008) 19-39. [7] D. Font Rotchés, F. J. Cantero Serena, Melodic Analysis of Speech Method applied to Spanish and Catalan, Phonica 5 (2009) 33- 47. [8] H. Strik, K. Truong, F. Wet, C. Cucchiarini, Comparing different approaches for automatic pronunciation error detection, Speech Communication 51 (2009) 845–852. DOI: 10.1016/j.specom.2009.05.007 [9] M. Mateo Ruiz, Protocolo para la extracción de los datos tonales y curva estándar en análisis melódico del habla, Phonica 6 (2010) 49-90. [10] M. Mateo Ruiz, Scripts en Praat para la extracción de datos tonales y curva estándar, Phonica 6 (2010) 91-111. [11] P. Oplustil, G. Toledo, Uso de una herramienta didáctica para la práctica de la entonación en hablantes no nativos de español, Sintagma: Revista de lingüística 31 (2019) 37–50. [12] K. Schwaber, J. Sutherland, La guía definitiva de scrum: Las reglas del juego, 2020. 87