English. In this paper, we present the enetCollect1 COST Action, a large network project, which aims at initiating a new Research and Innovation (R&I) trend on combining the well-established domain of language learning with recent and successful crowdsourcing approaches. We introduce its objectives, and describe its organization. We then present the Italian network members and detail their research interests within enetCollect. Finally, we report on its progression so far.
Italiano. In questo articolo presentiamo la COST Action enetCollect, un ampio network il cui scopo e` avviare un nuovo filone di Ricerca e Innovazione (R&I) combinando l’ambito consolidato dell’apprendimento delle lingue con i piu` recenti e riusciti approcci di crowdsourcing. Introduciamo i suoi obiettivi e descriviamo la sua organizzazione. Inoltre, presentiamo i membri italiani ed i loro interessi di ricerca all’interno di enetCollect. Infine, descriviamo lo stato di avanzamento finora raggiunto.
In this paper, we present the COST network enetCollect that aims at kick-starting an R&I trend for combining language learning with crowdsourcing techniques in order to unlock a crowdsourcing potential for all languages, consisting in learning and teaching activities. This potential will be used to mass-produce language learning material and language-related datasets, such as NLP resources.
1European Network for Combining Language Learning
with Crowdsourcing Techniques, Web:
EnetCollect connects to ongoing
crowdsourcing research, including Games With A Purpose
approaches
This paper aims at fostering the participation of the Italian NLP community while further allowing it to benefit from the research and collaboration opportunities enetCollect offers (e.g. research stay grants) for its remaining 2.5 years of funding. Sections 2 and 3 present enetCollect’s ambition, and its organization while Section 4 introduces the Italian members and their research interests. Sections 5 and 6 report on achievements up to now and the current state of affairs. 2
Started in March 2017, enetCollect will pursue, until April 2021, the long-term challenge of fostering language learning in Europe and beyond by taking advantage of the ground-breaking nature of crowdsourcing and the immense and ever2https://www.dict.cc 3https://www.wiktionary.org/ 4https://www.zooniverse.org/ 5http://crowd4u.org/en/ growing crowd of language learners and teachers6 to mass-produce language learning content and, at the same time, language-related data such as NLP resources. The prospect of mass-producing language-related data can vastly impact domains such as NLP, which in turn will impact back on language learning by fostering support from various language-related stakeholders (e.g. see Section 4 for NLP-related crowdsourcing scenarios).
As intensifying migration flows (due to economical and geopolitical reasons) increase the diversification of language learner profiles and the demand for learning material, the launch of such an R&I trend is very timely. Indeed, the effectiveness of the existing material runs the risk of gradually falling behind and the varied combinations of languages taught and target groups can hardly be addressed by small-scale initiatives. EnetCollect timely kick-starts an overarching R&I trend to continuously foster various initiatives. Funding-wise, the timing is also favorable as both the increasing need for learning solutions and the problem-solving nature of crowdsourcing are widely acknowledged.
The creation of a new R&I community is addressed through formal Research Coordination Objectives aiming at creating a shared knowledge of the subject, at carrying out prototypical experiments and at disseminating promising results while formal Capacity-Building Objectives aim at creating the core R&I community, communication means and new initiatives. In Section 5, we report on progress regarding these objectives. 3
EnetCollect makes a working distinction between explicit and implicit crowdsourcing approaches: while for explicit crowdsourcing the crowd intentionally participates (e.g. Wikipedia), for implicit crowdsourcing the crowd is not necessarily aware of its participation (e.g. reCaptcha7). EnetCollect is organized along five working groups (WG) and three support groups called coordinations.
Whereas WG1 focuses on explicit
crowdsourcing approaches to create data or learning content
(e.g. collaboratively creating lessons), WG2
focuses on implicit crowdsourcing approaches for
the same purpose (e.g. generating exercise
con621% of the Europeans aged over 14 years (9˜0 millions
people, Eurobarometer report,
The five WGs are different content-wise and can be pursued in a parallel fashion. Nonetheless, they remain interdependent in the overarching objective. For example, the boundary between explicit and implicit crowdsourcing (WG1 and WG2) is sometimes difficult to draw when the crowd is explicitly involved while their actions are being implicitly crowdsourced8. Also, any crowdsourcing approach will fail if there is no crowd to rely on (WG3), no technical solution to support its functional needs (WG4), and no appropriate ethical or legal contexts to implement it (WG5). Alongside the WGs, three coordination groups on Dissemination, Exploitation and Outreach are providing standardized support for WG-transversal tasks. 4
The Italian members are currently among the most numerous and active participants to the Action and its events. In addition, the Action coordination (Chair and Grant Holder) is carried out by two Italian members from Eurac Research (see below). Being all related to NLP, enetCollect’s Italian partners have a common interest in combining language learning with implicit crowdsourcing (WG2) so as to extend and correct NLP datasets. All crowdsourcing scenarios described hereafter share the same overarching approach: the NLP partner uses an NLP dataset to generate exercise content and both crowdsources and cross-matches the learners’ answers in order to validate/discard the data used to generate the exercise content, just like GWAP players validate/discard data while playing. Deriving expert knowledge from crossmatched learners’ answers is a challenge enetCollect aims at addressing. Relying on a crowd of 8E.g. crowdsourcing learner essays and their corrections by teachers to create annotated corpora. learners is however promising in two ways. First, learners should be mostly confronted with exercise content generated from reliable NLP data so as to not undermine their efforts. Their constantlyevaluated proficiency levels thus provide a reliability score for their answers. Second, as a crowd of learners renews itself over time, the set of crowdsourced answers for each question is potentially infinite and their “inferior” reliability is thus compensated by their “superior” quantity.
Eurac Research is particularly concerned with
research on the three official languages of South
Tyrol (Italian, South Tyrolean German and the
minority language Ladin). As regards NLP, Italian is
the best covered while South Tyrolean is
approximated by adapting solutions for standard German
and Ladin has barely any coverage. To improve
this situation, the IAL aims at crowdsourcing
varied NLP resources for South Tyrolean German and
Ladin, starting with wide-coverage Part-of-Speech
(POS) lexica. The foreseen crowdsourcing
scenario is to use POS lexica to generate exercise
content for widely adopted exercises such as the one
for grouping words according to their properties
(e.g. “select all verbs among these five words”)
or for identifying words within a grid of random
letters (e.g. “select five adjectives in the grid”.
By crowdsourcing the learners’ answers, the IAL
aims at gradually improving the lexica while
continuously adding new entries. As for the targeted
crowd of learners, the IAL will build on its
longstanding collaborations with schools
MT technologies supporting both human
translators and multilingual applications. The creation of
dedicated language resources is thus a core
activity. Within enetCollect, HLT-MT aims at
enriching existing parallel corpora and at enhancing MT
evaluation by crowdsourcing multiple translations
of the same sentence
the Department of Literary, Linguistic and
Comparative Studies, University of Naples
“L’Orientale” aims at improving linguistic
representativeness, precision, robustness and
computational efficiency of NLP applications
The Institute of Computational Linguistics ‘Antonio Zampolli’ (CNR-ILC) carries out research at the international, European, national and 10https://sites.google.com/view/ parseme-it/home
11Groups of words composing one lexical unit, such as
’tirare le cuoia’ (En. kick the bucket)
regional level since 1967. It participated in
several EU initiatives on language resource
documentation and recently took the lead of the
national CLARIN-IT12 consortium. Its main
areas of competence also include Text Processing,
NLP, Knowledge Extraction, and Computational
Models of Language Usage. Among ILC’s
resources, ImagAct13, a multimodal resource about
action verbs, represents a starting point for
crowdsourcing experiments, where words denoting
actions could be explained through videos sharing
a semantic core. Crowdsourcing could be used
to build these datasets by asking learners to
label actions shown in short videos. As shown with
middle school pupils
Bruno Kessler (FBK) is working on Automatic Speech Recognition (ASR) and addresses computer assisted language learning as an application field. In a first project, it aims to automatically assess children’s reading capability at primary school. ASR is used to align a given text with the speech read out by a pupil, to highlight its errors and score it. A second project concerns the use of ASR and classification tools to automatically check the proficiency of Italian students aged between 9 and 16 years, in learning both English and German. Both written texts and spoken utterances have to be evaluated, using reference scores related to some proficiency indicators (e.g. pronunciation, fluency, lexical richness) given by human experts. In the first project, corrections of ASR errors can be crowdsourced and used to build more reliable models for assessing reading capabilities of children. Similarly, in the second project crowdsourcing could help both to transcribe and to score the answers uttered by the students. In both cases, crowdsourcing could allow to adapt ASR models and produce more reliable gold standards. 5
In this section, the most relevant achievements14 related to the overall progression of the network 12www.clarin-it.it 13www.imagact.it 14See more information on http://enetcollect.eurac.edu. are reported in relation to the formal Research Coordination and Capacity-Building Objectives outlined earlier in Section 2.15
The already large initial number of 68 individual members for 34 participating countries has increased by 67% to 114 members and by 10% to 38 countries. The people subscribed to enetCollect’s mailing list have increased by 149% from 79 to 197. Also, 15 financed research stays, lasting 152 days overall, led to intense cooperations. Building the theoretical framework. The 30 presentations and 39 posters at network meetings and 15 research stays have contributed to the first building blocks of the foreseen theoretical framework, especially with regards to the state-of-theart review. So far, 3 meetings and 1 training school were organized (168 participations in total). Communication and outreach. EnetCollect’s intranet and website are online for 9 and 7 months and host already a substantial amount of information. 11 mailing lists targeting subsets of members were created and used. 4 calls for research stays and 5 calls for meeting participation were distributed and drew attention (and members) to enetCollect. Aside from one invited talk, several early activities for publications at conferences of related research communities are ongoing.
were supported early on, e.g. through the advertisement of specific opportunities or dedicated internal campaigns (e.g. for Marie SklodowskaCurie Individual Fellowships). Three applications for mid-sized projects were already submitted in the first year, of which two got positively evaluated, and one got funded by a Swiss agency. 6
We presented enetCollect, outlined its key aspects and introduced both its Italian members and their research interests. By harnessing even a fragment of the crowdsourcing potential existing for all languages taught worldwide, enetCollect could trigger changes of noticeable impact for language learning and language-related R&I fields, such as NLP. The fast uptake and overall progression of enetCollect within its first year indicate its relevance and the potential magnitude of its ambition.
15We do not report on content-related results as these are too numerous and varied and, more importantly, they are (or will be) the focus of different publications authored by the members having achieved them.