We describe an adaptive neural machine translation platform which integrates continuous learning and supports multiple use-cases in the translation industry. The application is being developed and evaluated within the applied research project ITAI. Research within the project has shown the potential of the platform to cover the main identi ed use cases and provide rapid adaptation via continuous learning.
Neural Machine Translation (NMT)
This technology is being increasingly integrated in the translation industry to support growing translation needs in the digital era.
Providing adequate support to the translation industry requires taking two main aspects into account.
First, actual practices in the industry feature a wide array of scenarios depending on the IT infrastructure at hand and the network of translators working for a speci c company in the eld. Translation may thus be performed via computer-assisted translation (CAT) tools such as SDL Trados Studio or Wordfast, to name two of the main ones, in Content Management System (CMS) environments, or directly performed in document editors such as Libre O ce or MS Word. This disparity makes it di cult to bring MT technology to a signi cant portion of the translation industry.
Secondly, MT systems are usually only
updated periodically when signi cant volumes
of new training data become available and,
therefore, do not provide timely adaptation
of MT output corrections generated via
postediting. This limitation can result in a loss of
productivity and increased frustration on the
part of translators tasked to repeatedly
correct identical errors over time when querying
MT engines. Continuous learning (CL)
addresses this issue via continuous updates of
MT models on the basis of post-edited
machine translation output fed back to model
training processes. In NMT, CL usually takes
the form of Online Learning (OL), where
each new pair of source sentence and
postedited translation is used to update the
corresponding model
One key aspect in CL is determining the proper trade-o between rapid adaptation of the models from user corrections and model stability over time, a topic which has only Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). been partially explored so far. Optimal integration of CL methods remains a matter of active research and is of key importance to provide useful adaptive machine translation technology.
In this paper, we describe ITAI, an adaptive neural machine translation platform which integrates continuous learning and supports multiple use-cases in the translation industry. The platform is being developed within the applied research project ITAI, partially supported by the Department of Economic Development of the Basque Government (Spri). The project started in April 2019 and will nalise in December 2021.
It is carried over by the following consortium: MondragonLingua1 (project coordinator), Emun2, iAmetza3, Mixer4, Tai Gabe5 and Vicomtech6. The project takes into account the translation requirements from each company and continuous translators' feedback across development cycles. 2
The architecture of ITAI is described in Figure 1. The application consists of the following main elements: • Front-ends, from which users may interact with the application. The front-ends include a web-based user interface (UI), plugins for speci c CAT tools, and supports CMS integration. • A REST API, which exposes the functionality of the back-end and handles user authentication. • A back-end, which includes the required components to perform machine translation, manage the data generated from the use of the system, and manage the training and selection of continuously updated NMT models.
The core work ow involves users requesting machine translation of texts or documents, post-editing the automated translations as needed, and sending validated translations to the system. These validated translations are fed back to the NMT models via 1https://www.mondragonlingua.com/en/ 2https://www.emun.eus/en/ 3https://iametza.eus/ 4http://www.mixer.com.es/es/mixer/ 5https://www.naiz.eus/ 6https://www.vicomtech.org continuous learning to produce incremental improvements of the models.
The ITAI UI is implemented in Angular and the back-end components in Go. We describe the main components and functionality in more detail in the following sections. 2.1
As noted in the introduction, translation activities in the industry cover a wide array of usage. To provide support for the main identi ed use cases, the application supports different entry points.
We rst addressed the most commonly
used frameworks for multilingual content
generation. CAT tools are an important part
of the translation ecosystem, and we
developed a speci c plugin to connect the
popular SDL Trados environment to the
application, similarly to
We also developed a Web-based user interface to provide an additional access point to the functionalities of the system. Such an environment was identi ed as necessary for two main reasons. First, translation is also being carried out professionally outside dedicated environments such as CAT tools and without any technological support to improve productivity. Secondly, some proprietary CAT tools do not support the transfer of postedited translations to external applications and users' feedback cannot be re ected in the MT models. In either case, users are thus limited in their interaction with supporting MT technology.
To address these issues, the ITAI UI offers a full- edged access to the MT technology supported by the back-end. Users with little or no access to MT technology may thus upload documents and retrieve automatically translated documents maintaining the original format. The translated documents can then be post-edited in an external environment and the resulting validated translations uploaded via the UI, where the content will 7https://www.ubiquo.me/ UI CAT
CMS FRONTENDS
Translate text | document Post-edit and validate
API be extracted to feed the MT models. Dual use-cases are also supported, where users may work with a CAT tool that supports the integration of ITAI MT services, and upload translation memories or documents containing their post-edited data via the ITAI UI. Additionally, the UI provides a simple editor where users can directly post-edit machine translated output that has been automatically segmented and ltered, as a default environment to correct and validate translations prior to sending them for integration in the MT models.
The user interface also provides dashboards to monitor volumes of translated and validated data, and list content that is pending validation to dissociate use of MT from the provision of post-edited data, as priorities usually di er for these two activities.
The ITAI back-end provides support for three main types of functionality, which we describe in turn in the next sections. 2.2.1 Machine translation Machine translation is carried out with Vicomtech's Itzuli MT toolkit, via its two main components: itzuli-translator and itzulidoctrans.
The former performs text translation and
can be deployed in scalable Kubernetes mode
or as a standalone platform in a dedicated
server. It integrates MarianNMT
(
Document translation is done via itzulidoctrans, a robust component for translation of documents in a variety of formats (odt, docx, xlsx, pptx, html or xli , among others). It performs content extraction, text translation via itzuli-translator, and document reconstruction with format preservation.
Itzuli is a validated platform which supports large-scale translation services and provides ITAI with robust MT functionality.8
All NMT models currently deployed in
ITAI are Transformer models
As one of the main goals of the project is to gradually increase the quality of MT models via continuous learning from user-generated corrections and validations, data management is a key functionality of the platform.
Since the platform allows for the provision
of post-edited data via documents, in
addition to the provision of segment-level data,
the component supports sentence alignment
with a combination of the metrics generated
8It notably supports the internal and
public MT services of the Basque Government
(https://www.euskadi.eus/traductor/),
MondragonLingua's commercial MT services for Basque
(https://lingua.eus/eu/itzultzailea) and
domainspeci c translation, and Vicomtech's public platform
for the improvement of Basque translation technology
(https://www.batua.eus/).
by the HunAlign
Finally, the component also generates translation memories from validated aligned data, which users can download as a byproduct of the data management processes.
Previously unseen data validated by users reach the model management component, where continuous learning takes place. New pairs, consisting of a source sentence and its validated translation, are used to adapt the relevant models, with a single update for online learning using the appropriate learning rate for the selected optimiser. Automatic evaluation then takes place to measure the impact of the update on both the new pairs and static test sets for the models at hand.
Although online learning is a relevant framework to adapt MT models on the y, it is still an open matter to determine an optimal balance between aggressive adaptation, required for online learning to take e ect on the basis of single data points, and model stability over time, necessary to maintain the overall quality of the models.
Several experiments are being carried out within the ITAI project to determine the appropriate con gurations in this respect. Current results tend to favour a hybrid approach, with online learning performed for rapid MT adaptation useful to the users of the system, and batch ne-tuning over prior model training checkpoints once the volumes of accumulated new data reach a signi cant threshold. Model management processes for continuous learning will be adapted as necessary as nal conclusions are reached within the project regarding continuous learning for neural machine translation. 3
In this paper, we described a neural machine translation platform which supports continuous learning and multiple use cases in the translation industry. The application is already operative within the applied research project ITAI and will be nalised in 2021. Research within the project has shown the potential of the platform to cover the main identi ed use cases and provide rapid adaptation via continuous learning. It also uncovered the need to further explore continuous learning for neural machine translation to reach an optimal balance between rapid adaptation and model stability over time.