In Natural Language Processing, more complex business use cases and shorter delivery times drive a growing need of smoother, more exible and faster implementations. This trend also requires integrating and orchestrating di erent functionalities delivered by services belonging to di erent technological platforms. All these needs imply raising the level of abstraction for NLP components development. In this paper we present a Model Driven Architecture approach suitable to develop an open and interoperable UIMA-based NLP stack. By decoupling UIMA NLP models from other solution speci c platforms and services, we obtain major architectural improvements.
As Natural Language Processing (NLP) approaches complex tasks such as Question Answering or Dialog Management, the capability for NLP tools to seamlessly interoperate with other software services, such as knowledge bases or rules engines, becomes crucial. Such level of integration may require linguistic models to be shared among a variety of di erent platforms, each of which comes with its own information representation language. Platforms like UIMA1 or GATE2 consist of middleware and tools for designing and pipelining NLP speci c tasks, including support for modeling data structures for text annotation, such as lexical, morphological and syntactic features, which may be embedded in interprocess communication protocols. However, while perfectly suited for annotation purposes, NLP speci c schema languages, such as the UIMA Type System, fall short on ful lling solution-level modeling needs. Model-Driven software Architectures (MDA), on the other hand, are speci cally aimed at tackling the complexity of modern software infrastructures, with emphasis on the integration and the orchestration of di erent technological platforms. The MDA approach is based on providing formal descriptions (models) of requirements, interactions, data structures, protocols, and many other aspects of the desired system, which are automatically turned into technical resources, such as schemes and software modules, by activating transformation rules.
Based on this consideration, we adopted an MDA approach to develop a
\Watson ready"3, UIMA-based NLP stack for Italian, as part of the activity
of the newborn IBM Language & Knowledge Center for Advanced Studies of
Trento4. We wanted our stack to be as open and interoperable as possible, to
help users leveraging the availability of NLP resources and tools in the Open
Source / Open Data space. In addition, our stack aims at being independent
from language speci c issues and domains, to facilitate its reuse across projects
and within our (multinational) Company. The basic idea was to design a highly
modularized general model including all the required structures, and to obtain
technical platform-speci c resources from a suitable set of model-to-model
transformations. Also, we embraced the idea of abstracting semantic information away
from the UIMA Type System, as in [
Here we present an overview of the basic ideas behind our approach, introduce our project, and discuss future directions. At the present stage of development, we can share our vision on MDA positioning and motivation with respect to NLP development (section 3), and we can report our rst implementation experiences (section 4). Finally, we outline some related topic and introduce future works. 2
Natural Language based solutions may require the NLP stack to cooperate with other components in a complex system. Such cooperation typically involves data exchanges with reference to a shared information model. The picture 1 shows the integration of an NLP stack with a Knowledge Base (e.g. an Ontology-based Data Access System) and a Rule Engine.
An UIMA-based NLP pipeline produces an annotated text (step 1 in the picture 1) contained in an UIMA CAS (Common Annotation Structure). A wrapper of the UIMA Type System de nes all the operations needed for a consumer (the Rule Engine in this case) in order to access the CAS and invoke the appropriate operations within the cooperating subsystem when needed (see 4.2). When developing and maintaining the solution, an Engineer builds a rule set (see step 3) in order to process linguistic structures and interact with a Knowledge Base (step 4), which, in turn, uses the annotated text to store assertions as the result of an Information Extraction process (step 5). In a separate ow, the Knowledge Base can be queried by a User through a Question Answering System based on a suitable query language (step 6). The integration of all components involved is guaranteed by a common abstract model (Platform Independent Model) that contains the overall conceptualization of the system. The transition from one
platform speci c data structure to another is handled by a set of Model-toModel transformations (steps 7 and 8 ). The gure also shows the link to legacy (possibly huge) conceptual models, such as the KB ontology (step 9). 3
Model Driven Architecture (MDA) [
The key idea behind MDA is to provide a higher level of abstraction so that software can be fully designed independently from the underlying technological platform. More formally, MDA de nes three macro \modeling" layers: { Computation Independent Model (CIM) { Platform Independent Model (PIM) { Platform Speci c Model (PSM)
The rst one can be related to a Business Process Model and does not necessary imply the existence of a system that automates it. The PIM is a model that is independent from any technical platform; the third (PSM) layer is the actual implementation of the model with respect to a given technology and it is automatically derived from the PIM. Notice that the PIM allows a comprehensive representation of the structure and behavior of the system being developed.
The modeling language is typically UML or EMF7, but it could actually be any other Domain Specic Language (DSL).
Developing powerful NLP tasks, such as Question Answering systems,
requires combining a great variety of analytic components, which is what UIMA
has been designed for. We consider UIMA the standard solution for document
work ow analysis. Within this framework, MDD tools can be e ectively used to
better manage the UIMA Type System. In particular, we decided to look at it
as a PSM dedicated to text annotation. The motivation for leveraging MDD (in
the NLP eld) can be summarized as follows:
{ Formalization: MDA languages are well studied in logics and reasoning
mechanisms can be developed upon. [
In particular, with respect to our architecture, we modeled UIMA annotations by de ning classes rather than just (data) types, so that a consumer is able to invoke operations designed for those objects. Access to UIMA annotation is then achieved by means of automatically generated wrappers. Another motivation for a model driven approach was the need to represent complex linguistic data, and exploit existing tooling and resources for generating training data for a statistical parser.
In sum, we tried to exploit the maturity and exibility of MDD tools while keeping up the power of UIMA as a framework for component integration, pipeline execution, and work ow management in general. As the PIM language, we chose EMF because it is already integrated with UIMA and provides powerful and mature model driven features. Once also the code is generated (by UIMA JCASgen), the type system correspond to an implementation of a (business) domain model, limited to the structural aspects (as opposed to behavioral aspects).
At PIM level, we also have to represent those properties that, once transformed against a target model, give speci c characteristics on that model. For instance, in order to generate the UIMA Type System (PSM) starting from the PIM, we have to represent on the source model whether a class (that is a root in a hierarchy on the PIM model) will be generated as an UIMA annotation or not (UIMA TOP). Here we have taken two possible scenarios into account: { Having an UML PIM, this speci cation is easily accomplished by using an UML pro le8. Pro les de ne stereotypes that can be further structured with custom properties. This way, we have a generic "Unstructured Information" pro le that at least, encompasses an Annotation stereotype; thus a class that is thought to become an annotation will be simply "marked" with this stereotype.
{ Having an EMF PIM (such as our current implementation), we can represent the same thing as an EMF annotation. Therefore, (we apologize for the words con ict) we will have a class annotated as Annotation.
In any case, a class stereotyped as Annotation on the PIM will take the role of a generic annotation for document analysis, independently from the underlying framework.
The main bene t of our approach is the ability to represent NLP objects independently from any particular implementation: we are using di erent (generated) PSMs (that are better explained in section 4) all deriving from the starting (PIM) model, as shown in the picture 2.
These bene ts have certainly a price, that is essentially represented by the cost of developing the necessary transformations. However, following basic assumptions of the MDD approach, we estimate that those cost are well paying back, especially when heterogeneous components have to be integrated, development is managed iteratively, and models are subject to high volatility. 4
In order to better clear up how we are leveraging the Model Driven approach, we list here the artifacts (PSMs and code) we are generating through appropriate transformations that we have developed. Starting from our \application" model: { UIMA type system (we modi ed the existing transformation from EMF in order to avoid any further modi cation on the UIMA type system) { EMF wrapper of UIMA type system { this wrapper also acts as the input for creating the model for the Rule engine as explained below Starting from (our) models of common standard data for parser training such as CONLL, PENN and others we generated all necessary (OpenNLP-speci c) data for training the parser on: { Tokenization { Named Entities { Part of Speech tagging { Chunking { Parsing
To represent the model (PIM), we use the Eclipse Modeling Framework9 (EMF), which represents a de facto Java-based standard for meta-modeling. Informally, we may say EMF represents a subset of UML (the structural part) with very precise semantics for code generation. In the future, we could move this representation to a pro led UML, as mentioned above (see section 3). Furthermore, EMF o ers very powerful generation features. Summarizing, in the current implementation we use EMF in two ways: 1. A language to represent the model 2. A PIM model to generate di erent target PSM 4.1
The NLP Parser component is implemented using Apache OpenNLP10 and UIMA11; it is based on a UIMA Type System built from the Syntax and the Abstract models using the UIMA transformation utility. The training corpora for the parser has to be provided in a speci c format required by OpenNLP. Since the data that we had available for training were in standard formats such as PENN12, CONLL13 and others, some transformations were required. Ecore models have been created for the purpose of representing source formats. Furthermore, some simple JET14 transformations has been developed in order to generate our corpora (in speci c OpenNLP formats)
Compared to other solutions, this makes our infrastructure extremely exible: should the parser be replaced or the data formats changed, the only operation we will have to make is to modify the JET template accordingly. 4.2
As anticipated in 2, in the higher layers of our architecture, we have a Rule Engine that acts as a reasoner on annotation objects coming from the UIMA pipeline. We wanted this layer to be able to call operations implemented on those objects (as explained in section 3) and those objects always implementing the exact interfaces of the (Ecore) PIM model. Given these requirements, we developed a transformation that generates a wrapper of the UIMA type system and that 9 http://www.eclipse.org/modeling/emf/ 10 http://opennlp.apache.org/ 11 http://uima.apache.org/ 12 http://www.cis.upenn.edu/~treebank/ 13 http://ilk.uvt.nl/conll/#dataformat 14 http://www.eclipse.org/modeling/m2t/?project=jet fully re ects the starting PIM model, including operations. Once implemented, the code will be kept up also against future re-generations, thanks to merging capabilities of this transformation. Thus, as shown in gure 1, the Rule Engine \consumes" instances of this wrapper, and still can access the underlying UIMA annotation. We considered the possibility of directly adding these operations on classes generated by UIMA (via JCAS generation utility) but this would not be consistent with our model-driven approach since those operations would not be part of a general, system-wide model. 4.3
As far as the Rule Engine is concerned, we chose IBM Operational Decision Manager (ODM)15. ODM rules have to be written against a speci c model, called Business Object Model (BOM), that allows a user-friendly business rule editing; ODM provides tools to set up a natural language vocabulary: users can use it to write business rules in a pseudo-natural language. Once de ned, the rules are executed on a BOM-related Java implementation named Execution Object Model (XOM). We obtained the BOM by reverse engineering the XOM, and the XOM directly from Java classes (implementing the type system wrapper) generated from our PIM (EMF) model. Therefore, the BOM model can be seen as just another manifestation of our PIM model. 4.4
Our architecture is backed by a Knowledge Base Management System which
stores and reasons on information extracted from many sources. Leveraging on
the Knowledge Model included in the PIM, we were able to integrate an external
pre-existing system, named ONDA (Ontology Based Data Access) [
We have outlined here an innovative approach to NLP development, based on the idea of setting UIMA as the target platform in a Model-Driven development process. A major bene t of this approach consists in giving NLP models a greater value, especially in terms of generality, usability, and interoperability. 15 http://www-03.ibm.com/software/products/us/en/odm/ 16 http://www.w3.org/TR/owl2-profiles/
While developing this idea, we understood that a suitable Model-Driven
machinery for NLP should be supported by speci c design patterns for concrete
models. In particular, the model we have developed has been abstracted both
from morphosyntactic speci city and from semantic aspects. The former
(including part-of-speech classes, genders, numbers, verbal tenses, etc) may
significantly vary among di erent languages; the latter (including concepts like
persons, events, places, etc) are related to speci c application domains. By
decoupling these layers, we achieved a lightweight \generic" UIMA type system[
We implemented a rst prototype of a Knowledge Base query system based on the Eclipse Modeling Framework (EMF). For the future, we are considering the possibility of representing the model in UML, in order to have a greater representational power (such as modeling sequence diagrams).
The work presented here is still at an early stage. More work is needed to complete the linguistic model, for instance in the area of argument structures, such as verbal frames. From an implementation standpoint, our priority is to consolidate, improve and extend the set of Model-to-Model transformations, and to further exploit MDD tools.