The CROSSMINER project is an open source project, which is motivated by the increasing dependence on existing open-source software (OSS) to develop new complex systems. The project is a follow-up of the previous OSSMETER project. The complexity and diversity of new CROSSMINER components and existing OSSMETER ones raised challenges related to the integration and the communications among heterogenous built-in components as well as addressing security aspects for the whole project. In this paper, we present a microservice architecture, which is implemented by relying on Docker to support the integration and deployment of the CROSSMINER components.
Project data – – – – – –
Open-source software (OSS) is computer software distributed with a license that allows
access to its source code, free redistribution and the creation of derived works. Unlike
commercial software which is typically developed within the context of the organization
with a well-established business plan and commitment to the maintenance,
documentation and support of the software, OSS is very often developed in a public, collaborative,
and loosely-coordinated manner. This has several implications to the level of quality of
OSS as well as to the level of support that OSS communities provide to users of the
software they produce. Consequently, developing new software systems by reusing
existing open source components raises challenges related to at least the following
activities: i) searching for candidate components, ii) evaluating a set of retrieved candidate
components to find the most suitable one, and iii) adapting the selected components to fit
the specific requirements. Dependence on OSS projects can either be a blessing or a
curse [
The paper is structured as follows: The next section provides a review of related works about the project. Section 3 gives the integration challenges of the heterogenous and legacy components. Section 4 presents the planned integration tasks of the CROSSMINER components. Section 5 describe the outline deployment approach of the CROSSMINER platform. 2
The EU OSSMETER FP7[
This paper is a follow up of the work already done on the EU CROSSMINER 2017
STAF-RPS [
Overview
The CROSSMINER project integrates a set of components, which built using completely a legacy technology stack. Furthermore, other components are added in the current project following the same architectural approach, which includes new source code and NLP (Natural Language Processing) measurement tools, system configuration DevOps tools, workflow-based knowledge extractors and advanced IDE (Integrated Development Environments).
Figure 1 presents the logical architecture of the CROSSMINER platform, by identifying the functional modules that compose the platform and identifying the contribution from the OSSMETER project.
These tools available on [
To handle all these issues, we explored some technologies inspired from microservice architecture to assembly those heterogenous components into a unified platform.
Microservices are essentially independent software services that provide a specific business functionality in a software application. Each microservice is loosely coupled with other services. These services are self-contained and serve a single functionality. The first integration task focuses on identifying the principal macro components, by delineating their perimeters and administering interactions between them, as defined in Figure 2. In this context, we have identified the macro components that support the services provided by the CROSSMINER platform and the clients composed of applications, which consume these services. In our integration approach, we have made the choice of not imposing a common technology for the implementation of the different services that compose the platform and did not make any assumptions about where and how CROSSMINER components will be deployed. Our only requirement is that the components must provide a REST endpoint for their services and that all communication between services and clients or between services itself must be implemented through such APIs.
The second integration aspect is to address the high-level complexity due to the
communication between the front clients and the macro components. Here, the key concept is to
provide a unified REST API that aggregates all the services provided by the other
CROSSMINER components. The new component was added to the architecture: API
Gateway. It is a pattern, which comes from microservices ecosystem and that represents
a single point of control for frontend clients by allowing them to consume services
provided by the different platform components (see Figure 2). Also, it acts as a reverse
webproxy that redirects client’s requests to services provided by the platform components.
With this approach the client only has to know the URL of the API Server to access all
the services provided by the platform. At the same time, the REST API of the
components can be refactored with no changes to the API provided by the API Gateway.
The last challenge is confronted with the need to secure access to the services provided
by the CROSSMINER platform. To this end we setup an authentication system at the
level of the API Gateway based on the JSON Web Token (JWT) security mechanism [
For the deployment of the CROSSMINER platform, as each of the CROSSMINER components has its own procedures and requirements for building and deployment, it is planned to follow a deployment strategy based on containers. This kind of approach provides several advantages related e.g., to the isolation of the components processes inside standalone containers, speed up the container’s deployment, etc.
Regarding the container technologies, Docker has been considered as the favorite standard for this kind of technologies. This is due the fact the Docker execution engine as currently assuming a very stable behavior on the major operation systems (i.e. Windows, Mac OS, and several Linux distributions). Moreover, Docker images are supported by the majority of Cloud Platforms. Docker provides a tool for defining and running multicontainer Docker applications – Compose. Compose uses a file to describe the services that compose the distributed application, by identifying aspects like: the images used by each service, the network configuration behind the services, the dependency between the services, the data volumes required by the distributed application, etc. Compose allows to create containers, configure them and the underlying network, and start the distributed application with a single command.
Figure 3 shows a diagram representing how the current CROSSMINER distributed application is structured.
Metric-Platform – Image with the components of the Metric Platform and some metric providers developed in CROSSMINER. This image can be run as different containers; Metric-Platform Database (oss-db) – MongoDB image to store the data of the metric platform. Currently it runs as one container but may be replaced by a replication or shared MongoDB cluster if needed;
CROSSMINER Admin – a web application developed to administrate the CROSSMINER platform e.g., to create projects, configure analysis tasks, etc; Knowledge Base Service – Image aggregating the components implementing the Knowledge Base. It is used to run the cross-project analyses and provides the API to give access computed data; Knowledge Base Database (kb-db) – MongoDB image to store the data from the analyses performed by the Knowledge Base Service.
DevOps Dashboard [
The CROSSMINER project is an OSS project focus on increasing dependence on opensource software (OSS) and it permits to take educated decisions. The project is a followup to a previous project called OSSMETER. The complexity of its legacy components raised challenges related to the integration and the communications among the heterogenous built-in components. To handle all these integration issues, we explored some technologies inspired from microservice architectures to assembly those heterogenous components into a unified platform and to address the security aspects of the whole platform.
Finally, we adopted a container-based distributed strategy based on the Docker Compose
technology [