This position paper discusses the design of an approach to enable trusted data integration in a multi-cloud environment in the presence of heterogeneous and large data sources. This approach is based on mechanisms to compute trust in data and its providers by applying statistical and probabilistic methods on its provenance. The result is a solution expressing analytics queries as services coordination that can be enacted on multi-cloud settings. This paper describes the associated challenges and possible ways of addressing them. The approach and challenges are based on concrete requirements stemming from a medical scenario related to understanding, modelling, and predicting patients' conditions associated with sleep apnoea.
patient Alice? The number of apnoea intervals of my patients during the last 10 days? The number of apnoea The digitalization of companies implies the explosion intervals/night when the patient Bob slept 5H and had a of data to be integrated and analyzed to answer difer- 200 glucose level 2H before going to sleep? ent types of questions for retrieving and analyzing data. These queries can be answered by composing on deOver the past decade, service-oriented environments mand and stream data services. Beyond the methods have made it easier for many users to access data through described by the APIs, the services are tagged with QoS data services. A data service is a software entity accessi- measures such as data freshness, response time, execuble through APIs that describe the methods that provide tion price, among others. As a result, the data services data either on-demand or continuously. composition includes a combination of QoS preferences
Companies willing to make data-driven decisions cur- that can be interpreted as constraints. rently use services and process large amounts of data The notion of service level agreement (SLA) can define and resources. However, these companies have to deal the agreement between the users and the set of services with the eficiency and cost of processing linked to this used. From this contract, the user’s preferences and reavalanche of complex multi-source data deployed on sev- quirements emerge. For example, a user can: eral clouds for economic reasons. In this context, several economic actors are led to cross-reference this volumi- 1. privilege the data freshness of the data used to nous data through data integration, guided by queries answer her/his query, like 1month old data; that specify the data required by an application, a user, 2. the availability of the service; or a community of users. 3. the latency of data delivery constraint to millisec
For example, in the e-health context, there can be ser- onds. vices providing information about physiological metrics In this context, the challenges and questions that drive of people, apnoea events, during sleeping hours, glucose the our work are: measures by day, dietary and training sessions information. Through these data services, applications can be, for example, access training programs performed by the • What models, mechanisms, algorithms are suitable for data integration when qualitative and quantitative criteria guide it? • What are the criteria, constraints & requirements on the data that guide queries evaluation and data integration across multiple providers? • When we invoke machine intelligence, the question is what intelligence mechanisms to consider for making integration intelligent? • Finally, does the cloud bring specific challenges when making data integration intelligent?
This position paper discusses the interest of treating munity of users. The classical view of data
integraanalytics queries as trustworthy service-based coordi- tion, where data sources are known in advance, has
nations and considering them as first-class citizens. We been widely addressed in databases: data model
equivaintroduce an approach to answer analytics queries on lence and transformation, schema integration, and query
medical data by building trustworthy data integration. rewriting algorithms. In data integration in the presence
The approach considers that data is provided by services of services that act as data providers, the starting point is
deployed on diferent cloud providers. Therefore, the a query that expresses needs in terms of the data required
paper highlights the challenges to running service-based and must search for services that can meet these needs.
analytics queries in a multi-cloud environment. The ap- In the approaches [
Accordingly, the remainder of the paper is organized ing problem using matching and service composition
as follows. Section 2 discusses related work regarding mechanisms [
Heuristics and "best-efort" approaches have already
been proposed in the fields of databases [
The classical view of data integration has been widely
addressed in databases through data model equivalence and
transformation, schema integration, and query rewriting
algorithms. The main feature of these approaches is the
prior knowledge of the data sources when performing
the integration process. The emergence of data services
has revolutionised the problem of data integration
because services provide data whose sources and format are
not known. Data integration with data provider services
starts with a query expressing the data requirements and
searches for services that can provide them. Also, in
the approaches [
Data integration [
<patient, provider, time, d1, … dn>
Rhône SLA
provider
provider
OSF (Sleep Observatory of the
The approach proposed in this paper has been proposed in the context of the project SUMMIT. As a technology transfer exercise, the project has a partner, the startup Datamedcare, that promotes the platform ADEL that integrates diferent actors that intervene in treating a condition called sleep apnoea.
Neumologists treat the condition, and part of the treatment includes using devices called CPAPs intended to be used every day during people’s sleep. The device is technically calibrated according to patients’ physical and physiological characteristics. Doctors receive records that include the use of the device and other information from the patient and decide how to adjust the treatment if things are not going well.
The platform ADEL integrates the information and provides a global view of the protocols implemented to follow patients. The integration is loosely coupled, preventing doctors from automatically analysing conditions by correlating data. Besides, given the medical context, it is essential to ensure the respect of SLA so that doctors, patients, and Care providers can be confident about the quality of the data and the data services used to exchange them among these actors. The data provision strategy of ADEL does not technically consider this SLA notion. 3.1. Servitization of the ADEL platform First of all, behind the scenes, we worked on a servitization phase of the current setting of the ADEL platform (see Figure 1). The objective of this phase was to ensure the independence of the actors and the type of data and infrastructure they use to produce and manage this data.
Servitization was important because we included a QoS dimension considering that the function of data providers and consumers is not enough to be able to expose SLAs.
We defined five services groups hosted in secure private or public cloud providers (Azure, Google Cloud and AWS) to ensure diferent quality guarantees. The groups consist of services related to the following data: (1) Observations collected by CPAPs (the masks used by patients to treat their apnoea). (2) Questionnaires used by neumologists to follow patients. (3) Health care providers’ data regarding patients’ subscriptions. (4) Pharmacies data about treatments for apnoea patients. (5) Data collected by the association of neumologists studying apnoea.
In this context, analytics queries issued by actors can specify SLA contracts regarding, for example, the availability of the services managing data, their response time, and the quality of the data like freshness, frequency of upload, etc.
In this setting, we assumed that analytics queries with
their SLA specifications are rewritten into services
compositions enacted to answer them [
Given the complexity and non-triviality of the apnoea case, the questions and requirements can be summarized in two categories: Apnoea questions to answer. In the context of the Apnoea application to which we transfer our SLA guided intelligent data integration through queries we have a kind of baseline questions provided by Apnoea specialists. These are essentially analytics questions willing to understand the conditions and evolution of the disease, particularly concerning the CPAP’s use.
Questions include an analysis of the way CPAPs are used. The idea is to determine the extent to which patients adapt to a given CPAP model and how this adaptation determines whether they will be assiduous. Assiduity can result in a positive evolution of their condition.
Other questions concern the analytics by the companies that have to know how they perform interacting with patients to adopt their product. - data quality evaluation module which implements our observability protocol; - data service trust measuring module which collects both data quality and performance measurements and computes data services trust scores. This meta-data is not included in SLA’s models. In our future work, we will propose the extension of SLA’s for including such measurements.
SUMMIT (http://summit.imag.fr) is a technology transQuery Rewriting as a Service Considering our ob- fer project funded by the Auvergne-Rhone-Alpes region jectives, we have proposed a rewriting service consisting that addresses multi-clouds, intelligent data integration, of three main components (see the centre in Figure 1)): service level agreement and focuses on the context of • Rhone a services composition module guided by multi-device environments in the medical context.
SLA requirements We have the following services available implemented • An SLA observer that monitors the services con- during the servitization process of the ADEL platform tinuously and evaluates QoS metrics (see Figure 1)): (i) the Healthcare providers with informa• A registry which is a services classifier that tags tion about the patients’ insurances; (ii) a service dealing and ranks services with a quality index used by with information about patients’ medical procedures; (iii) Rhone to choose the services to be composed CPAPs data services managing the observations about given a query and its SLA specification. apnoea episodes during sleep collected when patients use their devices; and (iv) computing services that provide 3.2. Selecting trustworthy services analytics functions deployed in diferent cloud providers. We implemented experiments as service-based queries As aforementioned, we target a trustable integration; to coordinate the ADEL platform’s data services with therefore, we describe the SLA aspects and how we inte- data processing operators to answer analytics questions grated them into services compositions to answer ques- regarding the apnoea condition. Experiments focus on (1) tions. Recall that we need to choose the services that will classifying patients according to their CPAP frequency provide data, ensuring SLA expectations for answering of use (i.e., compliance). The objective is to observe the a question. For addressing SLA and given that we work evolution of their physiological metrics as they use their for medical applications, we considered a definition of CPAP. (2) Other experiments address the study of metrics Trust that includes service performance and data quality, regarding the apnoea condition and the use of CPAPs in knowing that there is no or few access to meta-data what time seeking behaviour patterns. (3) Finally, experiments we call black-box services. are devoted to predicting the evolution of patients’ conIn this context we addressed three questions: dition according to the evolution of their physiological status and the use of their treatment. • P1. What is the appropriate model for describing individual data services trust using service performance and data quality factors? • P2. How to collect the necessary information for
this trust evaluation model? • P3. How to define data quality metrics using the
collected information?
To this end, we propose a data quality observability
protocol [
We have used reference questions of the SUMMIT project partners and translated them into service coordinations that implement queries. In this project, we do not address the problem of processing natural language specifications.
Still, we assume that we have a query language like the
one proposed in our previous work [
Classification queries. Q-1: Are there any key characteristics that diferentiate patients whose compliance is less than 2 hours/night, between 2 and 4 hours/night and more than 4 hours/night on average? Are there any key characteristics that differentiate patients whose compliance is less than
2 hours/night, between 2 and 4 hours/night and more than 4 hours/night on average? rop eH ivd lta
h re -ca r e rop eH ivd lta
h re -ca r e
P r e s t a t a i r e P r e s t a t a i r e
Avg frequence by patient patients(P) from allPatients fetch union
AVG +
GroupBy
AVG/ groupby
Other questions that we implemented concern aggregation queries like Q-3: where physiological metrics collected by CPAPs are analysed to observe their evolution along time (see below). Filters can be applied to select data items for specific intervals. Data preparation can be done first to see the data distribution along time and include assiduity factors.
• Q-3: Evolution of pressures, leaks, AHI over time
under treatment. • Q-4: How does compliance changes over time since initiation of treatment. Which factors influence compliance (including type of mask and brand of CPAP).
Modelling and correlation queries. We expressed queries that included joining and filtering data provided by diferent services as data preparation tasks. Then the resulting data sets are used as input to identify the role of several attributes in a level of apnoea condition and discover their role in this value.
The first part of Q-4: is implemented under the same principle of the Q-3 . Then other operations can be applied to address the second part and determine the factors that can influence compliance. To answer the second part of Q-4: a service of type health care provider must be used to fetch the history of the CPAPs versions and brands that every patient has tested. • Q-2: Correlation between Epworth1 and the apnoea- The first tasks of all the service coordinations expresshypopnea index (AHI) at diagnosis. Figure 3 shows ing queries (see Figures 2 and 3) start by fetching and the corresponding service coordination that im- preparing data by selecting, filtering (e.g., items with(out) plements this question. Data are fetched in par- specific values) or projecting data and then applying opallel from two providers, managing the apnoea erations like the union. These operations can be executed data collected from the CPAPs and data from the in parallel or sequentially become the input operators patients’ records. The coordination in the fig- (tasks) that can infer/discover/model or compute aggreure is generic, but there can be other possibili- gations. ties. For example, having two parallel sequences of "fetch, filter and project operations" for each service provider, joining both results and finally applying a model that can estimate correlation.
Prediction queries. On the basis of clinical data, including history and self-administered questionnaires (OSFP), is it possible to predict the severity of the condition with AHI<15; 15 <AHI<30; AHI>30? Having data providers with labelled data collections can support prediction queries. The tasks can include analysing the properties of the attributes/variables of the data (e.g., linear 1The Epworth Sleepiness Scale (ESS) is a scale intended to measure daytime sleepiness that is measured by use of a very short questionnaire. This can be helpful in diagnosing sleep disorders
M C ranage aPAPD t a
P a it e n t C a r n e t C P A P A p p
A P I fetch fetch
Epworth, IAH
Date of the diagnosis join project or in only one cloud? Therefore, trust aspects can concern the cloud providers hosting the execution of a query. The decision making associated with the deployment is an open issue that we are currently addressing. 4.2. Open Challenges: discussion and
position Having worked on the design and implementation of service coordinations that can implement the queries, the next step to address is the cloud or environment in which queries are executed. Besides, several services can be used and available for a given task or a given type of data. There is a decision making problem to address to choose the services that will execute the tasks of a query. Our project can deploy services in diferent clouds and guarantee various services, including trust guarantees. Thus, this concerns a rewriting process that can generate a solution space for a given query rather than one coordination.
In our current work, we have proposed data quality
observability protocol and its associated service TUTOR
[
The open challenge is to include the query rewriting process within an optimisation process that can use a cost model to rank or propose the coordinations that can potentially provide expected results in the best conditions possible. These conditions can include trust aspects associated with the services participating in a services coordination.
Another critical challenge is to reason about deploying
a services coordination that implements a query. The
coordination process can be executed in a distributed
setting. Should the coordination run on diferent clouds
5. Conclusions and Future Work
This paper introduced open challenges and possible
directions for integrating data for answering analytics queries
on multi-cloud environments. The problems discussed
are inspired by a concrete use case related to the analysis
of medical data to understand, model and predict the
condition of sleep apnoea. Our current work concerns the
stabilisation, profiling, testing, and scaling of Rhone [
We are also consolidating services monitoring for
computing services trust indices based on technical metrics
and data quality metrics [
We observe the following main perspectives: (1) Evolving towards trust-based data services recommendation (multi-cloud). (2) Addressing more database challenges like capitalising on case observations to design deployment patterns for data services compositions. (3) Monitoring for collecting knowledge. (4) Proposing enactment and optimisation strategies.
This work is funded by the project SUMMIT, pack ambition program of the region Auvergne Rhône Alpes - P089 - 0718-184-ARA, https://summit.imag.fr.