Data quality evaluation and improvement is an important asset in every system, particularly in systems which aim is to analyse data, such as those that are based on multidimensional data models. When talking about data quality the main approach found in literature is fitness for use which means that data quality cannot be evaluated nor improved without taking context into account. Evaluating data quality over systems with a multidimensional model is clearly context dependent. However, there is not enough generality in the solutions found in the literature for context-based data quality management, which means that for every particular case the problem needs to be redefined. In this PhD proposal we aim to reach to a formal definition of every concept mentioned above and their interactions. As a result it would be possible that, given a particular multidimensional model and its context, a set of Data Quality rules can be generated in a simple way.
adopted [1], accepting that DQ cannot be evaluated nor improved ignoring the information about the context Data Quality (DQ) is a multifaceted concept, since there where data will be used. In the case of DWs, context are many aspects that can be taken into account when can be useful for compensating missing data, correcting trying to define and measure the quality of data. These errors, detecting inconsistencies, and many other qualityaspects are called DQ dimensions, while DQ metrics are related tasks. defined in order to measure them [ 1]. As DQ is context dependent, DQ dimensions and met
Data Warehouse (DW) systems are decision-oriented rics are specific for each domain and use case, therefore, information systems and as so, a fundamental asset for solutions are highly dependent on each particular case. decision making. DWs are populated with data extracted Formalization is needed to provide an abstraction level from heterogeneous sources which is transformed to be that gives generality to solutions, allowing the instantiaqueried and analyzed with a multidimensional perspec- tion of them for each particular case. tive, allowing aggregations by diferent criteria. Multi- Although there has been certain progress in research dimensional data model is typically used for designing about context-oriented DQ for DW, we believe that there DWs and for doing analysis on top of them. The main is still a deep gap for arriving to well-formalized integral concepts of this model are dimensions, hierarchies, facts and robust solutions. There are few works that propose and cubes, also including as an essential tool, a set of formalizations for these concepts, and they do not address multidimensional operations that allow navigating and DQ as an integral discipline, including DQ dimensions aggregating data. and metrics management, and diferentiating the tasks in
In these systems DQ is an unavoidable issue, since it is DQ management, mainly evaluation and improvement. compromised at diferent moments of the DW lifecycle, This work is a step forward the formalization of DW, such as ETL and multidimensional operations. Specific DQ and context, in a general way, so that it allows manDQ problems appear due to multidimensional model char- aging context-oriented DQ in DW for particular cases, acteristics (described above). DQ management allows DQ in a robust and systematic way. Among DQ dimensions, improvement when it is possible, and also DQ awareness we focus on consistency, accuracy and completeness [1], by the user, ensuring decision making is not biased by as they illustrate very common DW quality problems. poor quality data. The rest of the document is structured as follows: in
There is consensus in the literature about the impor- section 2 we mention some works related to DW, DQ and tance of considering context in DQ management. The context focusing on existing formalizations, in section well-known fitness for use approach has been widely 3 we present the PhD proposal in terms of the problem and solution approach, and in section 4 we conclude and mention the next steps to be followed. is presented. The authors show that, although there are many works that consider the context for DQ, and many As said before, DQ is context dependent [2, 3, 4, 5, 6, 7], that consider the context for DW, very few address the as it is perceived diferently according to the application problem of managing DQ in DW considering the context domain of the data, the user or even the location in which (i.e., relating the three issues simultaneously). it is being used. For this reason, the context becomes an In [18] a model in which DQ is addressed at the ETL essential part of DQ definition. On its own, data context process is presented. Domain ontologies are used to is an ambiguous concept and in general it is specifically model the process and business rules are mapped to defined for each particular application. Commonly, it those ontologies through diferent quality metrics. Even refers to user and location aspects [8, 9, 4, 10], but many though it is not explicitly said, both ontologies and busiother aspects can be considered for its definition. ness rules are considered as context.
What is left of this section is focused on the formaliza- Both [19, 20] are based on [21], where Hurtadotion of context and of the solutions for DQ over Multi- Mendelzon’s multidimensional model for DW is predimensional Data using context, as these are the main sented, making minor adaptations to it, according to the aspects addressed in our work. specific needs of each work. Even if the main goal is, in both of them, context based DQ evaluation, the use of Context Formalization. Considering works that pro- the multidimensional data model difers. pose formal models for contexts, two interesting ap- In [19], the multidimensional model is used to model a proaches were found: using ontologies [11, 12, 13, 14, 15] part of the context that includes relationships between its and using first order predicates [ 16]. components. To give context to relational databases, the
When using ontologies, some works present formal authors adapt the model of [21] to a relational schema specifications that are absolutely domain dependant. In and combine it with quality predicates. [12], the proposal of an access control mechanism is con- On the other hand, in [20] quality evaluation over a structed by the context and the user profile, both modeled multidimensional model is specified with logical rules using ontologies. The main drawback of this approach that includes context. In this case, the specification preis that it only considers the user context and that it is sented in [21] is adapted to be used in the rules definition. proposed for a specific domain. In [ 11] the context is Although there is a lot of work done in order to formalspecified in a more general way. The authors identify ize the context for a dataset, there are two main aspects components that may belong to any context: people, ac- that remain unsolved: the context formalizations are not tivity, location and computational entity. Each specific general enough so that they can be instantiated for the domain is modeled with a particular ontology that is diferent particular cases, and there is very little work merged with the identified components. A similar ap- on formally specifying the context for DQ management proach is presented in [15], where a context model is in DW. Our work proposes a general formalization of presented considering diferent elements that should be a DW and its context that enables the instantiation for present, such as the local context or the surrounding con- any particular case, and on top of this, it proposes the text. Each of these concepts are later mapped to domain definition and formalization of context aware DQ rules ontologies in order to contextualize DWs. With the idea for evaluation and improvement of the DW quality. of mappings, in [14] domain ontologies are formalized Our work share some aspects with many of the preand used in order to give context for a particular user sented above. We particularly inspire on the multidithat is modeled using an ontology. To do so, a formal mensional model proposed in [19] and the quality rules mapping between both ontologies is proposed. Finally, idea presented in [20]. We find specially relevant the with the aim of obtaining context from an ontology, in mappings ideas presented in [14, 15] and the idea of de[13] a mathematical model is proposed. The authors’ termining the context of a dataset within a particular approach is to calculate the distance between diferent ontology presented in [13]. concepts in an ontology in order to determine the context of particular data.
In [16] first order predicates are used to formalize the 3. Thesis Proposal context. However, the authors do not present a general context formalization that can be instantiated for particular cases. As in [12], the main problem with the approach is the lack of generality.
Existing work about contexts, DW and DQ in general, is analyzed in [17], where an exhaustive literature review This section presents the research problem addressed by this work and the solution approach. First, we illustrate the problem with an example, then we state the problems to solve in a general way and finally, we present the main aspects of our approach, through specific parts of the solution and examples of our proposal, trying to cover the whole picture of the proposed solution. 3.1. Running Example information and some sales in Sales Fact Table will not be considered when the roll-up from Subgenre to Genre takes place. However, when the roll-up is done directly from Book to Genre, no information is lost.
over data attributes, but also includes problems related to multidimensional operations results. To illustrate both type of problems we use an example, whose main concepts are shown in Figure 1. 3.2. Research Problem
The example refers to a Sales DW, implemented in The research problem addressed by this work is the definia relational star schema, which consists of a fact table tion of formal rules for DQ assessment and improvement Sales, related to many dimension tables with data about for a DW, taking the context into account. books, authors, cities, dates, etc. Figure 1 shows Sales In order to tackle this problem, we state the following Fact Table and Book Dimension Table. Additionally, the sub-problems to be solved: (i) formal definitions for both ifgure presents the conceptual representation of one hi- DW and context, which allow the instantiation of any erarchy of Book dimension. This hierarchy is composed particular DW or context, (ii) definition of a mechanism by three categories, named Books, Subgenre and Genre. for the interaction between DW and context, enabling We consider all hierarchies to be homogeneous [21], i.e., the use of diferent formal languages to represent each every member from a category has exactly one parent in one, (iii) definition and formalization of DQ assessment the category above. and improvement rules for the DQ dimensions: accuracy,
Diferent DQ problems may arise in this system, such consistency and completeness [1], and (iv) solution imas the ones showed in Figure 1: plementation, which integrates all the components in a
DQ problems in attributes data. Rectangle 1 shows both unique system. an inconsistency between the attributes language_id In order to test and validate the solution, a real use and language_name and also a semantic accuracy prob- case consisting of a particular DW and its context should lem because “Harry Potter and the Sorcerer’s Stone” is be designed and implemented. Afterwards, metrics and written in English. In rectangle 2 a syntactic accuracy cleaning tasks for consistency, accuracy and completeproblem is presented, it should say “High Fantasy” in- ness, should be implemented, as instantiations of the stead of “High Fantsy” . proposed DQ rules. Finally, we should carry out a com
Summarizability problem. Rectangle 3 shows summa- parison between the obtained results with our solution rizability problem [22] over Book dimension. Ideally, a and results obtained with an analogous solution that does roll-up from Book to Genre and the composition of the not consider context in DQ evaluation and improvement. roll-up operations from Book to Subgenre and from Subgenre to Genre should return the same result. However, due to a DQ problem they may not return the same result. 3.3. Approach When looking at Book Dimension Table, book with id 1 We use the formalization presented by Hurtado and does not have a value in the subgenre attribute. This Mendelzon [21] to formalize the DW, making some mimeans that a roll-up from Book to Subgenre will loose { ,
} { ,
} nor extensions and modifications in order to adapt the model to our goals. is a data type. { {
The context is modeled through domain ontologies: given an OWL ontology , we consider its classes named = {
1, … }; its object properties named = 1 … }, where ( ) and ( ) are its domain and range; and its data properties = 1, … }, where ( ) is a class and ( )
Mappings are defined as a mechanism for the interaction between DW and context (issue (iii) of previous section). They are ternary relations, where the first argument is the DW element, the second argument is the ontology element and the third is a Boolean that indicates if the mapping is total, which means that both the element of the DW and the context represent the same real world entity. We introduce as an example the definition of mappings for Dimensions and Categories.
Dimensions: ⊆ { [1], … , []} × ×
maps DW dimensions, using the specification taken from [21], to ontology classes.
Categories: ⊆ (
1 ∪ … ∪ ) × ( ∪ ) × maps DW categories, using the specification from [21], either to ontology classes or to ontology data properties. If a category is mapped to a data property , then there must exist a mapping between either the dimension to which the category belongs or another related category of the same dimension, and the class ()
The ontologies chosen to give context to Book Dimension presented in section 3.1 are “The British National Library”1 ontology and “The Book Vocabulary Metadata”2 both ontologies represent information about books and other aspects related to them such as authors or languages. it includes more classes and consequently more informa- text and along with it, a way of managing the context
2http://www.ebusiness-unibw.org/ontologies/opdm/book.html
model, the context and the mappings. A set of rules for syntactic accuracy for the category of the Book dimension according to the property dct:title of the “British National Library” ontology is presented in equations 1 and 2, where in the predicate (, ) particular book and ∈ {0, 1} is the result of the metric. , is a ∈ ∧ . ∈ ( ∶ ) ∈ ∧ . ∉ ( ∶ ) → (, 1) → (, 0) (1) (2) owlready24 for managing ontologies.
sented above are implemented in Python using PyDatalog3 for managing the DW model and DQ rules and
and interaction between ontologies and Datalog, such that their reasoning power can be exploited for DQ rules.
used before for this kind of solutions.
Up to now we completed first proposals of the literature review; a formalization for the DW based on [21] model; and a definition and formalization of the context based on domain ontologies. Following these first steps we proposed a mapping between the DW and the conscope, as a way of determining how much of the domain is being taken into account to give context to a DW. We worked on the implementation of each of the formalized solutions. Finally, we defined and implemented a simple DQ metric for syntactic accuracy dimension in order to test the viability of the proposed solution.
The main focus of our ongoing work is to reach a level of abstraction in the formalization of the DW, the context and their interactions that makes it possible to evaluate certain DQ dimensions for any DW in any context. Currently, we are working on the definition and formalization of complex and generic DQ rules for consistency, completeness and accuracy.
Next steps will concentrate in the implementation of the complete solution based on the proposed formalizations, which will allow the definition of any DW, context and DQ rules set, as well as the application of the solution to a real case study.
formation Retrieval 10 (2007) 143–172. URL: https: //doi.org/10.1007/s10791-006-9018-0. doi:10.1007/ s10791-006-9018-0. [15] O. Barkat, S. Khouri, L. Bellatreche, N. Boustia, Bridging context and data warehouses through ontologies, in: Proceedings of the Symposium on Applied Computing, SAC ’17, Association for Computing Machinery, NY, USA, 2017, p. 336–341. URL: https://doi.org/10.1145/3019612. 3019838. doi:10.1145/3019612.3019838. [16] A. Ranganathan, R. H. Campbell, An infrastructure for context-awareness based on first order logic, Personal and Ubiquitous Computing 7 (2003) 353–364. [17] F. Serra, A. Marotta, Context-based Data Quality Metrics in Data Warehouse Systems, CLEI Electronic Journal 20 (2017) 3:1–3:23. URL: https://www. clei.org/cleiej/index.php/cleiej/article/view/22. doi:10.19153/cleiej.20.2.3, number: 2. [18] S. Abdellaoui, L. Bellatreche, F. Nader, A qualitydriven approach for building heterogeneous distributed databases: The case of data warehouses, in: 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2016, pp. 631–638. doi:10.1109/CCGrid.2016.79. [19] L. Bertossi, M. Milani, Ontological Multidimensional Data Models and Contextual Data Quality, J.
Data and Information Quality 9 (2018) 14:1–14:36. [20] A. Marotta, A. Vaisman, Rule-Based Multidimensional Data Quality Assessment Using Contexts, in: Big Data Analytics and Knowledge Discovery, Lecture Notes in Computer Science, Springer, Cham, 2016, pp. 299–313. [21] C. A. Hurtado, A. O. Mendelzon, OLAP Dimension Constraints, in: Proceedings of the Twenty-first ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, ACM, NY, USA, 2002, pp. 169–179. [22] C. A. Hurtado, A. O. Mendelzon, Reasoning about Summarizability in Heterogeneous Multidimensional Schemas, in: J. Van den Bussche, V. Vianu (Eds.), Database Theory — ICDT 2001, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2001, pp. 375–389. doi:10.1007/ 3- 540- 44503- X_24.