Traditional Data Warehouse (DWH) architectures are challenged by numerous novel Big Data products. These tools are typically presented as alternatives or extensions for one or more of the layers of a typical DWH reference architecture. Still, there is no established joint reference architecture for both DWH and Big Data that is inherently aligned with business goals as implied by Business Intelligence (BI) projects. In this paper, a work-in-progress approach towards such custom BI architectures, the GOBIA method, is presented to address this gap, combining a BI reference architecture and a development process.
design science approach [
The following fundamentals explain the basic concepts regarding architectures
and Business Intelligence required for the proposed solution and outline the
problem statement to be addressed. De nitions of and further reading on the
basic terms Data Warehouse and Big Data can be found in [
The term Business Intelligence (BI) is used to describe a holistic enterprisewide approach for decision support that integrates analytics systems (e.g., a DWH), but also strategy, processes, applications, and technologies in addition to data [2, p. 13]. Besides that, BI is also said to postulate the generation of knowledge about business status and further opportunities [2, p. 13]. More importantly, a crucial aspect of BI is its alignment to its business area of application [2, p. 14]. This implies that BI and also its parts (including an analytics system) should be aligned to the respective business in order to support decision making for business operations.
While a traditional DWH architecture has well-de ned layers such as the
staging area (Extract-Transform-Load, ETL) or data marts [
However, no BI reference architecture has been established yet that is inherently technology-independent, i.e., usable for both DWH and Big Data, and addresses the business-alignment of BI. Such goal-orientation aids the selection of customized architectures, since speci c goals can be considered in the process.
As several combinations of technologies and products can be placed in an analytics architecture nowadays, the potential complexity of architectures is increased. For instance, certain Apache Hadoop distributions (e.g., by MapR2 or Hortonworks3) present all of their o ered product options in a single package, 2 http://doc.mapr.com/display/MapR/Architecture+Guide 3 http://hortonworks.com/building-an-enterprise-data-architecture/ where no process to a customized architecture is outlined and the necessary architectural choices are left to the implementer. For instance, if a weather prediction BI application should be implemented using these Hadoop distributions the tting products have to be chosen. While these choices could possibly be made with certain e ort, e.g., Apache Storm for streaming weather data processing and MapReduce for batch analytics, the process of arriving at these decisions cannot be supported best solely by considering a (simple) classical layered view as with the DWH reference architecture before. Previously, this view was su cient as typical products were located mainly in the DWH sphere, but to match todays complexity and heterogeneity from an architectural point of view, the classical layered view needs to be further re ned.
Reference architectures used in computer applications typically exhibit a
layering of services [
An example for a service hierarchy is a high-level telecommunication service provided to an end-user that can be comprised of several lower-level services in the back-end. In a data analytics scenario, high-level analytical services could be placed in a core analytics layer (e.g., "Cluster customer groups" or "Sentiment analysis of product-related tweets") and be consumed by BI applications on top, possibly supplied to by a middle-ware (e.g., data marts). These services are provided for by data preprocessing services (such as "Cleanse customer data" or "Filter tweets") at a lower layer and are ultimately based on several data sources (e.g., "Twitter" or "ERP"). Each of these services can be allocated, respectively be backed, by a novel or traditional product. However, the mentioned challenge of actually allocating these heterogeneous products to layers or services in a speci c scenario remains and needs to be addressed (cf. Fig 1). We do so using a service hierarchy within a layered architecture that serves as a guide towards a nal implementation of a customized architecture, since it allows for a clear structuring of complex architectures in a modern heterogeneous product landscape.
Abeck et al. see a layered architecture as a foundation for (software) reference
architectures, as software systems development would be based on layering [
The proposed approach is termed the "Goal-oriented Business Intelligence Architectures" (GOBIA) method and consists of a BI reference architecture (GOBIA.REF) and development process (GOBIA.DEV). In the following, both artifacts are brie y presented.
GOBIA.REF aims to address the architectural gap outlined above and is intended as a layered, technology-independent BI reference architecture. It is accompanied by a development process (GOBIA.DEV) that aids in its customization, so that the outcome is aligned to the goals and requirements of a speci c scenario or application. This inherently supports the principle of BI to be business-aligned. The resulting architecture is a high-level conceptual model resembling a service hierarchy, which is not yet focused on technical details, but aims to alleviate the challenge of implementing the architecture (i.e., assigning speci c products to the de ned roles and functions). 3.1
The proposed BI reference architecture (see Fig. 2 on the right) as a layered architecture generalizes DWH and Big Data in the analytics layer as "BI functionality" as common denominator. The customized architecture is built based on this reference architecture and should be seen as a service hierarchy.
Data sources of the architecture reside at the bottom of the reference architecture. These can be located internally or externally (e.g., in a cloud). While this is comparable to other architectures, no restrictions are imposed on data formats or delivery and persistence modes. For instance, data source blocks could simply be "Mapping data" or "Transportation routes". The "Data Preparation and Preprocessing" above it ful lls a similar purpose as the staging area in a DWH, but the tasks should be more coarse-grained and mostly omit technical details. For instance, a task in this layer could be to "Transform mapping data" or to "Complete disease data". Instead of having a DWH and/or Big Data tools in the core analytics layer, this layer contains BI functionality in general, which is technology-independent and focused on the results of BI. For instance, BI functionality could include high-level functionalities such as "Classify customer into types" or "Identify sales patterns". Data marts, as in a DWH, can ful ll the role to provide subsets of data to the BI-speci c applications. As the layers are conceptual, a decision whether to materialize any of these subsets is not made at this point.
BI-speci c applications consume the BI functionalities delivered through the data mart layer to deliver applications to a client or end-user, much as in many other architectures. The di erence, however, is that GOBIA.REF aims to clarify on the actually needed BI functionality so that the choice of selecting suitable technological artifacts afterwards becomes less complex. 3.2
The proposed development process of the customized architecture, GOBIA.REF (see Fig. 2), is designed so that actual goals and requirements on a target BI system are derived from a more coarse-grained strategy, which is assumed to be already de ned. The latter, indicated as (0) in Fig. 2, allows to derive application domain(s) and scenario(s) (use cases) from it in step (1). The underlying domain should de ne the playing eld laid out by the strategy (e.g., nance, health care...). The scenarios, set in the domains, de ne the requirements and goals of the customized architecture (2), and business-relevant information such as costs, expected value, or revenue. A de ned goal could, for example, be to "Analyze customer behavior to map his characteristics to products that the he might nd interesting". At this point, the BI-speci c applications required at the top-most layer are determined.
This is followed by a co-alignment step (3). The main outcomes are BI functionalities to be placed in the architecture, as well as necessary data preparation or preprocessing tasks, and data properties of suitable data. For this, requirements and goals are aligned together with BI functionality and data properties. The result should be that, eventually, suitable BI functionalities realize the goals and adhere to the requirements set before and that these BI functionalities and data preparation tasks t the data properties. If, e.g., a goal was to di erentiate groups of customers, BI functionality for a suitable clustering method must be de ned.
Data properties are characterized by using the "V's" [
Notably, co-alignment can also mean that requirements and goals are adjusted as well in the course of an iterative de nition process. For example, if data properties for an initial set of requirements and goals are characterized and the data is of higher quality than expected , goals could be re ned to explicitly exploit this data. This re nement, then, could lead to a further adjustment of BI functionality or data preprocessing tasks.
Input to the requirements and goals in step (2) is a BI reference architecture (GOBIA.REF). Also, there can be a direct strategic impact on it, e.g., a decision not to have any data marts in the nal architecture. Moreover, domainspeci c template reference architectures could be possible like, e.g., a set of typical nance-algorithms as BI functionality templates.
Finally, in step (4), the customized architecture is assembled by assigning the outcomes of the co-alignment (e.g., BI functionalities) to the respective layers and by building a service hierarchy. This high-level conceptual output can be used further in the implementation of the target BI system. 3.3
For illustration purposes, a sample use case is brie y discussed and its outcome presented (cf. Fig. 3). This ctitious case is tailored towards a global organization concerned with the health of people, e.g. the World Health Organization (WHO). Firstly, the GOBIA.DEV process is executed to determine the goals and requirements and to present the functionality at the di erent architectural layers. Then, GOBIA.REF is used to assemble this into a layered and hierarchical form that can be used as a blueprint for a subsequent implementation.
The set strategy (0) could be the objective of the WHO, which "is the attainment by all people of the highest possible level of health" 4. Naturally, healthcare is set as application domain (1). In this scenario, global disease management should be the speci c application that allows for disease monitoring and pattern recognition to facilitate appropriate mitigation or containment procedures and that supports the set objective of the WHO.
The goal (2) of the BI system should be recognizing disease patterns around the globe to allow description and comparison of current disease spread patterns to eventually allow for enhanced monitoring. To achieve this, several data sources are required. These could be con rmed disease cases ("disease occurrence data"), incoming reports of potential, uncon rmed diseases ("emergent disease information") and knowledge about diseases like symptoms ("disease data"). To create a map and to predict endangered areas in the future location data | ranging from basic maps to more re ned data as health infrastructure and environmental data, which could in uence disease spreadings and potential dangers | is needed as well as, e.g., public and private transportation routes such as ight routes or roads. Furthermore, appropriate algorithms are required to map health dangers of certain levels to appropriate mitigation or containments procedures.
The results of the subsequent co-alignment lead to a service hierarchy or layered architecture as shown in Fig. 3. To conduct co-alignment, the properties of the required data sources should be assessed. For instance, because of the extensive mapping data, potential data volumes could be regarded as "high". Besides technical properties (volume, variety, velocity), qualitative properties as "value" can be assessed. In this case, value could be rather low for uncon rmed, emergent disease reports due to the uncertainty and poor initial data quality and be potentially high for actually con rmed disease cases from, e.g., hospitals. With this, the required BI-speci c functionality can be formulated that supports the goals of the application as well as the necessary tasks that, e.g., deal with the data properties (such as the cleansing of uncon rmed disease cases). These tasks could include a classi cation of emergent disease information (i.e., if its actually a con rmed case or an irrelevant report). By clustering the cleansed and completed diseases occurrences, these can be integrated into the preprocessed mapping data. Lastly, disease patterns could be recognized and eventually plotted on an interactive map or assembled into regional health reports. 4 http://apps.who.int/gb/DGNP/pdf_files/constitution-en.pdf
This work has tried to outline gaps in "universal" reference architectures that did arise as a result of moving into the age of Big Data. A proposal for a BI reference architecture based on a basic concept in Computer Science was made. A development process has been proposed to support a goal-oriented creation of a customized BI architecture, yielding a possible prerequisite for choosing suitable analytic tools.
Future work should address various parts of the proposed method. Firstly, the proposal is to be re ned. For example, the semantics in the development process needs to be elaborated upon, and inputs and outputs be speci ed in more detail.
Secondly, the steps following an execution of the development process are to be elaborated, since the high-level conceptual model output cannot be directly operationalized. The challenge to select technological artifacts (e.g., from a Hadoop distribution) and connect these to realize the concept is not resolved yet. Here, best practices or generalizations of architecture setups could be derived in order to address this challenge. It should also be elaborated how ndings from these can be generalized to templates to enhance the method itself. For instance, best practices could be used to derive domain-speci c reference architectures or prede ned building blocks for the co-alignment step in GOBIA.DEV (e.g., common data processing tasks that address certain data properties or speci c BI functionalities).
Thirdly, both reference architecture and development process should be evaluated empirically to test if they t their intended usage. Such an evaluation should build, for example, a customized architecture based on a Hadoop framework (e.g., MapR) to verify whether the process is indeed less complex when using the GOBIA method. Also, such evaluation should include a comparison to other existing approaches (e.g., for reference architectures) to better assess to which extent GOBIA.REF and GOBIA.DEV can utilize the proposed advantages in practice.