Data Warehouse technology addresses business process analysis. However, it ignores upstream decision-making like formulating policies and policy enforcement rules. We provide a requirements engineering approach for building an integrated data for all types of decisions in an organization. To do this, we develop a two level generic platform with the bottom level having generic models of decisions, information, and decision-information association as well as information elicitation techniques for eliciting information for decisions. The higher level is the source of decision for the lower layer and is exemplified by policy enforcement rule decisions as well as operational decisions for managing the business process. Each source produces its own data warehouse requirements specification and these are integrated together using our integration technique.
Data Warehouse (DW) failure statistics highlight the crucial role of RE in mitigating system failure [1]. Hayen [2] refers to studies that indicate the typical cost of a DW project to be one million dollars in the very first year. However, one-half to two-third of these projects fail. One of the causes of this failure [1] is inadequate determination of the relationship of the DW with strategic business requirements. These statistics reinforce the need for RE for DW systems.
DWRE techniques identify DW structures like facts, dimensions and finally arrive at star schema. DW structures are identified from existing systems, information gathered from users’ of the DW or a combination of the two. DWRE techniques are classified into three broad categories based on the nature and phases of the process. By nature they can be top-down, bottom-up or mixed and by phases they can have a singlederivation phase or multiple phases in which the process is performed. A more detailed comparison of top-down, bottom-up and mixed approach is as follows:
available, information, needs to only be converted into the multi-dimensional form.
Thus, the starting point is existing data bases and data sources. Desired facts and
dimensions are then imposed on these available sources. There are two basic
approaches, (a) Database Driven approach that starts from existing databases [3], and
(b) ER schema driven approach that starts from ER schema [
contents of a DW To-Be from scratch. These approaches directly adopted model driven techniques developed in software/information systems RE like goal-orientation and scenario-orientation. User driven and Goal driven approaches of DWRE belong to this category.
Some User driven approaches include techniques developed by [
Goal driven approaches [
III. Mixed Driven Approaches In purely demand driven techniques, the information needed for decision making may not necessarily be available in existing data sources, whereas in purely supply driven techniques decision making may require information outside of that available in existing data sources. This led to the development of mixed driven techniques where the needed information was identified and the available data was determined.
According to the approach of [
1. Lack of DW support for upstream decision-making in an organization:
According to [
In [
OMG, in its Business Motivation Model [
Decisional and Information perspectives have been introduced by [
There is a need to explicitly model the decision-information relationship and treat both decision and information as first class concepts of DWRE.
Limited techniques specific to Information Elicitation:
DWRE techniques are highly oriented towards arriving at information in the
form of Facts, Dimensions and Measures. This is either done directly without
analyzing information and or without sufficiently exploring information before
structuring it. Techniques like [
While arriving at MD structures is essential, it is equally important to elicit, examine and analyze information that is unstructured and also to elicit information in a guided manner.
To sum up, there is need to treat decision and information as first class concepts of RE models, develop decision and information models for conceptual clarity and effective guidance, and to lay emphasis on eliciting early, unstructured information before arriving at multi-dimensional structures.
With three inter-related DW systems, policy formulation, policy enforcement, and operational, there may be common data across them. Therefore, need for integrating these arises. Existing approaches do star schema/data mart integration by identifying conformed dimensions. However, this implies long lead times due to first arriving at the star schemas and then integrating them. Requirements may change during this period and the integrated system may be out of step with desired one. We can now define the following research questions: 1. What are the different kinds of decisions, the applications from which they originate and their inter-relationship? 2. Can we define information elicitation techniques that are neutral to the types of decisions? 3. Having determined information relevant to decisions of each application, do we keep separate Data Warehouses application-wise or do we maintain one integrated form? 2.
In order to answer the research questions, we propose a solution divided into the following steps: 2.1.
We start by establishing the ‘Decision Environment’ and derive a typology of decisions. We define two broad categories of decisions: Imperative decisions and Managerial decisions. Managerial decision-making is upstream and is of two kinds. One kind of Managerial decisions’ is those that deal with formulation of norms and standards that are to be followed in organizations. These decisions are Policy Decisions. The other kind are concerned with the enforcement of given policies. The decision problem here is that of defining an appropriate set of rules that the organization will follow during its operations. These decisions are Policy Enforcement Rule (PER) Decisions. Imperative decisions are derived from policy enforcement rules and consist of operational actions. The imperative decision making problem is that of selecting the most appropriate action in a given situation such that it also does not violate policy enforcement rules. Imperative decisions can be taken once Policy enforcement rule decisions are taken. Policy enforcement rule decisions are taken once policy decisions have been taken. Thus, there is a continuum between policy, policy enforcement rule decisions and operational decisions.
Policy formulation is done in a number of contexts with varied stakeholders [
Our RE process is rooted in Decision Requirement which we model as a tuple <decision, information>. Decision Requirement implies that RE process has two steps, first to determine the choice set of decisions and then to elicit information to choose one from the choice set.
This and our treatment of decision and information as first class concepts leads us a two level generic platform. The bottom of Fig 1 shows the generic platform having generic models of Decision Requirement, decision and information. Information elicitation techniques for decisions are defined in this layer. The higher layer is the source of decisions for this lower layer. Decisions are obtained from PER formulation and operational decisions. This means that the lower layer is neutral to the source of decisions and can be used for any kind of decisions. It is generic.
PER Formulation technique
Operational Decision Source: Higher Layer
Generic Decision Requirement Model Generic Decision Model
Generic Information Model Generic Platform for Decision Information
Fig 1: The Generic Platform For the bottom layer, we propose three information elicitation techniques namely, CSFI, ENDSI and MEANSI as a set of generic techniques for eliciting information for the DW to-be. The elicited information has to be converted to multi-dimensional form and conforms to a generic information model developed in the Generic Platform. For the higher layer, we propose to develop RE approaches for our two sources of decisions. Once this is done, an integration process has to be developed to arrive at the integrated DW. 2.3.
Our process begins with first developing Policy Enforcement Rule DW and operational DW. For the former, we propose a PER life cycle thus address the issue of providing support for strategic decision making. For the latter we propose the operational life cycle. Now the remaining question is to integrate the two DWs into one comprehensive DW for the entire organization. .This makes the integrated DW a single enterprise resource that supports ALL forms of decision making. The integration is done using our Vertical integration technique.
PER life cycle creates a DW for PER decision making, DWper, with a rule base and information in its own operational early information base. The input to the PER life cycle is operational policies. The PER life cycle has two parts; one is to formulate rules and the other to elicit relevant information. We assume that organizational policies are represented in our extended first order logic. Thereafter, we propose guidelines to arrive at PERs. PERs are of the form “WHEN triggering action IF condition THEN correcting action”. In general there can be more than one correcting action for a given condition and the decision maker must formulate the appropriate set of correcting actions. In order words, the decision maker works with the choice set {select action, modify action, reject action}. Choosing one of these constitutes the decision problem.
We now need to elicit information that the decision maker will refer to in the decision making task of choosing the appropriate correcting action from the choice set. The generic early information elicitation approaches namely ENDSI, MEANSI and CSFI, defined in the platform of Fig 1 are used. This constitutes early information, EIper,
1. a source of information for the vertical integration life cycle
2. converted to late, structured information
For the latter we have proposed guidelines to convert early information into ER
diagram. Subsequently, ER can be converted to star schema by applying existing
algorithms of [
Operational life cycle creates a DW for operational decision making, DWop. Actions are first extracted from PERs and each action, a, of the PER layer is treated as a decision, d, to be taken at the operational level. This yields the initial set of decisions and is the input to the operational life cycle. Applying the generic decision model mentioned in Fig 1, each decision is subjected to AND/OR decomposition and generalization-specialization process and leaf nodes are determined. The next task is to elicit information for every decision using CSFI, ENDSI, and MEANSI. The elicited information is EIop.
Again, EIop, can be either be converted to ER diagram, for which we propose guidelines, or can be used as a source of early information for vertical integration. In the case of the former, once the ER diagram is converted to star schema we obtain a stand-alone DWop.
We found that there are two problems with keeping separate Data Warehouses. These arise if there is common data in them.
Difference in refresh cycles between DWper and DWop cause common data to have differen values in the two DWs. Thus, rule formulators and operational decision makers end up taking decisions on different data in this temporal window. The larger this window, the longer this inconsistency exists. Loss of business control occurs when data of an operational DW calls for decision makers of the policy enforcement DW to take decisions, but the decisions are not taken because daa in the latter do not suggest this need. Thus, integration is required to maintain compatibility between PER and operational level.
We show that there are in fact two forms of integration that can exist, horizontal and vertical. While the former integrates data marts at the same level of decision-making, the latter integrates data marts across PER and operational levels. For vertically integrating DWper and DWop, we propose a ‘build by integrating’ approach that works pair-wise pair-wise. When a new data mart is to be built, its requirements specification is integrated with an existing one. The integrated requirements specification then goes through the development cycle. Thus, the point of integration is moved upstream into the requirements stage. In other words, early information is integrated. The advantages of integrating upstream and in a pair-wise fashion are: downstream development effort is minimized. it never allows un-integrated data marts to be built. Thus, pre-empting our two problems.
A complete logical DW is available for decision making.
Integration is done as a four-step process, Metadata reading, Correspondence Drafting, Information Mapping and Conflict Resolving. The integrated early information obtained is then converted into ER schema and finally into star schema. Through vertical integration, an integrated DW is obtained that can be used for both forms of decision making.
A summary of the contributions made is as follows: 1. Addressing full decisional making continuum: DW support has been extended from providing just operational support to providing policy enforcement and operational support. 2. Elicited Information can be traced back to members of the choice set thereby facilitating decision making. In the case of PERs the choice set is {select A, modify A, delete A} where A is an action and {select A1, select A2, select A3} for operationa decisions. For each alternative, information is elicited, thus relating information to a particular member of the choice set.
Discovery of early information: Information is obtained in a two-step process, early information elicitation step, using ENDSI, MEANSI and CSFI and late information elicitation step where early information is converted into ER diagram which is subsequently converted to a star schema.
Development of a requirements integration approach that pre-empts the problems of inconsistency in decision-making and loss of business control.
This work addresses the issue of providing support to different kinds of decisional needs in a unified, enterprise wide DW system. A two level generic platform is proposed with generic models at the bottom level and decision sources at the higher level.
We develop RE process to arrive at separate DWs for the PER and operational decisions respectively. The two DWs are integrated upstream in the requirements engineering phase. The integrated requirements specification is then converted into multi-dimensional form.
Future work includes developing a policy life cycle for eliciting information for policy decisions and integrating it with ‘lower’ levels of the decision continuum. 3.