The performance measurement method introduced in this paper is based on the five-step indicator lifecycle that covers formal definition of indicators, measurement, analysis, reaction, and reformulation of indicator definitions. Performance measurement framework is introduced that support this performance measurement method and that enables the indicator lifecycle. The goal of this research is to provide a method for performance measurement that ensures timely and to given context appropriate decision making process. For the purposes of storing the information necessary for decision support a data warehouse is used as a component of the process measurement framework.
Effective organization of business processes ensures the achievement of institution’s goals. Performance measurement compares the measurement results with target values to discover the progress. An important aspect is how to choose appropriate measures and how to define an appropriate measurement framework.
Performance measures [
To perform effective measurement and adequate reaction on discovered situations,
not only different perspectives, but also different aspects (e.g. connection to success
factors, reporting, reaction, responsibilities) of performance indicators could be
modeled and documented. In our previous research [
To implement performance measurement according to the strategies of the company and some management model, the companies develop and use measurement systems. A data warehouse is an option to build a Performance Measurement System. An advantage of using a data warehouse for the implementation of a performance measurement system is the possibility to use existing infrastructure of the company’s data warehouse.
Traditionally data warehouses store customer and financial indicators of the
companies, but other perspectives are typically not covered. Some attempts to integrate
the perspective of internal business processes into a data warehouse have been made in
[
We do not try to incorporate another perspective of measurements into the data warehouse. We propose to use the data warehouse as an integral part of a performance measurement system, which can store indicators of different perspectives and can be used according to proposed measurement framework.
The measurement framework describes different measurement aspects to bring order within this important undertaking of the organization. Thereby, the quality of measurement is improved, for example, by means of performing the analysis of right indicators in right time and undertaking the right actions as a result.
The usage of an existing data warehouse gives additional advantage to the performance measurement. The analysis of indicator values can be performed using existing OLAP tools, reports and dashboards.
We start with related work described in section 1. Section 2 explains the concept of indicators life-cycle that forms the basis for proposed measurement framework. In section 3 the reporting tool and its metadata is described that is one of the ready-made data warehousing components used within the measurement framework. The architecture of performance measurement system is given in section 4. In section 5 the conclusions are given.
Performance measurement systems implemented by means of a data warehouse are given in several works. The existing approaches concentrate mostly on how to build an appropriate dimensional model of the data warehouse according the process perspective of measures to be stored.
The Process Data Warehouse [
A category of data warehouses for performance measurement can be distinguished,
where the business process execution data is stored. The systems already mentioned
use workflow data as one of data sources, but workflow data warehouse [
Methodologies how the performance should be evaluated also are a subject of
research. For example, methodology [
Our approach uses the advantages of an existing data warehouse – ETL processes, analysis tools, data storage schemas – that allow to prepare and store indicators according to the different perspectives, as well as integrates the data warehouse with a performance measurement framework that is based on the life-cycle of indicators, which ensures the quality of the performance measurement by supplying necessary information for each measurement task.
In our previous research [
Indicator definition step describes the information needs of the user. In the measurement step the indicators get the values. The analysis step represents the process, when indicators are used to make decisions. The reaction step represents the implementation of decisions. The life-cycle ends with the evaluation of indicator definitions and predefined values of indicator properties during the improvement step.
The properties of indicators are grouped in aspects according to the particular step
(Figure 1). The explanation of the meaning of properties can be found in [
TYPE KRI, RI KPI, PI
TIME •Orientation (Past, Current , Future)
DATA SOURCE •Existing •To be gathered
LEVEL • Organization • Individual
Indicator PROCESS (for PI and
KPI)
Indicator
TIME • Frequency Measurement aspects of indicators
PERSPECTIVE • Customer • Finance • Internal Processes • Learning and Growth • Environment /Community • Employee satisfaction
SUCCESS FACTORS • Success factor • Critical success factor
MODEL • Indicator dependencies • Indicator hierarchy • Base / derived measures
METHOD • Automatically/calculated • Manual/surveys
REPORTED TO •Management •Responsible for processes
TIME •Reporting time
ANALYST Analysis aspects of
indicators
PROCESS • Rebuild • Improved
TIME • Reaction time Reaction aspects of indicators
Indicator Indicator
TARGET VALUE • Decision criteria
ANALYSIS MODEL • Analytical functions • Representation
ACTIONS • To be performed
RESPONSIBLE FOR • Reaction VALUES OF ASPECTS • To be changed Improvement aspects of indicators
Indicator
DEFINITION
• To be adjusted
• Accepted/Rejected
One of the questions raised by the proposed measurement framework with
indicators as the central element of interest was how the indicators should be
formalized to bring the maximum of clarity into the measurement process – what, why
and how is measured. Our previous research [
The type of an information system to be developed has some impact on a way of
formulating sentences that express requirements. We assumed that requirements for
data warehouses and information requirements particularly have a similar structure or
pattern. We based the proposed model on the structure evaluation of the sentences that
formulate performance indicators taken from the performance measures database [
All indicators have common structure, for that reason it is possible to determine a
pattern for re-writing business requirements formally. The requirement formalization
may be represented as a metamodel. The detailed description of the metamodel and the
algorithm how the sentences expressing the indicators are reformulated according to
the given metamodel can be found in [
One important and integral part of our process measurement framework is a reporting
tool developed at the University of Latvia. This reporting tool is developed as the part
of the data warehouse framework [
Semantic, Logical and Physical metadata describe the data warehouse schemata in different levels of abstraction, starting from the business understanding of the schema elements, describing it by means of OLAP concepts at logical level and ending with the physical storage of the data warehouse tables. OLAP Preferences metadata is introduced to describe the user preferences on reports’ structure and data and is used for OLAP personalization purposes. Reporting metadata contains definitions of reports on data warehouse schemata.
OLAP Preferences Metadata
Semantic Metadata Logical Metadata
Physical Metadata
Metadata levels are interconnected by associations between particular classes of
metadata. In the context of this research, the Logical and Reporting metadata are of
particular interest; so both the levels as well as the connections between them will be
described in more detail here. Detailed description of the rest of the metadata levels can
be found in our previous research [
The logical level metadata describes the data warehouse schema from the
multidimensional viewpoint (Figure 3) and mostly is based on the OLAP package of
Common Warehouse Metamodel (CWM) [
Fact tables contain measures, but dimensions consist of attributes and hierarchies built from hierarchy levels. Fact tables and dimensions have FactTableDimension association. Only dimensions and fact tables having FactTableDimension associations can be used simultaneously in one report. More about connections with reporting metadata is given in section 3.3.
The standard OLAP package of CWM is extended by the class AcceptableAggregation that allows only meaningful definitions of aggregate functions (e.g. SUM, AVG) for each measure and dimension. This metadata is used to ensure correct queries by the reporting tool.
1..*
FactTable -Name -Description
1 1..*
Measure -Name -Description 1 Schema -Name -Description -FactTableDimension
1 0..* 1..*
AcceptableAggregation -Aggregation 0..* 1..* 1..*
Dimension -Name -Description -IsTimeDimension 1..*
Attribute -Name -Description -corresponds 1 1
Level -Name 0..* -Description
0..* * 1 Hierarchy
-Name -{ordered} -Description
Reporting metadata describes the structure of reports (Figure 4). In the meaning of this model, reports are worksheets. Worksheets contain data items defined by calculations. Calculations in their turn specify formulas containing parameters and table columns that correspond to schema elements of the underlying data warehouse. Reports also are based on joins between tables and may have user-defined conditions.
Reports in the tool are created by choosing desired elements of a data warehouse schema and defining conditions, parameters etc. Only measures and attributes belonging to one schema could be included in the definition of one report. Subquery -QueryText
Constant -Value ConditionObject
ConditionSet -Formula -IsVersionCondition 1 *
1 3.3. Connections between Logical and Reporting Metadata The models of logical and reporting metadata are interrelated. Report items are defined by formulas from calculation parts. If a calculation part corresponds to a particular dimension attribute or measure, then this schema element from Logical metadata is connected to the class CalculationPart by the association ‘corresponds’ in the reporting metadata. 4. Construction of Performance Measurement Framework We propose an approach of building Performance measurement systems by substantially exploiting existing data warehouse technologies.
The proposed Performance measurement framework is grounded on the following
principles of design and operation:
• processing of indicator information is performed in conformance with the
lifecycle of indicators and formal indicator metamodel, defined in [
Indicator life-cycle support database (detailed information is given in the next section) stores links to the formal definitions of indicators from the Indicator formal definition database, which is built according to the formal model for indicator definition described earlier in section 2.2. These indicator definitions are collected and formalized during the requirements gathering process for obtaining the precise and appropriate indicators for process measurement, as well as for documenting the information requirements of a data warehouse. Performance management component
Reporting tool
Indicator editor is an administrative tool and is meant for two purposes: (1) to establish the links between the Indicator layer of the system and the Indicator formal definition database and (2) to configure the Indicator measurement (or ETL) metadata.
The measurement process, during which indicators get their values, is performed through the ETL processes, which use corresponding ETL metadata for measurements. The ETL component is an external part of the performance measurement framework. In the context of this research we assume that a set of procedures is defined for performing the data warehouse data renewal according to the values of ETL metadata for measurements (e.g. according to the planned timing schedule). During the ETL processes data from external Data sources are processed and loaded into the data warehouse that represents in our framework the Measurement data.
The remaining part of the data warehouse layer of the proposed framework is the Data warehouse and reporting metadata component that is developed according the previously described metadata layers in sections 2.1 and 2.2 that describe respectively the logical level of data warehouse schema and the reporting metadata.
Performance management component is the main part of the framework that is provided to coordinate the monitoring of business processes by analyzing the measurement results of indicators. Component is based on descriptions of different properties of indicators that are stored in the Indicator life-cycle support database and that allow the user to analyze the indicator values in the most appropriate way by means of two other components of the framework – Dashboard module and the Reporting tool.
The Dashboard module visualizes the most important values of indicators
comparing them to the stored target values of indicators. The reporting tool provides
more detailed information to the user by calling predefined reports linked to particular
indicator definition. An existing reporting tool is used, which is built according to the
previous mentioned reporting metadata (more information about this tool can be found
in [
Indicator formal definition database
IndicatorMeasurement Indicator TimingSchema
Link to ETL DW metadata IndicatorDefinition Indicator Perspective={cust,fin,...} SuccessImpact Level={org,indiv} TimeOrientation
IndicatorType={KRI,RI,KPI,PI} Indicator life-cycle support metadata Indicator life-cycle support execution data
Indicator IndicatorName FormalMetamodel Definition Measurement Analysis Reaction
1..n
Notification User Indicator NotificationTime Status={unprocessed,delayed,processed} Message ReportConfig RequiredReactionTime ReactionTime
Dashboard module 1..n
IndicatorAnalysis Indicator TargetValue ActionValueDefinition DecisionOperator={=,<,>,...} TimingSchema MessageTemplate ReportDefinition
IndicatorReaction Indicator Type={none,react} TimingSchema ResponsibleUser ActionToPerform Performance management component
Reporting tool
As the duty of performing measurements is fully assigned to the data warehouse, ETL metadata are prepared and stored separately from Indicator life-cycle support database. Actually, this is one of the key points of the framework to fully connect data warehouse for such functionality.
Workflow status is stored in the ‘Indicator life-cycle support execution data’ and is accessible directly by users via Dashboard module. Workflow status is controlled both by Performance management component and by Dashboard module. It incorporates information about notifications by the system sent to users and the reaction of users to them. 4.3. Indicator Life-Cycle Steps in Performance Measurement Framework According to the indicator life-cycle definition in section 2, Performance measurement framework should support all five steps of the life-cycle. This section is to describe the proposed framework according to the life-cycle steps.
Indicator life-cycle support metadata set metadata
Definition Measurement
Analysis
Reaction
Improvement Indicator life-cycle
LEGEND
ETL metadata
ETL Measurement data analyze react Notification Reaction
Data sources
Data warehouse actual value 3. User’s reaction is obtained from the Dashboard module (Figure 9) and can be of two types: • A request for the detailed notification. Reporting tool is used here to obtain a report that describes the actual measurement in detail; • Reaction. If the description of an indicator provides for a response to the notification, user is required to assert this in time and in a special way. 4. In control step Performance management component checks whether users have responded to the notifications, if such reactions were appointed in the analysis step (Figure 10).
The above described processing of indicator data by Performance measurement framework is performed in conformance with the life-cycle of indicators.
Table 1 shows mapping between the indicator life-cycle and its implementation by the Performance measurement framework. Indicator life-cycle aspect group 1. Definition
Description of implementation by the performance measurement framework Indicator definition is described in indicator formal definition database, as well as Indicator life-cycle support database. Indicator definition includes preparation of the metadata required to ensure the whole process.
Measurement process is fully delegated to the data warehouse and its appropriate ETL process.
Analysis is coordinated by Performance management component. Measurement data are processed and displayed to the users.
A user reads and, if required, reacts to the notification.
Performance management component controls the reaction.
Indicator improvement technically matches indicator definition. Procedure react Begin
Foreach user
Display all from Notification in the dashboard Where User = user Foreach notification From Notification Where User = user Do
Wait for user action Do
Case user asks to show detalized information Do
run report according to notification.ReportConfig and display it Case performs an action according to report.Indicator.Reaction.ActionToPerform Do
notification.ReactionTime = current time 4.4. Integration with Data Warehouse Components ETL metadata for measurements (IndicatorMeasurement class) is a part of Indicator life-cycle support database (Figure 6). The Indicator attribute identifies a particular attribute that is measured, whereas TimingSchema describes the time parameters of measurement (e.g. frequency, exact starting time). The last attribute – ETLprocess – points to the data warehouse meatadata repository, particulary to the ETL metadata part of the repository that describes mappings between the source and data warehouse schemas. This metadata also contains calls to corresponding procedures that implement these mappings and necessary data transformations. For the proposed measurement framework we can assume that the IndicatorMeasurement class contains the procedure call that renews the data warehouse data schema that contains data necessary for calculation of the given indicator.
The IndicatorAnalysis class of Indicator life-cycle support database (Figure 6) and its ReportDefinition attribute is planned to be a pointer to the report definition stored in accordance to the metamodel of the reporting tool.
Reporting metadata (Figure 4) contain the Worksheet class that identifies a particular report that can be invoked when analysis of measurement results is performed. The report can be simple, when one particular value is retrieved to compare it with a target value, or complex, when the report is used for the detailed analysis. The complexity of the report depends on the definition of the particular report.
Using data warehouses in performance measurement systems has been already extensively explored. The proposed Performance measurement framework has been designed to obtain the maximum benefits from matured data warehouses technologies in implementing indicator life-cycle support.
The applied model of indicator life-cycle serves as a theoretical means of quality assurance for the performance measurement. The use of data warehouses as integral part of the framework covers two important aspects of ensuring the indicator life-cycle: (a) indicator measurement, and (b) part of indicator analysis (performed by Reporting module).
The provided method for performance measurement ensures timely and to given context appropriate decision making process. The indicator life-cycle support database stores metadata that define and schedule the measurement and control processes of indicators, including timing schemas, responsibilities and actions to be performed. The proposed framework provides the option to build performance control on the activities initiated from the side of the measurement system, as soon as the system recognizes the problem and so the need for more detailed analysis.
Preliminary works of implementing the framework are already in progress, so we expect the first experimental results in the near future.
This work has been supported by European No. 2009/0216/1DP/1.1.1.2.0/09/APIA/VIAA/044. Social
Fund (ESF) project