Data-Driven Strategy Maps: A Hybrid Approach to Strategic and Performance Management Combining Hard Data and Experts' Knowledge

Abstract

A Strategy Map is a tool that depicts the interrelationships between the key performance indicators of a company. Strategy Maps are considered as Decision Support Systems by allowing the user to understand the consequences of a decision on other indicators of the business which is crucial in decision-making. To this date, the majority of the practical development of Strategy Maps is based on the knowledge and intuition of experts of the company regardless of the methodology used. These "soft data" present a number of drawbacks when implementing Strategy Maps: in accuracy, in completeness and a lack of longitudinal perspective. Currently, technological innovations enable to collect, store and analyze more data. These "hard data" are a powerful source of information used in Decision Support Systems to enhance strategic decision-making. We suggest to integrate hard data in the development process of the Strategy Maps in order to increase their reliability. This paper presents the outline of a research project related to the use of hard data in Strategy Maps. Five research questions are presented in order to contribute to the current literature with theoretical conclusions, methodological propositions and empirical demonstrations.

Keywords

• Strategy Map Data Mining Performance Measurement Models Strategic Management Strategic Decision-Making

-

measures used [ 3 ] and participatory aspects [ 4 ] are among examples of what in uences the rm performance through decision-making e ectiveness.

Decision Support Systems (DSS) are information systems designed to facilitate decision-making activities. A number of DSS tools have been developed in the context of strategic and performance management. One of the most popular tools is the Balanced Scorecard (BSC) which integrates both nancial and non- nancial indicators into four perspectives: Learning and Growth, Internal Business Processes, Customer and Financial [ 5 ]. In 2000, the two creators of the BSC have extended the concept and have developed a second tool called Strategy Map (SM) [ 6 ]. The added value of SMs comes from the presence of cause-and-e ect relationships between the indicators, it is the core element of the tool. The indicators of the BSC perspectives are linked to each other to form a visual causal map. Capturing the causal relationships that occur between the key indicators of a company is very essential for two main reasons. First, it helps the decision-makers to con dently understand the impact of a decision on others indicators. Second, it is an opportunity to be able to in uence intangible indicators (e.g. employees' satisfaction) by playing on causing indicators which act as levers. According to the rule of the creators of the SM, the causal links can only happen within the same perspective or toward any upper perspective. A generic SM is illustrated in Fig. 1, the arrows between the indicators represent the causalities.

Financial Customer Internal Business Processes Learning & Growth

Ind_Fin_1

Ind_Fin_2 Ind_Cust_1

Ind_Cust_2

Ind_Cust_3 Ind_BP_1

Ind_BP_2 Ind_LG_1

Ind_LG_2

Ind_LG_3

Creating a SM has been recognized as essential for rms in the literature. Indeed, not linking indicators nor validating those links is the cause of failure of performance models such as the BSC and companies that successfully build their SMs experience higher ROI and ROE [ 7 ]. In practice, SMs (and BSCs) are utilized by company for multiple reasons including: to formulate the strategy [ 6 ], to control [ 8 ], to communicate [ 9, 10 ] and for decision-making purposes [ 11, 12 ].

One the one hand, although proposed methods have evolved during the last two decades, building SMs still relies mainly on human inputs even when quantitative methods are carried out. On the other hand, the rise of digitalization has allowed companies to collect more data than ever before. In order to stay competitive, companies use information contained in the data to elaborate strategies and make decisions. In this research project, we explore the SM tool under a new perspective. We want to integrate data in the process of SM building. We will investigate a hybrid approach to SMs compared or as a complement to more traditional human-driven SMs. The contributions of the project to the scienti c literature are the following: (i) a state-of-the-art review of the literature regarding methods and context of SM development, (ii) an exploration of the issues related to human inputs in the process, (iii) new methods for building SMs based either solely on data or on a hybrid approach combining data and traditional methods and (iv) the comparison of SMs built using di erent methodologies. Resulting managerial contributions from this work can be highlighted as the methods to develop SMs are intended to practitioners.

The remainder of this paper is organized as follows: Section 2 states the problem and highlights the research questions of this project. Section 3 describes the research approaches including methods, current achievement and preliminary results of each study. Section 4 presents the challenges related to this research project and Section 5 concludes the paper. 2 2.1

Research Objectives

Problem Statement

In practice, SMs are built based on the intuition and experience of the experts of the company. Information sourced from human knowledge can be called "data" as well as information measured objectively. In order to distinguish the two types of data, we refer to all type of information sourced from human knowledge as "soft data", even when quanti ed, as opposed to "hard data". The prevalence of soft data in the SM literature can be explained by two main reasons. First, the authors created the tool in the early age of information systems democratization when hard data was rarely available. Second, soft data collection o ers advantages such as lower cost, easier availability and less time consumption. Nowadays, the technological advances allow companies to collect, store and analyze more (hard) data than ever before and we believe that it could be used to develop SMs that are more reliable. Fig. 2 shows the preponderance of soft data in the literature of SM development and the positioning of this research project.

We point out three major issues related to the use of soft data to build SMs: 1. Accuracy: under uncertainty, decision-makers use heuristics in order to assess the probability of an event and produce a judgment which can lead to systematic errors and biases [ 13 ]. Assessing causal relationships su ers from human cognitive limitations [ 10 ]. 2. Completeness: evaluating all possible causal relationships between indicators is long and complex for the experts of the company which can lead to simple or incomplete SMs. Qualitative methods

Soft data Literature on SM development

Literature gap addressed in this research project Hard data

Quantitative methods 3. Longitudinal perspective: collecting data on a long period of time is di cult and costly. This leads to mostly cross-sectional articles in the literature. However, it is incorrect to talk about causality in SMs due to the lack of time dimension. Indeed, a lagging variable X must precede a leading variable Y in time in order to talk about a causal e ect [ 14 ].

Some authors have acknowledge issues related to soft data in the development of SMs and attempt to counter them by carrying out more laborious methodologies (see for example fuzzy methods in [15{17]). However, the use of hard data sources methods are seldom presented or discussed in the literature. 2.2

Research Questions

In this research project, we propose to integrate hard data in the creation of SM to overcome the issues highlighted in the previous subsection. We divide the research project into several stages and we have developed the following research questions: { RQ1: What is the state-of-the-art of the literature regarding the practical and methodological development of Strategy Maps? { RQ2: How are soft data based Strategy Maps perceived by practitioners? { RQ3: How can we build a Strategy Map based on hard data? This third research question proposes a methodological contribution to the literature to build SMs using only hard data. { RQ4: How can we combine soft data and hard data to create a hybrid method to build Strategy Maps? { RQ5: How does a hybrid method based Strategy Map performs compare to a traditional soft data based Strategy Map?

Research Approach

This broad research project is currently composed of four ongoing studies. Each one has speci c methodologies and outcomes: 1. A systematic literature review; 2. A qualitative research; 3. A methodological proposition using hard data; 4. A hybrid methodology proposition combining soft and hard data and comparison study;

In the two latter studies, we will work in collaboration with skeyes as case study. Skeyes is the Belgian public autonomous company in charge of the air tra c control of the ve airports in Belgium and two radar stations. Hereafter, we present each study with respective objective, methods, related work as well as the current achievement and preliminary results if available. 3.1

Study I - Systematic Literature Review

Objective. Two decades have passed since the formalization of SMs by Kaplan and Norton and many researchers have studied their development with either a theoretical or practical aim. Quite surprisingly, no paper reviews the current state-of-the-art of the SM development literature leaving practitioners ooded with 20 years of research without proper structure. The aim of this systematic literature review is to synthesize prior research in SMs development and answer to the rst research question of this project: "What is the state-of-the-art of the literature regarding the practical and methodological development of Strategy Maps?". More speci cally, we aim to answer the following sub-research questions: 1. What research methods were used in the literature to support the development of SMs? We plan to examine how SMs have been developed in the literature by classifying the research methods and to determine when those methods appeared and were used. 2. What organizational contexts were studied in the SM development literature? We decompose this question to explore who what type of organization was developing a SM as well as why the reasons and motivation behind the SM development. 3. What are the research gaps and challenges in the SM development literature and what are the resulting future research directions? This last question is constructed as a consequence of the two previous ones.

Methods. We intend to perform this systematic literature review following the recommendations of [ 18 ] and based on the protocol proposed by [ 19 ]. Such protocol is essential to document the whole process, guide and organize the SLR [ 20 ] and diminishes the eventuality of having researcher bias as it makes the review more transparent and replicable [ 19 ]. Related Work. The closest work in the literature is the study from [ 21 ] reviewing documents to synthesize guidelines for SMs development. While his study is essentially normative, our study focuses on the positive view of SMs development. Current Achievement and Preliminary Results. The automated and manual searches have led to respectively 131 and 23 documents including journal articles and conference papers. After a quality check, a total of 75 documents can proceed further for the analysis which is currently ongoing. 3.2

Study II - Qualitative Research

In Section 2, we explained how soft data can lead to issues and biases in the SM. Although scienti c authors are aware of these, no study exist from the point of view of the practitioners to understand their attitude toward soft data based SMs.

Objective. In order to investigate the second research question of our research project: "How are soft data based Strategy Maps perceived by practitioners?", we intend to perform a qualitative study and analyze: (i) to what extent are the (potential) users of SMs aware of possible biases introduced by soft data and (ii) what is the attitude of the (potential) users of SMs toward the possible biases introduced by soft data in the context of decision-making. The conclusion of this study will enable to contribute to the literature by highlighting the aspects that are the most feared or misunderstood by the practitioners in order to harmonize their needs with future proposed methodology.

Methods. We perform this analysis using semi-structured interviews with users and potential users of SMs. The strength of this study will emerge from the heterogeneity of the sample as we interview participants with very diverse pro les working in di erent sectors and company types and sizes. An inductive approach will be used in order to analyze the data after the saturation threshold will be reached, meaning that new interviews would not bring new information. Related Work. Potential biases introduced by judgmental heuristics concept has been developed by Tversky and Kahneman [ 13 ]. While heuristics in decisionmaking (see for instance [ 22 ]) has been largely studied in many contextualized studies such as politics, medicine or consumer behavior, it has not yet been explored in the speci c context of SMs.

Current Achievement and Preliminary Results. We interviewed 10 participants with heterogeneous characteristics and have collected an important amount of information to explore our research questions. The inductive approach for data analysis is currently under progress. Preliminary results show that decision-makers are well aware of biases introduced by human involvement in the process of creating SMs. However, they are not utterly con dent in a process discarding all human interaction for the creation of SMs.

Study III - Methodological Proposition Using Hard Data

Objective. In this study, we will try to answer to the third research question of our project: "How can we build a Strategy Map based on hard data?". We will contribute to the literature by proposing a methodology for building SM purely based on hard data. The method we will propose could also lead to the automation of SM developments.

Methods. In order to build the data-driven SM, we propose to use time series Vector Auto-Regressive (VAR) models [ 23 ] combined with Granger causality [ 24 ] tests. In fact, skeyes data is collected on a daily basis and form time series which justify the use of VAR models to explore how one indicator can cause another one. Additionally, in order to talk about causality, it has been discussed that the variable must include a temporal perspective [ 14 ].

Related Work. Hard data quantitative method studies are very scarce in the literature. For instance, [ 25 ] use SEM methodology in order to build the SM causalities. However, they base their quantitative methodology on both hard ( nancial indicators) and soft (survey measures) data while our study is aiming at using hard data only. Another di erence occurs as these authors try to answer the temporal critic addressed to the SMs, they only use two data points in time whereas we suggest to use time series.

Current Achievement and Preliminary Results. A preliminary test for causality estimation and validation with hard data has been successfully carried out with four indicators of skeyes future SM and the results are presented in [ 26 ]. 3.4

Study IV - Hybrid Methodology Proposition Combining Soft and Hard Data and Comparison Study

This last study answers to the resilience from practitioners against the use of hard data only in a model with decision-making purpose as discussed in the preliminary results of study II.

Objective. This study will address the fourth and fth research questions of this project. The fourth research question: "How can we combine soft data and hard data to create a hybrid method to build Strategy Maps?" will be explored with a two-stage process. First, the creation of a soft data based SM with traditional methodology. Second, the validation of the obtained map with hard data. Regarding the fth research question: "How does a hybrid method based Strategy Map performs compare to a traditional soft data based Strategy Map?", we will compare the SMs obtained at the end of each stage: the soft data based SM and the hybrid SM. We will contribute to the literature in two ways: by proposing a unique hybrid methodology for building SM combining soft and hard data and by comparing the SM resulting from our proposed methodology with the SM built with traditional methods.

Methods. This study will imply mixed-methods and a case study. The rst stage will be carried out using DEMATEL (Decision Making Trial and Evaluation Laboratory) methodology as this is one of the most represented in the literature on SM development and its implementation procedure ts the requirements of skeyes, the company which provides us the necessary data. In order to (in)validate the causalities suggested by skeyes' experts at the rst stage, we plan to use VAR models and Granger causality tests for the same reasons that we explained in the previous subsection. For the comparison between the softdata based SM and the hybrid SM, we will use k-means clustering method. Related Work. There are numerous examples in the literature of related work for the DEMATEL procedure in SM development (see for instance [ 15, 27, 28 ]). The validation of soft data suggested causalities with hard data has been slightly explored. A few article related to hard data validation with quantitative methods make use of Structural Equation Modeling (SEM) models as in [ 29, 30 ]. Regarding the comparison between the two SMs, Moraga and his colleagues propose to use k-means clustering method to compare a quantitative and a qualitative SM, however both SMs in the latter article are still soft data [ 31 ].

Current Achievement and Preliminary Results. The selection of indicators and distribution between the four perspectives of the SM has been carried out. An online questionnaire has been developed in order to conduct the DEMATEL process remotely. The study is currently awaiting the experts assessment on potential causalities between the selected indicators. 4

Challenges

This section presents the challenges that need to be taken into account throughout this whole research project. Those cross-cutting challenges are at an early re ection stage and are thus not yet as far developed as the previous sections: { Indicator selection and SM visualization optimizations. The aim is to explore the optimal number of indicators to include in the SM without being overloaded and thus ine cient for decision-making purpose; { Causalities' strengths and directions. Data-driven SMs lack interpretability of causal links in terms of strength and direction. Those could be integrated in the tool for interpretation purpose; { Indicators combination for causality. Currently, only one-to-one direct causality e ects are represented in a SM. It would be interesting to explore other types of causal relationships between the indicators of the SM; { Data quality and availability. One of the main practical challenge with the proposed methods for the integration of hard data in the SMs is the quality and availability of data in companies.

Conclusion

This research project starts with the observation that SMs are subject to accuracy issues and biases related to the use of soft data during their development. Consequently, we suggest to integrate hard data to create more reliable SMs for decision-makers. The research project will be divided into four studies which tackle the identi ed problem, contributing to the current literature with theoretical conclusions, methodological propositions and empirical demonstrations. Acknowledgment. I would like to thank the supervisor of this research project, Dr. Corentin Burnay, for his review and support on this paper.

References

1. Kunc , M. H. , & Morecroft , J. D. : Managerial decision making and rm performance under a resourcebased paradigm . Strategic management journal 31 ( 11 ), 1164 { 1182 ( 2010 )

2. Robert Baum , J. , & Wally , S. : Strategic decision speed and rm performance . Strategic management journal 24 ( 11 ), 1107 { 1129 ( 2003 )

3. LeRoux , K. , & Wright , N. S.: Does performance measurement improve strategic decision making? Findings from a national survey of nonpro t social service agencies . Nonpro t and Voluntary Sector Quarterly 39 ( 4 ), 571 { 587 ( 2010 )

4. Carmeli , A. , Shea er , Z. , & Halevi , M. Y.: Does participatory decisionmaking in top management teams enhance decision e ectiveness and rm performance?. Personnel review ( 2009 )

5. Kaplan , R. S. , & Norton , D. P.: The balanced scorecard: measures that drive performance . Harvard business review 83(7) , 172 ( 2005 )

6. Kaplan , R. S. , & Norton , D. P. : Having trouble with your strategy? Then map it . Harvard Business Review 49 ( 5 ), 167 { 176 ( 2000 )

7. Ittner , C. D. , & Larcker , D. F. : Coming up short on non nancial performance measurement . Harvard business review 81 ( 11 ), 88 { 95 ( 2003 )

8. Malina , M. A. , Nrreklit , H. S. , & Selto , F. H. : Relations among measures, climate of control, and performance measurement models . Contemporary Accounting Research 24 ( 3 ), 935 { 982 ( 2007 )

9. Ritter , M.: The use of balanced scorecards in the strategic management of corporate communication . Corporate communications: An international journal 8(1) , 44 { 59 ( 2003 )

10. Kasperskaya Riabenko , Y. , & Tayles , M.: The role of causal links in performance measurement models . Managerial Auditing Journal 28 ( 5 ), 426 { 443 ( 2013 )

11. Wiersma , E.: For which purposes do managers use balanced scorecards?: An empirical study . Management accounting research 20(4) , 239 { 251 ( 2009 )

12. Cheng, M. M. , & Humphreys , K. A. : The di erential improvement e ects of the strategy map and scorecard perspectives on managers' strategic judgments . The Accounting Review 87 ( 3 ), 899 { 924 ( 2012 )

13. Tversky , A. , & Kahneman , D. : Judgment under uncertainty: Heuristics and biases . science 185 ( 4157 ), 1124 { 1131 ( 1974 )

14. Norreklit , H.: The balance on the balanced scorecard a critical analysis of some of its assumptions . Management accounting research 11(1) , 65 { 88 ( 2000 )

15. Jassbi , J. , Mohamadnejad , F. , & Nasrollahzadeh , H.: A Fuzzy DEMATEL framework for modeling cause and e ect relationships of strategy map . Expert systems with Applications 38 ( 5 ), 5967 { 5973 ( 2011 )

16. Mohamadnejad , F. , & Jassbi , J.: A fuzzy inference system platform for simulating cause and e ect relationships of strategy map . In: IEEE International Conference on Fuzzy Systems , pp. 1 - 8 . IEEE ( 2012 ). https://doi.org/10.1109/FUZZIEEE. 2012 .6251314

17. Glykas , M. : Fuzzy cognitive strategic maps in business process performance measurement . Expert Systems with Applications 40 ( 1 ), 1 - 14 ( 2013 )

18. Webster , J. , & Watson , R. T.: Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly, xiii-xxiii ( 2002 )

19. Kitchenham , B. , & Charters , S. : Guidelines for performing systematic literature reviews in software engineering . Technical report, EBSE Technical Report EBSE2007-01 ( 2007 )

20. Vom Brocke , J. , Simons , A. , Riemer , K. , Niehaves , B. , Plattfaut , R. , & Cleven , A. : Standing on the shoulders of giants: Challenges and recommendations of literature search in information systems research . Communications of the association for information systems 37(1) , 9 ( 2015 )

21. Islam , S.: A practitioners guide to the design of strategy map frameworks . Paci c Accounting Review 30 ( 3 ), 334 { 351 ( 2018 )

22. Gigerenzer , G. , & Gaissmaier , W. : Heuristic decision making . Annual review of psychology 62 , 451 { 482 ( 2011 )

23. E. Zivot and J. Wang .: Vector autoregressive models for multivariate time series . Modeling Financial Time Series with S-Plus , 385 { 429 ( 2006 )

24. Granger , C. W. : Investigating causal relations by econometric models and crossspectral methods . Econometrica: journal of the Econometric Society , 424 { 438 ( 1969 )

25. Slapniar , S. , & Buhovac , A. R. : Identifying temporal relationships within multidimensional performance measurement . Journal of Business Economics and Management 15 ( 5 ), 978 { 993 ( 2014 )

26. Pirnay , L. & Burnay , C. : Data-Driven Causalities for Strategy Maps . In: Cher S., Perini A. , Nurcan S . (eds) Research Challenges in Information Science. RCIS 2021. Lecture Notes in Business Information Processing 415 , 409 { 417 . Springer, Cham. ( 2021 ) https://doi.org/10.1007/978-3- 030 -75018-3 27

27. Lpez-Ospina , H. , Quezada , L. E. , Barros-Castro , R. A. , Gonzalez , M. A. & Palominos , P. I.: A method for designing strategy maps using DEMATEL and linear programming . Management Decision 55 ( 8 ), 1802 - 1823 ( 2017 )

28. Quezada , L. E. , Lpez-Ospina , H. A. , Palominos , P. I. & Oddershede , A. M. Identifying causal relationships in strategy maps using ANP and DEMATEL . Computers & Industrial Engineering 118 , 170{ 179 ( 2018 )

29. Huang , S. , Lee , C. & Kao , A. Balancing performance measures for information security management . Industrial Management & Data Systems 106 ( 2 ), 242 { 255 ( 2006 )

30. Saghaei , A. , & Ghasemi , R. : Using structural equation modeling in causal relationship design for Balanced-Scorecards' strategic map . World Academy of science, engineering and technology 49(1) , 1032 { 1038 ( 2009 )

31. Moraga , J. A. , Quezada , L. E. , Palominos , P. I. , Oddershede , A. M. , & Silva , H. A. : A quantitative methodology to enhance a strategy map . International Journal of Production Economics 219 , 43 { 53 ( 2020 )