=Paper=
{{Paper
|id=Vol-2728/doctorate3
|storemode=property
|title=Ontology of Value, Risk and Cognitive Biases for an IT Portfolio Decision Making
|pdfUrl=https://ceur-ws.org/Vol-2728/doctorate3.pdf
|volume=Vol-2728
|authors=Eduardo da Costa Ramos,Maria Luiza M. Campos,Fernanda Araújo Baião
|dblpUrl=https://dblp.org/rec/conf/ontobras/RamosCB20
}}
==Ontology of Value, Risk and Cognitive Biases for an IT Portfolio Decision Making==
https://ceur-ws.org/Vol-2728/doctorate3.pdf
Ontology of Value, Risk and Cognitive Biases for an IT
Portfolio Decision Making
Eduardo da Costa Ramos1, Maria Luiza M. Campos1, Fernanda Araújo Baião2
1
Graduate Program in Informatics – Universidade Federal do Rio de Janeiro (UFRJ)
Rio de Janeiro – RJ – Brazil
2
Department of Industrial Engeneering – Pontifícia Universidade Católica do Rio de
Janeiro (PUC-Rio)
Rio de Janeiro – RJ – Brazil.
ramoseduardoc@gmail.com, mluiza@ufrj.br, fbaiao@puc-rio.br
Abstract. Decisions about whether or not to kill an ongoing project of an IT
portfolio may be critical. A bad decision to continue with a project with a high
probability of making things worse, in exchange of a small chance of avoiding
a large loss, often turns manageable failures into disasters. However, these
decisions can be negatively impacted by cognitive biases of decision makers,
such as the loss aversion, which are enhanced when uncertainty is substantial
and information incomplete. As a result, it is important to reduce the biases in
these decisions. This paper introduces an ongoing research that aims to create
an ontology to improve the understanding of these cognitive biases. It can be
used as part of a strategy to reduce biases in a risk and value analysis.
1. Introduction
Decisions over the project portfolio are often referred to as the point where strategy is
put into action and is therefore crucial for the companies in order to reach their strategic
goals. This is achieved through the successful selection and execution of an appropriate
mix of projects, through a process known as project portfolio management (PPM). As
the dependency on information technology (IT) for organizational performance
increases, organizations must use IT portfolio management techniques to ensure that the
IT projects are aligned with the organizational strategic objectives. Making a portfolio
decision is far from trivial. This decision process is characterized by uncertain and
changing information, dynamic opportunities, multiple goals and strategic
considerations and interdependence among projects (LEVINE, 2007, p.319).
In this research proposal, we focus on making Go/Kill decisions on individual IT
projects on an ongoing basis (LEVINE, 2007, p.319), in other words, deciding to keep
or to remove an ongoing project from the IT portfolio. A bad decision to continue with a
project with a high probability of making things worse, in exchange of a small
possibility of avoiding a large loss, often turns manageable failures into disasters
(KAHNEMAN, 2011).
These decisions can be negatively impacted by cognitive biases (KAHNEMAN,
2011). Cognitive biases are systematic errors that recur predictably in particular
circumstances, such as decision making under uncertainty (KAHNEMAN, 2011). In
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
this research, we consider the loss aversion, risk seeking and framing effects cognitive
biases and will be detailed in section 2.
Therefore, to improve the decision making, it is important to better understand
how these biases occur, how to deal with them, and how to help to reduce their negative
consequences. In this sense, a considerable number of studies have been done in
strategic decision making (e.g. KAHNEMAN et al., 2011; KAHNEMAN, 2011;
CRISTOFARO, 2017; ABATECOLA et al., 2018). However, despite the relevance of
IT PPM to organizations, there is a lack of empirical studies relating portfolio
management to cognitive biases, specifically in the software projects context (DA
CUNHA & DE MOURA, 2015). Besides, there are still fewer studies about removing
or reducing cognitive biases in IT portfolio management (e.g. IDLER & SPANG, 2019;
PEDERSEN, 2016). Therefore, our research question is:
How can we (de-bias) mitigate or reduce the negative consequences of the
cognitive biases (loss aversion, risk seeking and framing effects) in the Go/Kill
decisions of ongoing individual projects in an IT portfolio?
This paper is structured as follows. Section 2 presents behavioral economics.
Section 2.1 details the heuristics and biases and how it occurs; Section 2.2 shows the
cumulative prospect theory and how it is used to predict the decision making. Section 3
defines IT portfolio concepts. Section 4 presents one ontology of value and risk. Section
5 presents the design science research as the research method used in our research
proposal, and section 6, our proposed ontology.
2. Behavioral Economics
Behavioral economics (BE) is a descriptive theory that focuses on how decisions are
actually made while a normative decision theory is about how decisions should be made
in order to be rational. One of the main foundations of BE is the dual process theory. It
aims to explain the decision-making process from the view of decision makers through
a general framework comprising two distinct systems of thinking: System 1 and System
2. System 2 corresponds to reasoned thinking, that has effortful mental activities;
System 1 corresponds to intuitive thinking and operates automatically and quickly, with
little or no effort, and no sense of voluntary control (KAHNEMAN, 2011). BE research
aims to understand how the intuitive and the reasoned thinking fail and how they are
effective (KAHNEMAN, 2011).
2.1. Heuristics and Biases
To reduce difficult mental tasks, such as assessing probabilities and predicting values,
decision makers often use heuristics. These procedures include computational shortcuts
and editing operations, such as eliminating common components and discarding
nonessential differences for arriving at satisfactory solutions with modest amounts of
computation (KAHNEMAN, 2011). While these broadly accurate heuristics work most
of the time and can generally be described as very powerful (GIGERENZER, 1999),
they are also the source of systematic errors, known as psychological biases or cognitive
biases, and that recur predictably in particular circumstances, such as decision making
under uncertainty (KAHNEMAN, 2011).
Therefore, value and risk analyses are subject to several cognitive biases,
because they are human-centric activities that require estimating likelihood of
occurrence and potential impact, under uncertainty (NOYES et al., 2012). The
significant role cognitive biases play in risk management has been acknowledged by
both ISO31000:2018 and COSO:ERM 2017, that are important risk management
guidelines (SIDORENKO, 2018).
In this research proposal, we use the cognitive bias typology of Arnott (2006, Table 1)
that was proposed to support the development of decision support systems (DSS). The
specific cognitive biases studied in our research are loss aversion, risk seeking and
framing effects. Framing effects (KAHNEMAN, 2011) occur when events framed as
either losses or gains may be evaluated differently. Perceptions of risk can be affected
by the way in which a situation is presented and they are particularly vulnerable to
framing effects, which influence the way individuals approach a risk, and have biases in
the decisions they make (NOYES et al., 2012). Loss aversion is one of the basic
phenomena of choice under both risk and uncertainty in which losses loom larger than
gains (KAHNEMAN, 2011). Loss aversion has been used to explain other biases, such
as the endowment effect and sunk cost fallacy (KAHNEMAN, 2011). Finally, risk
seeking integrates one of the core achievements of cumulative prospect theory (CPT)
that is the fourfold pattern of risk attitudes. This pattern is presented in the next section,
that introduces the CPT. CPT allows a better understanding of the cognitive biases.
2.2. Cumulative Prospect Theory (CPT)
CPT is a descriptive theory and extends prospect theory to uncertain as well as to risky
prospects with any number of outcomes while preserving most of its essential features
(KAHNEMAN, 2011). It is based on the dual process theory.
Three cognitive features, all operating characteristics of System 1 (intuitive
thinking), are the foundation of cumulative prospect theory (Kahneman, 2011, p. 281-
2). They are: (1) evaluation is relative to a neutral reference point and this point defines
what is a gain and what is a loss; (2) there is diminishing sensitivity to both increasing
values and increasing gains or losses; and (3) losses loom larger than gains in decision
maker’s minds (ARNOTT and GAO 2019).
One of the core achievements of CPT is the fourfold pattern of risk attitudes: risk
aversion for gains and risk seeking for losses of medium and high probability; risk
seeking for gains and risk aversion for losses of low probability (KAHNEMAN, 2011).
3. IT Project Portfolio Management
Project portfolio management (PPM) consists of two different parts or phases: (1)
project portfolio selection and (2) portfolio management. The first phase is to select
projects for the pipeline. This phase includes a structured process to deal with project
proposals and how they can be evaluated. The second phase is to maintain the project
pipeline. Once the selected projects are being executed, they need to be monitored and
evaluated to ensure that they still fulfil the conditions of the original selection criteria. If
the conditions change it needs to be reconsidered if the project should remain in the
portfolio or be terminated (LEVINE, 2005). Our focus in this research proposal is on
this last decision that is called Go/Kill decision.
In the information technology (IT) context, IT portfolio management provides
the discipline of balancing risk against expected returns, evaluating the performance and
utilization of existing systems, analyzing and assessing alternatives and trade-offs, and
removing waste resulting in significant efficiencies and cost savings. To be able to make
this complex decision, experienced IT-projects portfolio managers use many methods to
quantify risks, costs, value, and performance of the IT projects (MAIZLISH and
HANDLER, 2005, p.66), and combine deliberate rational analysis (reasoned thinking)
with experienced-based intuition (intuitive thinking) (PEDERSEN, 2016). Therefore,
risk and value analyses are important activities for the Go/Kill decision in IT PPM.
4. Ontology of Value and Risk
Sales et al. (2018) have presented an ontological analysis of risk which makes explicit
the deep connections between the concepts of value and risk. They have also proposed a
concrete artifact, namely the Common Ontology of Value and Risk, formalized in
OntoUML. They formally characterized the process of ascribing risk as a particular case
of the process of ascribing value (in the sense of use value).
This ontology considers three perspectives: (1) an experiential perspective; (2) a
relational perspective; and (3) a quantitative perspective, which projects value and risk
on measurable scales and it is our focus in this research proposal.
Risk and value have the following similarities: (a) goal dependency; (b) context
dependency; (c) uncertainty and impact in the conceptualization of both: the
computation of the likelihood of an event times its impact on one’s objectives and
preferences fits the quantitative analysis of both risk and value. The impact for the value
is related to achieving goals. This opens the possibility of applying methodologies and
techniques developed for value analysis in marketing and economics to the case of risk
analysis, and vice versa. Risk analysis is traditionally accepted as a complex and critical
activity in various contexts, such as strategic and project planning, finance, engineering
of complex systems, and software development (SALES et al., 2018).
5. Research Method
We selected design science research (DSR) as the research method, because of its
strength in solving a real problem using applied research (PEFFERS et al. 2007) in the
field of information systems; in our case, an ontology to help analysts identify the
existence of some potential biases in a risk and value analysis of a Go/Kill decision in
an IT portfolio.
We will evaluate the ontology by focus group analysis. These groups are formed
by IT portfolio decision makers and DSS analysts. We will use the pragmatic
perspective to evaluate the perception of utility of the ontology in a de-biasing strategy.
6. Solution Proposal
De-biasing requires the decision support systems (DSS) analyst to understand the
mechanisms underlying the particular bias that is subject of change. Each bias can have
a different de-biasing approach, and this makes de-biasing difficult in practice as it
requires significant effort from the analyst (ARNOTT and GAO 2019).
To answer the research question, we propose an ontology representing the
Go/Kill decisions associated to IT projects and their values, risks and cognitive biases.
The proposed ontology extends the Common Ontology of Value and Risk (SALLES et
al., 2018) because value and risk analyses are at the core of the Go/Kill decisions and it
uses cumulative prospect theory (CPT) as the decision making theory under uncertainty
to support the value and risk analysis. We use CPT because it may allow a better
understanding of the cognitive biases considered in our research problem.
Furthermore, the true semantics of what is value and how the cognitive biases
sometimes negatively affect decision making are sometimes not well understood. Its
semantic essence can be better explored and represented to identify other potential
associated elements. In the literature review, we found just one ontology (LORTAL et
al., 2014) used to reduce cognitive biases. The aim of this ontology was to quantify the
risk of occurrence of a cognitive bias in the intelligence domain. This ontology can
model the probability and severity parameters for risk assessment. Moreover, although
there are a few proposals of ontologies (KORNYSHOVA and DENECKERE, 2010;
NOWARA, 2017) that deal with intuitive decision making, which are the source of
cognitive biases, they do not consider the cognitive biases. So, to the best of our
knowledge, there are no ontologies considering the loss aversion, risk seeking and
framing effects cognitive biases for a decision making under uncertainty.
In particular, we plan to extend the ontology of Salles et al. (2018) by, initially,
(i) including the concepts of decision making according to CPT to measure value and
risks relative to a reference point (rather than absolute outcomes), according to the
pattern of risk attitudes; (ii) taking into account cognitive biases related to value and
risk; and (iii) adapting the concepts of risk assessment of a cognitive bias from the
ontology of Lortal et al. (2014) to loss aversion, risk seeking and framing effects for IT
portfolio Go/Kill decisions. Most decision makers are unaware that they use reference
points, so they are likely to decide without realizing how it can bias their choice of
action (FRENCH et al, 2009, p. 38).
Our proposed ontology aims to improve the understanding by the analysts of
these decision biases and the situation in which they occur. This ontology can be used as
part of a strategy to reduce the biases in the Go/Kill decisions of an IT portfolio. More
specifically, our focus is on developing DSS as a de-bias strategy (LARRICK, 2004).
We use the debiasing process of Arnott (2006) because it was developed to support DSS
development projects. Our focus is on the first step of this process: to identify the
existence and nature of the potential bias. As value and risk are very important to the
goal achievements of an IT portfolio and as their measurements may be subject to
cognitive biases, thus it is of fundamental importance that these elements and their
relations be well understood to support the first step of this debiasing process. So, we
argue that one ontology representing the Go/Kill decision of a project in an IT portfolio
with its values, risks and possible biases considered in our research can help DSS
analysts identifying the existence of these biases.
References
ABATECOLA, Gianpaolo; CAPUTO, Andrea; CRISTOFARO, Matteo. (2018)
Reviewing cognitive distortions in managerial decision making. Journal of
Management Development.
ARNOTT, D. (2006). Cognitive biases and decision support systems development: A
design science approach. Information Systems Journal, 16(1), 55-78.
ARNOTT, D., & GAO, S. (2019). Behavioral economics for decision support systems
researchers. Decision Support Systems, 122, 113063.
CRISTOFARO, M. (2017). Reducing biases of decision-making processes in complex
organizations. Management Research Review.
DA CUNHA, J. A. O. G. and DE MOURA, H. P. "Towards a substantive theory of
project decisions in software development project-based organizations: A cross-case
analysis of IT organizations from Brazil and Portugal," 2015 10th Iberian Conference
on Information Systems and Technologies (CISTI), Aveiro, 2015, pp. 1-6.
FRENCH, S., MAULE, J., & PAPAMICHAIL, N. (2009). Decision behaviour, analysis
and support. Cambridge University Press.
IDLER, B. D., & SPANG, K. (2019). IT project decisions: conclusions and
recommendations for corporate practice. International Journal of Managing Projects
in Business.
KAHNEMAN, D. (2011) Thinking, Fast and Slow, first ed. Farrar, Straus and Giroux,
New York.
KAHNEMAN, D., LOVALLO, D., & SIBONY, O. (2011). Before you make that big
decision. Harvard business review, 89(6), 50-60.
KORNYSHOVA Elena; DENECKERE, Rebecca. Decision-Making Ontology for
Information System Engineering. ER 2010 29th International Conference on
Conceptual Modeling, Nov 2010, Vancouver, Canada. Springer, 6412/2010.
LEVINE, H. A. (2007). Project Portfolio Management: A Practical Guide to Selecting
Projects, Managing Portfolios, and Maximizing Benefits. San Francisco, CA.: John
Wiley & Sons.
LORTAL, Gaëlle; CAPET, Philippe; BERTONE, Alain. Ontology building for
cognitive bias assessment in intelligence. In: 2014 IEEE International Inter-
Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision
Support (CogSIMA). IEEE, 2014. p. 237-243.
MAIZLISH, B., & HANDLER, R. (2005). IT (information technology) portfolio
management step-by-step: Unlocking the business value of technology. John Wiley
& Sons.
NOWARA, Piotr. (2017). Decision-O: The Decision Ontology. Online Git repository.
GitHub. Com. https://github.com/nicholascar/decision-o.
PEDERSEN, K. (2016). IT project selection: Politics, experience and good
friends. Electronic Journal of Information Systems Evaluation, 19(1), 55.
PEFFERS, K., TUUNANEN, T., ROTHENBERGER, M. A., & CHATTERJEE, S.
(2007). A design science research methodology for information systems
research. Journal of management information systems, 24(3), 45-77.
SALES, T. P., BAIÃO, F., GUIZZARDI, G., ALMEIDA, J. P. A., GUARINO, N., &
Mylopoulos, J. (2018). The common ontology of value and risk. In International
Conference on Conceptual Modeling (pp. 121-135). Springer, Cham.