Risk management is a critical process for organizations to manage and navigate environments that are uncertain, complex, and dynamic. The first step of the risk management process is risk identification, which has the goal of identifying a diverse space of specific and relevant potential risks. Despite the central role of risk identification in the risk management process, limited work has investigated cognitive processes in risk management. This paper conceptualizes risk identification as a type of anticipatory thinking-the process by which we imagine alternative states of the world. It explores how three anticipatory thinking metrics (novelty, specificity, diversity) can be used to assess risk identification.
Risk management has increasingly become a required
process for organizations, within both public and private
sectors, that are attempting to navigate uncertain, complex,
and dynamic environments (Baird, Skromme, and Thomas
1986; Hood and Rothstein 2000). It is employed across as
diverse topics as information security (Gerber and Solms
2005), product development (Chin et al. 2009), construction
The exploratory nature of risk identification is similar to
that of anticipatory thinking, which is the process by which
Copyright © 2020 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
an individual imagines alternative futures and is a critical
component for successfully navigating complex
circumstances
This paper presents an analysis of linkages between the mechanisms and processes of risk identification and anticipatory thinking to support a deeper understanding of assessing risk identification. It considers how metrics devised for the assessment of anticipatory thinking can be used to measure the quality of risk identification.
Risk management is a widely used technique in management, engineering, finance, defense, and public health, to determine the allocation of resources in order to monitor and minimize the impact of unfortunate events and maximize the potential of opportunities (Hubbard 2009). It is a cyclic process that involves identification, evaluation, and prioritization of risks. Among these, risk identification is the first step of the process and often introduces a bottleneck for the success of following steps due to the vast problem space (Department of Defense 2017). During risk identification, an analyst employs detailed knowledge and systematic methods to generate a set of risks and their impacts, which are sometimes accompanied by other features of the identified risks such as vulnerability, speed of situation development, potential gain from taking the risk, and others depending on the context. The risks could be threats, opportunities, or uncertainties in general. All the information is gathered for subsequent qualitative and quantitative analyses.
Two qualitative risk assessment methods, bow-tie analysis and risk classification charts, are of particular relevance in the application of anticipatory thinking. A bowtie analysis aims to identify the causes and preventive measures of a particular risk (Hancock 2016), making links among risk, impact, and cause (which could be the impact of another risk). This method, wherein an analyst generates risk-impact pairs, shares commonalities with the anticipatory thinking methodology. A risk classification chart is a grid plot of impact against likelihood for each particular risk. It is created to quantify the diversity of identified risks and their impacts.
Despite the challenges of risk identification, research has shown several promising methods that can improve risk identification performance. For example, there is evidence that risk identification is a trainable skill and that part of this skill may be domain general as experience within a domain is not sufficient to fully support superior performance (Maytorena et al. 2007). In addition, based on observations by risk and project practitioners, assembling a panel of individuals with relevant but diverse backgrounds can yield better risk identification outcomes (Emmons et al. 2018). These methods generally align with ways that may support divergent and anticipatory thinking.
Anticipating how situations may evolve into the future is a significantly challenging task, yet this form of anticipatory thinking plays a central role in strategic decision-making and risk identification activities in areas such as military planning, business planning, and medicine, where individuals must generate ideas about the conditions under which events occur, identify second and third-order effects, and develop explicit potential alternatives to a given scenario in order to avoid tactical or strategic surprise.
Anticipatory thinking relies on many connected cognitive components including attention, memory, executive function, situational awareness, and domain expertise (Koziol, Budding, and Chidekel 2012; Mullally and Maguire 2014). Each of these components serves an important role in perceiving the status, attributes, and dynamics of relevant elements in the environment and projecting how these elements could lead to different future states.
Anticipatory thinking can take three distinct forms: prospective branching, backcasting, and retrospective branching (Figure 1). Prospective branching involves anticipating future system states and identifying indicators that may lead to these system states. Backcasting involves examining a particular future system state and thinking back in time to identify warnings and indicators that lead to its occurrence. Retrospective branching is the identification of possible unknown past system states and their paths towards the present one. All forms of anticipatory thinking focus on the mapping of alternative system states and paths towards them through uncertain conditions, and the goals of the analyst influence where the uncertainty is mapped out.
Divergent thinking is central to anticipatory thinking. Individuals with strong divergent thinking skills are hypothesized to be able to generate creative ideas by exploring many possible solutions. Strong divergent thinking skills may be particularly useful during the generative phase of anticipatory thinking wherein individuals anticipate potential futures and generate indicators tied to those events. In fact, recent research shows strong correlations between performance on anticipatory thinking activities and divergent thinking skills (Geden et al. 2019).
Anticipatory thinking is essential for effective risk identification. Prior to assessing and weighing risks, organizations and individuals must identify high and lowlikelihood events and determine the level of risk associated with each event. Identifying vulnerabilities and risks requires individuals to think across time and identify causal links between events, causes, and consequences. For instance, if a risk has been realized, then a risk management team may need to engage in retrospective branching to identify indicators that led to the risk. Conversely, if a team is engaging in a strategic risk identification exercise to reduce vulnerability, then team members will need to engage in prospective branching to identify leading indicators and causal dependencies of future scenarios.
Geden et al. (2019) developed an anticipatory thinking assessment that may be relevant for assessing risk identification skills. The assessment presents respondents with a future-oriented prompt (e.g., “The impact of smart home technologies on older adults in 10 years”), and asks them to generate as many pairs of potential future events (uncertainties) and their subsequent consequences (impacts) as they can within a short ten-minute window.
The format of the assessment uses a similar dyadic pairing form that risk identification can take (i.e., cause → risk; risk → consequences). Individuals are able to generate and reuse multiple impacts and uncertainties to generate a list of novel and specific outcomes tied to the scenarios (Table 1). This simple methodology allows for significant flexibility while also assessing the extrapolation component of anticipatory thinking. Individuals’ anticipatory thinking performance is assessed using three metrics that aim to capture the novelty and uniqueness of each response, the level of diversity across responses, and the level of detail in the description of the responses.
Traditional risk identification metrics typically focus on assigning each risk with characteristics such as likelihood and impact ratings. These numeric ratings are then used to produce rankings or visualizations, such as heat maps or scatterplots, to categorize the most important risks for further analyses (Figure 2). Risk plots can provide information about which risk categories are not being sufficiently explored and regions of unexplored risk space (e.g., high impact / low likelihood). These metrics, while informative, miss out on the actual quality of the ideas being generated, providing a limited view into the quality of the risk identification process.
The AT metrics complement this process, as they can be used to assess the quality of individual risks, and provide a more complete picture of the set of risks identified. Overall, three AT metrics were identified that related to risk identification (Geden et al. 2019). They are meant to broadly investigate the depth of the ideas generated and the breadth across the search space that individuals explored.
Novelty is an AT metric that describes the level of uniqueness of a given response. Ideally, this would be assessed relative to other responses for a given prompt, though practically it can often only be assessed relative to a portion of all generated responses. In divergent thinking research, novelty is also sometimes referred to as originality (Guilford 1967).
An important goal of the risk identification process is to identify risks that may be unexpected so that proper monitoring or identification of risks can take place. Novelty is an important metric for this goal, as it can provide a measure of how similar identified risks are, and demonstrate that the ideation process has not shifted toward premature convergent thinking and evaluation.
While novelty/uniqueness are important characteristics of a response, even the most creative response is not useful if it is not clearly elaborated and appropriate to the problem. Specificity attempts to capture this by rating how clearly a given response is described.
This metric relates to risk identification, as a risk needs to be clearly described in order to enable a proper evaluation of its likelihood and impact, as well as how it relates to other risks. Experts in a given domain may score higher on specificity due to extensive knowledge in the given context compared to novices. In a practical context, a low level of specificity across responses could lead to difficulties later in the risk assessment process, when trying to determine mitigation and monitoring strategies or more directly quantify the severity of potential impacts. Diversity seeks to measure how well a set of responses covers the breadth of the problem space. For AT, this was measured by looking at how many different categories a participant generated a response for. This metric helps to contextualize the quantity of submissions generated, while also helping to identify areas of the problem space that may have not been fully explored in the ideation process.
For application to risk identification, a key challenge is
identifying categories for a particular domain. While
individual organizations or domains may have their own
categorization structure, there are also more generalizable
paradigms such as PESTLE (Political, Economic, Social,
Technological, Legal, Environmental) or PMESII (Military,
Infrastructure, Information Systems). General
categorization schemes can be used across domains
To illustrate the application of anticipatory thinking metrics, consider an example of risk identification in an industry that regularly employs risk management: construction. The construction industry is an inherently dynamic, risky, and unpredictable field with risks able to detrimentally impact the productivity, quality, and budget of a construction project (Maytorena, Winc, and Kiely, 2007). For this example, we will take the perspective of a construction company, which we will refer to as Build, working on a site in the northern panhandle of Texas.
As part of Build’s typical risk identification process, the company considers environmental risks such as fires or flash flooding. One employee notes the increased risk of earthquakes due to fracking (Magnani et al. 2017) in the northern panhandle and suggests that earthquakes should be added to the list of environmental risks, even though they historically have been atypical for the region. The novelty metric would identify this suggestion as being new and creative, and due to its lack of previous consideration worth further exploration.
This risk sparks a conversation about liability and safety regulations involving earthquakes, and whether Build would be at fault for any accidents due to insufficient design for environmental factors. The specificity metric would identify this precise new liability risk as being useful, as there is enough detail for further exploration as opposed to a vaguely identified risk, such as “legal liability”.
As part of Build’s risk identification process, they
categorize risks according to
This example, while simplified, illustrates how the anticipatory thinking metrics could be applied toward real circumstances employing risk identification. In a real risk assessment exercise, many risks would be identified and the novelty, specificity, and diversity of the generated risks would be evaluated.
The AT metrics described here (i.e., novelty, specificity, diversity) have several limitations. First, they are resource intensive to calculate as they are hand coded, which limits their scalability. A second limitation is that it is not clear how to calculate a single score for each analyst based on the response level metrics. One potential method is to take the mean of the top n responses, which unlike the total mean, would not punish for analysts who create many low/medium quality responses. However, this does not account for overlooking key risks, such as environmental impact of a nuclear meltdown. Ideally, individual metrics should account for both the presence and absence of relevant risks, but it is currently unclear how to create a composite that provides this more holistic perspective.
Risk identification is the critical first step in risk management. However, current understanding of the cognitive processes underlying risk identification is limited. There appears to be a strong relationship between anticipatory thinking and risk identification, and anticipatory thinking metrics originally developed for anticipatory thinking hold promise for assessing the quality of a risk assessment. These metrics may serve as powerful research tools to develop an empirical understanding of the cognitive process of risk identification.
Future studies should be conducted to evaluate the psychometric validity of these metrics within the domain of risk identification beyond the construct validity detailed here. Another promising direction for future work is improving the generalizability of the proposed metrics by developing natural language processing models to support the automatic assessment of identified risks. Additionally, an important extension of this work is to use these metrics to investigate how the quality of risk identification can impact the downstream phases of risk management, such as risk assessment, planning, and system resilience.
This work was supported by the North Carolina State University Laboratory for Analytic Sciences. Department of Defense 2017. Department of Defense risk, issue, and opportunity management guide for defense acquisition programs. Office of the Deputy Assistant Secretary of Defense for Systems Engineering.Washington, DC: Author.
Hancock, B. 2016. The Bow-Tie Analysis: A Multipurpose ERM Tool. Available at: https://erm.ncsu.edu/library/article/the-bowtie-analysis-a-multipurpose-erm-tool Hines, A., & Bishop, P. J. 2006. Thinking about the future: Guidelines for strategic foresight. Washington, DC: Social Technologies. Hubbard, D. W. 2009. The Failure of Risk Management: Why It’s Broken and How to Fix It. Hoboken, NJ: John Wiley & Sons, Inc. Klein, G., Snowden, D., & Pin, C. L. 2007. Anticipatory thinking, Proceedings of the Eighth International NDM Conference, Pacific Grove, CA, June 2007
Magnani, M. B., Blanpied, M. L., DeShon, H. R., & Hornbach, M. J. 2017. Discriminating between natural versus induced seismicity from long-term deformation history of intraplate faults. Science Advances, 3(11), e1701593.
Maytorena, E., Winch, G. M., Freeman, J., & Kiely, T. 2007) The influence of experience and information search styles on project risk identification performance. IEEE Transactions on Engineering Management 54(2):315-326.
McLennan, J., Elliot, G., & Holgate, A. 2009. Anticipatory thinking and managing complex tasks: Wildfire fighting safety and effectiveness. In Proceedings of the Industrial & Organisational Psychology Conference, 90-95.