In this paper, we propose to use the Lean Methodology to reduce losses due to the uncertainty of possible solutions during the execution of a process that results in a valuable product. According to the Lean principle of amplify learning, risk management is implemented using feedback from process participants in short time intervals. Each such interval is represented by a cycle with stages build-measure-learn; the reaction of process participants in the learning stage improves the build stage in the next iteration. We propose to perform a hierarchical decomposition of risks and introduce two categories of risks: final risk, which corresponds to losses due to uncertainty in the outcome of decisions, and indicated risk, which means the deviation of process characteristics from the planned normative values. Two types of characteristics are considered: observable characteristics, that can be directly measured when the increment is reached, and unobservable characteristics related to consumers' perception of the increment and can be evaluated through surveys. The mechanism for evaluating individual characteristics of the process iteration, aggregating the critical values for all characteristics, and obtaining the indicated risk level based on them is proposed. Options for determining the final risks based on the obtained levels of indicated risks are proposed.
Decision-making under uncertainty is typical for many domains [
The education has long been considered a domain protected by the government at
the legislative level and can have only particular problems. There exist a whole set of
Copyright © 2020 for this paper by its authors. This volume and its papers are published under
the Creative Commons License Attribution 4.0 International (CC BY 4.0).
risks here such as the risk of deterioration in the provided quality of educational
services, the risk of unsuccessful implementation of new educational projects, the risk of
reputation loss of an educational institution and advantages loss on the education
market, personnel risks, shortage financing and much more [
When choosing an alternative decision, a decision-maker is guided, on the one
hand, by his risk preference, and on the other hand, by the appropriate criterion for
decision choice according to the payoff table. The general approaches used in the
decision-making process under uncertainty are Wald’s maximin strategy, maximax
strategy, Hurwicz’s pessimism-optimism index, Savage’s minimax regret criterion
[
According to the Project Management Body of Knowledge [
In this paper, we examine an approach to reducing losses caused by uncertainty through the use of Lean methodology. Lean methodology, by definition, is focused on the client and his needs and has the task of optimizing the production process in such a way as to create a valuable product while reducing costs. 2
The complications of the decision-making process due to the existence of uncertainty
have long been recognized. Uncertainty concerns determining the available solutions,
assessing their capabilities, assessing the impact of the environment, et cetera [
Work towards risk-based decision-making led to the formalization of the process
[
Next comes the risk assessment – identifying potential problems and ordering them regarding the degree of risk. Then decision-makers can develop a risk management plan and start implementing it.
We need to monitor the success of the planned measures. Therefore, the next component is the collection and analysis of data about the primary process to identify and rank changes in risk because of risk management activities.
The impact assessment stage is intended to determine if the risk controls are
adequate. In lean methodology, it implements the Lean principle of amplify learning,
which works by providing feedback from stakeholders in short iterations [
Effective implementation of these components requires effective communication with stakeholders, during which the information necessary for analysis is collected.
Consider the known problems arising from the application of this approach. When
using new technologies, experience provides only a partial guide. Risks can be linked
to each other through processes with strictly limited total resources (e.g.,
power/mass/volume or budget/time/production volume) [
Of particular importance is the work in the direction of risk reduction for critical
systems. Failure of a safety-critical system could result in significant economic
damage or loss of life. “It is essential to employ rigorous processes in their design and
development, and software testing alone is usually insufficient in verifying the
correctness of such systems” [
From the fact that risks are directly related to uncertainties in the outcomes of
various solutions, it follows that domains with high degrees of uncertainty are subject to
risks mostly [
It should be noted that the means of minimizing losses in lean make it possible to
minimize losses, including from the realization of risks. This approach is called a lean
start-up, and it was proposed for activities in an environment of high uncertainty –
innovative entrepreneurship. [
One of the main elements of a lean start-up is the build-measure-learn cycle. Initially recognized as a product concept based on assumptions, each of which is a source of risk. Therefore, working on a plan to get a product is very likely to fail. Usually, the product is developed incrementally to prevent failure. Each increment is designed to test a specific subset of hypotheses. The critical point is to test hypotheses on a working product, not on a model or prototype. Accordingly, the concept of the minimum value product (MVP) is introduced into consideration – a product that provides the minimum set of capabilities sufficient for its assessment. Next, the MVP is launched into use, and data on its success is collected. For assessments to be informative, they must be performed using suitable scales. Collected ratings are analyzed, which means the study of the perception of the product by the consumer. The result of training can impact on the further direction of product development or a decision to change the concept (pivot).
The work aims to reduce losses in the Risk-based decision-making process by hierarchical decomposition of risks and Lean Design Technology to manage these risks.
The application principle of amplify learning to the risk management process may reduce the loss due to unsuccessful solutions. Further, the proposed approach is considered in more detail. 3
The process is considered as a set of activities that lead to the task solution and described by quadruple where CNin is the set of internal conditions, CNout is the set of external conditions.
The changes in the process state based on the results of risk analysis concerns the resources and the conditions that can be specified as where RQ is a set of requirements for the process result, R is a set of identified risks, PS is a set of process states at various design iterations, and ТК is a set of tasks to be solved by the process.
The process state at the ith iteration is defined as follows where T is a current task to solve, B is a current process state, L represents changes in the process state based on the results of risk analysis, and M is a set of tools for evaluating the current process state.
The current process state is explained as
P = {RQ, R, PS, ТК},
PSi={T, B, L, M},
B={RS, CN}, rs=<t, qcur, qmax, mc>,
CN=CNin U CNout,
L={∆RS, ∆CN}, where RS is a set of resources allocated for executing the process, and CN is a set of conditions under which the process is executed.
Each resource rs∈RS can be detailed as where t is a type of resource, qcur is a current value of the resource (can be represented as a numeric value, period, or set), qmax is the maximum possible value of the resource, and mc is the control channel resource.
The set of conditions is composed as follows (1) (2) (3) (4) (5) (6) where ∆RS is the changes for process resources, and ∆CN is the changes to process conditions.
Tools for evaluation are explained as follows m∈M=<QP, stus, stact>, (7) where QP is the set of tools for evaluating the state of the process, stus is the extent of satisfaction with the state process on the part of end-users, and stact is the extent of satisfaction with the state process on the part process participants.
The result of each iteration of the process can be described by a set of characteristics cp=<K, V, tst, tend>, (8) where K is the set of used metrics, V is the set of acceptable values according to metrics, tst is the time when the iteration started, and tend is the time of completion of an iteration. 4
Risk is a consequence of a decision and is related to the subject who not only makes a choice but also evaluates both the probability of possible events and the size of their consequences. Usually, risks are evaluated and analyzed as a whole. However, each risk is a complex system due to various influencing factors. Accordingly, as with any complex system, a hierarchical decomposition can be performed for a risk. As result, we obtain a risk breakdown structure of the entire project (Fig. 2). The root node corresponds to the most common risk – the failure of the project as a whole. Let introduce the concepts of final and indicated risk. The final risk is the possibility of losses due to the random nature of the decisions taken. The nodes of the first level of the hierarchy correspond to the final risks. The negative consequences of decisions are not always manifested at once; in some processes, they can accumulate gradually. The indicated risk can be defined as the likelihood of deviation from the planned values due to the random nature of the decision results. Leaf nodes and nodes of intermediate levels, except the first, correspond to the indicated risks.
We will use a risk assessment matrix to assess indicated risks before the process starts. Take the simplest matrix 3х3: we will consider three levels of risk likelihood (likely, unlikely, highly unlikely) and three levels of severity (slightly harmful, harmful, extremely harmful). Accordingly, three levels can be identified for indicated risks – low (green), medium (yellow), high (red).
Based on indicated risk assessments, decision-makers can assess the risk that combines them. This evaluation corresponds to the procedure of coloring the parent node of the tree in the case when all children are painted. Coloring rules depend on the risktaking of the decision-maker and the criticality of the projected results. Here are examples of rules: ─ Pessimist rule – the parent’s node is assigned a risk level corresponding to the maximum risk level of child nodes; ─ Majority rule – the parental node is assigned a risk level corresponding to the risk level of most child nodes; ─ Ostrich rule – the parent’s node is assigned a risk level corresponding to the minimum risk level of child nodes.
Fig. 3 shows the build-measure-learn loop for the Lean iterative process. Using the build-measure-learn cycle allows paying more attention to the indicated risks. Let introduce the concept of iteration. Iteration is the time interval in the project during which a result that is valuable for stakeholders is developed. We will call this result an increment. Indicated risks within a single iteration can be considered as independent. In the multidimensional feature space that describes the iteration result, the decision-maker has to define the limits of the expected values. Going beyond the expected values signals the implementation of indicated risk and the need to respond to the risk. At the end of each iteration, based on the information obtained from the collected measurements, the indicated risk evaluation D is performed as a function:
D = f (A, R), where А is the result of process state analysis. The analysis can be performed based on a set of RL predefined comparison rules with quantitative values and/or based on surveys using a set of questionnaires QN.
We will distinguish two types of increment characteristics – observable OV and unobservable OU. The observed characteristics are all those that are directlymeasured on the product increment. For example, if the increment is a new article in a corporate blog, then the observed characteristic may indicate audience engagement. The unobservable characteristics are related to the perception of increment by consumers and do not allow direct measurement. For example, in this example, an unobservable characteristic might be that readers agree with the content of the article.
In the case of observed characteristics, decision-makers usually use a quantitative scale of assessment for the unobservable – nominal or orderly. Measurements are used in the first case, and surveys are used in the second case.
In the case of non-quantitative scales, it is necessary to move to quantitative measurement. The simplest solution is to attribute quantitative values to categories and calculate the weighted average value.
For each characteristic Oi, we will introduce an estimation value x, for which two xmin and xmax thresholds need to be set, which are a risk level l(Oi) for the increment for this characteristic.
l (Oi ) green risk, if 0 ≤ x < xmin =yellowrisk, if xmin ≤ x ≤ xmax .
red risk, if x > xmax Then we can calculate the determinative value for each characteristic: di =
xi − xi.min xi.max − xi.min . (9) (10) (11) (12) Aggregation of determinative values for all characteristics that are relevant to the risk under consideration gives a determinative iteration value:
n D = ∑ wi di ,
i=1 where wi is the weight coefficient that determines the importance of the ith object of measurement.
Then the level of each indicated risk is defined as red risk,
if D > 1 green risk, if D < 0 l ( IR) =yellow risk, if 0 ≤ D ≤ 1. (13) Branches that are predicted to worsen the risk level are problematic and require a response from the decision-maker. The actions taken are implemented at the build stage of the next iteration.
The increment is built following the requirements set out under the influence of certain external and internal conditions. The determines of the conditions allows us to take into account their influence on the process. Stochastic components of impacts require the introduction of a “reserve coefficient” to compensate for possible damage.
Process iteration metrics are an indispensable component because they allow decision-makers to organize process management. They measure the results of the iteration, and these measurements should be then compared to particular expected values.
Risks as a combination of adverse consequences and their probabilities can be uncritical – those that can still be corrected by any process changes – and critical, which means the failure of the process. Note that fixing the failure of the entire process – the point of no return reached on an arbitrary iteration – is possible even if the process resources – time, material reserves, budget funds, et cetera – remain unused for subsequent iterations. For example, working with a focus group shows that using the MVP of a software product under development does not lead to solving consumer problems. This is the implementation of critical risk; a further investment of resources in developing the product will not lead to its demand. If the result of working with a focus group determines that the MVP can solve the problem, but work with it is inconvenient, the risk is uncritical. Improving the UI/UX (User Interface / User Experience) will lead to satisfying consumer expectations.
To collect data on the results of using MVP in accordance with the amplify learning principle, we suggest using surveys of participants in the process iteration. In this way, we get an idea of the problem that has not yet occurred by indirect indicators. Surveys reflect the subjective perception of respondents’ reality, so the sample of respondents based on the survey results should be representative. Note that the composition of participants and, therefore, the composition of respondents may differ in different iterations. The survey collects data on any questions that are derived from the requirements for the corresponding process iteration.
Surveys are usually performed using questionnaires. The questionnaire is a set of questions that can be answered using certain scales (most often, it is a Likert scale, but others are available). The form of question-giving and answering should be in line with the target audience of respondents. Survey Experts are responsible for designing questionnaires and interpreting responses. They must have information about the subject area and possess a high level of logic, coherence of questions and answer options, as well as the language of communication with the Respondent.
A comparison of process iteration metrics with their expected values is performed according to rules that are individual for each process, taking into account its specifics. The comparison results allow us to detect the presence of indicated risks. In this case, the risky decision-maker must make changes to the build stage in the next iteration; the changes may relate to the external and internal conditions of the process, its participants, and resources.
Let the solution of the problem T be performed on the ith iteration of the process P, and match to a set of metrics Ki={Ki1,…, Kim}; the number of metrics may differ for different iterations. Measuring of observed characteristics are performed directly on the increment, measuring of unobservable characteristics are performed using surveys. When compiling a questionnaire to avoid its redundancy should be investigated such a subset of metrics Ki*⊆Ki, which will allow clarifying the situation with all indicated risks RIi thoroughly. Thus, the necessary and sufficient conditions must be met for subset metrics Ki*: ─ a complete set of metrics of this subset is necessary to assess the full set of indicated risks, and no metric can be excluded without violating the evaluation of one or more indicated risks; ─ having a complete set of subset metrics is sufficient and guarantees an evaluation of the full set of indicated risks.
Measuring a specific unobservable characteristic is obtained as a statistical generalization of responses to the corresponding questionnaire question.
Fig. 4 shows the process of forming the questionnaire as a measurement tool. Survey Resources is the set of all possible tools to check the values of Kij* by surveying. Selected Metrics (SM) includes metrics that will be controlled by a survey, but in general, not all metrics can be controlled in this way: SM ⊆ Kij*.
With Selected Metrics and their measurement scales, Survey Expert forms prototypes of questionnaire questions.
The next stage is an adaptation, where Survey Expert takes into account Special Requirements: ─ the size of the questionnaire (the maximum allowable number of questions determines the estimated time of the survey); ─ order of questions (if there are several semantic groups of questions and several questions in each group); ─ target audience (age, special requirements for people with disabilities, et cetera) After considering all the above requirements, a Designed Questionnaire is created based on the prototype questions.
Thus, in order to perform risk management using Lean Methodology, it is necessary to pre-process the available information about the process in order to formalize the task of evaluating indicated and, subsequently, final risks (Fig. 5).
Process P
Identification of
the tasks TK Identification of iterations CP Identification of risks R
Determination the correspondence between
TK and CP Determination the correspondence between R and CP
Identification of the measuring objects
OV and OU
Identification of the rules RL
Creation questionnaires
QN
The assessment of the risks
Of the various processes, one can single out those in which iterations are activities repeated in time with practically the same meaning. Further, the expediency of conducting surveys as tools for working with indicator risks was investigated.
An example of a typical process is finding and choosing a tone of voice for a product company that wants to increase sales of its own product. A well-chosen tone of voice allows conveying the company’s product values to the audience, detach the company from competitors and find contact with the audience, speaking with it “in the same language” in accordance with age, social status, life values, et cetera.
Each iteration of the process is associated with the publication of a message intended for reading by the target audience on the corresponding social network (Facebook, Instagram, LinkedIn, et cetera). The success of the tone of voice selection can be monitored by the level of engagement by the target audience: the number of reactions, comments, reposts, clicks, use of an offer, photo, or video views. It is useful to survey to manage the increase in the success of the tone of voice selection. In this case, one should understand the opacity of the scheme of interaction of the end-user with a specific publication in accordance with the policy of the social network, i.e., not all potential clients will be able to take part in the survey.
Conducting a survey reveals the following risks: ─ r1: unique selling proposition will not provide value to the end consumer; ─ r2: a style and language of the publication will not establish an emotional connection with the consumers of the product; ─ r3: brand values will not match the values of the target audience.
Fig. 6 shows the final risk tree for a product company’s advertising publication. The post contains a clearly articulated unique selling proposition, responding users generally support the brand values. However, emotional engagement turned out to be at a low level due to the inconsistency of the style and terminology of publishing the topics expected by the target audience. If the Democrat’s rule is used to assess the overall risk (as shown in the figure), then this risk will go unnoticed, and problems will be identified at the stage of calculating the conversion rate. Simultaneously, even if there were assumptions about a weak emotional connection, it would be useful to understand what exactly the user did not find in the publication: consistency, emotionality, incentive, et cetera. The details of the final risks should give just indicator risks. Let look at a more complex example, which is a process that has unique iterations. Let us show on this process the risk structure with indicator and final risks. 5.2
In March 2020, due to quarantine, the university was forced to switch from full-time to online education instantly. However, either a package of teaching materials for fulltime education or materials for blended learning accompanied all courses. This situation has generated some risks: ─ r1: there will not be enough teaching materials; ─ r2: the learning environment will not allow realizing the planned activities; ─ r3: the learning load will be too hard; ─ r4: there will be poor communication with teachers.
All risks are associated with high uncertainty due to external factors, namely infrastructure capabilities and properties of student groups. Therefore, it was advisable to apply the proposed approach. The risk breakdown structure is described in Table 1. ID Content r1 There will not be enough teaching materials r1.1 The teacher will incorrectly determine which materials require revision r1.2 The teacher will not have time to prepare additional materials r2 The learning environment will not allow realizing the planned activities r2.1 The teacher will ineffectively use the capabilities of the learning environment r2.2 The learning environment does not provide the required capabilities r3 The learning load will be too hard r3.1 Critical accumulation of not completed works will occur r3.2 The load in another course (in other courses) will peak increase r4 There will be insufficient communication with teachers r4.1 The teacher will not be able to devote as much time to communication as the students need r4.2 Communication channels will not allow organizing adequate communication We applied the Lean Iterative Process (Fig. 3), under which we distinguished observable and unobservable characteristics at the Increment stage. A feature of the full-time educational process at the university is that the solution to a particular task of the course can be completed in two weeks. Therefore, it was decided to limit the buildmeasure-learn cycle by time and to determine its duration as two weeks.
First, we formed a set of metrics that stayed the same for all iterations. The observed characteristics included the follows: ─ percentage of students who completed tasks, metric K1 – the percentage of completed work; ─ the successfulness of students in the task, metric K2 – the average mark for the performed work; ─ the ability to invest the teacher’s time, metric K3 – the estimate in hours for time that can be spent on the course; ─ adequacy of the online learning environment, metric K4 – the probability that the available tools will be sufficient.
Fig. 7 shows the mapping of metrics of observed characteristics to a set of indicated risks. Each indicated risk is associated with one or more observable characteristics that reflect the teacher’s point of view. It is necessary to conduct a survey to take into account the students’ point of view. The questionnaire contained six questions: ─ q1: would you like to have more guidelines and teaching materials? ─ q2: is it comfortable to work in an online environment? ─ q3: is the scope of work within the course acceptable? ─ q4: did other courses interfere with this week’s assignments? ─ q5: did the teacher help you with the course material? ─ q6: is it convenient for you to communicate with the teacher? For each question, students could give one of two answers – “yes” or “no.”
For each characteristic, we defined the threshold values. That gives us possibility to determine the risk levels according with (10) based on results of direct measurements or surveys. As well, we calculated the levels of indicator risks with (11)–(13).
Consider what happened in the first two iterations. We will not present the results of measurements and calculation of determinative values, and we will only consider the changes in coloring. We used the pessimist rule to color the structure; the tree painted at the lockdown start is shown in Fig. 8.
Coloring the structure after the first iteration is shown in Fig. 9. As we can see, the teacher accurately assessed most of the risks. However, the volume of work was underestimated, which increased the importance of risks associated with a lack of time. Let pay attention to the risk r1. It requires more attention than it seemed during the initial assessment. Moreover, if the decomposition had not been performed, we would not have known about it. The situation did not deteriorate significantly, so it was decided not to make changes in the online course.
The coloring of the structure after the second iteration is shown in Fig. 10. Let pay attention to the fact that the second iteration was completed during the period of midterm control. After the completion of the second iteration, the three indicated risks turned red. Let pay attention to the risk r3, for which there was a deterioration due to the influence of external factors. If we had not performed the decomposition and performed the estimation at the end of the iteration, it would not have been possible to catch the deterioration and understand its causes. Accordingly, it was decided to devote the next iteration to working with risk r3.1, which should also affect the level of risks r1.2 and r3.2.
Thus, in the experiment, two types of processes were considered: with iterations of the same type and with iterations of different types. It is expedient for all processes to build a colored risk breakdown structure, in which indicated risks allow taking early measures to eliminate losses leading to project failure.
Decision-making is associated with reducing the risk of loss. The traditional riskbased decision-making process consists of goals setting, risks assessing, potential problems identifying and ordering, risk management, and assessing management effectiveness. The risk breakdown structure results from a focused risk assessment, which differentiates the negative impacts of activities that lead together to project failure.
The paper proposed to apply the lean start-up approach and consider the process of obtaining a useful product in the form of short build-measure-learn cycles. Each cycle provides an increment of the product. The meaning of the increment depends on the goals of the whole process. In software development, an increment could be new features of a software product; in the case of the learning process, an increment could be a set of developed skills, relevant to learning goals.
Some measured values characterize the product. The observed characteristics are assessed with the results of measurements on a quantitative scale. Unobservable characteristics are assessed with surveys using a nominal or ordinal scale. A comparison of the measured and expected values for characteristics makes it possible to assess the level of risk for each characteristic on the green-yellow-red scale.
We examined the proposed approach for the process of transition to online learning. From the beginning, we built the risk breakdown structure with the allocation of final risks that affected achieving the goals of the project and indicated risks that move the current state of the project from the planned state. Next, we defined the observable characteristics that ensure the current state of the project from the planned one and the set of questions to assess the unobservable characteristics of the process. Finally, we demonstrated the coloring of a risk breakdown structure for sequenced iteration of the long-term process.