Advances in machine intelligence methods over the recent years are bringing the accuracy and confidence of AI systems to the level of regular human operator, and in a number of cases, exceeding it offering opportunities for improvements in the quality, performance and cost efficiency of decision-making systems. To address concerns and challenges in application of artificial systems in critical decision-making areas, the concept of collaborative human-AI decision-making system is proposed, aimed at utilizing the strengths of human and machine intelligent methods to maximize the performance in a cost-efficient process. It is demonstrated that multi-channel human-AI systems can have a number of advantages compared to conventional systems and produce significant improvements in both accuracy and performance. Applicability criteria of single and multiple stage decision-making systems are defined and discussed. It is demonstrated that collaborative human-AI decisionmaking systems have significant potential in improving quality and effectiveness of decisions in many areas of application.
The advances in machine intelligence methods and technologies over the recent years have
brought the performance of machine systems in a number of tasks and areas of application to the level
of regular human operator, human expert or exceeding it [
However, introduction of such complex artificial intelligent systems in critical areas and
applications including aviation; public safety and security; health care and others can be associated
with essential challenges of its own, not in the least in the areas of public trust and confidence in
critical decision-making systems that employ such components [
Another driver for introduction of higher efficiency decision-making systems is the rising cost of
errors in critical decision areas, for example the cost of diagnostic errors in primary healthcare system
that has been found to contribute significantly to the overall cost of public healthcare [
Often, the problem of decision making can be reduced to selection from the set of acceptable
options [
On the other hand, methods of machine intelligence in application to decision-making in practical
domains and applications include, clustering methods [
Taking into consideration challenges as well as the opportunities, an investigation and discussion of methods and strategies of safe and efficient application of machine intelligence technologies to decision-making problems in practical domains and applications with the objective to harness the advantages of human and machine intelligent systems to improve the quality and performance of resulting decisions without compromising safety and retaining full trust and confidence of the public in the system. 1.1.
With human and AI decision-making systems having clear complementary strengths as illustrated
in Table 1, the advances in machine intelligence methods over the recent years bringing the accuracy
and confidence of decision to the level of regular human operator, and in a number of cases,
exceeding it, provide both an opportunity and a foundation for introduction of Collaborative
HumanAI decision-making systems (CHAI DMS) harnessing the synergy of strengths of human and machine
intelligent systems and as a result, improving the performance in multiple domains and tasks of
application in quality, confidence and capacity of decision-making system.
comprehensively verified systems and technology as they deal with the issues of trust and confidence
in machine-based technology by the general public that cannot be taken as assured [
The challenge therefore lies in creating collaborative human-machine intelligent decision-making methods, models and systems that are capable of combining the benefits and strengths of human and machine expertise, while minimizing their respective shortcomings and ensuring confidence and trust in the produced decisions.
Consider a decision-making system that has multiple channels C1,...,Cn and the final decision on an input X is obtained from decisions of the channels by a summation process described by a decision function D based on the decisions of the channels:
D( X ) D(c1( X ),c2 ( X ),..,cn ( X )) (1) In the general case, the decisions of the channels ck can be of the following types: Categorical, with values in a certain set of valid values V. Numerical, integer, rational and other numerical types. Binary i.e. True or False value that will be assumed for illustration in the rest of this article. Other types, such as verbal, tokens, etc.
We will assume that the decision produced by a channel Ck(X) on input X comprises a typed value
c(X) as defined above, and the confidence A(X), measured by a factor in the range [
Each decision-making channel Ck can be characterized by the following parameters: accuracy Pk and error Ek as combined measure of statistical errors of both types; and confidence, Ak. The goal of a multi-channel decision-making system is to maximize the accuracy of the decision function D(X) and minimize the error on the trial set of inputs { X }.
In addition to the decision function, a “conflict function” C(X) is defined as measure of agreement or disagreement between decisions of channels on a given input:
C(X) = C(c1(X), .., cn(X)) (2)
Together, the results of decision and conflict functions D(X) and C(X) for specific input evaluate how close the channels are to a common decision about the input.
Definition of decision functions are specific to the system. Several examples are provided below.
Logical decision function: in the simplest cases, the decision function can be defined as logical conjunction (AND) or logical disjunction (OR) of the channel decision. For example, a two-channel system with logical disjunction decision function has the accuracy and error that can be derived from the channel factors:
Confidence: the decision with the highest confidence among the channels is selected:
D(X) = ck(X), Ak = max(A1, .., An)
Ranked Confidence: the decision with the highest confidence taking into account ranks of the channels is selected:
D(X) = ck(X), Ak = max(w1 A1, .., wn An) where wk, weight or rank of the channel Ck.
Voting and Ranked Voting: the decision with the highest number of votes of confident channels is selected.
And a number of other strategies, as discussed in the following sections. 2.1.
An example of multi-channel Human-AI decision-making system in application to diagnostics of
medical conditions was proposed in [
Even relatively simple parallel system as multi-channel two-stage CHAI may have a number of
important advantages over conventional single-channel decision-making models. As indicated by the
results obtained with published accuracy benchmarks in medical diagnostics for a number of common
conditions, employing a two-stage CHAI DMS with a human arbiter has the potential for improving
the accuracy of the diagnostics up to and above 10% [
Another essential advantage is efficiency and performance. If both human and machine channels are in agreement, the expert channel does not need to be activated. Additionally high operational efficiency of automated system and the fact that they can operate continuously in the 24×365 mode means that the incremental cost of producing the cumulative decision based on the results of the individual channels in a modern information system would be negligible.
Thirdly, the system allows to employ highly knowledgeable expert resources only for complex cases where higher level of expertise is warranted. Limited expert resources can be employed in a highly efficient distributed system on a regional or national level with remote access to all necessary data, tests and history.
Not in the least, the cost of deployment of a trained and verified in the related area of application multi-channel CHAI system can be minimal, comparable to that of a routine operation of installing or upgrading software while offering considerable value over time in improved quality of results.
Finally, further advantage of such an integrated system is that complex cases can be tracked to the outcome and eventually added to the training set increasing the quality of both human and machine decisions.
On the other hand, two stage CHAI DMS require certain time window, te for adjustment of the final decision in the scenario of conflict or uncertainty of the channels. In some cases, this is not a problem, but certain situations and / or environment may not allow even for a short window for the second stage decision. This case is considered in Section 2.2.
≥ ℎ
Parallel Human-AI DMS can be used when average accuracy of machine channel achieved that of a regular human operator. That condition is realized in the growing number of tasks and applications where Pm, Ph the accuracy of machine and human operator, respectively.
It is also assumed, as commented earlier that the accuracy of the expert channel in the initial, parallel processing stage of the decision sequence is higher than that of either of the human or the machine channels in the parallel stage.
Finally, the time factor of the second stage decision te must be acceptable for the operational environment of the system Top, i.e. Top ≥ te.
If the accuracy of machine channels achieves the level where it consistently exceeds that of a
regular human operator (Chess, Go [
Independent machine channels ensure that cases are evaluated to sufficient level of detail, and only if the channels are in agreement the decision of the first phase is accepted as final. Otherwise, if a conflict is detected between the decisions of channels, human expert makes the final decision with complete information about the case.
The applicability criteria of the multichannel AI with Expert Arbiter will be: where Pk, Pe the accuracy of the machine channels and expert channel, respectively; Ph: the accuracy of a regular human operator.
Such a system allows to maximize the quality of decisions made by individual channels while utilizing expert resources efficiently and effectively and retaining complete human control over critical decisions.
= ( ) > ℎ; ≥ (3) (4) 2.2.
In some cases, even a short window for expert decision te cannot be allowed by the dynamics of the situation and / or operational environment. In these scenarios multi-channel CHAI model has to be adapted to produce decisions in a single pass within time allowance of the operational environment. This can be the case in dynamic operational environments dominated by automated processes, for example aviation; cyber-security; financial system and others. The diagram of a single-stage multichannel DMS is shown in Figure 2.
Let us consider application of a multi-channel CHAI system described in Section 2.1.2 above in a critical domain such as aviation or public security. It is clear that applicability of the system in these situations will be determined by the relationship of the time factor of the second stage decision te and the minimum time allowed by the situation, tm. If the range of the former intersects with that of the minimum operational response time, i.e. for at least some X, te ≥ tm it is clear that the second stage decision could not be used in such cases.
As before, let ck, Ak be the decision and confidence of the k-th channel Ck. The essential objective of a single-stage DMS is then finding the decision function based on summation of partial channel decisions that: 1. Minimizes the error of the decision function D on the set of possible inputs { X }; and 2. Can be achieved within a time window shorter than operational time, tm.
As before, we will limit consideration to the case of binary channel decisions, ck = {True; False}, k = 1 .. N; the cases with differently valued channel decisions (Section 2) can be considered in a similar manner.
A number of solutions can be proposed for single-stage decision functions D(X), that include, among other possible definitions: 1. Combined confidence:
True, if D i1,n:Ci True False, otherwise.
Ai
i1,n:Ci False
Ai ;
The decision function is based on the sum of confidences of the channels that selected a decision, so that the decision with the highest confidence is selected.
2. Difference of confidences:
True, if max A max A ; D i1,n:Ci True i i1,n:Ci False i
False, otherwise; where Δ ∈ (-1, 1]: const, confidence threshold.
The function selects the decision if the best confidence among the channels that detected the condition exceeds that of those that did not detect it by a minimum value of a defined constant threshold (confidence threshold).
3. Voting: Let Then the decision function where Δ ∈ (0, n]: const, voting threshold.
1, if Ci True; (Ci )
0, if Ci False. n True, if (Ci ) ; D i1 False, otherwise;
The rule is based on the number of channels that detected the condition that has to exceed a defined constant threshold (voting threshold).
4. Average confidence: Let the decision function, D(X) be:
This decision function detects the condition if the average confidence among the channels that detected it exceeds that of the channels that did not detect the condition.
Generally, the choice of the decision function depends on the specific task or application. To improve flexibility and versatility of the system, an option to pre-configure the system with a decision function from pre-defined standard set can be useful to support wide range of tasks and applications.
In most of considered standard decision functions confidence plays key role in choosing correct decision. Because appellation to second case is not available in this type of systems, confidence of the identified decision has to be sufficiently high, particularly in critical applications. We will return to further discussion of this question in Section 3. 2.3.
An important advantage of multi-channel AI systems is that they can be applied to tasks and applications with very short decision window (down to micro to milliseconds range), well beyond the range of capacity of a human operator. Essential criteria of their use in operational practice are: 1. Verified ability to achieve certain minimum threshold of accuracy Pmin on a representative set of decisions. 2. Short operational time factor Top ≤ te 3. Sufficient level of confidence of the channels that can be measured by a minimum average confidence threshold Amin on a trial set of inputs { Xtr }: Ak ≥ Amin
As pointed out earlier, confidence is an essential factor in the decision that has to be considered in selection of specific decision function for the area of application.
The analysis in the previous sections demonstrated that multi-channel human-AI systems can be
effective in improving accuracy, cost and performance of decisions in many areas of application. Let
us consider potential applications of CHAI DMS in critical areas such as aviation and public security.
Applications in public health care were considered previously in [
A critical decision in a developing situation can be taken by the system based on combination of factors as: 1) operational time frame; 2) impact, or cost of wrong decision and 3) confidence of the decision produced by the system. For high-impact decisions such as in-flight issue or developing public security situation, the trade-off is between the response time and the confidence of the decision. In such scenarios two-stage systems with human expert arbiter have highest confidence, if operation time frame allows it. Note that the actual scale of the time frame is determined by the situation and can be different among the domains of application, from days in diagnostics, to minutes in operational flight control. Decision-making system Operational time Ts ~ Top or Ts > Top
Ts < Top
As was commented earlier, not all domains of application allow response time frames where human arbiter can be involved. An example can be a public information security situation or financial system, where action and response time can be measured in fractions of a second. In such situations, the response of an automated intelligent system such as multi-channel AI DMS discussed in Section 2.2 has to be determined by the confidence A(X) that has to be above the minimal threshold of confidence Amin defined for a high-impact response.
As it cannot be ascertained with a practical system that every possible critical situation will produce a confident decision satisfying the condition of the minimum confidence, with exception cases that may include, for example, novel scenarios not encountered in training, a special, “default” scenario has to be defined as well for cases where DMS could not come to a decision with sufficient confidence within the operational timeframe. Such a scenario may constitute for example, “freezing” the situation at a safe level while triggering an alarm for immediate expert attention.
Applicability rules of single and two-stage collaborative DMS can be summarized in the following table:
Rapid development of machine intelligence technologies offers opportunities for their applications to improve quality and performance of decision-making systems. In this work, models of collaborative decision making with participation of machine intelligence were studied. Two types of multi-channel decision-making systems were considered: two-stage system with an expert arbiter and single-stage multi-channel system with principal objective to improve accuracy and performance over conventional systems while retaining maximum human control over critical decisions.
Decision selection methods developed in the study take into account both the decisions and confidence of the channels that is essential in identification of optimal decisions in the critical cases and domains of application. An additional advantage of the proposed approach is the ability to configure decision rules in the pre-operational phase taking into account the context of the task and domain, as well as additional considerations of the decision maker. This approach allows to maximize flexibility of the system and employ it in a wide range of tasks and domains of application.
The proposed parallel multi-channel architecture combining human and machine expertise into a single synergetic system offers a number of essential advantages over conventional “single-chain” decision-making models, including: A significant improvement in overall accuracy of decisions. For two-stage systems, it does not introduce additional delays in the decision process due to high operational capacity of the machine intelligence channel, whereas for single-stage ones, offers significant improvement in performance. Flexibility: the system is highly adaptable and transferrable to different areas / domains of application. It allows optimal use of limited expert resources only in the situations that require expert attention. Is fully compatible with distributed, high-performance performance models of service delivery. Combines strengths and advantages of the human and machine intelligences for an optimal outcome. Allows to retain complete human control over critical decisions. With minimal incremental cost of development and implementation.
The essential benefit of incorporation of machine intelligence methods into decision-making systems is the ability to produce better decisions more efficiently in a broad range of tasks and applications. For these reasons it is expected that collaborative and synergetic human-machine intelligent systems, including of the type considered in this work, will be finding more applications in a wider range of tasks and domains with the potential of significant improvement in the quality, performance, reliability and efficiency of decisions in both everyday and critical tasks and applications.
The authors are grateful to the colleagues at the Department of Information Technology, National Aviation University and Department Information Technologies, Uzhhorod National University for productive discussions of the topic and results of this work.