Machine Learning (ML) models have been widely applied for clinical decision making. However, in this critical decision making field, human decision making is still prevalent, because clinical experts are more skilled to work with unstructured data specially to deal with uncommon situations. In this paper, we use clinical experts' privileged information as an information source for clinical decision making besides information provided by ML models and introduce a collaborative human-ML decision making model. In the proposed model, two groups of decision makers including ML models and clinical experts collaborate to make a consensus decision. As decision making always comes with uncertainty, we present an interval modelling to capture uncertainty in the proposed collaborative model. For this purpose, clinical experts are asked to give their opinion as intervals, and we generate prediction intervals as the outputs of ML models. Using Interval Agreement Approach (IAA), as an aggregation function in our proposed collaborative model, pave the way to minimize loss of information through aggregating intervals to a fuzzy set. The proposed model not only can improve the accuracy and reliability of decision making, but also can be more interpretable especially when it comes to critical decisions. Experimental results on synthetic data shows the power of the proposed collaborative decision making model in some scenarios.
Machine Learning (ML) has experienced a surge in recent
years. They have been used to develop models for
facilitating decision making processes in different areas. The
development of these models is based on the idea that computers
can process big data and make predictions whilst it is often
hard for human experts. However, humans are more skilled
to work with unstructured information and deal with
uncommon situations. That is the reason that human decision
making is still prevalent in many areas like clinical decision
making, defence commanding, criminal punishment
predicCopyright © 2021 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
tion, etc. In some of these areas like clinical decision
making, ML models have been applied largely. However, the
reliability of these models is always under question. It is
reported that ML models are not enough in critical clinical
decision making
Human-in-the-loop ML models pave the way to
implement human expertise in ML models. In these models,
human experts can interact and collaborate in different stages
of ML process to improve the performance of ML models.
Clinical experts can collaborate in three stages of data
producing and data processing
ML models are generated using training data and used to predict the test samples. So, these models decide based on training data information. However, there are some information that are available at the training stage but not available for test data. This information called privileged (hidden) information (Vapnik, Vashist and Pavlovitch 2009), (Vapnik and Vashist 2009). In this study, we introduce another type of privileged information that is available at the testing stage but not available (recorded) for training. For example, at the time of diagnosing a patient’s disease, clinical experts have an estimation about the diagnosis (with different level of confidence) based on their experience, patient’s appearance and reviewing documents and test results. These estimations are not normally recorded so that they cannot be used in training models. However, they can be captured for each patient at the diagnosis (testing) stage. In this paper, we propose a framework to capture experts’ privileged information and integrate with trained ML models in a collaborative decision making.
As ML models and clinical experts use different sources of information to make decision, a consensus decision making approach can improve decision making. In this paper, unlike human-in-the-loop ML models, we introduce a collaborative human-ML decision making model where both clinical experts and ML models are decision makers. Clinical decisions should not be made individually. We believe clinical experts and ML models can help each other through a group decision making process. Collaborative decision making approach can provide a trust between ML models and clinical experts to have trustable and explainable models. In addition, this approach provides an opportunity to have a precise investigation on the ML models’ and clinical experts’ performance in clinical decision making process individually and jointly. Besides, this approach improves decision making especially when a decision maker is not available due to disruption in connectivity or information is provided intermittently for decision making.
Uncertainty is inevitable in decision making. Decision
makers may have different levels of uncertainty based on
their knowledge and level of access to information. When it
comes to human decision makers, decision making
accompany with two kinds of uncertainty named inter-expert
uncertainty and intra-expert uncertainty. Inter-expert
uncertainty shows the variation among different decision makers
and intra-expert uncertainty is related to the change of the
mind of each decision maker during the time on the same
situation
In ML models, data uncertainty and model uncertainty are
two important sources of uncertainty. To capture uncertainty
of ML models by intervals, we recently have introduced an
interval modelling technique to capture uncertainty in
ensemble learning
Interval Agreement Approach (IAA) is an aggregation
method that generates fuzzy sets from interval-valued data
to minimize the loss of information in aggregation process.
Thus, in this paper, we make two important contributions: (1) we present a collaborative human-ML decision making model to use two important sources of information in clinical decision making from two groups of decision makers, ML models and clinical experts, and (2) we measure uncertainty of both decision maker groups through intervals and capture decision uncertainty in the collaborative model using interval modelling and IAA.
The structure of the paper is as follows. In the next section, we describe the technique to generate intervals as the outputs of ML models to capture ML models’ uncertainty. Also, IAA is described in this section. Then, the proposed collaborative decision making model is explained. After that, the performance of the proposed model is investigated using a synthetic dataset. Finally, conclusions of the paper are presented.
To capture uncertainty of clinical experts’ opinion, we use
interval data. When decision makers are ML models, we can
capture the uncertainty of ML models using prediction
intervals. Here, we describe how we can generate intervals as
the prediction of ML models based on the approach we
introduced in
Suppose 1, 2,…, as probabilities determined by b classifiers (generated using bags), an UI for ML model is generated by calculating the first quartile ( 1) and the third quartile ( 3) of the probabilities as (1).
IAA is an aggregation function to aggregate decision makers’ opinions while the opinions are presented as intervals. IAA aggregates intervals to a fuzzy set.
Let = { 1, … , } be a set of intervals given by m decision makers as their opinions where = [ , ] (i = 1, 2, …, m). Ag gregating intervals of set A generates a Type 1 Fuzzy Set (T1 FS) in IAA with the membership function of which is defined as (2) (Wagner et al. 2015). μA= ∑mi=1 yi⁄(⋃mj1-=i+11 ⋃jm2-=i+j12+1 … ⋃mji=ji-1+1 ( Aj1∩… ∩ Aji))
In (2), yi= i⁄m is the degree of membership and ‘/’ refers to assignment of degree of membership. The degree of membership in IAA is related to the number of intervals overlapped in a point. So, the value of one for the degree of membership in a point shows all intervals overlap at that point. To simplify (2), can be written as (3) for a point like x.
A μ (x)= Where ∑mi=1 μ̅Ai(x)
m ̅( ) = {1 ̅ ≤ ≤ ̅ 0 and three clinical experts as decision makers, the process of collaborative decision making is depicted in Figure 2. In this process, decision makers determine the probability that a test sample belongs to the main class and present it as an interval. For example, in a cancer diagnosis problem, decision makers determine the probability that the test sample is malignant.
In the proposed model, both classifiers and clinical experts have access to the electronic health records. The records are used to train classifiers. Using generating uncertainty intervals explained before, we have an interval as the output of each classifier. This interval shows the probability of belonging a test sample to the main class. Also, clinical experts are asked to give their opinions about the test sample as intervals. IAA aggregates all intervals to a T1 FS. This fuzzy set shows how much all decision makers are in agreement. A T1 FS can be shown as a list of tuples which each tuple indicates different region of change over the membership function as (4).
= [ 1, 2, … , ], = ([ , ], ℎ) Where is the left point, is the right point and ℎ is the (4) height or the membership function value of the tuple calculated using (3).
To make the collaborative decision about a test sample, we calculate the centroid of this fuzzy set. The centroid of the fuzzy set is calculated using (5). (
If the value of the centroid is more than 0.5, it shows the test sample belongs to the main class. For example, in the cancer diagnosis problem, if the centroid is more than 0.5, it shows the test sample is malignant.
In the proposed approach, accompany with making a collaborative decision, we can evaluate the performance of both groups of decision makers separately. Specially in high risk conditions, this provides us important information to make decision in a timely manner.
In the proposed collaborative decision making approach, we can calculate the width of the intervals presented by clinical experts and ML models as a measure for uncertainty. The width of intervals provides us information about how much certain clinical experts and ML models are in their decision making separately and collectively. So, we can recognize decision makers that are highly deviated from the consensus to extract informative insights for updating the decision-making process.
In Algorithm 1, the steps of the proposed collaborative decision making approach for m ML models (classifiers) and n clinical experts is described.
Algorithm 1: Collaborative expert-ML decision making process Input:
Electronic health records Test sample u Collection C1, C2, …, Cm of classifiers Clinical experts’ opinions about the test sample u (n clinical experts) The number of randomly picked samples in bagging (V)
The number of bags generated using bagging (H) Output: The decision about getting infected to the disease for test sample For i from 1 to m do
For j from 1 to H do end end Else
Select a bag of training samples (V samples with replacement) using bagging Train classifier Ci on the selected bag Compute the probability for the given test sample u using the trained classifier and assign it to Pj Compute the first quartile of {P1, P2, …, PH} and assigns it to Q1i Compute the third quartile of {P1, P2, …, PH} and assigns it to Q3i Determine the uncertainty interval [Q1i, Q3i] for classifier Ci regarding test sample u Ask the probability of belonging the test sample u to the main class as an interval from clinical expert j and named it [DLj, DRj] (j = 1, 2, …, n) Aggregate [Q11, Q31], [Q12, Q32], …, [Q1m, Q3m], [DL1, DR1], [DL2, DR2], …, [DLn, DRn] using IAA and generate a T1 FS Calculate the centroid of the T1 FS If the centroid >= 0.5
Return ‘Test sample u belongs to the main class’
Return ‘Test sample u belongs to the subordinate class’
To show how the proposed collaborative approach works,
we use Liver Disorders dataset from
UCI as electronic
health records
In this experiment, three classifiers of Decision Tree (DT), Logistic Regression (LR) and Gaussian Naïve Bayes (GNB) are considered as ML models. We split the dataset to training dataset and test dataset with the ratios of 75% and 25% respectively. Regarding the proposed approach, we calculate uncertainty intervals for classifiers. For clinical experts’ opinion, we generate synthetic data as their decision (the probability of assigning one sample to class 1, here as the main class) to examine different scenarios in collaborative decision making approach.
In the first scenario, we assume that both groups of decision makers (ML models and clinical experts) make similar decisions and have close uncertainties. So, we construct three synthetic intervals as clinical experts’ opinions using three uncertainty intervals generated by classifiers. To generate one opinion interval from one uncertainty interval, we randomly select two numbers from the range of numbers between endpoints of the uncertainty interval added by an and these numbers are the endpoints of the opinion interval. Results of this scenario in three different experiments are shown in Table 1.
Experiments 1 2 3
Performance Width of the measure Accuracy F-score G-mean intervals Accuracy F-score
G-mean Width of the intervals Accuracy F-score
G-mean Width of the intervals
Collaborative model 0.790 0.870 0.563 0.156 0.775 0.859 0.560 0.150 0.773 0.858 performance of these three groups of decision makers is not significant. According to this table, the decision-making results in terms of accuracy, F-score and G-mean are similar to each other for three decision maker groups in all experiments. Also, in this scenario the widths of intervals are close together for all groups. We know that always the width of the intervals for collaborative model is between the width of the intervals for clinical experts and ML models. Totally, we can say that the performance of the collaborative model in the case that the performance of two individual groups is like each other is better than at least one of them.
In the second scenario, we examine the effect of the bad performance of one of decision makers’ groups on collaborative decision making. We generate opinions’ intervals so that the predictions for clinical experts are far from the real class of the test samples to some extent. In experiment 1 of this scenario, we generate intervals with wider width and in experiment 2, we generate intervals with narrower width. The results related to this scenario are shown in Table 2.
The results in Table 2 shows the power of interval modelling as well as IAA as interval aggregation function in decision making. In the cases that one group of decision makers makes bad decisions, results show it does not affect the collaborative decision significantly. So, the proposed model is robust and is suitable for critical collaborative clinical decision making. Considering Table 2, we conclude that when it comes to the width of the intervals as a measure for uncertainty, we can investigate the decision made by the group with high value of width of intervals. Decision making by this group of decision makers can affect decision quality. So, we can ignore them in decision making process.
In the third scenario, we investigate the effect of interval modelling in the proposed model on decision making performance. We believe that the proposed interval modelling improves decision making through capturing uncertainty. To test it, we compare the performance of the proposed collaborative model with its equivalent point prediction model. In this scenario, we use majority voting as the most common used aggregation function in point prediction approaches. To create point prediction model, for each ML model like DT, we create different classifiers using bagging, then we use majority voting to determine the class label of each test sample using classifiers. To create synthetic point prediction data as experts’ opinions, we use the strategy mentioned in previous scenarios to generate opinions intervals. Then, we consider the middle points of the intervals as clinical experts’ point predictions to determine the class label of test samples. Finally, using majority voting on point predictions generated by ML models and clinical experts, we determine consensus point prediction for test samples. The results on comparing the proposed collaborative model to its equivalent point prediction model are shown in Table 3. In the first experiment, the opinion intervals are generated according to scenario 1 and in the second experiment opinion intervals are generated according to scenario 2.
The results in Table 3 show better performance of the proposed model than its equivalent point prediction model in terms of accuracy and F-score. It means that interval modelling in the proposed collaborative model improves decision making through capturing uncertainty.
Regarding all described scenarios and experiments, we conclude that the proposed collaborative model can be an effective model to capture uncertainty, use clinical experts’ privileged information and implement the power of ML models in clinical decision making. However, real datasets and scenarios can present more accurate analysis on the performance of the proposed collaborative human-ML decision making model.
In this paper, we use expert’s privileged information in addition to ML models in clinical decision making and present a collaborative human-ML decision making model. In the proposed model, both clinical experts and ML models are considered as decision makers. To handle the uncertainty of decision making in collaborative model, we use intervals. Clinical experts’ opinions are asked as intervals, and we develop an approach to have intervals as the outputs of ML models. IAA as a powerful interval-based aggregation function is used to aggregate decision makers’ interval predictions to a T1 FS. The generated FS is used to determine the final consensus decision. In the proposed collaborative model, the width of the intervals is a measure for decision makers’ uncertainty. So, it provides information about the uncertainty of each group of decision makers to improve decision making. To show how the proposed model works, we consider different scenarios and experiments using synthetic data and test the performance of the proposed model. Results show the power of intervals and interval modelling in the proposed collaborative model to capture uncertainty and making more effective and robust decision in clinical decision making. Also, as a significant result, we observe that weak performance of a group of decision makers does not affect the collaborative decision in the proposed model significantly. For the future work, we are collecting real data to examine our proposed model in different scenarios. Also, we will develop the proposed method for multi-class classification decision making problems.
Maadi, M.; Khorshidi, H.A.; and Aickelin, U. 2021. A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications. International Journal of Environmental Research and Public Health 18 (4): 1–27.
Vapnik, V.; Vashist, A.; and Pavlovitch, N. 2009. Learning using Hidden Information (Learning with Teacher). In the international Joint conference on Neural Networks, 3188-3195