MCDM methods have proven to be a powerful tool for selecting, prioritizing and sorting ambiguous data. In emergency situations, when resources are in short supply, such methods can be used to provide said limited resources to the most in need, without human bias. In this paper, a scenario, where there is a large number of patients awaiting treatment is considered. Each patient suffers from several COVID-19 symptoms and has an AI-assessed probability of COVID-related pneumonia. Said attributes are used to simulate real-life scenarios where patients with different symptoms and test results are ranked by the state-of-art MCDM method. To further enhance the performance of the method, different weighting methods have been tested, analyzed and compared. In the experiments, a MultiMOORA decision-maker was chosen. Among the tested weighting methods, Standard-deviation, MEREC and CRITIC methods performed better, than the basic mean method. However, the entropy method improved the performance of the MCDM exceptionally. MCDM methods, MultiMOORA, weighting methods, medical data, patient ranking Proceedings
The enormous growth of big data in past decades has resulted in an ever-growing need for better and
more reliable methods of data analysis. Finding an optimal solution may prove to be a challenging task,
given a large number of usually ambiguous criteria for assessment. Such a problem significantly limits
the human ability to efficiently make decisions. Over recent years, more and more attention has been
brought to MCDM (Multiple-Criteria Decision-Making) methods and techniques, facilitating this task.
From choosing the best and worst option, through highlighting the most important criterion, to ranking
all alternatives, MCDM methods have proven to be highly effective and vaguely universal. In recent
years, the focus has been on the use of aggregation or averaging methods. This is evident in one of the
core federated learning solutions. Federated learning is based on the aggregation of models in order to
obtain a single, common one. However, not all models can be used as there may be an attack on one of
them. Through the development of federated learning, more and more attention is being paid to MCDM
and bringing a consensus of different results. An example is a fuzzy controller that analyzes the results
from a large number of methods [
2023 Copyright for this paper by its authors. CEUR
ceur-ws.org
search for the most optimal solution in a finite time. Such solutions are used in the Internet of Things,
which allows you to create a network of connections between various objects that can download data,
process it and communicate. In [
This section describes the proposed method along with the description of methods and techniques used in the framework. 2.1.
In order to extend the accessible data and unify it into one, attribute-extensive database, artificial samples can be created by mixing samples from each dataset together, creating a new set of artificially created samples. In the case of class-distinguishable samples, objects of the given class are mixed only with the samples of the same class. In this paper, only two classes are considered: healthy and ill. 2.2.
Given a decision matrix , with criteria and alternatives (see Eq. 1), each criterion needs to be assigned weight and objective. The objective of the criterion is to maximize the values if it is beneficial or to minimize the values when it is not. For each criterion in , its objective needs to be chosen manually. For example, if one were to consider buying a car, he would aim to lower the cost, therefore the price criterion would be non-beneficial and its objective set to minimize the value. Determining weights, however, is usually a very subjective matter, requiring expertise. Nevertheless, there are methods of criteria weighting using sole information contained in the decision matrix.
The mean method is the most basic one. It assigns to all criteria the same value, equal to the fraction of the number of criteria. So in decision matrix with criteria , each weight is determined as:
In the standard deviation method, first, we need to normalize the values in the Decision matrix. In order to do so, for each criterion , each value is normalized using min-max scalar:
Next, for each criterion , the standard deviation of its corresponding set of normalized values is counted. By calculating the standard deviation for the criteria we consequently determine their weights.
In this method, again, the decision matrix first needs to be standardized. We can do this by calculating standardized values for each criterion set in the decision matrix by the formula:
With the normalized matrix formed, we can now calculate the entropy value for each criterion , which can be described as:
Given the decision matrix , normalization of its values must be performed. We can do so by scaling values of each criterion set accordingly with its objective using. For each element of the matrix , this process of linear normalization can be presented as follows:
Next, the overall performance of the alternatives is calculated. This is done by measuring the values of logarithmic function Eq. 9 for each sum of alternative values, with mean weights applied to them. The given formula is:
Proceeding further, the performance of the alternatives with each criterion separately removed is measured. Thus, for every alternative set ′ (see 10) of size is obtained. This set containers the overall performances of the alternative concerning each case of ignoring the -th criterion .
Lastly, for each criterion the sum of absolute deviations of the overall performances is calculated. This can be presented by the formula:
In the CRITIC method, first, we need to normalize the values in the Decision matrix using the minmax scaler, the formula of which is shown in Eq. 3. Next, weights for each criterion are calculated accordingly to the following equation:
For this research three different databases were used, each including covid positive and negative
cases:
1. Covid-19 Symptom patient dataset with information about 2575 patients suffering
symptoms with different intensities. The 5 criteria are:
a. Fever [94, 108] °
b. Body Pain {0, 1}
c. Age [
This data is accessible at Kaggle, under this link. 2. Coughing Mel-Spectrogram Image dataset, which is documented under this link. An example of a sample from this database is shown in fig. 2. 3. Lung X-ray dataset, along with image-matching masks. This data can be accessed on
Kaggle, under this link. An example of a sample from this database is shown in fig. 2. 3.2.
In order to obtain a usable piece of information for multiple-criteria decision analysis from imagetype data, a simple convolutional neural network was used. By processing the images throughout the model, a set of floating-point numbers in the range between 0 and 1 indicating the possibility of each patient being Covid-positive was gathered.
Input layer consisted of dual input of two 256x256 grayscale images, one being a lung image and the other being its matching mask, which is then combined in the minimum layer. Then a signal of the same size as the images (that is, a lung image with the mask applied) is passed onto the hidden layers.
In the model, the following layer configuration, was used: Convolutional 2D layer (filters = 64, kernel size = 3, activation function = ReLU)
Max Pooling 2D layer (pool size = 3x3) Convolutional 2D layer (filters = 64, kernel size = 3, activation function = ReLU) Max Pooling 2D layer (pool size = 3x3) Convolutional 2D layer (filters = 128, kernel size = 3, activation function = ReLU) Max Pooling 2D layer (pool size = 3x3) Convolutional 2D layer (filters = 256, kernel size = 3, activation function = ReLU) Max Pooling 2D layer (pool size = 3x3) Flatten Layer Dense layer (64 neurons, activation function = ReLU)
Dense layer (8 neurons, activation function = ReLU)
As an output layer a single neuron with a sigmoid activation function was used. So the output of the model for each case was a floating point number.
The training was performed on a dedicated dataset with the Adam optimization algorithm. After five epochs, the model reached satisfactory accuracy of 94% on a validation set.
Input layer consisted of a single input of shape 256x256x4 RGBA images. The signal is then passed onto the hidden layers.
In the model, the following layer configuration, was used: Convolutional 2D layer (filters = 64, kernel size = 3, activation function = ReLU) Max Pooling 2D layer (pool size = 3x3) Convolutional 2D layer (filters = 64, kernel size = 3, activation function = ReLU) Max Pooling 2D layer (pool size = 3x3) Convolutional 2D layer (filters = 128, kernel size = 3, activation function = ReLU) Max Pooling 2D layer (pool size = 3x3) Convolutional 2D layer (filters = 256, kernel size = 3, activation function = ReLU) Max Pooling 2D layer (pool size = 3x3) Flatten Layer Dense layer (64 neurons, activation function = ReLU) Dense layer (8 neurons, activation function = ReLU)
As an output layer a single neuron with a sigmoid activation function was used. So the output of the model for each case was a floating point number.
In order to study the capabilities of MCDM methods well, three additional databases were created. The symptoms dataset, as a core, was expanded by probability tables obtained in Sec. 3.2.1 and Sec. 3.2 in the following fashion: each positive sample from the core database was assigned a positive sample from one of the probability tables. That way, two new databases were created (with their size grown exponentially). The third database was obtained by simply assigning to each positive sample from one of the already mixed datasets, another positive probability table (so the mix of symptoms and x-ray datasets was again, an expanded probability table obtained from the cough database). It should be noted, that the same was done with negative cases.
Due to the large quantity of data in the X-ray and Symptoms set and their imbalance towards exceptionally sparser data in the Cough dataset, only 48 positive and 73 negative samples were randomly drafted from the former datasets, in order to match the exact numbers of the ladder one. Only those selected samples were then shuffled together, using the fore-mentioned fashion. Having explained the process of creating new datasets of artificial patients, the databases used in the experiments can be listed as follows: Symptoms only database (48 positive and 73 negative samples) Symptoms x Cough database (482 = 2, 304 positive and 732 = 5, 329 negative samples) Symptoms x X-ray database (482 = 2, 304 positive and 732 = 5, 329 negative samples) Symptoms x X-ray x Cough database (483 = 110, 592 positive and 733 = 389, 017 negative samples)
After assembling the databases, for each one of them (except for the Symptoms only database, where 48 samples of each class were chosen) only 1000 positive and 1000 negative samples have been randomly chosen for further assessment.
In each dataset, the objective of every single criterion was to maximize the value. That is because the goal was to detect Covid-positive patients and greater values of the criteria were beneficial for such purpose. 3.4.
As for the MCDM method, MultiMOORA was chosen for the experiments. The robustness and
objectiveness of the method makes it perfect for universal application [
In the experiments, I wanted to create a list of patients, ordered by the severity of Covid related symptoms. It is worth mentioning, that the main goal was to separate Covid-positive patients from the negative ones suffering similar symptoms. In other words, the desired ranking would have Covidpositive patients at the very top of it and the majority of negative cases at the bottom.
Figure 9: Weighting methods comparison for Symptoms x X-ray x Cough database. Figure 10: Weighting methods comparison for Symptoms x Cough database Figure 11: Weighting methods comparison for Symptoms x X-ray database Figure 12: Weighting methods comparison for Symptoms database
In order to visualize the effectiveness of ranking, TOP x accuracy graphs for ranking vector were created which can be presented as a function:
The TOP(x) function graph representing ideal ranking should then look like Fig. 3. This function, and metrics directly derived from it were used to assess the effectiveness of weighting methods. 4.2.
All of the obtained results can be listed as: TOP x% accuracy. Mean Square Error (MSE) (for the TOP(x) function). Mean Absolute Error (MAE) (for the TOP(x) function). Difference between the perfect and the obtained area under the first half of the TOP(x) function.
As can be observed in Fig. 4 - 8, using more sophisticated weighting techniques before the decisionmaking module was beneficial for the final ranking in most cases. Even though all of the methods did very poorly with the symptoms-only database, considerable improvement was made with the other three. In terms of Symptoms x Cough and Symptoms x X-ray databases, mean and standard-deviation methods scored seemingly and both MEREC and CRITIC methods did slightly better than the previous two. However, the entropy weighting method performed exceptionally better (despite the initial drop in the Symptoms x Cough database), than any other method. The same conclusion can be drawn from the ranking of Symptoms x Cough x X-ray database, where, while all the other methods performed equally or better than mean weighting, the entropy weighting method performed far better than any other technique, with the mean squared error (comparing to the ideal ranking in Fig. 3) equal to 7e-6, while mean method had the mean squared error of 5.52e-4, as can be read in Tab. 3 and 1.
The proposed solution can be used to solve real-life problems, such as the one considered in this paper. MCDMs can be used as accurate, unbiased ranking agents, which can be further enhanced by using adequate weighting methods prior. Based on the obtained results, the use of entropy weighting along with the MultiMOORA method is one of the best criteria methods. The proposal can be used in various types of systems where there is a lot of information on the basis of which a decision should be made, e.g. medical systems or federated learning. In future work, we plan to focus on adapting new multicriteria methods in the federated learning process.
This work is supported by the Rector’s mentoring project at the Silesian University of Technology.