The problem of predicting the success of organ transplantation is critical in the field of medicine. The use of a probabilistic neural network is of considerable interest in this context. In this study, the authors compared the speed of work of four popular methods for optimizing the parameter of a probabilistic neural network in the case of analyzing a short medical dataset collected by Lviv Regional Clinical Hospital. All three algorithms have demonstrated efficiency, reaching the optimum performance point. The use of optimizers provided a significant saving of time and computing resources compared to grid search.
Classical models of artificial intelligence do not demonstrate sufficient efficiency in the case of processing small medical datasets. This is due to a number of reasons, the main of which is the problem of overfitting.
The use of a probabilistic neural network in such cases can improve performance compared to traditional models. However, the selection of the optimal network parameter by brute force method requires a lot of time and computing resources. That is why the use of optimization methods is appropriate for this task.
still needs development.
New approaches to working with small datasets appear every year. However, this area of research
The issue of using a probabilistic neural network for classification problems was analyzed in [
2023 Copyright for this paper by its authors. CEUR
ceur-ws.org
In the above-mentioned works, the selection of the optimal parameters of the neural network was carried out using a grid search. Thus, optimizing the parameters of a probabilistic neural network is relevant.
The purpose of this study is to compare the performance of three popular methods for optimizing the parameter of a probabilistic neural network in the case of analyzing a short medical dataset. 2.1.
A probabilistic neural network is often used to solve a wide range of tasks, including classification
[
The work of this neural network in the case of binary classification can be described as follows: 1. Let there be k vectors of class 1 and m vectors of class 2 in the sample. We denote the j-th component of the i-th vector as X i1, j for class 1 and as X i2, j for class 2. The task of the model is to classify the input vector X . Therefore, it is necessary to determine the probability that the vector X belongs to class 1. 2. Canberra distances between the input vector and all sample vectors are calculated:
X i1, j − X j n n Ri1 = , Ri2 =
j=1 X i1, j + X j j=1 X i2, j + X j Gaussian distances are calculated based on the obtained values:
X i2, j − X j The probability that the input vector belongs to class 1 is calculated by the formula: k Di1
Similarly for class 2: y pred = 0, if max Pc = P1 , c = 1, 2 1, if max Pc = P2
k
So probabilistic neural network predicts a class of the input vector using the following rule: 2.2.
For models that are built using unbalanced datasets, the F1 score is an appropriate measure of performance [13].
Let yi, i ∈ 1..N denote belonging to a certain class in the test sample of size N, then yipred, i ∈ 1..N is the value predicted by the model.
Precision of the model will be equal to:
Precision = i=1
Recall = i=1
N ( y pred * y )
i i N y pred
i i=1 N ( y pred * y ) i i N yi i=1
N N ( yipred * yi ) ( yipred * yi ) Thus, the problem of maximizing F1-score can be presented in the following form: N N ( yipred * yi ) ( yipred * yi ) (6) (7) (8) (9) f 1_ score = 2 * i=1
N y pred
i i=1 i=1 N ( y pred * y )
i i N y pred
i i=1 * i=1 + i=1
N ( y pred * y ) i i N yi i=1 N yi i=1 2 * i=1
N y pred
i i=1 i=1 N ( y pred * y )
i i N y pred i * i=1 + i=1
N yi i=1 N yi i=1 N ( y pred * y ) i i
→ max i=1 with the restrictions 0.001>sigma>10. 2.3.
We applied three popular optimization algorithms: • Bayesian optimization • Differential evolution • Dual annealing
These methods do not require the calculation of derivatives and can perform optimization in case the objective function is a “black box” [14].
Bayesian optimization uses Gaussian process to model the black-box objective function [15].
We defined the upper confidence bounds function as acquisition function to balance exploitation and exploration. Also we used the following optimization parameters: • number of initial points – 5; • number of iterations – 10.
Differential evolution is a stochastic method. It applies the key concepts of genetic algorithms [16]. The first step of the algorithm is to create a generation of candidates that are the objective function arguments. At each iteration, a new generation is created by mixing with other candidates.
We applied a “best1bin” strategy for creating trial candidates. According to it: • the difference between two randomly chosen candidates is used to provide a mutation of the best member of the population; • a binomial distribution is applied for recombination.
We defined following key algorithm parameters: • population size – 10; • mutation – [0.5;1); • recombination – 0.7; • maximum number of generations – 10.
Dual annealing is also a stochastic approach. It combines the generalization of Fast Simulated Annealing and Classical Simulated Annealing coupled to a strategy for carrying out a local search on accepted locations [17]. This approach describes an advanced method to improve the solution that was found by the generalized annealing process. A distorted Cauchy-Lorentz visiting distribution is used in this optimization algorithm.
We used the following optimization parameters: • parameter for visiting distribution – 2.62; • parameter for acceptance distribution – -5.0; • maximum number of global search iterations – 10.
For all algorithms, the optimization was performed on the interval σ ∈ [0.00001;10].
The imbalanced dataset collected by Lviv Regional Clinical Hospital (Department Hospital Nephrology and Dialysis) was used in this study. It contains data on 164 patients who received HLAmatched renal allografts between 1992 and 2020 by 42 attributes (such as age, sex, glucose level, etc.). Among them, 64 (42.1%) were women and 88 (57.9%) were men. The age of the patients at the time of transplantation was 32.6 ± 8.7 (in the range of 18–60) years. 152 patients were transplanted for the first time, 12 (5 women and 7 men) were transplanted again. 3.2.
Three optimization algorithms described above were used to optimize the parameter. The implementation of optimizers from the scipy.optimize and bayesian optimization libraries of the Python programming language was used. The optimization results are shown in Table 1.
F1 score
As can be seen, all three optimizers have reached the point from the intervals where the value of F1score is maximal. The precision value indicates a quite large proportion of false-positive results, while the recall is 100%.
All algorithms showed quite good optimization speed. The shortest execution time was demonstrated by dual annealing. A visualization of the optimization duration can be seen at Fig. 1.
In terms of the number of evaluations of the objective function, Bayesian optimization is the most effective (a visualization can be seen at Fig. 2). However, other steps of this algorithm also cause computational costs, which is reflected in the duration of execution.
The selection of the optimal value of the parameter was also carried out using a grid search on the interval σ ∈ [0.00001;10] with a step Δ=0.001. The execution time was 41.852 seconds. The number of objective function calculations was 10000. As a result of the experiment, it was found that there are two intervals on which F1 score reaches a maximum of 0.842 (Fig. 3).
So the use of each of the optimizers provides a significant reduction in execution time and computational costs, compared to the grid search.
The problem of predicting the success of organ transplantation is critical in the field of medicine. The use of a probabilistic neural network is of considerable interest in this context. In this study, the authors compared the performance of three popular methods for optimizing the parameters of a probabilistic neural network in the case of analyzing a short set of medical data collected by Lviv Regional Clinical Hospital. All three algorithms have demonstrated efficiency, reaching the optimum performance point. The use of optimizers provided a significant saving of time and computing resources compared to a grid search.
Further research may concern the optimization of model parameters, where the probabilistic neural network is used in combination with other machine learning methods.
This research is supported by the EURIZON Fellowship Program: “Remote Research Grants for Ukrainian Researchers”, grand № 138.
[12] Oleksiv I, et al. Quality of Student Support at IT Educational Programmes: Case of Lviv Polytechnic National University. In: 2021 11th International Conference on Advanced Computer Information Technologies (ACIT). 2021. p. 270–5. [13] Martyn Y, et al. Optimization of Technological’s Processes Industry 4.0 Parameters for Details Manufacturing via Stamping: Rules of Queuing Systems. Procedia Comput Sci. 2021 Jan 1;191:290–5. [14] Ganguli C, et al. Adaptive Artificial Bee Colony Algorithm for Nature-Inspired Cyber Defense.
Systems. 2023 Jan;11(1):27. [15] Ljaskovska S, et al. Optimization of Parameters of Technological Processes Means of the FlexSim Simulation Simulation Program. In: 2020 IEEE Third International Conference on Data Stream Mining & Processing (DSMP). 2020. p. 391–7. [16] Mochurad L. Optimization of Regression Analysis by Conducting Parallel Calculations. In: COLINS-2021: 5th International Conference on Computational Linguistics and Intelligent Systems, April 22–23, 2021, Kharkiv, Ukraine, 982-996 p. [17] Basystiuk O, et al. Machine Learning Methods and Tools for Facial Recognition Based on Multimodal Approach. In: Proceedings of the Modern Machine Learning Technologies and Data Science Workshop (MoMLeT&DS 2023). Lviv, Ukraine, June 3, 2023. CEUR-WS.org, pp. 161170 (2023).