classes. This article is a summary of our work, which goal was to predict diseases based on given symptoms using diferent machine learning classifiers from sklearn library as well as to compare their accuracy to algorithms developed by us from scratch. Based on the characteristics of the learning set, the tree model uses a set of predefined questions to identify the sample machine learning, naive Bayes classifier, decision tree, random forest, disease prediction ICYRIME 2022: International Conference of Yearly Reports on Inforthat can be classified based on that data. It is possible to
In recent years, a wide range of applications of artificial
intelligence methods can be observed in various
information systems [
Our program aims to predict diseases on the basis of symptoms. Three diferent classifiers were used for this manner: naive Bayes classifier, decision trees and random forest. We have used both „sklearn” algorithms as well as classifiers written from scratch by us. An additional premise is the verification of accuracy of all of those algorithms. Often when the data are noisy they are filtered in order to reduce the noise [27, 28]. The program takes into account 132 symptoms marked with either a „0” or a „1”. There are 42 diferent diseases
CEUR htp:/ceur-ws.org ISN1613-073
CEUR
Workshop Proceedings (CEUR-WS.org) enter the symptoms manually (for the presentation we assumed that the numbers will be randomly selected by a computer). The output will be the name of the disease to which these symptoms are related.
The Naive Bayes classifier is a simple probabilistic classifier. Its activity is based on the assumption of the mutual independence of the predictors. They often have no relation to reality and are therefore called naive. The Bayessian analysis uses a priori probabilities derived from previous observations. A priori probabilities allow you to classify a new object based on those probabilities. The formula for the Bayes probability: (A|B) = (A) (B|A) (B) The naive Bayes method provides the user with several modeling approaches for a given theme. The probability distribution can be Gaussian, lognormal, gamma or Poisson.
For our project we used the Gaussian distribution, which formula looks as follows, where is a standard deviation and is the expected value: ( | ) = √︀2 2 1
︂( exp − ( − )2 )︂ 2 2 The naive Bayes classifier is very often used in spam ifltering. These classifiers are relatively easy to implement, computationally efective and are ideal for relatively small amounts of data compared to other algorithms. (1) (2) index (), entropy ( ) and classification error ( ). Definition of entropy for non-empty classes (|) ̸= 0: () = − ∑︁ (|) log2 (|), =1 The decision tree model is attractive if we interpret the data correctly. As the name suggests, this model can be viewed as a classification of data by Decision-making based on a set of responses Based on the characteristics of the learning set, the tree model uses a set of predefined questions to identify the sample classes. The drawing (fig. 1) represents an intuitively understandable case, but one can scale such a model to larger problems and also take numerical data into account. The algorithm generates a tree root and separates the data based on the information growth. Through repeated iteration, we can repeat this step in each child node until we get the leaves themselves, which means that all the leaves of a node belong to a specific class. This approach leads to large multi-node trees, which can lead to an overtraining of the model. To avoid this, you should cut the trees by setting their maximum height.
If you want to separate nodes with the most informative features, you need to define a target function that is optimized with the tree learning algorithm. In our case, the function of the target is to maximize the information gain in each branch, which can be written with an equation: (, ) = () − ∑︁ ( ), =1 (3) The parameter is the property on which branching is performed, and are records of the parent node and the j-th child node, I is the degree of contamination, defines the total number of samples in the parent node and in the j-th child node. In binary trees, three measurements of impurity are most commonly used: the Gini
As with entropy, the highest value is used for perfectly mixed classes, e. g. for a binary configuration of ( = 2): () = 1 − ∑︁ 0.52 = 0.5, =1 Often, the Gini index and entropy produce similar results, and it is not worth evaluating the tree according to diferent criteria, but to experiment with the cut-of. The third measure of contamination is the classification error: () = 1 − {(|)},
The random forest method is characterized by good classification performance, scalability and user-friendliness.
Random Forest can be intuitively interpreted as an ensemble of decision trees. The concept is to combine weak learners to build a more robust model, a strong learner, that has a better generalization error and is less susceptible to overfitting. The random forest algorithm can be summarized in four simple steps: 1. Draw a random bootstrap sample of size (randomly choose n samples from the training set with replacement). 2. Grow a decision tree from the bootstrap sample. entries, which are examples of the occurrence of sympAt each node: toms of a specific disease. There are 42 unique diseases. a) Randomly select features without re- The database can be visualized as a matrix: b) Spplalcitemtheennt.ode using the feature that pro- ⎡ 1,1 1,2 . . . 1, 1 ⎤ vides the best split according to the objec- ⎢ 2,1 2,2 . . . 2, 2 ⎥ ttihveeifnufnocrmtioanti,ofnorgianisnt.ance, by maximizing = ⎢⎢⎢⎣ .. .. .. .. .. .. .. .. .. .. .. .. .. .. ⎥⎥⎥⎦ , 3. Repeat the steps 1 to 2 times. ,1 ,2 . . . , 4. Aggregate the prediction by each tree to assign the class label by majority vote.
where M is the number of symptoms and N is the number of samples. Based on the symptom x, the program predicts the disease y.
There is a slight modification in step 2 when we are training the individual decision trees: instead of evaluating all features to determine the best split at each node, we only consider a random subset of those. Although Random Forest can’t interpret the results as strongly as individual decision trees, the big advantage is that it’s less important to choose the right hyperparameters. Normally, it is not necessary to prune a random forest, as the model is quite insensitive to tree sounds. The only parameter that interests us is – number of trees. In most cases, the accuracy of the classifier increases as the number of trees increases, but this also increases the computing power required for the classification.
The database is called “Disease Prediction Using Machine Learning” and was downloaded from „kaggle.com”. The base consists of 133 columns, each of which is responsible for a symptom, and the last is the predicted disease resulting from those symptoms. The database has 4692
In order to visualize the data, we have created a diagram showing the frequency of appearance of each symptom (fig. 2). As predicted, fatigue was the most common symptom that occurred in almost half of the diseases.
The second most common symptom was vomiting, and the third most common symptom was high fever. The occurrence of these symptoms may indicate that they do not have a significant impact on the classification of the disease. However, most symptoms did not occur in up to 10% of cases. Symptoms such as weight gain or pus filled pimples, which are the least common, are almost immediately suggestive of a certain disease, or of a narrow spectrum of diseases. To check if there is a correlation between the symptoms, we have created a heatmap of the symptom correlations.
The heatmap (fig. 3) is in dark tones, indicating a weak and low correlation, but between some symptoms there is a very high and in some cases almost complete corre- The naive Bayes classifier had an efectiveness of 98,37% lation, e. g. between cold and sinus pressure symptoms. which was a satisfactory result (fig. 4).
All the classifiers described above worked with the same data, so comparing the results gives a good overview of their efectiveness for our database. Both handwritten classifiers and classifiers from the library are analysed. Their accuracy is compared to see how our classifiers perform compared to „Sklearn” algorithms.
The worst result was achieved by the decision tree with 97,4%. The Random Forest successfully predicted the disease in 99,9% of cases, and the naive Bayes classifier with 100% was error-free (fig. 5).
The worst of the three disease prediction algorithms was When we started this project, we aimed to get our acthe Decision Tree classifier with 62,3%. The improved curacy close to Sklearn classifiers. However, the results version – the Random Forest was the best and predicted show that we have exceeded initial expectations. The the disease in every case, resulting in accuracy of 100%. biggest diference is in the classifier decision tree, where our algorithm was 35,1% more efective. In the random forest method, the diference was less than 0,0001% in favour of Sklearn, with both classifiers having very good accuracy of disease prediction and were almost error-free.
In the naive Bayes classifier, the accuracy was again better in our implementation, where the algorithm proved to be error-free, resulting in a score 1,63% higher than Sklearn (fig. 6).
The occurrence of symptoms was entered randomly by a computer as either a “0” or a “1”. The new samples were then classified using the same three classifiers from the Sklearn library. The first two diseases were classified in the same way by all classifiers, but for the subsequent ones the algorithms are no longer compatible. Such results may be may be due to the fact that the occurrence of symptoms was entered randomly by the computer, so they may not have made sense from a medical point of view, because they have created combinations of symptoms that never occur together in real life, resulting in samples with strange data, not similar to those given in the training set.
The accuracy of classifiers at the level of 100%, at first glance pleasing, makes us wonder whether the algorithm really did so well, or maybe it is caused by the data on which it worked. In our case it seems to be the second thing. Two conclusions come to mind when thinking about this: there were too few samples (4692); there is a pattern and not enough combinations between the diferent symptoms. Having 132 symptoms gives us 2132 = 5, 4 * 1039 possible combinations. With such a limited number we can be almost sure that there are some patterns and relations in the base which cause such a high eficiency.