There are lots of diferent machine learning algorithms, and every one of them has its use. However, not every one of those algorithms will be a good choice for every scenario that we want to test - some are simply superior for one case, while others are more suitable for another. In this short paper, we compare the accuracy for predicting diabetes in Pima Indians patients of two soft set classifiers and one fuzzy classifier. SYSYEM 2022: 8th Scholar's Yearly Symposium of Technology, Engin"eekrianmgiagnizd35M9aptohlesml.palti(cKs., BGriuzinńeskk,iJ);ublyar2t3r,o2l005202polsl.pl (B. Rolnik); 2.1. Data normalization domisig095polsl.pl (D. Sigulski) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License In select cases, data has been normalized using one of CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) two types of normalization:
between algorithms, two soft set classifiers and a fuzzy
classifier to be exact, to see how they perform - both in
One of the most important tools of artificial intelligence terms of their accuracy and the time required to produce
are neural networks [
Disease detection is becoming a very important field know whether our choice of the membership function is
in machine learning [
Since not every machine learning algorithm is a good may be harder for a human to make a decision based on
ift for every problem, it is important to test diferent many parameters, while it is certainly much easier to
algorithms to see how they fit in our scenario. David make a choice based on a single, crisp value produced
H. Wolpert and William G. Macready have themselves by a classifier using a fuzzy set. However, it also means
indicated that the first theorem in their paper "state[s] that a decision made based on a soft set reasoning can be
that any two algorithms are equivalent when their per- much more complete, with more parameters available.
formance is averaged across all possible problems"[
That is why we have decided to focus on the comparison
Standard deviation normalization − ¯
− − • - sample value • - column standard deviation • ¯ - mean column value Min-max normalization • - sample value • - minimum column value • - maximum column value where: where: where: where: • - the beginning of the lower base of the trapezoid, membership takes the value 0 Triangular membership function (; , , ) = ⎧ 0, ⎪⎪⎨ − , − −
, − ⎪ ⎪⎩ 0, if ≤ . if < ≤ . if < ≤ . if < . • - the beginning of the base of the triangle, membership takes the value 0 the value 1 takes the value 0 • - the center of the triangle, membership takes • - the end of the base of the triangle, membership
We have also used a handful of equations in our fuzzy where: classifier. (3) where: (; , , , ) = ⎧ 0, ⎪ ⎪ ⎨ ⎪⎪ − , − 1, ⎪ − , ⎪⎪ ⎪⎩ 0, − if ≤ . if < ≤ . if < ≤ . if < ≤ . if < .
• - the end of the lower base of the trapezoid, membership takes the value 0
= { }
() = { 1, 2, ..., } • - antecedent fulfillment degree Center of gravity of a triangle • afd - antecedent fulfillment degrees (i.e.: afd=(0.6,
The dataset used for this project was the Pima Indians Diabetes Database [24]. According to the source, "This dataset is originally from the National Institute of Diabetes and Digestive and Kidney Diseases. [. . . ] In particular, all patients here are females at least 21 years old of Pima Indian heritage." The dataset consists of several medical predictor variables and one target variable. (4)
Here is a short description of each column.
1. Pregnancies — number of times pregnant; 2. Glucose — plasma glucose concentration after 2 hours in an oral glucose tolerance test; zoid, membership takes the value 1 membership takes the value 1 • - the beginning of the upper base of the trape3. BloodPressure — diastolic blood pressure (mm • - the end of the upper base of the trapezoid, 4. SkinThickness — triceps skin fold thickness
Hg); (mm); 5. Insulin — 2-Hour serum insulin ( IU/ml); 6. BMI — Body mass index (weight in kg/(height in m)ˆ2); 7. DiabetesPedigreeFunction — a function that describes the likelihood of diabetes based on family history; 8. Age — age in years; 9. Outcome — class variable (0 or 1). 268 of 768 are 1 and 500 are 0;
However, we had to clean the data first - as seen in Tables 2, 3 and 4, there were many incomplete records which in turn would result in the classifiers producing incorrect predictions. Which is why we removed all zero values outside the Pregnancies and Outcome columns. The column statistics after removing the incomplete records can be seen in Tables 5, 6 and 7.
We have also shufled the dataset, normalized it in select cases (either min-max or standard deviation normalization) and divided it into a training set and a validation set (70:30 split).
The values for both sick (marked with yellow) and healthy patients (marked with red) are not too distinct, apart from the glucose column - as seen in Figure 1. As shown in Figure 2, the correlation between the outcome values and other columns is not particularly strong either - only the glucose column shows a correlation above 0.5, with almost all other correlations not even exceeding 0.3.
In this subsection, we are going to describe the columns in the database from the medical point of view.
Glucose is a simple sugar, which is the basic source of energy in the human body. Measuring blood glucose levels on an empty stomach can help determine if a person has diabetes.
The approximate norm of blood glucose concentration is: which suggests that there was an error in the dataset - or perhaps a diferent testing method was used.
Diabetes pedigree function Family history of diabetes was shown to be a significant predictor of diabetes prevalence.There is no additional information about this function in the dataset, apart from the short explanation that it is "a function that describes the likelihood of diabetes based on family history", so we can only assume that the higher the value, the higher the risk is for a patient.
BMI is the body mass index. It is calculated by comparing height with weight. Its value is helpful in assessing the risk of overweight-related diseases such as atherosclerosis, diabetes or ischemic heart disease. The lower the BMI value, the lower the risk of disease development.
Norms for BMI:
Age According to the American CDC agency, patients are at risk if they are 45 years or older.
The next column is blood pressure. Blood pressure is the amount of pressure that flowing blood exerts against the walls of your arteries. The measurement is made with a pressure gauge and the obtained values are given in mm Hg, i.e., millimeters of mercury.
Blood pressure in terms of the norms is divided into: • 70-99 mg/dL — correct values; • 100-125 mg/dL — abnormal values, oral glucose tolerance test is required;
• above 126 mg/dL — abnormal values, repeating the test is required, patient is diagnosed with dia- The first soft set classifier uses a mean column value to betes after getting such a result twice. create the weights table.
Firstly it gets the mean value, then it counts how many values in the column were below and above the mean.
If there were more values above the mean, the program
adds a pair [
All the algorithms were written and executed in Python 3.9.5 using Jupyter Notebooks. • normal: <80 mm Hg; • elevated phase 1: 80–89 mm Hg; • elevated phase 2: >90 mm Hg.
The thickness of the skin fold on the triceps is one of the determinants of body fat level. We have not managed to nfid any information regarding the accepted norms.
Insulin is a hormone responsible for regulating blood sugar (glucose) in the body. The norm 2 hours after a meal is up to 30 mIU/ml.However, in our dataset the median for insulin was 125.5 IU/ml - 4 times over the norm, if we assume the units are the same exact ones
The second soft set classifier uses minimum and maximum column values to calculate how a sample value compares to the minimum and maximum values, to then create the weights table.
Firstly it gets the minimum and maximum values, then
it calculates how far they are from minimum to maximum
in percentage - i.e. if a sample value is equal to minimum,
the algorithm appends [
Firstly, we need to explain how we chose the membership functions for the antecedents and the consequent.
There is no clear-cut threshold for the number of pregnancies, so we have decided to pick 2 as our middle threshold and with the rest of the domain divided based on the middle triangle, as seen in Figure 3.
We have decided to divide the glucose column domain according to the glucose norms. with the middle being at 112, since this is the middle of the abnormal range of values that do not yet warrant a diabetes of diagnosis. The chart of its membership function can be seen in Figure 4. BMI has quite clear norms, so we used them to divide the values in the BMI column - as seen in Figure 8.
As seen in Figure 9, the diabetes pedigree function values were divided using its median, since no medical norms were available.
As seen in Figure 10, age values were divided according to CDC’s norm that people who are 45 years or older are at risk of developing type 2 of diabetes.
Finally, the outcome consequent was divided in half, with values that fall in the middle being marked as ’sick’, which can be seen in Figure 11.
After creating the membership functions, we wrote the rules connecting all the antecedents to the consequent. An example of such a rule can be seen in Table 8
The algorithm starts by getting the membership degrees for each column and then fuzzifying the sample values based on these degrees. Once we have all the fuzzy values, we can calculate the rule fulfillment and then pick the highest fulfillment degrees for each consequent label. Then out of these labels we pick the one with the highest degree - if there is a "draw", i.e., the degrees are equal, we pick the worst case - so in our dataset we mark the patient as sick, since it is obviously better to mark one too many, than one too few.
As seen in Figure 12, the time performance of the classiifers varied quite a lot, with the soft set classifier using percentages being more than three times slower than the soft set classifier using mean values and almost twice as slow as the fuzzy classifier. The time performance is an average of 50 timed runs to average out any abnormal situations.
The soft set classifier using mean values was the worst performing one, with less than 50% accuracy for data normalized with standard deviation and around 50% accuracy for data normalized with percentages and unnormalized data. Table 11
The soft set classifier using percentages did not per- Fuzzy Classifier accuracy summary (20 samples) - mean and form much better, either, with its accuracy slightly ex- standard devitation ceeding 50%. As seen in Table 9, the results were quite in all cases. On the contrary to the mean soft set classifier, Value it performed best with data normalized using standard mean 63.43% deviation - but the diference was not stark at all. std dev 4.19
The fuzzy classifier came out on top, with its accuracy averaging out at more than 60%. As seen in Table 11 and Figure 13, its accuracy did not drop below 55% - so in almost all cases it performed better than other classifiers did on average. Even taking the standard deviation into account, ass seen in Table 11, the results were significantly better.
In conclusion, the soft set classifiers were not a good choice for this problem and dataset. They probably would have worked better with a more uniform outcome distribution - but with more samples being marked as ’sick’, the results were skewed.
The fuzzy classifier produced superior results, and with improved, more fine-tuned rules it could achieve an even better accuracy. Similarly to the other soft set classifier, it would have benefitted from a more uniform data distribution.
As seen in Figure 14, the fuzzy classifier did evidently better, but the results can definitely be improved further.