This article presents and explains how we have used Naive Bayes Classifier and date base to predict the weather. We compare the dependence of accuracy on the probability of diferent distributions for various types of data. SYSYEM 2022: 8th Scholar's Yearly Symposium of Technology, Engi- It is one of the most important probability distributions, neering and Mathematics, Brunek, July 23, 2022 playing an important role in statistics. The formula for " agnilut814@polsl.pl (A. Lutecka); zuzarad785@polsl.pl the probability density is as follows: (Z. Radosz) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) (3)
Many IT systems use the widely understood artificial
intelligence [1, 2]. Artificial intelligence methods also
apply to the data processing [
The task of our program is to predict the weather using the naive Bayes classifier. It is especially suitable for problems with a lot of input data, so it is perfect for our project. It uses a conditional probability formula that looks like this: (|) = (|) * () () (1) • is our hypothesis • is the observed data (attribute values) • (|) is the probability that we would observe such data if the hypothesis were true • () is the a priori probability that the hypothesis is true • () is the probability of the observed data Importantly, the naive Bayes classifier assumes that the influence of the attributes is independent of each other, therefore (|) can be written as (1|) * (2|) * ... * (|). (2) The naivety of this classifier follows from the above assumption.
We started the project with an analysis of data from the database. Initially, we shufled and normalized the data to the 0 − 1 range to operate on smaller numbers, thus increasing the performance of our program. After dividing into validation and training sets, we move on to the main part of our program, i.e. the use of the naive Bayesian classifier. Its task is to return the name of the most probable weather for a given sample.
This algorithm uses a probability distribution defined by a density function that describes how the probability of a random variable (x) is distributed. We implemented 5 diferent probability distributions to compare the algorithm’s eficiency for diferent probability density formulas. We use the following distributions: Gauss, Laplace, log-normal, uniform, triangular.
exp( − (2− 2 )2 ) 1 √2 The probability function plot of this distribution is a bellshaped curve (the so-called bell curve).
Where is the mean value and is the standard deviation.
The density function graph is as follows:
Where is the expected or mean value and standard deviation. The red line corresponds to the standard normal distribution.
It’s a continuous probability distribution named after Pierre Laplace. The probability density is given by the formula: 1 exp( −| − | ) (4) 2 Where is the expected value, i.e. the mean, and b> 0 is the scale parameter. The function graph looks like this:
It is the continuous probability distribution of a positive random variable whose logarithm is normally distributed. Pattern:
1 √2 exp( − ( − )2 2 2 ) * 1(0,∞) (5)
It is a continuous probability distribution for which the probability density in the range from a to b is constant and not equal to zero, and otherwise equal to zero. We column j, sr [j] - the mean value of the column j and can see it in the formula below std [j] - standard deviation of the values from column j.
These methods process the input data through the formu () = ⎨⎪⎧02√d13la dl<a − − √√33 ≤ ≤ + √3 (7) loafstfhoer pthroebparboibliatybidlietynsdiitsytrfiubnuctitoionnaantdthreetupronintthesavmalpulee ⎪⎩0 dla > + √3 [j], that is, the probability of sample [j] occurring under the conditions sr [j] and st [j]. Finally, the algorithm Where is the mean value and is the standard devia- returns the name of the weather most likely to occur at tion. the sample input.
The function graph is as follows: Each probability distribution has a diferently defined density function. Therefore, the distributions may difer in the results. Below we present the pseudocode of NaiveClassifier class methods with an emphasis on processing the input data by probability distributions.
As we can see, the formulas difer significantly, which will definitely have a big impact on the efectiveness of the program. We tried to choose such formulas for the probability density so that the values were not very divergent, as we will present in the next paragraphs.
The naive Bayes classifier uses probability density functions to compute the probability of a given start condition. The NaiveClassifier class has 6 static methods: „laplace”, „logarytmiczny”, „jednastajny”, „trojkatny”, „gauss”, „bayes”, where the first 5 are diferent probability distributions . We use as many as 5 to compare the algorithm’s efectiveness for diferent formulas on the probability density. The "bayes" method accepts the following input data: data – training set, sample – a set of values in the range 0-1 that represent successive database columns, name – the name of the probability distribution (Gauss, Laplace, log-normal, uniform, triangular). At the beginning, the algorithm extracts the records with the given weather. Then, using the loops, the program goes through all the records of the training set, calculating the mean values and standard deviation of each column for each type of weather. Using the given "name", the algorithm calls the appropriate method, where the input data is: sample [j] - where j is the sample value for the
Data: Input , , Result: Weather Name Extract weather records ; Enter the weather record sets into the list ; := 0; for < () do = []; := 1; for < 7 do
Calculate the mean value of the column j ; Calculate the column standard deviation j ; if == ′ then tr.append(NaiveClassifier.laplace (sample[j],sr[j-1])); Data: Input , Result: The value of the density function at x := 2; return ((1/(2*b))*(math.exp(-(math.fabs(x-mean)/b))));
Algorithm 2: Laplace’s algorithm Data: Input , , Result: The value of the density function at x if > 0 then return (1/((math.pi*2)**(1/2)*std*x))*math.exp(((math.log(x)-sr)**2)/(2*std**2)); end else end return 0;
Data: Input , , Result: The value of the density function at x if < − 3 * * (1/2) * then
return 0; end if >= − 3 * * (1/2) * && <= + 3 * * (1/2) * then
return 1/(2*(3**(1/2))*std); end if > + 3 * * (1/2) * then return 0; return 0;
Data: Input , , Result: The value of the density function at x if < − 6 * * (1/2) * then
return 0; end if >= − 6 * * (1/2) && <= then return (x-sr)/(6*std**2)+1/(6**(1/2)*std); end if > <= + 6 * * (1/2) * then return -(x-sr)/(6*std**2) + 1/(6**(1/2)*std); end if > + 6 * * (1/2) * then
return 0; end else end end else end return 0;
Data: Input , , Result: The value of the density function at x return (1/(dev*np.sqrt(2*np.pi)))*np.exp(-((xmean)**2)/(2*dev**2));
For our project, we use the Istanbul Weather Data database, downloaded from the kaggle website. The database has 3896 records. It contains the following data columns: DateTime, Condition, Rain, MaxTemp, MinTemp, SunRise, SunSet, MoonRise, MoonSet, Avg
Wind, AvgHumidity, AvgPressure.
We analyzed the data using a matrix graph that shows the relationships between weather variables.
The chart is dominated by warm colors, which means that most of the records in our database are sunny and slightly cloudy.
It can be seen that the data is presented in compact groups. This means that the parameters for diferent weather conditions do not difer much from the others.
This can make our algorithm that determines the weather based on these parameters not very accurate. There may be situations where the algorithm will calculate the weather "Sunny" because it was the most probable, but the actual weather will be diferent. In the Experiments section, we will test and analyze the obtained results of the algorithm’s accuracy.
We also analyzed the data using a violin graph for all weather conditions and maximum temperature as we can see below:
In the attached photo we can see how the temperature To normalize the data to the range 0-1, we changed int64 value changes for a given weather, for example for "Mod- to float64. erate rain", i.e. moderate rain, the maximum temperature ranges from 5 to 20 degrees.
DateTime, SunRise, SunSet, MoonRise, MoonSet will not be used in our project, so we can get rid of them. d a t a . d r o p ( ’ DateTime ’ , a x i s = 1 ,
i n p l a c e = True ) d a t a . d r o p ( ’ S u n R i s e ’ , a x i s = 1 ,
i n p l a c e = True ) d a t a . d r o p ( ’ S u n S e t ’ , a x i s = 1 ,
i n p l a c e = True ) d a t a . d r o p ( ’ MoonRise ’ , a x i s = 1 ,
i n p l a c e = True ) d a t a . d r o p ( ’ MoonSet ’ , a x i s = 1 , i n p l a c e = True )
Our project consists of two files: a file containing the program code - "Pogoda.ipnb" and the database - "Istanbul Weather Data.csv". After analyzing the data from the database, we went to the "ProcessingData" class, in which we created 3 static methods: shufle, splitSet and normalize.
It takes base as input, i.e. our database. We use for loop to go through it selecting records and swapping them.
It takes as input x - database and k - division of the set. In the variable n we write the length of the set x multiplied by k to know how to divide this set. Then, to two xTrain variables, we write the data from the database to the value n, creating the training set, and to the variable xVal - all data following the value n, creating the validation set. Finally, we return both of these sets.
Program code
Data: Input , Result: Training and validation set = (() * ) = [: ] = [ :] return xTrain, xVal;
Algorithm 8: SplitSet algorithm d e f s p l i t S e t ( x , k ) : n= i n t ( l e n ( x ) ∗ k ) x T r a i n =x [ : n ] x V a l =x [ n : ] r e t u r n x T r a i n , x V a l
Takes x, which is a database that will have records scrambled using the shufle method. At the beginning, we enter all data from the database into the variable values, except for the string value, and the values of the column names into the variable columnNames. We loop through all the columns in the column, and then take all the rows in the column column and store them in the variable data. Variables max1 and min1 are assigned maximum and minimum values from date. Using the next loop, we go through all the rows and assign to the variable val the formula for normalization min-max, that is, we subtract the coordinate database record [row, column] from the value min1, and then divide this diference by the diference between max1 and min1. Finally, we write the value after normalization to the database. The method returns us a normalized database. Data: Input Result: Standardized database = .( =, , ′′) = ..() for ): do take all the rows from the column column 1 = () 1 = () for row in range(0,len(x) do we go through all the lines = (.[, ] − 1)/(1 − 1) .[, ] = end end return x;
Algorithm 9: The normalize algorithm
The NaiveClassifier class has 6 static methods: "laplace", "logarithmic", "uniform", "triangle", "gauss", "bayes". The ifrst 5 are methods describing the functions of diferent probability distributions. The "bayes" method has been described in general in the "Algorithm" section, so now we will look at the details. First, the method extracts the database records with the given weather name into separate lists. Then each of the lists created above is put into the "names" list. Additionally, we create a stringnames list with string weather names and a values list that will store the calculated probabilities for each weather. Data: Input , , Result: Weather name Extract weather records ; Enter weather record sets into ; Enter weather names in the list ; := [] := 0; for < () do = []; = []; = []; := 1; for < 7 do
Calculate the mean value of the column j ; Calculate the column standard deviation j ; if [ − 1] == 0 then
sr[j-1]=0.0000001; end if [ − 1] == 0 then
std[j-1]=0.0000001; end if == ′ then tr.append(NaiveClassifier.laplace (sample[j],sr[j-1])); end if == − then tr.append(NaiveClassifier.logarytmiczny (sample[j],std[j-1],sr[j-1])); end if == then tr.append(NaiveClassifier.jednostajny (sample[j],std[j-1],sr[j-1])); end if == then tr.append(NaiveClassifier.trojkatny (sample[j],std[j-1],sr[j-1])); end if == then tr.append(NaiveClassifier.gauss (sample[j],std[j-1],sr[j-1])); end end values.append (np.prod(tr)*len(names[i])/len(names)); end = .(()); return value from at index ;
Algorithm 10: Bayes algorithm
The next step is to loop through all the values of the @ s t a t i c m e t h o d names list in sequence. Then we create auxiliary lists: tr d e f a n a l i z e ( T r a i n , V a l , name ) : [] - to store 6 probability values that correspond to the c o r r e c t =0 next database column, sr [] - to store the mean values f o r i i n r a n g e ( l e n ( V a l ) ) : from each column, std - to store the standard deviations i f N a i v e C l a s s i f i e r . for each column. We pass the next loop through all the c l a s s i f y ( T r a i n , V a l . columns one by one. Then we calculate the mean and i l o c [ i ] , name ) == V a l . standard deviation for a given column. The next step is i l o c [ i ] . C o n d i t i o n : the conditions that prevent sr [] and std [] from occurring. c o r r e c t +=1 The time has come for timetables. Depending on the in- a c c u r a c y = c o r r e c t / l e n ( V a l ) ∗ 1 0 0 put data "name" to the list tr [] we add the result of the r e t u r n a c c u r a c y function of the given distribution. After going through the inner loop, we compute the value from Bayes’ theorem. Based on the formula for conditional probability, we 7. Tests multiply the values in the tr list, then multiply that product by the list of names [i]. We divide the whole thing by We started our tests by checking the algorithm’s operathe length of the "names" list. We add the obtained result tion using various samples. to the "values" list. After going through both loops, we determine the index from the values list with the highest value. Finally, we return the name of the weather with the given index from the stringnames list. The above code shows us that depending on the value in the sample, the algorithm returns diferent values, which proves the correct operation of the algorithm.
The next step was to determine the accuracy of our algorithm for various probability distributions. For this we used the AnalizingData class with the analysis method.
We called the method for each of the 5 types of probability distributions for the training and validation division in the ratio of 7: 3, and then, using the plt package, we displayed the graph.
Another class in our project is AnalizingData with the Analyze method. This method measures the accuracy as a percentage of the Bayes classifier. The input data is: Train - training set, Val - validation set and name - name of the probability distribution. The algorithm ifrst sets the value of the corrtect variable to 0. Then it goes through the iterator loop and goes through all the records of the Val set. If the value returned by the bayes classifier at the input: Train, Val [i], name is the same as the weather name for the Val [i] record, increase the variable correct by 1. Finally, the algorithm returns the accuracy, which we calculate by dividing correct by the product the length of the validation set and 100.
Data: Input , , Result: Accuracy of the bayes algorithm := 0 := 0 for < ( )): do if . ( , .[], ) == .[]. then
correct+=1; end end accuracy=correct/len(Val)*100; return x;
Algorithm 11: Analyze algorithm Program code c l a s s A n a l i z i n g D a t a : As you can see in the attached picture, the algorithm has diferent accuracies depending on the probability distribution. The Gaussian distribution definitely exceeds other distributions with its accuracy, which is over 60%. This means that more than 60% of the weather names returned were valid. However, the Laplace and uniform distributions do not lag far behind. Their values are in the range 50-60%. Log normal and triangular distributions are the least eficient because their accuracy is less than 20%. Additionally, we measured the execution time of the algorithm. It was almost 4.512 minutes.
We tested the operation of our program for both normalized and non-normalized values and we determined the algorithm execution time for both data sets. The graphs below show the dependence of the accuracy on the probability of individual distributions.
The first plot shows the Bayesian operation for unnormalized data in a ratio of 7: 3, while the second plot shows the operation for unnormalized data with the same partition. The program launch time for the first graph was 4.512 minutes, while for the second graph it was 4.433 minutes. As we can see, the only significant diference was when using the Laplace distribution, the accuracy of which decreased by almost 10%. The remaining results are similar for both types of data. The time diference is insignificant as it is only 5.023s.
The following charts show the eficiency of the algorithm for normalized data for various divisions into training and validation sets:
The algorithm execution time decreases with the reduction of the training set. For the last execution of the algorithm, where the division was in the ratio of 9: 1, the
We can conclude from this that the Gauss distribution is the best probability distribution for our database. The algorithm with this distribution, with each modification, correctly determines about 60% of weather names, which is a good but unsatisfactory value. This is due to the way the data is distributed in the database. In the case of more diferent values for diferent weather conditions, this algorithm could become much more eficient. time was only 1.467 minutes. However, the first calculations, where the division was in the ratio of 1: 9, took as long as 9,517 minutes. This is because by reducing the training set, we increase the number of records in the validation set. As a result, the algorithm will be called more times and the most time-consuming elements such as extracting records with a given weather or loops will be performed many times.
Analyzing the above, we can see that the accuracy of the Gaussian distribution is superior to all of them, its value is practically unchanged. The Laplace distribution is in second place, almost tapping 60%. The value of the uniform distribution ranges from 50-55 %. It achieves the most on the last chart where the training set is 0.9. Triangular and log normal distributions reach much lower values than the previously mentioned distributions. The jump is quite big, around 30%. However, the log-normalized distribution only slightly exceeds the triangular distribution once, and it is in the third graph. Nevertheless, the accuracy values of both distributions never exceed 20%.