-Data classification is one of important topics in information technology. There are many methods used in this process. This article presents classification of iris flowers by the use of decision tree. In the system was implemented a procedure to used an open data set for classification of applied types of flowers by numerical features describing them. Results show that proposed model is very simple but also efficient in numerical classification.
In information processing very important role is for data analysis and decision processes. From information hidden in data we can get knowledge about various things. In general we can use various methods to process the data. Artificial intelligence gives us many interesting approaches to data science.
In general input data is organized in smaller groups called
classes in which final classification is done. By the use of
this kind of decision making processes we can estimate many
things. In [
In [
In this article i show how to implement a simple python procedure based on decision tree classifier.
Implemented idea was used to recognize iris flowers from open data set. Results show that proposed implementation works well returning very good results.
©2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Ronald Aylmer Fisher was born on February 17, 1890 in London, and died on July 29, 1962 in Adelaide. He was a British geneticist and statistician. The Briton graduated from Gonville and Caius College at the University of Cambridge and worked as a professor of eugenics at the London School of Economics in 1933-1943 and professor of genetics at the University of Cambridge (1943-1957). Anders Hald described him as ”a genius who almost created the foundations of contemporary statistics”, and Richard Dawkins as ”the greatest heir of Darwin” and a member of the Royal Society in London (Royal Society). The geneticist created, among others, maximum likelihood, analysis of variance (ANOVA) and linear discriminant analysis. He also dealt with methods of hypothesis verification using statistical methods (in anthropology, genetics, ecology) and was one of the creators of modern mathematical statistics. He is also known for the development of experimental results at the Rothamsted Institute of Agricultural Research (1919 - 1933) and as the author of the Statistical Methods for Research Workers (1925), Statistical Methods and Scientific Inference (1956).
The principle of proposed decision model is based on decision tree.
1) Decision Tree: A decision tree is a (graphical) method of supporting the decision-making process. It is used in decision theory and in machine learning. In the second application, it helps in acquiring knowledge based on examples. Algorithm - it works recursively for each node. We have to decide whether the node will be:
2) a branch node according to the values that the given attribute takes and for each child node we create a recursive call to the algorithm with the list of attributes reduced by the attribute just selected.
Building a tree - The tree consists of nodes: decisions and states of nature and branches. Rectangles are decisions, and states of nature are circles. We start with the root. At the very beginning we have the first given sepal length (figure 1), then we analyze the second variable - sepal width (figure 2). In this way, we continue the construction of the whole tree. A. Coding from sklearn import tree import numpy as np iris = open("iris.txt","r") list = [] for line in iris: data = line.split(",") list.append(data) list2 = [[float(column) for column in row] for row in list] print(list2) iris_res = open("iris-result.txt","r") res = [] for line in iris_res: data = line.split(",") res.append(data) res2 = [[int(column) for column in row] for row in res]
The code was written in Python. As you can see, we have introduced the data of IRIS databases to the program. Then we entered the function to learn our network through a library that Python has: sklearn.tree. DecisionTreeClassifier
The IRIS database contains a set of iris flower measurement and was first made available by Ronald Fisher in 1936. This is one of the most well-known collections, in addition, as we’ll see in a moment is also very simple. The set of irises consists of 4 measurements of flower petals and a leaf: width and length. There are three types of flowers:
Versicolor - This flower is found in North America and develops up to a height of 80 centimeters. The leaves of this plant have a width of more than one centimeter, and the roots form large and thick clumps. A well-developed plant has 6 blue petals and blooms from May to July, while large seeds can be observed appearing in autumn.
Setosa - the flower is found in Canada, Russia, north- east Asia, China, Korea, southern Japan and Alaska. The plant has half-green leaves, high branched stems and purpleblue flowers similar to lavender (there are also pink and white flowers). The roots are shallow, large and rapidly spreading.
Virginica - this flower is native to North America. The leaves are 1 to 3 centimeters long and sometimes longer than the flower stalk. The plant has 2 to 4 erect or arching, bright green. The roots are spread underground. The seeds are light brown and differently shaped, and are born in three-part fruit capsules. The petals vary in color from dark purple to pinkish- white. These plants bloom from April to May and have from 2 to 6 flowers.
Our database has 150 data (50 for each type of flower), of which 120 we used to learn the artificial neural network, and 30 (10 for each species) to test the artificial neural network. The question then arises: Do the data groups we have received correspond to the three species of iris? To see this, let’s look at the error matrix.
A. Confusion Matrix
Confusion Matrix is the basic tool used to assess the quality of the classification. In our table, we consider three classes of abstraction as a result of which we get a 3 x 3 matrix. The
sensitivity, recall, hit rate or true positive rate (TPR) T P R = T P = T P +F N = 10+0 = 1100 = 1
T P 10
P T P R = 1 F N R specificity, selectivity or true negative rate (TNR) T N R = T N = T N+F P = 20+0 = 2200 = 1
T N 20
N T N R = 1 F P R precision or positive predictive value (PPV) P P V = T P +F P = 10+0 = 1100 = 1
T P 10 P P V = 1 F DR negative predictive value (NPV) N P V = T N+F N = 20+0 = 2200 = 1
T N 20 N P V = 1 F OR textitmiss rate or false negative rate (FNR) F N R = FPN = F N+T P = 0+10 = 100 = 0
F N 0 F N R = 1 T P R fall-out or false positive rate (FPR) F P R = F P = F P +T N = 0+20 = 200 = 0
F P 0
N F P R = 1 T N R false discovery rate (FDR) F DR = F P +T P = 0+10 = 100 = 0
F P 0 F DR = 1 P P V false omission rate (FOR) F OR = F N+T N = 0+20 = 200 = 0
F N 0 F OR = 1 N P V accuracy (ACC) AC10C+20 = T PP ++TNN = T P +TTNP ++TF NP+F N = 10+20+0+0 = 3300 = 1 F1 score - harmonic mean of precision and sensitivity F1 = 2 PPPPVV+TTPPRR = 2T P +2FTPP+F N = 20+200+0 = 2200 = 1 Matthews correlation coefficient (MCC) M CC = p(T P +F P ) (TTPP+TFNN)F(TPNF+NF P ) (T N+F N) = 10 10 20 20 = 320000 = 32
300 informedness or Bookmaker Informedness (BM) BM = T P R + T N R 1 = 1 + 1 1 = 1 Markedness (MK)
M K = P P V + N P V 1 = 1 + 1 1 = 1 2) For Setosa sensitivity, recall, hit rate or true positive rate (TPR) T P R = T P = T P +F N = 10+0 = 1100 = 1
T P 10
P T P R = 1 F N R specificity, selectivity or true negative rate (TNR) T N R = T N = T N+F P = 20+0 = 2200 = 1
T N 20
N T N R = 1 F P R precision or positive predictive value (PPV) P P V = T PT+PF P = 101+0 0 = 1100 = 1 P P V = 1 F DR negative predictive value (NPV) N P V = T NT+NF N = 202+0 0 = 2200 = 1 N P V = 1 F OR textitmiss rate or false negative rate (FNR) F N R = FPN = F NF+NT P = 0+010 = 100 = 0 F N R = 1 T P R fall-out or false positive rate (FPR) F P R = FNP = F PF+PT N = 0+020 = 200 = 0 F P R = 1 T N R false discovery rate (FDR) F DR = F PF+PT P = 0+010 = 100 = 0 F DR = 1 P P V false omission rate (FOR) F OR = F NF+NT N = 0+020 = 200 = 0 F OR = 1 N P V accuracy (ACC) AC10C+20 = T PP ++TNN = T P +TTNP ++TF NP+F N = 10+20+0+0 = 3300 = 1 F1 score - harmonic mean of precision and sensitivity F1 = 2 PPPPVV+TTPPRR = 2T P +2FTPP+F N = 20+200+0 = 2200 = 1 Matthews correlation coefficient (MCC)
T P T N F P F N M CC = p(T P +F P ) (T P +F N ) (T N +F P ) (T N +F N ) 10 10 20 20 = 320000 = 23
300 informedness or Bookmaker Informedness (BM) BM = T P R + T N R 1 = 1 + 1 1 = 1 Markedness (MK)
M K = P P V + N P V 1 = 1 + 1 1 = 1 3) For Virginica sensitivity, recall, hit rate or true positive rate (TPR) T P R = TPP = T PT+PF N = 101+0 0 = 1100 = 1 T P R = 1 F N R specificity, selectivity or true negative rate (TNR) T N R = TNN = T NT+NF P = 202+0 0 = 2200 = 1 T N R = 1 F P R precision or positive predictive value (PPV) P P V = T PT+PF P = 101+0 0 = 1100 = 1 P P V = 1 F DR negative predictive value (NPV) N P V = T NT+NF N = 202+0 0 = 2200 = 1 N P V = 1 F OR textitmiss rate or false negative rate (FNR) F N R = FPN = F NF+NT P = 0+010 = 100 = 0 F N R = 1 T P R fall-out or false positive rate (FPR) F P R = FNP = F PF+PT N = 0+020 = 200 = 0 F P R = 1 T N R false discovery rate (FDR) F DR = F PF+PT P = 0+010 = 100 = 0 F DR = 1 P P V =
Fig. 3. Confusion matrix for classification results by the use of implemented decision tree method false omission rate (FOR) F OR = F NF+NT N = 0+020 = 200 = 0 F OR = 1 N P V accuracy (ACC) AC10C+20 = T PP ++TNN = T P +TTNP ++TF NP+F N = 10+20+0+0 = 3300 = 1 F1 score - harmonic mean of precision and sensitivity F1 = 2 PPPPVV+TTPPRR = 2T P +2FTPP+F N = 20+200+0 = 2200 = 1 Matthews correlation coefficient (MCC)
T P T N F P F N M CC = p(T P +F P ) (T P +F N ) (T N +F P ) (T N +F N ) 10 10 20 20 = 320000 = 32
300 informedness or Bookmaker Informedness (BM) BM = T P R + T N R 1 = 1 + 1 1 = 1 Markedness (MK) M K = P P V + N P V 1 = 1 + 1 1 = 1 =
P - condition positive - the number of real positive cases in the data, N - condition negative - the number of real negative cases in the data, TP - true positive, TN - true negative, FP - false positive,
FN - false negative We present our error matrix in the form of a table in figure 3, in which the poems correspond to the species of iris, to which the data point belonged, while the columns tell the genre to which it was qualified. The elements inside the table specify the number of data points corresponding to the species of iris specified in the row header assigned to the data group specified in the column header.
Let us note the perfect compatibility between data groups and species of iris. Each point has been correctly classified.
Proposed reasoning was easy to implement. Results show decisions were very good and all inputs were classified correctly. The code of the reasoning in python used a library for artificial intelligence where the method was coded.
In future research i want to develop another method for data classification based on other probabilistic methods where decision between classes will be related to statistical measures.