425 Identification of Plants using Deep learning: A Review Rakibul Sk, Ankita Wadhawan Lovely Professional University (LPU University), Jalandhar-Delhi, G.T. Road, Phagwara,144411 Punjab Abstract Identification of plants is a very important field in the earth’s ecology to maintain a healthy atmosphere. Certain of these plants have significant medicinal properties. Nowadays of finding a plant is not easy by looking at its physical properties. This paper provides an academic database of literature between the duration of 2015–2020. It has been observed that the new generation of convolutionary neural networks (CNNs) in the space area of image recognition has produced remarkable performance. In this paper, techniques are discussed the concepts of Deep learning and different leaf recognition methods. Keywords Machine Learning, Artificial Intelligence, Fully Connected Neurons, Convolutional Neural Network, Deep Learning, Image Processing. 1. Introduction of results; it only includes identification of the data format and unlabelled inputs. Nowadays, Artificial Intelligence (AI) is the Deep learning offers superior results even most important part of our lives, it is used in on big data. Deep learning, which is used in the field of Computer Vision, Robotics, Digi- image identification and computer vision etc, tal Marketing Transformation, Medical field, utilizes artificial neurons identical to the neu- Banking, and business sectors. Artificial Intel- rons of man. Deep neural networks, recurrent ligence (AI) has been mainly designed to make neural networks, and deep belief networks machines for thinking and acting like a hu- are used to speak identification, language pro- man being and machine learning would be a cessing, translation software, audio recogni- sub-part of Artificial Intelligence (AI), as well tion, bioinformatics, and drug development. as a theoretical algorithm analysis and a math- Plants are indeed an important member of ev- ematical model that carries out a particular ery natural life [1] as well as the formal nam- check without explicit programming, on the ing of this will ensure that every natural life basis of the assumption and templates. Some is preserved and maintained. Plants are essen- basic forms of artificial learning strategies are tial for our medicinal purposes, as alternative Supervised learning, unsupervised learning, sources of energy such as biofuels, and also and reinforcement learning. The Supervised used to fulfill our numerous domestic needs Learning Algorithms contain data, Unsuper- such as wood, clothes, food, and makeup. The vised learning algorithm requires a collection present extinction trend is primarily the prod- uct of overt and indirect human activity. Cre- ISIC’21: International Semantic Intelligence Conference, February 25–27, 2021, New Delhi, India ating correct identification information and " rakibulsk.cse@gmail.com (R. Sk); plant geography propagation is key to the ankita.23891@lpu.co.in (A. Wadhawan) survival of ecosystems in the future. © 2021 Copyright for this paper by its authors. Use permit- ted under Creative Commons License Attribution 4.0 Inter- Many countries worldwide are now design- national (CC BY 4.0). CEUR http://ceur-ws.org CEUR Workshop Proceedings ing programs to create channel control sys- (CEUR-WS.org) Workshop ISSN 1613-0073 Proceedings 426 tems for national agriculture [2]. India will usually a straightforward job. In a fact, certain have a long tradition of utilizing plants as a plants cannot even have visible parts of the therapeutic source. This research is called leaf. On the other side, several research rele- ayurvedic [3]. Each plant on Earth has a cer- vant to utilizing machine vision approaches tain medicinal value according to Ayurveda. to address such issues were also performed. This is viewed as a worldwide type of sub- Aerial image recognition of landscapes that stitute to allopathic medicine. one of the big use machine learning techniques is an illus- bonuses of this is that it has no adverse effects. tration of computer recognition technologies Taxonomists systematically classify such medic- of agricultural computing. inal plants, that are susceptible to Miscar- In Section 2, we did a literature review of riages in certain situations. various research papers related to plants de- Image recognition strategies that have re- tection using deep learning and shown a ta- cently started to appear in an attempt to sim- ble (Table 1.) of the previous pattern method. plify that plant inspection process. With re- In Section 3, we discussed the identification spect to a plant identification research, meth- methodology. In Section 4, we discussed in de- ods focused by color characteristics were of- tail about the architecture of CNN. In Section ten used to establish a plant recognition method. 5, we discussed the process of leaf identifi- Color interpretation probably depends on color cation very preciously. At last in Section 6, distributions in such an image, although it is we gave the conclusion about our paper and not a safe function because here are certain discussed future work. situations where this feature’s temporal accu- racy is abused. The Shift of light, leaf move- ment through waves, camera jitter, changing 2. Literature Review of focus, sudden shifts of camera parameter The literature on Deep learning is very wide. contribute to incorrect plant category predic- The work done by various researchers in the tions. Giving numerous researches, the cate- field of plant identification using Deep learn- gory of plants dependent on digital images is ing is described in this section. Sapna Sharma. now seen as a difficult issue. Those researches (2015) used principal component analysis (PCA), were focused on the study of particular plant Hu’s moment invariant method, and morpho- leaves for the identification and classification logical features for classification and they have of plants [4]. In subsequent years, many stud- used sixteen different classes of the leaf. This ies were using them to develop a model for a Matlab measures the circumference by mea- plant leaf recognition system. suring the gap in each connected number of Gaber et al. used MCA to derive visual char- pixels along the area’s boundary [1]. T. Gaber. acteristics from plants [5] and linear discrimi- (2015) suggested a plant recommender method nant analysis. Multiple researchers also have that uses 2D visual photographs of plants. established the ability of broad convolution This program used the methodology of at- neural network (CNN) to outperform conven- tribute fusion and the process of multilabel tional object recognition or detection strate- classification. The experimental findings re- gies centered upon ordinary working light, vealed that the function fusion method’s accu- texture, and shape characteristics. Typically, racy was much higher than other individual the CNN systems are using in such big-scale applications. The tests showed their robust- plant recognizing activities consists of a trait ness in providing accurate recommendations extractor accompanied by a classifier. Despite [2]. T. J. Jassmann. (2015) designed a new of occlusions collecting plant crop does not 427 CNN they tested the usage of the newly imple- and efficacy of deep neural networks applied mented Exponential Linear Unit (ELU) rather to plant pathology and the in-depth study of than Rectified Linear Unit (ReLU) as CNN’s the topic, which illustrates the benefits and non-linearity method [3]. disadvantages, will contribute to more con- Hulya Yalcin (2016) suggested an architec- crete findings on plant pathology [8]. Barbedo, ture of the CNN to identify the form of plants J.G.A., (2018) explored the implementation from the picture sequences obtained from smart of issues in transfer learning and the use of agro-stations. the design is used as a pre- deep learning. They found that CNN is a processing stage to remove the picture prop- method used to classify plant biotechnology erties. Configuration of the CNN design and issues [9]. Zhu, X., (2018) uses CNN (Complex breadth are important points that should be Background) to recognize the small objected highlighted because they impact the recog- plant leaves. The designed methodology im- nizing capabilities of neural network architec- plemented sample-normalization founding V2 ture. They used 16 kinds of plants and com- which enhances the accuracy of Region CNN. pared them with other approaches; prelimi- For processing, the quality photos sub-samples nary findings show that the CNN centered ap- are split into a hundred and the residual im- proach’s classification performance outranks ages are returned to final production. The ap- other approaches.[4] proach suggested that it could be faster than Amala Sabu (2017) depicts that Universal conventional region convolutional neural net- Leaf Identification is a difficult Computer Vi- work [10]. Garcia-Garcia (2018) A writer of sion issue. Efficient leaf recovery method for this paper used deep learning techniques to Ayurvedic plant beneficial for other aspects of focus on high occupancy classification. They society including Medicine, studies in Botany. presented short information on the topics of Recognize the photographs of the leaf. The deep learning. Which offers the required rel- study of the Different approaches and classi- evant information on deep learning for the fications for leaf identification [5]. Lee, S.H., mission ahead [11]. (2017) gathered one of the pictures of plant Kaya, A., Keceli. (2019) suggested the con- leaves has also been discussed based upon cept of Transfer Learning for Plants Classifi- the leaf characteristics use as an input and cation focused on Deep Learning. This paper convolution neural network is being used to indicates the impact of four separate transfer- identify patterns for each plant depth infor- ence training models on plant classification mation. CNN was mainly utilized here just deals dependent on DNN for four available for the improved portrayal of the characteris- databases. Finally, their theoretical research tics and for effective studies of Leaf organisms reveals that Transfer Learning offers a ba- DN (Deconvolutional Network) used. It en- sis of plant classification self-estimating and ables greater recognition of plant leaves and analysing. They use certain common formats their populations [6]. Ghazi, M.M., (2017) im- including End-to- End, Fine modulation, Fine plemented three models of transfer learning modulation Cross Dataset, Deep Integrated to describe the identity of the various plants. Finetuning, Classification by RNN-CNN [12]. The Network was evaluated using LIFECLEF 2015. These three-model used GoogleNet, VG- GNet, and AlexNet for their suggestion here [7]. Barbedo, J.G. (2018) discussed the analysis of the key factors influencing the architecture 428 Table 1 Previous pattern recognition methods Author Technique used Dataset Size Results GLCM (grey level co-occurrence matrices: GLCM is a histogram at a given offset over an image with Sapna Sharma, Dr. co-occurring greyscale values.) and Principal com- 16 different Chitvan Gupta ponent analysis (PCA): the method measuring the classes of the Review (2015) [1] principal component and to perform on the data leaf. basis of change, Super Vector Machine (SVM): uses for classification problem methods. using Flavia a dataset which T. Gaber (2015) [2] Bagging classifier,2D based technique Accuracy: 95% 1907 colored images. Rectified Linear Unit (ReLU): is an activation func- Flavia dataset T. J. Jassmann (2015) tions and ReLu is the most used activation function contains leaves Accuracy: 60% [3] in the neural network, moreover in the CNNs. of 32 plants. Hulya Yalcin (2016) 16 plant Accuracy: Convolutional Neural Network (CNN) model. [4] species. 97.47% Amala Sabu (2017) K-Nearest Neighbor (KNN). Review Review [5] Using 113,205 Accuracy: Lee, S.H (2017) [6] CNN images 96.3% Transfer Learning using AlexNet GoogLeNet and Ghazi, M.M (2017) VGGN: VGGN is an object-oriented Model and sup- using 91,758 Accuracy: [7] ports 19 layers and VGGN is still the most popular images. 80.18% used architecture for image recognition. almost 50,000 Barbedo, J.G (2018) images use Using deep learning concepts Accuracy: 81% [8] from plantvil- lage dataset. Containing 12 different Barbedo, J.G.A CNN plants and Accuracy: 84% (2018) [9] 1383 images were used. For each species, 180 im- Zhu, X (2018) [10] RCNN ages are taken, Accuracy:99% of which 150 are taken for In this pa- Garcia-Garcia semantic segmentation using deep learning tech- per they use Review (2018) [11] niques. 2D and 3D datasets. Using total Kaya, A., Keceli Accuracy: Transfer learning method use on deep learning. number of (2019) [12] 98.70% 54,306 images. Noon, S.K., Am- jad, M., Qureshi, deep learning-based Review Review M.A., Mannan, A(2020)[13] 429 3.1.1. CNN CNN [7] is the part of deep neural network class. This is mostly used in Computer Vision to identify the given structure of the object being subjected. CNN ’s primary objective is to identify and forecast the sequence of the given input datasets. It delivers enhanced performance and accuracy. 3.1.2. Layers In CNN Figure 1: The architecture of our model (CNN) CNN is a controlled methodology in deep learning, and has developed a ground break- ing influence on numerous applications fo- 3. Identification cused on machine vision and images. The Methodology fields of which CNN is commonly employed include facial recognition, target identifica- For detecting the plans from the images of the tion, analysis of videos, etc. CNN platform leaf, we have discussed various strategies on components involve convection layers, pool- the CNN model utilizing various leaf dataset ing layers, completely linked layers, activa- to show the characteristics of the visualiza- tion functions, etc. tion approaches for CNNs. CNNs are neu- ral feed forward networks that are fully con- • Convolution Layer: For more pro- nected. CNNs are exceptionally effective in cessing the layer provides an RGB pic- decreasing the number of parameters with- ture or an output of another layer as out compromising layout efficiency. Images data. The obtained information is re- have a large size (as each pixel is regarded as ferred to as image pixels to produce a feature) that corresponds to the CNNs. it a function map reflecting characteris- has been developed to take account of images, tics of low levels, such as edges and but milestones were still reached in text pro- curves. Special characteristics at the cessing also. The edges of artifacts in each higher level can be defined via a se- picture are guided by CNNs. quence of further convolution levels. • Activation Layer: Nonlinearity makes 3.1. Architecture of CNN a network of neurons deeper. A Non- Deep Belief Network is used in various meth- linear activation layer shall be added di- ods of Language processing, Computer Vi- rectly after each layer Convolution stra- sion, Speech Reorganization, and many other tum. Specific nonlinear mechanisms applications. Deep Neural Network has a are used for Add non linearity. They three-layers Input, Hidden, and Output. The are: Deep Neural Network [6] processes in multi- Tanh: The range between [-1,1] takes ple NN. Fig.1 illustrates how the Neural Net- the real-valued number of this non-linearity. work nodes and layers are connected and share information. Sigmoid: The range between [0,1] takes the real-valued number of this non-linearity. 430 Rectified Linear Unit: It improves the model’s nonlinear property by altering the convolution layer’s receptive field by altering all the lower values to 0. • Pooling Layer: After the activation layer a downsampling layer was added to raising the spatial aspect without any alteration in size. Typically, a size 2x2 input filter is applied to produce an out- put based on the pooling process. It may be expressed either by peak pool- ing or by average pooling where the limit or average value is calculated for each sub-region used in the filter. There pooling is no restrictions. Then the pooling layer decreases the scale of the characteristic map, i.e., the width and length are limited but the dis- tance is not. It Decreases the number of Weights and Parameters, lowers the preparation period, and thus eliminates the computational expenses. It also re- quires overfitting controls. • Fully Connected Layer: This layer defines characteristics that are at a very large quality correlates to class or ob- Figure 2: Step of leaf Identification Process ject. Entering a fully connected field layer is a collection of features for pic- ture recognition without requiring to as seen in Figure 2. The demanding one that taking into consideration the spatial con- is part of the research is to determine leaf dis- text of the pictures. Fully connected tinguishing features for the identification of layer output is often a 1D vector achieved plant organisms. In this situation, a separate by compressing the final pooling layer classifier with high-performance statistical output. This is a method of organizing methods has been used to conduct leaf classi- 3D volume in a 1D vector. fication and function extraction of the func- tionality. The improvement in image analysis 4. Process of Leaf and CNN significantly aided researchers in the classification of plants by data analysis. Identification This is a basic image-based plant recogni- tion process is shown in fig.2 and some define Plants take a significant role in both human some general stages. and other life on earth. Recognition of the leaf design implements normally the phases 431 4.1. Image Acquisition 4.2.3. Binary conversion: Image acquisition is the process of collecting Create binary images on a gray image scale to datasets for identification leaves. Infected leaf use the threshold method. The Binary picture photographs of collected in managed settings is a visual image that contains just two poten- and are processed in JPEG format. Against tial meanings for each pixel. The two shades a white backdrop, the contaminated leaf is that a contrasting picture uses are typically put smooth, the light source was mounted white and black. on either side of leaves at a temperature of 45 degrees to remove each reflection and pro- 4.2.4. Noise Removal: vide even light wherever thereby increasing illumination and clarity. This crop is zoomed Digital photographs are susceptible to a plethora such that the photograph captured includes of Noise levels. Noise emerges from errors in just the crop and white backdrop the virtual method of image acquisition which results in pixels values They can be used to eliminate linear filtering those values noise 4.2. Image Pre-processing Styles. Few filters are ideal for this purpose, Image pre-processing procedures are essen- such as Gaussian filters or low pass filters, tially used to expose information that is hid- averaging. An ordinary filter, for example, is den or basically to show any features in such useful for having grain noise off the picture. A an image. Such methods are largely contex- median filter and averaging filter are used for tual and are structured to alter a picture and salt and paper noise removal from an image. taking advantages of the psycho-cultural di- mensions of the human sensory system. Equal- 4.3. Feature extraction ization of the histograms and electronic filtra- tion methods were used. mainly the characteristics of leaf color and form. The Specific plant leaf is generally iden- tical in color and form are considered for clas- 4.2.1. Histogram equalization: sification and so a specific function alone can- it’s some of those strategies for improving not achieve anticipated results. images. A certain approach allocates image intensities. Among this process, the contrast 4.3.1. Color features between the fields rises through local con- trast to greater intensity. The equalization of Dr. H.B. Kekre et al.’s suggested the approach histograms is used to enhance computational of scanning and retrieving photographs pri- complexity, clarity, and image consistency. marily focused on the production of the color function vector by measuring the mean. This three-color Red, Green and Blue are first di- 4.2.2. Grayscale conversion: vided in the suggested algorithm. Then means Gray scale conversion is used for converting and column mean of colors are determined images into grayscale. The grayscaled conver- for every plane side. For each plane the sum sion used the method of contrast feature and of all means of the row and all means of the intensity enhancement techniques for con- columns are determined. The characteristics verting the images and then placed them as of all 3 planes converge to create a matrix of pieces together for further processing. features. Until the function vectors for an im- 432 Figure 3: The basic geometric features age is created, they are contained in a database 4.4. Classification of features. Common statistical identification is the method of defining based on the previous information 4.3.2. Shape features such as a training dataset a group of groups, or Based on the of Geometric features, we de- classes to which a new phenomenon belongs. fined shape features: More precisely, classification in this work is a. Geometric features: We used the similar the method used to attribute a picture to a 5 geometric features (DMFs), define in fig. 3, certain plant genus, based on its collection of derived from the following 5 basic features: features. It is a subclass of more general statis- 2 tics and deep learning identification problems, including supervised learning. 1. Diameter: between any two points the diameter of the leaf is the longest distance on the closed contour of the 4.5. Testing leaf. Of this phase, we test the model by giving 2. Physiological Length: It’s the lengthtesting data to the model. Then check how is of the line which connects the two main it identifying the object and also, we get the vein terminals points in the vine. accuracy. 3. Physiological Width: This corre- sponds to the interval perpendicular to the physiological longitude between 4.6. Convolution Neural the two endpoints of the longest line Network Process section. CNN is one of the types of neural networks 4. Leaf Area: This is the amount of bi- which are widely used in computer vision nary representation 1 pixels on the smootharea. Its name stems from the form of secret image of the vine. layers it consists of. Usually a CNN’s con- 5. Leaf Perimeter: the count of pixels cealed layers comprise of convolutionary lay- in the leaf’s closed contour. 433 Figure 4: The architecture of our model (CNN) Table 2 For identification, Some Layer is used for our model Layer No of filters Filter size Stride Value 1st 16 2*2 1 2nd 16 2*2 1 3rd 32 2*2 1 4th 32 2*2 1 5th 64 2*2 1 7th 64 2*2 1 ers, pooling layers, completely linked layers,etc.), decreasing its dimensionality and mak- and layers of normalization. Here it simply ing conclusions about features found in dis- means that convolution and pooling functions carded sub-regions. There are 2 major forms are used as activation functions. of pooling generally recognized as pooling with max and min. Like the name indicates, 4.6.1. Dropout max pooling is focused on taking up the high- est value from the selected area, and min pool- A Dropout feature is employed in many CNN ing is based on picking up the selected re- works. That can result from the problem of gion’s minimal value. overfitting in our model. By randomly remov- ing those connections that exist between the nodes. 5. Conclusion and Future Fig 4 this architecture is our work. On the very last layer, the dense function is used. Work 2 CNN performs so much more on pictures and videos than traditional neural networks, since 4.6.2. Pooling the convolutionary layers take advantage of Pooling is a discretization method dependent the image’s intrinsic properties. Simple neural on the samples. The aim is to down-sample an feedforward networks see little structure in input data (image, hidden-layer output matrix, their inputs. When you combined all the im- 434 ages in the same way, the neural network will Conference on Agro-Geoinformatics, have the same success when trained on photos Tianjin, China, DOI: 10.1109/ Agro- that are not shuffled. But on the other side, Geoinformatics.2016.7577698, Spet, it optimizes local spatial picture coherence. 2016. This ensures they will significantly decrease [3] T. J. Jassmann, R. Tashakkori, and R. M. the number of operations needed to process Parry, "Leaf classification utilizing a con- an image by utilizing convolution on adjacent volutional neural network”, IEEE South- pixel patches as adjacent pixels are meaning- eastcon, Fort Lauderdale, FL, USA, DOI: ful together. We name it central connectivity 10.1109/ SECON.2015.7132978, April,2015. too. The Map is then loaded with the product [4] Amala Sabu, Sreekumar K, “Literature of a small patch of pixels converting, slid over Review of Image Features and Classifiers the entire picture with a window. There are Used in Leaf Based Plant Recognition many methods in the detection and classifica- Through Image Analysis Approach”, tion process of automated or computer vision International Conference on Inventive for plant identification but there is still a lack Communication and Computational of research in this field. Moreover, there are Technologies (ICICCT), DOI: 10.1109/ICI- currently no consumer options on the market, CCT.2017.7975176, March, 2017. even those that deal with the identification of [5] 5) T. Gaber, A. Tharwat, V. Snasel, and A. plant organisms dependent on photographs of E. Hassanien. "Plant Identification: Two the leaves. It has been concluded that a differ- Dimensional-Based Vs One Dimensional- ent approach using deep learning techniques Based Feature Extraction Methods”, Inter- are used to automatically identify and recog- national Conference on Soft Computing nize plants from the photographs of the leaf. Models in Industrial and Environmental The model established was able to sense the Applications, Springer, Cham, DOI: existence of a leaf and distinguish between http://doi-org443.webvpn.fjmu.edu.cn/ healthy leaves. 10.1007/ 978-3-319-19719-733, May, 2015. In the future research would be to raise the [6] Kaya, A., Keceli, A.S., Catal, C., Yalic, H.Y., size of the dataset by raising the samples and Temucin, H. and Tekinerdogan, B, “Anal- by adding different classes of the plants leaf. ysis of transfer learning for deep neural After doing the literature review of vari- network-based plant classification models.” ous papers, we conclude that CNN is the best Computers and Electronics in Agriculture, approach for detecting the plants and leaves Vol. 158, March, 2019. with very good accuracy. [7] Barbedo, J.G, “Factors influencing the use of deep learning for plant disease recog- nition”, Biosystems engineering, Vol. 172, References May,2018. [8] Ghazi, M.M., Yanikoglu, B. and Aptoula, E, [1] Sapna Sharma, Dr. Chitvan Gupta, “A Re- “Plant identification using deep neural net- view of Plant Recognition Methods and works via optimization of transfer learning Algorithms,” IJIRAE - International Jour- parameters”, Neurocomputing, Vol. 235, nal of Innovative Research in Advanced April, 2017. Engineering, Vol. 2, Issue no. 6, June,2015. [9] Lee, S.H., Chan, C.S., Mayo, S.J. and Re- [2] Hulya Yalcin, Salar Razavi, “Plant magnino, P., “How deep learning extracts Classification using Convolutional and learns leaf features for plant classifi- Neural Networks”, IEEE International cation”, Pattern Recognition, Vol. no: 71, 435 May, 2017. [10] Barbedo, J.G.A., “Impact of dataset size and variety on the effectiveness of deep learning and transfer learning for plant dis- ease classification”, Computers and Elec- tronics in Agriculture, Vol. 153, Oct, 2018. [11] Zhu, X., Zhu, M. and Ren, H., “Method of plant leaf recognition based on improved deep convolutional neural network”, Cog- nitive Systems Research, Vol. 52, Dec, 2018. [12] Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez- Gonzalez, P. and Garcia-Rodriguez, J., “A survey on deep learning techniques for im- age and video semantic segmentation, Ap- plied Soft Computing”, Vol. 70, May, 2018. [13] Noon, S.K., Amjad, M., Qureshi, M.A., Mannan, A., “Use of deep learning tech- niques for identification of plant leaf stresses: A review”, Sustainable Comput- ing: Informatics and Systems, Vol.28, Dec, 2020.