Machine Learning Concepts and Applications Yusifov Mahammad1, Lala Bakirova 2 1,2 Azerbaijan State Oil and Industry University 1, 34 Azadliq Avenue, Baku, AZ1000, Azerbaijan Abstract The thesis includes the main basic concepts of machine learning and its applications. For simplest possible explanation of machine learning concepts, handwritten number recognition software has been represented and on this application, concepts like artificial neurons, weights, edges, connections and others has been explained. Layered structure of artificial network, artificial neuron activation value. What is edges shapes? How the recognition process happening, what is neural network and neurons? What is weight and activation value, Sigmoid and ReLU function? What is the parameter of connections between neurons? These questions and others has been covered in this article. Keywords 1 Neural network, artificial intelligence, neurons, auto recognition, activation, weight 1. Introduction considered as a function that takes input as a number and passes as output to the next so-called artificial neuron [1]. Figure 1 is an example of Machine learning is a field of science that artificial neural network that consists of input, covers mathematical complex calculations and output and some additional hidden layers that can equations in order to make computers imitate the be considered as black box for now. “learning” process of humans. This process involves a computer algorithm that builds models based on given training data to make proper decisions without being manually programmed by a developer [2]. Machine learning is considered as an important functionality and branch of Artificial Intelligence and the applications of this interesting technology are very wide: Speech recognition and other image recognition techniques, computer vision, email spam, and malware filtering, virtual personal assistant, online fraud detection, self-driving cars, stock market trading, and medicine. And this list can be Figure 1: Artificial neural network expandable. In this thesis, main concepts and applications of machine learning will be covered. In every layer, neurons are connected to each other via weights that can be modified with result 2. Neural network of learning or training process. Next, we can take simple example application handwritten number recognition and go through these concepts of The artificial neural network is a set of neurons. connected units that each represents a neuron, like in the human brain. Every neuron can be ISIT 2021: II International Scientific and Practical Conference «Intellectual Systems and Information Technologies», September 13–19, 2021, Odesa, Ukraine EMAIL: yusifov.19@mail.ru (A. 1); lala_bekirova@mail.ru (A. 2) ORCID: 0000-0002-4295-8642 (A. 1); 0000-0003-0584-7916 (A. 2) ©️ 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 2.1. Handwritten number consists of 10 neurons that each represents a number as a result. The activation of neurons in recognition the output layer also represents grayscale value, the neuron with the highest activation number is Although it is not so challenging to humans to the “choice” of the machine. For example in our understand the number on the left which is sloppy case, the number is 9 and the neural network of written by me, for computers it is not an easy task. the decision process has been shown in Figure 4. Figure 2 shows an example for number recognition which has been taken from this site: https://www.i-am.ai/neural-numbers.html. Figure 2: Screenshot of number recognition AI How this software converts or translates the Figure 4: Artificial network with result “9” analog number given on the left to the digital mathematical number that has been given on the When we try to recognize a number, our brain right by the machine? Let’s start with looks for certain patterns like loops or shapes. For conceptualizing the term “neuron”. example, in our example, the number 9 has circle up top and a line on the right. The number 8 has 2 circles or loops one on top and the other at the bottom. So our next inner layer can be responsible for holding each one of these patterns shapes in a neuron. If the activation of the specific neuron is close to 1, the possibility of that corresponding pattern is high. For example, suppose there is one certain neuron that represents a circle pattern, and the activation is 0.9. That means the resulting number can be: 0, 6, 8, 9. Figure 3: Number on pixels or on “neurons” We can consider a neuron as a unit that holds a number between 0 and 1. In figure 3 we have 28 x 28 = 784 neurons and each one of these neurons holds a value that represents the greyscale value of the corresponding pixel. For example, the neuron that value is 0 means it is black and the one with value 1 means it is white. This number is called “activation” [3]. Figure 5: Second last layer holding patterns All of these 784 neurons represent the input layer of our artificial network and the output layer So we can suppose, our second last layer is neurons: weight which is also number. And from responsible for recognizing patterns as shown in mathematical approach: Figure 5. 𝑎1𝑤1 + 𝑎2𝑤2 + … + 𝑎𝑛𝑤𝑛 , (1) But, these patterns themselves also consist of This equation is weighted sum [4]. little shapes, which we can call edges. For example circle can consists of a bunch of edges. Figure 6: Little edges of patterns So we also need to have a layer that is responsible for recognizing these little edges. The layer that comes after the input layer can be responsible for that operation. So with 2 layered 16 neurons structure, we can suppose the working principle of our little AI is in this way: In the first Figure 8: Edge recognition with weights layer we get input as a handwritten number with 784 neurons, and then it sends a signal to the next Weight with the color green means positive layer in order to detect edges and then comes third vale and with the red color means negative so layer to detect bigger shapes as patterns and at the weights with the pixel color red means darker are output we have our result as a number. and it distinguishes darker and lighter areas on pixels so that it can detect edges. When we compute weighted sum in equation 1, we can come up with any number, but for our network, we need activation values to be in the range of 0 and 1 and here we need to use the specific function that minimizes real number to the range between 0 and 1. A common function is the sigmoid function. Figure 7: Overall of 2 layered neural network structure. This structure can be also applicable for other recognition tasks like image and voice Figure 9: Sigmoid function recognition. For example, detecting shapes in the pictures, or identifying syllables in human voice Negative inputs end up close to 0 and positive can be our “patterns” or “edges” in this structure. inputs end up close to 1 [5]. So the activation of a To be able to detect that edges, we need to neuron in our network is a measure of how assign new parameter to the connections between positive the corresponding weighted sum is. But also we need bias for inactivity. We just need to extract that value from the weighted sum before processing the sigmoid function [1]. 𝜎(𝑎1𝑤1 + 𝑎2𝑤2 + … + 𝑎𝑛𝑤𝑛 − (2) 𝑏𝑖𝑎𝑠) , So, the weights indicate what pattern, edges this specific neuron in the second layer is picking up on and the bias indicates how high the weighted sum needs to be before the neuron starts getting active. But there is another better function called Figure 11: Some numbers from MNIST database. ReLU – REctified Linear Unit. The more training data we give to the machine, the more it evolves itself by doing feedback: taking input from the picture does its operation and match the result with the real idealistic value behind the number and correct itself [6]. 3. Applications Very popular example of how powerful artificial intelligence can be is GPT-3 (Generative Pre-trained Transformer 3) launched by OpenAI company. This model uses deep learning to produce texts that are very similar to human words [9]. GPT-3 can even code by its own, you just Figure 10: ReLU function have to give input commands like create a button that looks like rectangular and when it is pressed Using sigmoid function, was very difficult to increase certain value or do some other train at some point, but ReLU solved this problem operations. For example debuild.co is interesting [10]. website that uses GPT-3 to write code. User gives Every neuron in the first layer is connected to orders as input and it generates JavaScript code the 784 neurons. And each one of these 784 according to orders and visually shows the result connections has weight and bias. So we have 784 at a time [10]. x 16 weights with 16 biases and the connection between other layers also has weights and biases according to them. Our network has 13002 weights and biases in total. Learning in here is, getting the machine to find a true setting for all of these many numbers so that it can solve the problem at a time [7]. The training process of the network can be done with the help of certain special databases that store collections of images of numbers handwritten. So the machine can train itself to become a better “predictor” with both having input and output from the database [8]. MNIST database is well-known example of Figure 12: Describing google UI to GPT-3 these databases which stores 70000 images of digits handwritten by people like high school GPT-3 functionalities are very wide and that students and employees of the American Census shows how amazing works can be done using Bureau that are published to help AI researchers. Machine Learning and Artificial Intelligence. In medical field, machine learning has so many applications and they are currently being developed. One of the proper examples is Microsoft's InnerEye Project that identifies differences between healthy cells and tumor cells analyses of human and better treatment for each by using 3D radiological images as shown in specific patient. There are 3 tables in this simple Figure 13. represented database: 1. patients: stores basic information for other relations 2. health_parameters: stores vital information about a human this data is essential for identifying potential diseases 3. other_parameters: stores information about daily activites, and possible decisions that the human can make. The second and third tables are connected to Figure 13: Screenshot from Inner Eye first table via foreign key on id of patient. With only this small data a well-structured AI can Pfizer uses machine learning to research about predict the cause of psychological disease and how the immune system of human body can fight develop more relevant therapy for patient. For cancer. Insitro is a startup uses data science and example: a married man with weight of 120kg machine learning to develop drugs that cures who is accountant may have heart disease and people much faster and with high level of success with having no hobbies is additional risk [11]. parameter that causes psychological disease on Besides medical field, there are many him. That one example can be extended widely interesting fields that use machine learning. with the growth of the database. More data Navigation programs like google maps analyzes provided to model, more it will be decisive and all roads and transportation ways and brings most correct. If there are 2 persons that have relation relevant choice of travel from location to another. they can also have same problem. For instance, a Social media programs has many features that use partner of current patient had this certain disease machine learning to process its functionality like 4 months ago as she registered in this database. So bringing “people you may know”, face detection, this AI can read all relations of humans even if image recognition and others. Self-driving cars they are not bound together but have same like Tesla works on deep learning and also workplace that other patient had, this can be includes IoT because of intelligent sensors. important information for model to decide. This Language translation apps, online video complex relation may be difficult for us to see and streaming apps and many others have a touch on decide but it is not the same for a machine. It can machine learning and expands quickly. perform big analysis and diagnostic on a human Another additional machine learning and identify multiple treatment or therapy application can be in psychology field. Model can methods. And a doctor who is psychologist can be established in order to read and store different choose one or more of those method even modify parameters of humans in a database. Database them. This kind of machine learning application is structure is shown in a following figure. not only about psychology field, it also can be about other specific fields of medical applications. 4. Conclusions In the new era of technology, Artificial Intelligence will help humans to solve so many problems in a short time without errors. It can affect the human life as given example about identifying the disease on human by giving database as a training data for machine to learn. In medicine, special surgeries, in oil manufacturing, Figure 14: Simple database for model to learn or in some dangerous places that humans should not be operate, machines will replace and make appropriate decisions like, or better than humans. This database can be input for model and it can And that complex operations is starting with use machine learning to improve more correct handwritten number recognition task. It is "Envisioning an artificial intelligence considered as “Hello-World” of Machine documentation assistant for future primary Learning. But as we know every big roads starts care consultations: A co-design study with with first little steps. general practitioners". Journal of the American Medical Informatics Association. 5. Acknowledgements 2020 [15] Kobie N. "DeepMind's new AI can spot breast cancer just as well as your doctor". Thanks to head of cathedra: Doctor of Wired UK. Wired. 2020 technical sciences Mrs. Lala Bakirova. 6. References [1] Lau, Suki. “A Walkthrough of Convolutional Neural Network – Hyperparameter Tuning": Medium. 23 August 2019. [2] Bishop, C. M. “Pattern Recognition and Machine Learning”, Springer, 2006. [3] Ethem Alpaydin. “Introduction to Machine Learning (Fourth ed.)” 2020 [4] "AI has cracked a key mathematical puzzle for understanding our world": MIT Technology Review 19 November 2020 [5] "Introduction to AI Part 1". Edzion. 2020-12- 08. Retrieved 2020-12-09. [6] "AN EMPIRICAL SCIENCE RESEARCH ON BIOINFORMATICS IN MACHINE LEARNING – Journal". Retrieved 28 October 2020. [7] Berlinski, David "The Advent of the Algorithm". Harcourt Books. 2020. [8] Zola, Andrew. "Interview Prep: 40 Artificial Intelligence Questions". Springboard Blog. 2019 [9] Sizing the prize: PwC's Global AI Study— Exploiting the AI Revolution". 11 November 2020. [10] "Automation and anxiety". The Economist. 13 January 2018. [11] Lohr, Steve "Robots Will Take Jobs, but Not as Fast as Some Fear, New Report Says". The New York Times. 2017 [12] Christopoulou F, Tran TT, Sahu SK, Miwa M, Ananiadou. "Adverse drug events and medication relation extraction in electronic health records with ensemble deep learning methods". Journal of the American Medical Informatics Association. 2020 [13] Brunn M, Diefenbacher A, Courtet P, Genieys W. "The Future is Knocking: How Artificial Intelligence Will Fundamentally Change Psychiatry". Academic Psychiatry. 2020 [14] Kocaballi AB, Ijaz K, Laranjo L, Quiroz JC, Rezazadegan D, Tong HL, et al.