Healthy eating is the cause of longevity. Rational consumption of products will help achieve high efficiency and activity throughout life. To improve the quality of your life, you do not need to read dozens of books and delve into medicine. Just use intelligent systems. Today, there are many different approaches to solving the problem of diet control in a fleeting world. The need for research on healthy eating in modern conditions is crucial to improving the physical condition of different age groups. After all, non-compliance with the basic rules reduces life expectancy and provokes the emergence of non-communicable diseases of various types: cancer, cardiovascular disease, and diabetes. The main goal is to create a system whose main task will help the end-user follow a healthy diet. The primary skills of the system include the ability to recognise dishes, read barcodes, decompose a dish into a set of ingredients, calculate calories, and calculate the energy value consumed per day. Calories counter, diet, nutrition, image analysis, food recognition, machine learning, food COLINS-2022: 6th International Conference on Computational Linguistics and Intelligent Systems, May 12-13, 2022, Gliwice, Poland ORCID: 0000-0001-7821-3522 (N. Oleksiv); 0000-0001-9149-4752 (O. Veres); 0000-0002-3666-7232 (A. Vasyliuk); 0000-0001-5727-3438 (I. Rishnyak); 0000-0002-9448-1751 (L. Chyrun)
Proper nutrition is one of the conditions for longevity and healthy living. After all, most health problems are related to what we eat. By following simple rules of healthy eating, we will feel much better. However, it is pretty challenging to track your diet daily because the day of an ordinary person is usually quite busy, and there is no place to calculate the caloric content of foods and monitor your diet. Moreover, you need to stabilise your diet and determine your diet to get the most out of constant physical activity. Therefore, creating a system that automates these processes is necessary for today’s reality. The main goal is to create a system whose main task will help the end-user follow a healthy diet. The primary skills of the system must be the ability to recognise dishes, read barcodes, decompose a dish into a set of ingredients, calculate calories, and calculate the energy value consumed per day.
A team of researchers from the University of Massachusetts has developed a neural network that can make a list of ingredients and form a recipe for food [1]. The input data for the system’s operation are images of a predetermined format [2-7]. The authors have created an open database of about a million different subject images to teach the system. The database was formed by collecting information from open sources [8-9]. The basis for this approach is culinary sites and access to food service databases. The article [1] follows that the neural network works on the principle of joint embedding: it
2022 Copyright for this paper by its authors. divides images into vectors with products or ingredients [10-14]. The neural network compares the formatted input image with the most similar picture of the recipe [15-17], which is compared with the input element under the influence of linear transformation [18-25].
The researchers used the neural network in actual conditions and compared the available results of the system with those that people themselves gave in a random survey. Under these conditions, the accuracy of the neural network is 83.6%, while the survey showed that the accuracy of recognition of products included in the dish is 81.6% [1]. Photographs with food are widely used in food magazines and social networks to share gastronomic experiences [26-30]. Many of these images were taken in restaurants. Recognition of dishes, in general, is challenging [31-36] due to different national dishes, varieties, cooking styles, and the inherent complexity of modelling food from its visual appearance [3738]. First, you need to adapt the photo frame based on the rejection of unlikely categories [39-41] located far from the test image. Then, reformulate the problem using a probabilistic model that combines dishes, restaurants, and places [42-55]. This model is used in three different tasks: food recognition, restaurant recognition, and space improvement [56-60].
To build a goal tree, we connect all the system’s goals. After that, we will select the main goal, which will serve at the top of the tree and the subordinate objectives of the first and second levels. The primary purpose of the developed system is to create an information system of dynamic and adaptive control of the human diet based on machine learning. To achieve this goal, you need to analyse the subject area, design the system and work on the implementation. The goal tree shown in Figure 1 allows you to create a compelling and complete hierarchical structure by decomposing the primary goal.
Creating an information system of dynamic and adaptive control of human diet based on machine
learning Subject area analysis
System design
System implementation e h t n i s e v i t an te ltr r e k fa am o h c r a e s e R Novelty
Recognition
quality
The subject area analysis allows you to analyse the current market situation, explore methods that will enable you to solve specific problems, etc. At the design stage, the necessary functions of the system are prioritised, and structural approaches are formed. At the maintenance, the scene is implementing the already researched and designed solution in an existing information system. It is known that according to the level of automation, information systems are divided into information retrieval, information reference, information management and intelligent information systems and decision support systems. To select the appropriate type of information system, we use the hierarchy A1 A2 A3 A4 a) Figure 6: Matrix a) pairwise comparisons of the main goal and b) comparison of alternatives
We create a matrix of alternatives comparisons and identify the information system type. The matrix of comparison of other options is shown in Figure 6b. Thus, the hierarchy analysis method is the chosen information system to support decision-making.
To develop and maintain an ontology describing the recommended system for monitoring the energy value of consumer food products based on machine learning technology, we use the standard that is part of the IDEF family in software development - IDEF5. IDEF5 method is an ontology by collecting statements about natural objects, their properties, and relationships and presenting this content intuitively and naturally reflecting the human perception of a particular area.
There are unique ontological languages to carry out the process of building IDEF5 ontologies: 1. Schematic language is a graphic language that is specifically designed to present primary data in the form of ontological information. This language allows you to naturally offer basic information about the initial development of the ontology and supplement existing ontologies with new data [6167]. This language will enable you to build various charts and diagrams in IDEF5. The purpose of graphs is to use visualisation to present basic ontological information. 2. Revisions and clarifications are a structured text language that allows you to describe the ontology elements in detail.
So, let us construct a classification chart using schematic language. Figure 7 shows a diagram of the classification of nutrients. As you can see, organic nutrients are made up of proteins, fats and carbohydrates. We believe that this classification within the system is necessary because a living organism cannot live and die without them.
Nutrient
Meal Proteins
Fats
Carbohydr ates
Breakfast
Lunch
Supper а) b) Figure 7: Diagram of classification of a) nutrient and b) meal
Figure 7b shows a diagram of the type of food intake. Each meal takes place in a different time range, so we use such categories as breakfast – early meal, lunch – midday meal, supper – a meal in the final phase of the day.
Any dish can be classified in the classification system because it is necessary to determine to which category the object that the user photographed with the lens of his smartphone belongs. Therefore, we distinguish the following dishes: main, sweet, salad, snack and drink. This classification is given in Figure 8a. Input resource – an object of the outside world with which the system works. We highlight the image with which the system can work with the dish, break it down into ingredients, and count each element’s calories that have undergone decomposition. The input also includes a barcode that allows the system to have information about the food composition without working with the product directly. Figure 8b presents a diagram of the classification of input resources.
Main
Sweet
Salad
Snack
Drink
Image
In the next step, we build a compositional scheme, which is shown in Figure 9:
Input resource
Next, we will show you some state transition diagrams to show the transition from one object to another. Figure 10 shows the transition from one state to another:
Perform decomposition
Time interval pasts Ingredient
Energy value
Breakfast
Lunch а) b) Figure 10: State diagram with a) “Perform decomposition” and b) “Time interval pasts” process
Process concept. After successful authorisation and authentication, the user sees the main page, which displays information for the current day: how many calories are consumed and how many you still need to get to comply with the norm, what foods are consumed, etc. To recognise a dish, the user must hover over the dish, take a photo and send it to the server where the recognition process occurs, using the available dataset. In case of failed recognition, similar options will be displayed. If they do not match, then to improve the system, the user can enter the necessary data and simplify the use of the application in subsequent attempts to recognise. The dish is broken down into ingredients, and the approximate total caloric content is calculated. The result is displayed on the user’s home page. The user can also monitor daily progress, which is shown in graphs based on daily statistics collected. The first time you run the IP, the user enters the necessary information to simulate his regular daily routine, which will be the basis for sending specific messages in case of deviation and reminding about the missed stage (for example, reminding the user that he missed breakfast). The stability and accuracy of the IP are proportional to the frequency of its use. If you use the system daily, its recommendations and tips will be more relevant. The combined scheme is given in Figure 11:
O
&
&
The implementation of IDEF0 diagrams was chosen to detail the structure of the information system.
The conceptual model of the process “Product recognition and diet control” is shown in Figure 12. The input parameters for the full functioning of the system are data on the caloric content of products, data on dishes, a dataset with images, user features (set in the settings) and an authorised profile for access to the API with barcodes. One of the key parameters influencing the formation of advice and a consumption plan is dietary norms. The initial result of the whole system can be the final calculation of calories and the establishment of recommended theses on nutrition and diet.
User features API with barcodes
The following steps are to build hierarchies of lower-level processes. Figure 13 shows the decomposition diagram of the first level of the “Product recognition and diet control” process.
Figure 14 shows the decomposition diagram Recognition”: of the second level of the process “Product
Formation of dietary advice Consumption recommendation
Датасет
Figure 15 shows the decomposition diagram of the second level of the “Determination of energy value of products” process:
Figure 19 shows the decomposition diagram of the third level of the process “Get data on the barcode”:
Figure 20 shows the decomposition diagram of the third level of the “Involve the user in recognition” process:
Save the entered data Recognition relationship
Recognized elements Update dataset
Figure 21 shows the decomposition diagram of the third level of the process “Match foods calories”: Data on caloric content of products
Recognized elements
Total calories Generate total calories Unknown products Bring the data to a normalized form
Calculation of caloric content of the product Get the caloric content of foods that are not in the database
Caloric content of new products
Figure 22 shows the decomposition diagram of the third process level “Get calories from the barcode”. Figure 23 shows the decomposition diagram of the third level of the “Update History” process. Figure 24 shows the decomposition diagram of the third level of the process “Update neural network dataset”.
Calculation of caloric content of the product Input object model transformation
Transfer dto
Create a new dataset
Formed dataset
Update information
Figure 25 shows the decomposition diagram of the third level of the “Create a popularity rating” process:
Figure 26 presents a diagram of the decomposition of the third level of the “Generate advice on change” process.
Figure 27 shows the decomposition diagram of the third level of the “Filtration according to preferences” process: Formation of the result
Dietary improvement data
Dietary improvement data User features
Bringing history to standard forms
Modified diet
Determining the best for the current period
Consumption recommendation
Based on IDEF0, we will build a hierarchy of processes that affect the system’s performance. In the All Fusion Process Modeller environment, we will generate a prototype of the already created order. The result is shown in Figure 28:
Recommended system for monitoring the energy value of food products Product recognition 1. Recognize the object 2. Break the dish into ingredients 3. Get barcode data 4. Engage the user in recognition
To understand how external forces affect our system, we will conduct a PEST analysis: The results are presented in the form of a matrix and are shown in table 2. We describe the information system as an ordered sequence of events with the simultaneous placement of objects directly related to the modelling process. The decomposition diagram is given in Figure 29.
Thus, the topic’s relevance is confirmed by the massive demand for similar systems of similar scale in the market of mobile applications [67-78]. The number approves of downloads on different platforms. The subject area was carefully researched, and ontologies were created following the standard of ontological research IDEF5 (Ontology Description Capture). The system’s primary goal is established, and its decomposition is carried out at different hierarchical levels. The method based on the IDEF0 hierarchy is specified. Within this structural methodology, there is a context diagram and its decomposition into three groups of scale. Based on the order, the structural content of the existing processes of the system is shown. The PEST analysis identified aspects of the external environment that affect the developed system. In Fig. 30 shows a block diagram of the methodology used in this study. First, the process of recognizing food images is based on a model for which the Food-500 dataset was used, which has 500 categories of dishes and 399,726 images. The image of the dish is processed by the method for determining the dish. To recognize a dish, it is necessary to decompose the image, which includes the detection of descriptors of recognized foods. Once the food names and descriptors are identified, the recognized products are ready for the next steps of the system algorithm. Thus, the end result is a product image, standardized with unique descriptors, which will count calories and output the energy content of the dish to the end user. To teach the deep learning model to recognize food, we first needed to pre-process the image manually. The main purpose of the pre-processing stage is to create an image of the product with the appropriate caption. The result of this step will be needed for the controlled learning of the deep learning model. Usually the simplest approach to creating such images is to label each image with one food class (food name) and teach the deep learning model so that it returns the product name as a result of providing the image to the ingredient as an input parameter. Because products often overlap on plates, and drinks can interfere with other items, we label each product at the pixel level. This means that the result of this step is a new image with the same width and height as the input image, with only one channel and not the three channels used in RGB images. This label image contains the class prediction for each individual pixel. Because creating such images without significant errors is a troublesome, non-trivial task, it is one of the main obstacles when trying to develop a solution for pixel classification.
We trained the model to recognize food using deep convolutional neural networks that work similar to human vision. In particular, we used fully convolutional networks (FCN) for adaptive learning based on research [79-93] and represent the current level of semantic segmentation. This process segments the input image into individual parts and then classifies each part into an output class; the network does this by performing a pixel-level classification. Therefore, the FCN outputs a pixel map instead of a class text label, and this pixel map contains model predictions for each individual pixel of the input image, not just one prediction for the entire image. This is important because it is the most accurate way to describe all the foods presented in one image. It took about 35 hours to learn the deep learning model. However, the classification of a single image takes much less time and computing power, making it possible to use deep learning models even in mobile applications. Once the training was completed the model was run once on a subset of the test. The essence of the developed model is to identify and recognize specific characteristics in images to apply them to a specific area. Visually, the process of recognizing the products of the system can be seen in Fig. 31. The figure clearly shows how the model distinguishes the object under study in the image from extraneous noise.
In order to measure the effectiveness of the learned model of deep learning, we used formulas that often serve as a guide to quality for fully convoluted semantic segmentation networks. Measurements are based on standard pixel accuracy and cross-sectional (IU) measurements, including the following: ℎ ∑ ∑ = 1 ∑
∑ = (∑ )−1 ∑
( + ∑ − )
( + ∑ − ) (1) (2) (3) (4) where ncl is the number of different classes of the trained model, nlj is the number of pixels of the class i for which the possibility of belonging to the class is provided j, ti is the total number of pixels of the class і [79-93].
The learning results of the model are shown in Table 3:
As expected, the model's performance is better on the training subset than on the other two subsets. However, the difference is not significant, which means that the model has mastered features that are well generalized. It is important to note that this performance was measured for all classes - this includes different times of the day (day and night) and different backgrounds on which the image of the dish is found. Because the subset of testing contains images new to the deep learning model, we consider the results of this subset to be the most representative of actual performance. From these results, we chose pixel accuracy as the final quality measure, as this measure is similar to the accuracy of classification in traditional convolutional neural networks, which classify the whole image into one class. The difference is that instead of calculating the accuracy at the image level, it is calculated at the pixel level. As can be seen from Table 3, the accuracy of the model of deep learning is at a high level, which indicates a willingness to implement this model in real conditions.
The software product is designed based on three levels of architecture, which consist of the following levels - presentation, business logic and database access. Figure 32 shows a visualisation of the system architecture, the relationships between the layers and the related technologies for implementation.
View Level Business logic level Data access level
Database
It is worth noting that the sequence of layers is appropriate and constant. The level of representation “knows” only about the existence of business logic. In turn, business logic interacts with the level of expression and access to the database. The database access level brings the results to specific objects and sends the result to a higher level using the scripting language and the specified connection. The advantages of this architectural solution are: • Code maintenance and support is each level is independent, so changes in one class do not lead to changes in another; • Flexibility is each level can be managed and scaled separately; • Reuse is independence allows you to use levels within different systems; • Faster development process is various specialists can deal with specific areas: front-end specialists is representation group. Back-end engineers - the introduction of business logic; database administrators is develop a database model.
The work of architecture is the following way. The user performs a set of operations that lead to a request to the server. Within the presentation level, projects are exchanged via the REST interface. A request comes from the mobile device and goes to the appropriate controller. The proper method of the controller calls the object from the level of business logic and instructs it to perform tasks. The business logic layer calls an object from the database access level to connect to the database to retrieve data. The last group forms the language of the scripts that are executed for a given database. After processing the data at each level (top-down), the response is returned to the controller. The result of the processing is sent using the HTTP protocol with the corresponding status code on the development of the operation. The information is provided to the user to process the body of the response that the device received from the server part. Let’s start with a description of the database directly related to the lowest level and is the central place to store data within the system. The database satisfies the 3NF requirements and consists of the following tables: Users, Roles, Genders, Meals, MealTimes, Ingredients and Measures. The scheme is shown in Figure 33:
The following relationships have been established between entities: • between Roles and Users - 1: N, as many users can register with a shared role; • between Genders and Users - 1: N, as one gender can belong to many users; • between Users and Meals - 1: N, because one user can create many unique dishes that differ in image, UPC, consumption time, etc.; • between MealTimes and Meals - 1: N, because, for example, many dishes can be breakfast/lunch/dinner; • between Ingredients and Meals - N: N (with intermediate table MealIngredients), because one ingredient can be in several dishes and vice versa - one dish can consist of many components; • between Ingredients and Measures - 1: N, which is true: one ingredient can be measured in grams, tablespoons, etc.
Database-level validation was created during development, which will not allow you to create a tuple if the required field is empty or the length of the area exceeds the allowable limits. Figure 34 shows an example of conditions that ensure efficient use of the system. Validation measures of this type help to reduce the size of the database and get rid of empty entries shown in the table at a distance.
The database is directly related to the level of access to the data. This level has a connection that allows you to connect to a database server that will run scripts in the future.
Because this system is highly dependent on the database, Dapper was chosen as the main ORM. The implementation of this package will allow you to directly write scripts in the code that the database server will process. The business logic level performs the basic operations within the system: saving files, bringing objects to a specific look, sending requests to open APIs, etc. There are two types of interlayer objects within a system: transport objects and model representations. Transport objects are identified with a specific database model, and model representation is a collection of many models and is an alternative to using temporary tables in the SQL standard.
The presentation level consists of a project in which the code for the user interface in mobile platforms is written and a project that receives requests from the previous one.
An example of a query with the return of a successful result is shown in Figure 35. Using such a powerful tool as Redux has created a single repository for the entire application. Applying this approach allows you not to send a request if we have done so before. First, we speed up the system and optimise the mobile device’s resources because we do not create new variables. The basic cycle of Redux is shown in Figure 36. Components are the building blocks of our system that send specific signals to the repository. Manipulations within the components cause specific actions that change the state in the storage and update it accordingly. A store is an object that contains all the states at a particular point in time. Repository registration occurs in the root, the main component, the first to run in the system. You can access the state using the getState () function. You can update the status using the dispatch () function. Figure 37 shows the system repository. Figure 38 shows the state of the history of consumption. It features the unique values of each product. The general state tree is given in Figure 39. Figure 38System storage states
For full use of the system, it is necessary to get acquainted with the technical characteristics and functionality of the program. The user manual helps in this context. The name of the program is “PocketDietCoach”. The logo is shown in Figure 38:
The program is written using the C # programming language and JavaScript. The SQL standard was used to write scripts to interact with the database. The following development environments were used during the development: Visual Studio 2019 and Visual Studio Code. Microsoft SQL Management Studio 2017 was used during the development. The Git version control system was used using Git Bash. The program is designed to control the daily diet and automatic calorie counting. This system’s focus suggests that its implementation is possible in various spheres of life. As for the system’s capabilities, you should make a list of needs that it can meet for the user. The main functions include recognition of products and QR-codes, determination of energy value, automatic calculation of caloric content, control over consumption, formation of a consumer diary, and tracking history. Additional features of the program are sending notifications about missed meals and recommendations for improving the diet.
The system solves the problem of determining the dish and its content, highlighting the exposure and recognising the dish in the image, calculating the energy value and recording processed information in the user profile, notification of missed meals and recommendations on consumption time.
One of the initial methods that help solve the tasks is authentication and authorisation methods. These procedures determine whether a user can log in and perform certain operations. Successful authentication results are in a temporary token that is active for 40 minutes. It means that it quickly becomes inactive, and the system updates it. It allows the system to make sure that requests are sent by a specific user, not by an attacker who changed the encrypted part of the key. The following method is to get statistics for the current date. A successful result allows you to process data and display information on the user’s screen. This method is parameterised, so navigating the home screen causes a new request to be sent with a variable date setting. An essential method for organising processes for the system is the formation of statistics related to nutrition. The processed information is submitted to the relevant sections, which can be further defined in the user profile.
In terms of temporal characteristics, it recognises a product such as pizza is approximately 5s. However, this time dependence depends on the speed of the Internet so that it may differ on the devices. Compared to analogues, this time is more than satisfactory for product recognition and calorie counting.
Currently, there is only one mode of operation in the program - custom. At this stage, the possibility of maintaining global statistics and creating an administrative method for closed users is being developed. The means of control is the freely available Clarifai API, which can also recognise dishes. If the local neural network could not match the received image, the unrecognised element will be sent along the described route. NutritionxAPI is used to recognise the barcode.
The limitation of the program is the presence of a device with Android / iOS version 8 and higher. The device must be equipped with a camera module and access the Internet for the whole operation. The system requires access to these device parts to comply with all usage rules during the first startup.
Information on functional limitations for use. A smartphone, camera module, and Internet access are required to use the system in full release mode fully. Requests can be sent via Wi-Fi or cellular data. The main condition is the optimal data rate. Also, an important condition is to use the Android or iOS operating system on your mobile device. Features for operating system versions are as follows: for Android, the Android SDK version must be at least 8.0. For iOS devices, the system will run successfully on iOS version 11 and above. You need to compile the project and get its build DLL in local development mode to run. Run the ASP.NET CORE project regardless of the launch port. The next step is to run React Native on your mobile device. The mobile project is launched using npm commands in the developer console. It must be installed on a user’s mobile device to access the application. The installation process meets the conditions of the platform community, which is confirmed by permission to use the camera and the Internet connection. Step-by-step instructions. To start, you need to log in to the PocketDietCoach application, shown in Figure 39a. Logging in the system in Figure 39b. Here you need to enter the login and password of the created account. The main page is shown in Figure 39с. The navigation menu is shown in Figure 40a. The photo mode is shown in Figure 40b. Activity statistics are presented in Figure 40c:
a) b) c) Figure 41: a) Image of the system on the home screen, b) Logging in, and b). Main page a) b) c) Figure 42: а) Navigation menu, b) photo mode and c) activity statistics
Consider the efficiency of the system and the compliance of the task to the results of work.
After the user downloads the application, he sees the page shown in Figure 41a. This page appears when the token has expired or does not exist. After a successful login process, the next login will redirect the user to the home page. The user can go to the registration page or login with a pre-created account on this screen. The user must click on the “Create Account” button and go through the account creation procedure to complete the registration process. Figure 41b user is shown to enter gender and body parameters: age, height, and weight. These parameters in the system allow you to determine how many calories the body needs. After entering the required parameters, the user must fill out the registration form, which is shown in Figure 41c. You need to fill in all available fields and enter your login, e-mail and password.
The order of registration is accompanied by two stages: introducing body parameters and creating a profile. If the user has created a profile in the system, he goes to the Login page, shown in Figure 42. The system prompts users to enter their e-mail and the corresponding password. If the response is successful, the user will be able to go to the home page.
а) b) c) Figure 44: a) Login page, b) validation message in case of blank fields and c) validation message in case of inconsistency of entered data
In case when the user has not filled in all the fields or entered a login/password that does not correspond to reality, the user will see the appropriate messages that will notify him of non-compliance with the validation conditions. In Figure 42b and Figure 42c. Examples of such messages are shown above. The main page is shown in Fig. 43a. The page is conditionally divided into two parts. The first contains information about the calories consumed, and the second is the dishes that the user consumed. The current date and the left and right navigation buttons are indicated at the top of the screen. They are responsible for moving by date and downloading the necessary information to the home screen accordingly. The starting point from which you can view the consumption history is the user’s registration date. The calorie ring clearly shows how many calories are left to consume to maintain the body’s energy balance.
More detailed information is given under the ring. It contains data on how many calories a user should consume, how many he consumes during the day and how many are left. The formation of the daily norm parameter is based on the user’s age, weight, and height. The home page also displays the products the user consumed during the day. The ability to add a dish or product is made possible by a special button. The “plus” sign switches the user to the mode of use of the camera in which you can point to the product and take pictures. Access to the camera is shown in Fig. 43b. After taking a photo, the system displays the image taken and asks us to wait a few seconds to process the information. During this time, the system recognises the elements in the picture, determines the caloric content and records the data in the database.
In case of successful recognition, the main menu is updated, and the information about calories consumed changes. The obtained results are added, and the development is displayed whether you have finished insufficient or exceeded. The photo taken is attached to the specific time range you ate the food and is supported by the dish’s name. The recognition result is shown in Figure 44a. Tips are also integrated into this system based on previous experience. For example, Figure 44b shows a message indicating that the user has not consumed food to date. Therefore, the plan encourages him to fix this because the user usually eats breakfast at this point.
Another page is the user profile, which is shown in Figure 44c. Initially, the information in the profile is filled with the data that were entered during registration. You can change them by clicking the appropriate option in the upper right corner. The user’s page lists the main parameters: name, age, country, weight and height. In addition, the profile shows user activity. The activity is represented by an activity schedule that indicates the number of actions during a given day. A graph is an array that consists of an object that contains the date and number of contributions for a given day. Accordingly, the richer the square, the more activity the user has spent during a specific date. Particular attention is paid to the analysis of consumed substances and dishes, so the information in diagrams is presented in the user’s diary. Figure 45a shows a line graph showing the number of calories consumed in the last seven days. The abscissa shows the days of the week, and the ordinate indicates the number of calories. It allows you to track a trend and draw conclusions about the improvement of the diet.
а) b) Figure 47: a) Graph of calories consumed and the percentage between the amount of food consumed and time and b) Bar chart of the average number of calories consumed and a pie chart of the most popular dishes
Also Figure 45a is a circle chart showing the relationship between the amount of food consumed and time. The system has four-time ranges, so the circles are also four. For example, breakfast should have the highest percentage under ideal conditions, and dinner before bedtime - the lowest. This chart allows you to determine the meal that covers the most calories and draw conclusions. With the data from the two charts above, you can choose the trend of calories consumed daily and over time.
In Figure 45b shows a bar chart showing the average number of calories consumed during the day over the last four months. The bar chart allows you to monitor progress and understand whether the user is moving in the direction of weight loss or increasing it. Figure 45b shows a pie chart showing the most popular dishes or foods during the last week and their share in the user’s daily diet.
The current market situation is analysed, and the need to create a system that will help automate the process of counting calories based on the recognition of ingredients contained in the dish’s image. The relevance of the topic is confirmed by the massive demand for similar systems of similar scale in the market of mobile applications. The number approves of downloads on different platforms. The advantages and disadvantages of alternative solutions are summarised in the table and summarise the result. Various architectural solutions and algorithms for neural network training methods are analysed to design the system. The efficiency of their implementation in different areas is compared. The appropriate form is selected, and the optimal initial data set is set. All information was analysed with the help of scientific articles, thematic literature, etc. The result of the system analysis is the creation of a goal tree based on an integrated approach. The system’s primary goal is established, and its decomposition is carried out at different hierarchical levels. The method based on the IDEF0 hierarchy is specified. Within this structural methodology, there is a context diagram and its decomposition into three groups of order. Based on the hierarchy, the structural content of the existing processes of the system is shown. The result of successful performance of work is the creation of a system that satisfies all tasks set: recognition of products, automation of the process of counting calories, formation of councils and sending of notifications. All results are supported in the software product description and user manual. The user manual describes the appropriate use of the system given the necessary conditions for a startup. The system’s functionality is supported by images that show the sequence of using the system step by step.
[1] O. Kryvoruchko, K. Khorolska, V. Chubaievskyi, Usage of neural networks in image recognition,
Foreign trade: economics, finance, law 3 (2019) 83–92. [2] Y. V. Bodyanskyy, A. O. Deyneko, Z. V. Deyneko, M. O. Shalamov, Adaptyvne navchannya neyronnoyi merezhi opornykh vektoriv naymenshykh kvadrativ, Informatsiyno-keruyuchi systemy na zaliznychnomu transporti [Adaptive learning of the neural network of support vectors of least squares, Information and control systems in railway transport] 2 (2015) 71–74. [3] S. I. Alpert, Suchasni kryteriyi otsinky tochnosti klasyfikatsiyi zobrazhen [Modern criteria for assessing the accuracy of image classification], Matematychni mashyny i systemy [Mathematical machines and systems] 4 (2013) 187–197. [4] V. K. Berehovyy, Osnovy naukovoyi orhanizatsiyi zdorovoho kharchuvannya [The basis of scientific orhanizatsii healthy eating], Efektyvna ekonomika [Efficient economy], 2011. URL: http://nbuv.gov.ua/UJRN/efek_2011_11_19. [5] S. V. Bilashenko, N. N. Shapovalova, O. H. Rybalchenko, Rozpiznavannya zobrazhen za dopomohoyu z·hortkovykh neyronnykh merezh z vykorystannyam biblioteky Keras [Image recognition with the help of cortical neural networks using the Keras library], Hirnychyy visnyk [Mining Bulletin] 10 (2018) 148–154. [6] Y. V. Bodyanskyy, N. O. Teslenko, A. O. Deyneko, Evolyutsiyna neyronna merezha z yadernymy funktsiyamy aktyvatsiyi y adaptyvnyy alhorytm yiyi navchannya [Evolutionary neural network with nuclear functions of activations and adaptive algorithm of its training], Naukovi pratsi Chornomorsʹkoho derzhavnoho universytetu imeni Petra Mohyly: Kompʺyuterni tekhnolohiyi [Scientific works of the Petro Mohyla Black Sea State University. Computer technology] 160(148) (2011) 53–58. [7] K. Y. Bortnyk, O. V. Olshevskyy, A. L. Kyrylyuk, Mashynne navchannya, yak osnova dlya rozvytku tekhnolohiy maybutnoho [Machine learning as a basis for the development of technologies of the future], Kompʺyuterno-intehrovani tekhnolohiyi: osvita, nauka, vyrobnytstvo [Computer-integrated technologies: education, science, production] 27 (2017) 85–88. [8] V. M. Brodkevych, V. Y. Remeslo, Alhorytmy mashynnoho navchannya ta hlybokoho navchannya (HN) i yikh vykorystannya v prykladnykh dodatkakh [Algorithms of machine learning and deep learning (H) and their use in application applications], Mizhnarodnyy naukovyy zhurnal «Internauka» [International Scientific Journal "Internauka"] 11(1) (2018) 56–60. [9] I. V. Butyrska, A. V. Manhul, Tekhnolohiya QR-kodu yak instrument pidvyshchennya efektyvnosti funktsionuvannya servisnykh system [NR-code technology as a tool to increase the efficiency of service systems], Visnyk Chernivetsʹkoho torhovelʹno-ekonomichnoho instytutu: Ekonomichni nauky [Bulletin of the Chernivtsi Trade and Economic Institute: Economic sciences] 1 (2015) 165–171. [10] V. V. Vitlinskyy, Shtuchnyy intelekt u systemi pryynyattya rishen [Artificial intelligence in the decision-making system], Neyro-nechitki tekhnolohiyi modelyuvannya v ekonomitsi [Neurofuzzy modeling technologies in economics] 1 (2012) 97–118. [11] G. Y. Voloshin, Metody raspoznavaniya obrazov [Methods of pattern recognition], VGUES, 2000. [12] S. Hadetska, V. Horokhovatskyy, Metody strukturnoyi klasyfikatsiyi zobrazhen na zasadakh bayesovskoyi teoriyi pryynyattya rishen [Methods of structural classification of images on the basis of Bayesian theory of decision making], Radio electronics, computer science, management 2(2018) 90–97. [13] I. Hamanyuk, Variant zastosuvannya bayyesivskykh metodiv dlya mashynnoho navchannya shtuchnoho intelektu systemy pidtrymky pryynyattya rishen u borotbi zi spamom [Option of application of Bayesian methods for machine learning of artificial intelligence system support decision-making in the fight against spam], Zvyazok [Communication] 6 (2018) 14–18. [14] N. V. Hlukhova, L. A. Pisotska, Rozrobka metodu analizu kolorovykh zobrazhen hazorozryadnoho vyprominyuvannya [Development of a method for analysis of color images of gas-discharge radiation], Systemy upravlinnya, navihatsiyi ta zvyazku [Management, navigation and communication systems] 2 (2018) 59–62. [15] N. V. Hlukhova, L. A. Pisotska, N. H. Kuchuk, Rozrobka systemy ekspres-klasyfikatsiyi vody na osnovi bazy danykh zobrazhen [Development of an express water classification system based on an image database] Zbirnyk naukovykh prats Kharkivskoho universytetu Povitryanykh Syl [Collection of scientific works of Kharkiv University of the Air Force] 3 (2015) 112–118. [16] L. Hlushchenko, Perspektyvy vykorystannya shtuchnoho intelektu dlya rozrobky funktsionalnomu kharchuvanni [Prospects for the use of artificial intelligence to develop functional nutrition] Visnyk Lvivskoho universytetu: Seriya biolohichna [Bulletin of Lviv University: Biological series] 73 (2016) 437. [17] V. A. Gorokhovatskiy, D. V. Pupchenko, Klassifikatsiya izobrazheniy vizualnykh obyektov po mnozhestvu deskriptorov osobennykh tochek na osnove neyronnoy seti Kokhonena [Classification of imaged visual objects by many singular point descriptors based on the Kohonen neural network], Sistemi upravlínnya, navigatsi ta zvyazku [Management systems, navigation and communication], 2 (2018) 68–72. [18] O. Hryhorenko, Naukovi pidkhody do formuvannya ratsioniv kharchuvannya [Scientific approaches to the formation of diets], Prohresyvni tekhnika ta tekhnolohiyi kharchovykh vyrobnytstv restorannoho hospodarstva i torhivli [Advanced techniques and technologies of food production, restaurant business and trade] 2 (2009) 210–218. [19] D. V. Hrynov, Metod rozpiznavannya zobrazhen obyektiv zasobamy vydovoyi rozvidky [Method of image recognition of objects by means of species reconnaissance], Systemy ozbroyennya i viyskova tekhnika [Weapons systems and military equipment] 4 (2007) 72–75. [20] V. V. Hrytsyk, Metod obrobky skladnykh zobrazhen ta yikh rozpiznavannya [Method of processing complex images and their recognition], Dopovidi Natsionalnoyi akademiyi nauk Ukrayiny [Reports of the National Academy of Sciences of Ukraine] 1 (2011) 28–32. [21] V. Hrytsyk, Obrobka skladnykh zobrazhen ta yikh rozpiznavannya v informatsiyno-analitychnykh systemakh kompyuternoho zoru [Processing of complex images and their recognition in information-analytical systems of computer vision], Dopovidi Natsionalnoyi akademiyi nauk Ukrayiny [Reports of the National Academy of Sciences of Ukraine] 7 (2009) 36–41. [22] I. H. Danylyuk, Aktualni problemy metodu hlybynnoho navchannya [Actual problems of the method of deep learning], Linhvistychni studiyi [Linguistic studies] 35 (2018) 155-158. [23] A. S. Dovbysh, H. A. Stadnyk, Informatsiyno-ekstremalnyy alhorytm navchannya systemy pidtrymky pryynyattya rishen z hipertsylindroyidnym klasyfikatorom [Informatsion-extreme algorithm for learning a system supporting decision-making with a hypercylindroid classifier], Radioelektronni i kompyuterni systemy [Radio electronic and computer systems] 3 (2015) 11–18. [24] A. S. Dovbysh, I. V. Shelekhov, Osnovy teoriyi rozpiznavannya obraziv [Fundamentals of image recognition theory], Sumy: Sumy State University, 2015. [25] I. Domanetska, O. Fedusenko, V. Khrolenko, Neyromerezhevi tekhnolohyi opratsyuvannya zobrazhen v adaptyvnykh systemakh navchannya [Neural network imaging technologies in adaptive learning systems], Shtuchnyy intelekt [Artificial Intelligence], 3 (2017) 24–31. [26] V. O. Drofa, T. M. Yefimenko, Informatsiyno-ekstremalnyy alhorytm rozpiznavannya nestatsionarnykh za yaskravistyu zobrazhen [Informatsion-extreme algorithm for recognizing nonstationary brightness images], Byonyka yntellekta [Bionics of intelligence] 2 (2015) 100–104. [27] D. O. Klymenko, O. A. Rudenko, Veb-dodatok dlya servisu skladannya ratsionu zdorovoho kharchuvannya ta dostavky produktiv [Web application for the service of compiling a healthy diet and food delivery], Systemy upravlinnya, navihatsiyi ta zvyazku [Management, navigation and communication systems] 2 (2019) 103–109. [28] V. H. Krasylenko, R. O. Yatskovska, V. I. Yatskovskyy, Modelyuvannya metodiv rozpiznavannya ta klasyfikatsiyi frahmentiv kolorovykh zobrazhen zemel silskohospodarskoho pryznachennya pry yikh dystantsiynomu monitorynhu [Modeling methods of recognition and classification of fragments of color images of agricultural lands in their remote monitoring], Systemy obrobky informatsiyi [Information processing systems] 5 (2017) 55–61. [29] T. V. Lendyuk, Modelyuvannya kompyuternoho adaptyvnoho navchannya i testuvannya [Modeling of computer adaptive learning and testing], Pratsi Odeskoho politekhnichnoho universytetu [Proceedings of Odessa Polytechnic University] 1 (2013) 110–115. [30] O. B. Leshchenko, A. S. Khlyupina, D. O. Bohdan, Veb-dodatok dlya vedennya shchodennyka kharchuvannya ta trenuvannya: vymohy, rozroblennya i vprovadzhennya [Web application for keeping a diary of nutrition and training: requirements, development and implementation], Radioelektronni i kompyuterni systemy [Radio electronic and computer systems] 3 (2018) 49–62. [31] H. V. Makarova, Stvorennya mobilnoho dodatku dlya optymizatsiyi vahy ta kharchuvannya lyudyny [Creating a mobile application for optimizing weight and nutrition], Systemy obrobky informatsiyi [Information processing systems] 2 (2017) 187–191. [32] L. M. Malyarets, I. Y. Ryzhykh, Zastosuvannya QR-rozkladu pryamokutnykh matryts Khauskholderovymy vidobrazhennyamy v rehresiynomu analizi [Application of QRdecomposition of rectangular matrices by Hausholder mappings in regression analysis], Ekonomika rozvytku [Development economics] 1 (2009). 16–20. [33] N. O. Matvyeyeva, Doslidzhennya zasobiv mashynnoho navchannya iz zaluchennyam suchasnykh mov prohramuvannya [Research of machine learning tools with the involvement of modern programming languages], Systemni tekhnolohiyi [System technologies] 1 (2018) 85–91. [34] L. Mestetskiy, Matematicheskiye metody raspoznavaniya obrazov [Mathematical methods of pattern recognition], MGU, 2004. [35] O. P. Mintser, Obriyi rozvytku adaptyvnoho navchannya [Horizons of development of adaptive learning], Medychna informatyka ta inzheneriya [Medical informatics and engineering] 1 (2017) 5–11. [36] O. A. Mokrintsev, Poperednya obrobka zobrazhen dlya avtomatychnoho rozpiznavannya odnovymirnykh shtrykhkodiv [Image pre-processing for automatic recognition of one-dimensional barcodes], Systemy upravlinnya, navihatsiyi ta zvyazku [Management, navigation and communication systems] 1 (2017) 111–113. [37] O. Y. Moroz, Shtuchnyy intelekt versus pryrodnyy intelekt? (maybutnye lyudyny v konteksti vyklykiv intelektualnykh supertekhnolohiy) [Artificial intelligence versus natural intelligence? (the future of man in the context of the challenges of intelligent supertechnology)], Politolohichnyy visnyk [Political Science Bulletin] 72 (2014) 18–35. [38] T. L. Mostenska, H. O. Kundyeyeva, Kharchuvannya yak skladova prodovolchoyi bezpeky [Nutrition as a component of food security], Naukovi pratsi Natsionalnoho universytetu kharchovykh tekhnolohiy [Scientific works of the National University of Food Technologies] 22(3) (2016) 11- 3–122. [39] Y. Motuzka, Kharchova ta enerhetychna tsinnist produktiv dlya spetsialnykh medychnykh tsiley [Food and energy cyanide products for special medical purposes], Tovary i rynky [Goods and markets] 2(1) (2017) 59–66. [40] M. Oldenderfer, R. K. Bleshfild, Klasternyy analiz. Faktornyy, diskriminantnyy i klasternyy analiz [Cluster analysis. Factor, discriminant and cluster analysis], Finance and statistics, 1989. [41] V. O. Pishvanova, Pryntsypy adaptyvnoho navchannya [Principles of adaptive learning], Visnyk Zaporizkoho natsionalnoho universytetu: Pedahohichni nauky [Bulletin of Zaporizhzhya National University: Pedagogical sciences] 1 (2015) 178–183. [42] V. S. Simakov, Y. V. Lutsenko, Adaptivnoye upravleniye slozhnymi sistemami na osnove teorii raspoznavaniya obrazov [Adaptive control of complex systems based on the theory of pattern recognition], KGTU, 1999. [43] V. V. Slastin, Ye. S. Samuseva, L. V. Moskalchuk, Sbalansirovannyy ratsion pitaniya [A balanced diet], Problemi kharchuvannya [Nutrition problems] 1 (2014) 33–39. [44] I. Tereykovskyy, Neyromerezheva metodolohiya rozpiznavannya Internet-oriyentovanoho shkidlyvoho prohramnoho zabezpechennya [Neural network methodology for recognizing Internet-oriented malware], Bezpeka informatsiyi [Information security] 19(1) (2013) 24–28. [45] T. V. Trachuk, T. P. Radishchuk, Metod matematychnoho modelyuvannya yak zasib realizatsiyi shchodennoho ratsionu [The method of mathematical modeling as a means of implementing a daily diet], Pedahohichnyy poshuk [Pedagogical search] 1 (2014) 34–36. [46] Y. O. Tupalo, Vykorystannya metodiv mashynnoho navchannya na praktytsi [Use of machine learning methods in practice], Kompyuterni zasoby, merezhi ta systemy [Computer tools, networks and systems] 17 (2018) 101–110. [47] O. V. Fedusenko, A. O. Fedusenko, I. M. Domanetska, Kontseptualna model adaptyvnoyi informatsiynoyi systemy navchannya [Conceptual model of adaptive information learning system], Upravlinnya rozvytkom skladnykh system [Management of complex systems development] 32 (2017) 86–90. [48] Y. Y. Yaninovych, H. V. Kachay, T. M. Shvets, Enerhetychna tsinnist produktiv [Energy value of products], Rybohospodarska nauka Ukrayiny [Fisheries science of Ukraine] 2 (2011) 122–126. [49] K. R. Muller, S. Mika, G. Ratsch, et al., An introduction to kernelbased learning algorithms, in:
Proceedings of the IEEE 108 Transactions on Neural Networks, 2001, pp. 181–202. [50] . B. T. Al-Janabi, Computer vision system for froth flotation based on centroid, Системи обробки інформації [Information processing systems] 9 (2014) 3–5. [51] N. Bolohova, I. Ruban, Image processing models and methods research and ways of improving marker recognition technologies in added reality systems, Suchasnyy stan naukovykh doslidzhenʹ ta tekhnolohiy v promyslovosti [The current state of research and technology in industry] 1 (2019) 25–33. [52] A. S. Dovbysh, D. V. Velykodnyi, O. B. Protsenko, V. I. Zimovets, Optimization of parameters of machine learning of the system of functional diagnostics of the electric drive of a shaft lifting machine, Radio electronics, computer science, management 2 (2018) 44–50. [53] R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification. John Wiley & Sons, New York, 2001. [54] J. Han, M. Kamber, J. Pei, Data mining: concepts and techniques. Elsevier, 2012. [55] S. Melnychuk, V. Gubarev, N. Salnikov, Using Information Features in Computer Vision for 3d Pose Estimation in Space, Kibernetika i vychislitel'naya tekhnika [Cybernetics and Computer Science] 190 (2017) 33–55. [56] I. Motuzka, D. Antiushko, Сlassification of products for enteral nutrition, Tovary i rynky [Goods and markets] 2 (2015) 17–24. [57] M. G. Pantelyat, Application of the finite element nethod to computer simulation of electromagnetic and thermal processes in induction cookers and heated dishes, Visnyk Cherkasʹkoho universytetu. Seriya: Fizyko-matematychni nauky [Bulletin of Cherkasy University.
Series: Physical and Mathematical Sciences] 1 (2017) 79–85. [58] R. J. Shalkoff, J. Digital image processing and computer vision. New York; Chichester; Brisbane;
Toronto; Singapore: John Wiley & Sons, 1989. [59] V. N. Vapnik, The nature of statistical learning theory. Springer, 1999. [60] E. Ziglio, The WHO cross-national study of health behavior in school aged children from 35 countries. New York; Chichester; Brisbane; Toronto; Singapore: J. School Health, 2001. [61] Y. Burov, V. Vysotska, P. Kravets, Ontological approach to plot analysis and modeling, CEUR
Workshop Proceedings Vol-2362 (2019) 22–31. [62] Lytvyn Vasyl, Vysotska Victoria, Dosyn Dmytro, Holoschuk Roman, Rybchak Zoriana, Application of Sentence Parsing for Determining Keywords in Ukrainian Texts, in: Proceedings of the International Conference on Computer Sciences and Information Technologies, CSIT, 2017, pp. 326–331. doi: 10.1109/STC-CSIT.2017.8098797. [63] V. Lytvyn, P. Pukach, І. Bobyk, V. Vysotska, The method of formation of the status of personality understanding based on the content analysis, Eastern-European Journal of Enterprise Technologies 5/2(83) (2016) 4-12. doi: 10.15587/1729-4061.2016.77174. [64] V. Vysotska, Linguistic Analysis of Textual Commercial Content for Information Resources Processing, in: Proceedings of the International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science, TCSET, 2016, pp. 709-713. doi: 10.1109/TCSET.2016.7452160. [65] V. Lytvyn, V. Vysotska, P. Pukach, M. Vovk, D. Ugryn, Method of functioning of intelligent agents, designed to solve action planning problems based on ontological approach, EasternEuropean Journal of Enterprise Technologies 3/2(87) (2017) 11-17. doi: 10.15587/17294061.2017.103630. [66] V. Lytvyn, N. Sharonova, T. Hamon, V. Vysotska, N. Grabar, A. Kowalska-Styczen,
Computational linguistics and intelligent systems, CEUR Workshop Proceedings 2136 (2018). [67] N. Oleksiv, O. Oborska, K. Mykich, S. Mushasta, Y. Pukach, O. Tereshchuk, Information System of Dynamic and Adaptive Control of Human Diet Based on Machine Learning Technology, CEUR Workshop Proceedings Vol-2917 (2021) 373-406. [68] T. Zohud, S. Zein. Cross-platform mobile app development in industry: A multiple case-study,
International Journal of Computing 20(1) (2021) 46–54. [69] M. R. Baharon, M. F. Abdollah, N. A. Abu, Z. Z. Abidin, A. Idris, Secure video transcoding in mobile cloud computing, International Journal of Computing 17(4) (2018) 208–218. [70] V. Inzillo, F. D. Rango, A. A. Quintana. A low energy consumption smart antenna adaptive array system for mobile ad hoc networks, International Journal of Computing 16(3) (2017) 124–132. [71] B. Rusyn, L. Pohreliuk, A. Rzheuskyi, R. Kubik, Y. Ryshkovets, L. Chyrun, S. Chyrun, A.
Vysotskyi, V.B. Fernandes, The mobile application development based on online music library for socializing in the world of bard songs and scouts’ bonfires, Advances in Intelligent Systems and Computing 1080 (2020) 734–756. doi: 10.1007/978-3-030-33695-0_49. [72] A. Rzheuskyi, A. Gozhyj, A. Stefanchuk, O. Oborska, L. Chyrun, O. Lozynska, K. Mykich, T.
Basyuk, Development of Mobile Application for Choreographic Productions Creation and Visualization, CEUR Workshop ProceedingsVol-2386 (2019) 340-358. [73] T. Teslyuk, V. Teslyuk, H. Lypak, N. Kunanets, N. Veretennikova, A Mobile Museum Guide
Application, CEUR Workshop Proceedings Vol-2631 (2020) 314-326. [74] O. Artemenko, V. Pasichnyk, N. Kunanets, K. Shunevych, Using sentiment text analysis of user reviews in social media for e-tourism mobile recommender systems, CEUR workshop proceedings Vol-2604 (2020) 259-271. [75] V. Lytvyn, V. Vysotska, A. Rzheuskyi, Technology for the Psychological Portraits Formation of Social Networks Users for the IT Specialists Recruitment Based on Big Five, NLP and Big Data Analysis, CEUR Workshop Proceedings Vol-2392 (2019) 147–171. [76] N. Shakhovska, S. Fedushko, M. ml. Greguš, I. Shvorob, Yu. Syerov, Development of Mobile System for Medical Recommendations, in: The 15th International Conference on Mobile Systems and Pervasive Computing (MobiSPC), 155, 2019, pp. 43-50.