=Paper=
{{Paper
|id=Vol-3171/paper97
|storemode=property
|title=Recommendation System for Monitoring the Energy Value of Consumer Food Products Based on Machine Learning
|pdfUrl=https://ceur-ws.org/Vol-3171/paper97.pdf
|volume=Vol-3171
|authors=Nazar Oleksiv,Oleh Veres,Andrii Vasyliuk,Ihor Rishnyak,Lyubomyr Chyrun
|dblpUrl=https://dblp.org/rec/conf/colins/OleksivVVRC22
}}
==Recommendation System for Monitoring the Energy Value of Consumer Food Products Based on Machine Learning==
https://ceur-ws.org/Vol-3171/paper97.pdf
Recommendation System for Monitoring the Energy Value of
Consumer Food Products Based on Machine Learning
Nazar Oleksiv 1, Oleh Veres 1, Andrii Vasyliuk 1, Ihor Rishnyak 1 and Lyubomyr Chyrun 2
1
Lviv Polytechnic National University, S. Bandera Street, 12, Lviv, 79013, Ukraine
2
Ivan Franko National University of Lviv, University Street, 1, Lviv, 79000, Ukraine
Abstract
Healthy eating is the cause of longevity. Rational consumption of products will help achieve
high efficiency and activity throughout life. To improve the quality of your life, you do not
need to read dozens of books and delve into medicine. Just use intelligent systems. Today,
there are many different approaches to solving the problem of diet control in a fleeting world.
The need for research on healthy eating in modern conditions is crucial to improving the
physical condition of different age groups. After all, non-compliance with the basic rules
reduces life expectancy and provokes the emergence of non-communicable diseases of various
types: cancer, cardiovascular disease, and diabetes. The main goal is to create a system whose
main task will help the end-user follow a healthy diet. The primary skills of the system include
the ability to recognise dishes, read barcodes, decompose a dish into a set of ingredients,
calculate calories, and calculate the energy value consumed per day.
Keywords 1
Calories counter, diet, nutrition, image analysis, food recognition, machine learning, food
dataset, IDEF, ontology
1. Introduction
Proper nutrition is one of the conditions for longevity and healthy living. After all, most health
problems are related to what we eat. By following simple rules of healthy eating, we will feel much
better. However, it is pretty challenging to track your diet daily because the day of an ordinary person
is usually quite busy, and there is no place to calculate the caloric content of foods and monitor your
diet. Moreover, you need to stabilise your diet and determine your diet to get the most out of constant
physical activity. Therefore, creating a system that automates these processes is necessary for today’s
reality. The main goal is to create a system whose main task will help the end-user follow a healthy
diet. The primary skills of the system must be the ability to recognise dishes, read barcodes, decompose
a dish into a set of ingredients, calculate calories, and calculate the energy value consumed per day.
2. Related works
A team of researchers from the University of Massachusetts has developed a neural network that
can make a list of ingredients and form a recipe for food [1]. The input data for the system’s operation
are images of a predetermined format [2-7]. The authors have created an open database of about a
million different subject images to teach the system. The database was formed by collecting information
from open sources [8-9]. The basis for this approach is culinary sites and access to food service
databases. The article [1] follows that the neural network works on the principle of joint embedding: it
COLINS-2022: 6th International Conference on Computational Linguistics and Intelligent Systems, May 12–13, 2022, Gliwice, Poland
EMAIL: nazar.oleksiv.mnsa.2020@lpnu.ua (N. Oleksiv); Oleh.M.Veres@lpnu.ua (O. Veres); Andrii.S.Vasyliuk@lpnu.ua (A. Vasyliuk);
Ihor.V.Rishnyak@lpnu.ua (I. Rishnyak); Lyubomyr.Chyrun@lnu.edu.ua (L. Chyrun)
ORCID: 0000-0001-7821-3522 (N. Oleksiv); 0000-0001-9149-4752 (O. Veres); 0000-0002-3666-7232 (A. Vasyliuk); 0000-0001-5727-3438
(I. Rishnyak); 0000-0002-9448-1751 (L. Chyrun)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
divides images into vectors with products or ingredients [10-14]. The neural network compares the
formatted input image with the most similar picture of the recipe [15-17], which is compared with the
input element under the influence of linear transformation [18-25].
The researchers used the neural network in actual conditions and compared the available results of
the system with those that people themselves gave in a random survey. Under these conditions, the
accuracy of the neural network is 83.6%, while the survey showed that the accuracy of recognition of
products included in the dish is 81.6% [1]. Photographs with food are widely used in food magazines
and social networks to share gastronomic experiences [26-30]. Many of these images were taken in
restaurants. Recognition of dishes, in general, is challenging [31-36] due to different national dishes,
varieties, cooking styles, and the inherent complexity of modelling food from its visual appearance [37-
38]. First, you need to adapt the photo frame based on the rejection of unlikely categories [39-41]
located far from the test image. Then, reformulate the problem using a probabilistic model that
combines dishes, restaurants, and places [42-55]. This model is used in three different tasks: food
recognition, restaurant recognition, and space improvement [56-60].
3. Methods and Materials
To build a goal tree, we connect all the system’s goals. After that, we will select the main goal,
which will serve at the top of the tree and the subordinate objectives of the first and second levels. The
primary purpose of the developed system is to create an information system of dynamic and adaptive
control of the human diet based on machine learning. To achieve this goal, you need to analyse the
subject area, design the system and work on the implementation. The goal tree shown in Figure 1 allows
you to create a compelling and complete hierarchical structure by decomposing the primary goal.
Creating an information system of dynamic and
adaptive control of human diet based on machine
learning
Subject area analysis System design System implementation
Construction of context diagrams
Choice of development tools and
Implementation of the designed
Analysis of image recognition
Research of alternatives in the
Building a hierarchy of tasks
Forming an initial sample of
System architecture design
images and barcodes
System testing
environments
methods
system
market
Recognition
Novelty Flexibility Productivity Speed Convenience
quality
Figure 1: Goal tree
The subject area analysis allows you to analyse the current market situation, explore methods that
will enable you to solve specific problems, etc. At the design stage, the necessary functions of the
system are prioritised, and structural approaches are formed. At the maintenance, the scene is
implementing the already researched and designed solution in an existing information system. It is
known that according to the level of automation, information systems are divided into information
retrieval, information reference, information management and intelligent information systems and
decision support systems. To select the appropriate type of information system, we use the hierarchy
analysis method. As alternatives, we choose four types of information systems: information retrieval
(A1), information retrieval (A2), decision support system (A3) and intelligent information retrieval
system (A4). The type definition of the information system will be evaluated to 6 characteristics of the
novelty of the developed product (K1), recognition quality (K2), the flexibility of implementation (K3),
productivity (K4), and speed (K5) and ease of use (K6). We use the relative importance scale of
priorities to calculate the expert assessment presented in table 1.
Table 1
The scale of the relative importance of priorities
Values High-quality characteristics
1 Equivalent elements
2 Insignificant priority
3 Weak priority
4 Moderate priority
5 Significant priority
6 Essential priority
7 Strong priority
8 Very strong priority
9 Unconditional priority
We depict the relative importance of one criterion compared to others in the matrix of comparisons
of characteristics in Figure 2.
C1 C2 C3 C4 C5 C6 PN EV
C1 1,00 1,50 1,17 1,20 1,33 1,25 1,33 0,10
C2 3,00 1,00 1,20 1,25 1,50 1,33 1,50 0,11
C3 7,00 6,00 1,00 3,00 5,00 4,00 3,99 0,29
C4 6,00 5,00 1,50 1,00 4,00 3,00 2,98 0,22
C5 4,00 3,00 1,25 1,33 1,00 1,50 1,79 0,13
C6 5,00 4,03 1,33 1,50 3,00 1,00 2,26 0,16
Figure 2: Matrix of comparisons of characteristics
We calculate the correct number using formula 1:
𝑛
PN = √∏𝑛𝑗=1 𝑎𝑖𝑗 ,
(1)
Calculate eigenvectors using formula 2:
𝑤
EV = ∑𝑛 𝑖 , (2)
𝑖=1 𝑤𝑖
where wi is properly number, ∑𝑛𝑖=1 𝑤𝑖 – the sum of correct values for a specific criterion.
We compile matrices of pairwise comparisons of the created alternatives A1, A2, A3, A4 for each
of the criteria and the corresponding goal in Figure 3 – Figure 6:
A1 A2 A3 A4 PN EV A1 A2 A3 A4 PN EV
A1 1,00 1,50 3,00 5,00 2,41 0,27 A1 1,00 1,50 1,25 3,00 1,71 0,19
A2 3,00 1,00 5,00 6,00 3,51 0,39 A2 3,00 1,00 1,50 5,00 2,41 0,27
A3 1,50 1,25 1,00 3,00 1,71 0,19 A3 5,00 3,00 1,00 6,00 3,51 0,39
a) A4 1,25 1,20 1,50 1,00 1,40 b) 0,16 A4 1,50 1,25 1,20 1,00 1,40 0,16
Figure 3: Matrix of pairwise comparisons for criteria a) "novelty" and b) "quality"
A1 A2 A3 A4 PN EV A1 A2 A3 A4 PN EV
A1 1,00 1,50 1,25 3,00 1,71 0,19 A1 1,00 1,50 3,00 5,00 1,71 0,19
A2 3,00 1,00 1,50 5,00 2,41 0,27 A2 3,00 1,00 5,00 6,00 2,41 0,27
A3 5,00 3,00 1,00 6,00 3,51 0,39 A3 1,50 1,25 1,00 3,00 3,51 0,39
a) A4 1,50 1,25 1,20 1,00 1,40 b) 0,16 A4 1,25 1,20 1,50 1,00 1,40 0,16
Figure 4: Matrix of pairwise comparisons for criteria a) “flexibility” and b) “productivity”
A1 A2 A3 A4 PN EV A1 A2 A3 A4 PN EV
A1 1,00 3,00 1,20 5,00 2,12 0,23 A1 1,00 3,00 5,00 6,00 3,51 0,39
A2 1,50 1,00 1,25 2,00 1,71 0,18 A2 1,50 1,00 3,00 5,00 2,41 0,27
A3 6,00 5,00 1,00 6,00 4,16 0,44 A3 1,25 1,50 1,00 3,00 1,71 0,19
а) A4 1,25 1,50 1,20 1,00 1,40 b) 0,15 A4 1,20 1,25 1,50 1,00 1,40 0,16
Figure 5: Matrix of pairwise comparisons for criteria a) "speed" and b) "convenience"
C1 C2 C3 C4 C5 C6
Generalization
A1 A2 A3 A4 PN EV 0,1 0,11 0,29 0,22 0,13 0,16
A1 1,00 1,50 1,25 1,20 1,40 0,16 A1 0,27 0,19 0,19 0,19 0,23 0,39 0,24
A2 3,00 1,00 1,50 5,00 2,41 0,27 A2 0,39 0,27 0,27 0,27 0,18 0,27 0,28
A3 5,00 3,00 1,00 6,00 3,51 0,39 A3 0,19 0,39 0,39 0,39 0,44 0,19 0,33
a) A4 6,00 1,25 1,20 1,00 b)
1,71 0,19 A4 0,16 0,16 0,16 0,16 0,15 0,16 0,16
Figure 6: Matrix a) pairwise comparisons of the main goal and b) comparison of alternatives
We create a matrix of alternatives comparisons and identify the information system type. The matrix
of comparison of other options is shown in Figure 6b. Thus, the hierarchy analysis method is the chosen
information system to support decision-making.
4. Experiment
To develop and maintain an ontology describing the recommended system for monitoring the energy
value of consumer food products based on machine learning technology, we use the standard that is part
of the IDEF family in software development - IDEF5. IDEF5 method is an ontology by collecting
statements about natural objects, their properties, and relationships and presenting this content
intuitively and naturally reflecting the human perception of a particular area.
There are unique ontological languages to carry out the process of building IDEF5 ontologies:
1. Schematic language is a graphic language that is specifically designed to present primary data
in the form of ontological information. This language allows you to naturally offer basic information
about the initial development of the ontology and supplement existing ontologies with new data [61-
67]. This language will enable you to build various charts and diagrams in IDEF5. The purpose of
graphs is to use visualisation to present basic ontological information.
2. Revisions and clarifications are a structured text language that allows you to describe the
ontology elements in detail.
So, let us construct a classification chart using schematic language. Figure 7 shows a diagram of the
classification of nutrients. As you can see, organic nutrients are made up of proteins, fats and
carbohydrates. We believe that this classification within the system is necessary because a living
organism cannot live and die without them.
Nutrient Meal
Carbohydr
Proteins Fats Breakfast Lunch Supper
ates
а) b)
Figure 7: Diagram of classification of a) nutrient and b) meal
Figure 7b shows a diagram of the type of food intake. Each meal takes place in a different time
range, so we use such categories as breakfast – early meal, lunch – midday meal, supper – a meal in the
final phase of the day.
Any dish can be classified in the classification system because it is necessary to determine to which
category the object that the user photographed with the lens of his smartphone belongs. Therefore, we
distinguish the following dishes: main, sweet, salad, snack and drink. This classification is given in
Figure 8a. Input resource – an object of the outside world with which the system works. We highlight
the image with which the system can work with the dish, break it down into ingredients, and count each
element’s calories that have undergone decomposition. The input also includes a barcode that allows
the system to have information about the food composition without working with the product directly.
Figure 8b presents a diagram of the classification of input resources.
Dish
Input
resource
Main Sweet Salad Snack Drink Image Bar code
а) b)
Figure 8: Classification chart of a) dishes and b) input resources
In the next step, we build a compositional scheme, which is shown in Figure 9:
Ingredient
Dish Nutrient
Organic
substances
Inorganic
substances
Carbohydrate
Proteins Fats
s
Figure 9: Compositional scheme
Next, we will show you some state transition diagrams to show the transition from one object to
another. Figure 10 shows the transition from one state to another:
Perform Time interval pasts
decomposition
Energy Breakfast Lunch
Ingredient
value
а) b)
Figure 10: State diagram with a) “Perform decomposition” and b) “Time interval pasts” process
Process concept. After successful authorisation and authentication, the user sees the main page,
which displays information for the current day: how many calories are consumed and how many you
still need to get to comply with the norm, what foods are consumed, etc. To recognise a dish, the user
must hover over the dish, take a photo and send it to the server where the recognition process occurs,
using the available dataset. In case of failed recognition, similar options will be displayed. If they do
not match, then to improve the system, the user can enter the necessary data and simplify the use of the
application in subsequent attempts to recognise. The dish is broken down into ingredients, and the
approximate total caloric content is calculated. The result is displayed on the user’s home page. The
user can also monitor daily progress, which is shown in graphs based on daily statistics collected. The
first time you run the IP, the user enters the necessary information to simulate his regular daily routine,
which will be the basis for sending specific messages in case of deviation and reminding about the
missed stage (for example, reminding the user that he missed breakfast). The stability and accuracy of
the IP are proportional to the frequency of its use. If you use the system daily, its recommendations and
tips will be more relevant. The combined scheme is given in Figure 11:
Read
information
Dish Determine the energy
value
O Ingredient
Make
O Nutrition
Product recommendations
Bar code
& Tips
History analysis
History
Data
&
gathering
Recomme
ndation
Figure 11: Combined scheme
The implementation of IDEF0 diagrams was chosen to detail the structure of the information system.
The conceptual model of the process “Product recognition and diet control” is shown in Figure 12.
The input parameters for the full functioning of the system are data on the caloric content of products,
data on dishes, a dataset with images, user features (set in the settings) and an authorised profile for
access to the API with barcodes. One of the key parameters influencing the formation of advice and a
consumption plan is dietary norms. The initial result of the whole system can be the final calculation of
calories and the establishment of recommended theses on nutrition and diet.
Dietary rules
Data on caloric content of
products Calculation of caloric content of the
Data on dishes product
Dataset Consumption recommendations
User features Recommended system for monitoring the energy value of
API with barcodes food products
Figure 12: Conceptual model of the “Product recognition and diet control” process
The following steps are to build hierarchies of lower-level processes. Figure 13 shows the
decomposition diagram of the first level of the “Product recognition and diet control” process.
Dietary rules
Dataset
Recognized elements
Data on dishes Product recognition
Calculation of caloric content of the
product
Data on caloric content of products Determining the History of nutrition
energy value of
API with barcodes products
Maintaining statistics
Consumption
Formation of dietary recommendation
advice
Figure 13: Decomposition of the first level of the “Product recognition and diet control” process
Figure 14 shows the decomposition diagram of the second level of the process “Product
Recognition”:
Recognized elements
Get barcode data
Bar code
Recognize the object
Picture of a
dish
Датасет
Break the dish into Difficult to recognize elements
Data on dishes ingredients
Engage the user in
recognition
Figure 14: Decomposition of the second level of the “Product Recognition” process
Figure 15 shows the decomposition diagram of the second level of the “Determination of energy
value of products” process:
Recognized elements
Data on caloric content of
products
Match foods - calories
Calculation of
caloric content of
API with barcodes Get calories from the products
barcode
Figure 15: Decomposition of the 2nd level of the “Determination of energy value of products” process
Figure 16 shows the decomposition diagram of the second level of the “Maintaining Statistics”
process:
List of most used Dietary rules
Create a popularity products and dishes
rating
Dietary improvement data
Generate dietary
advice
User features Consumption recommendation
Formation of dietary
advice
Figure 16: Decomposition of the second level of the “Formation of dietary advice” process
Figure 17 shows the decomposition diagram of the third level of the process “Recognise the object”:
Recognize barcodes in Barcode values
real time
Bar code
Send object to server Picture of a dish
Take a photo of the
dish
Image
Figure 17: Decomposition of the third level of the “Recognise the object” process
Figure 18 shows the decomposition diagram of the third level of the “Break the dish into ingredients”
process:
Picture of a dish
It is difficult to recognize the elements
Dataset
Carry out clustering
The result of clustering
Data on dishes Investigate the Recognition relationship
probabilities of
coincidences
Recognized elements
Process data
Figure 18: Decomposition of the third level of the “Break the dish into ingredients” process
Figure 19 shows the decomposition diagram of the third level of the process “Get data on the
barcode”:
Bar code
The result of the query
Send a request to the
open API
Recognized elements
Process the result
Figure 19: Decomposition of the third level of the “Get barcode data” process
Figure 20 shows the decomposition diagram of the third level of the “Involve the user in recognition”
process:
It is difficult to
recognize the elements
Ask the user to enter Updated data
the required data
Recognized elements
Save the entered data Recognition relationship
Update dataset
Figure 20: Decomposition of the third level of the “Involve the user in recognition” process
Figure 21 shows the decomposition diagram of the third level of the process “Match foods -
calories”:
Recognized elements
Data on caloric
content of Total calories
products
Generate total calories Calculation of caloric content of
Unknown products
the product
Bring the data to a
normalized form
Caloric content of new
Get the caloric content products
of foods that are not in
the database
Figure 21: Decomposition of the third level of the “Match foods - calories” process
Figure 22 shows the decomposition diagram of the third process level “Get calories from the
barcode”. Figure 23 shows the decomposition diagram of the third level of the “Update History”
process. Figure 24 shows the decomposition diagram of the third level of the process “Update neural
network dataset”.
Recognized elements
API with barcodes Response from the server
Send data to the server
Calculate the result Unstructured data
Calculation of caloric content of the
Bring the data to the product
standard view
Figure 22: Decomposition of the third level of the “Get calories from the barcode” process
Calculation of caloric content of the
product
Supplement the data Completely filled data
with additional
information
View diet Updated diet and intake
History of nutrition
Update information
Figure 23: Decomposition of the third level of the “Update History” process
Calculation of caloric content of the
product
Transfer dto
Input object model
transformation
Formed dataset
Create a new dataset
Conduct neural Learning outcome
network training
Log in for changes
Figure 24: Decomposition of the third level of the “Update neural network dataset” process
Figure 25 shows the decomposition diagram of the third level of the “Create a popularity rating”
process:
History of nutrition
Grouped history
Form categories
Normalized data
Categorize consumed
products
List of most used products and dishes
Print the result
Figure 25: Decomposition of the third level of the “Create a popularity rating” process
Figure 26 presents a diagram of the decomposition of the third level of the “Generate advice on
change” process.
List of most used
The norm of the diet
products and dishes
Analyzed history
History analysis
Processed data
Bringing history to
standard forms
Dietary improvement data
Formation of the result
Figure 26: Decomposition of the third level of the “Generate advice on change” process
Figure 27 shows the decomposition diagram of the third level of the “Filtration according to
preferences” process:
Dietary improvement data
User features
Bringing history to Modified diet
standard forms
Consumption recommendation
Determining the best
for the current period
Figure 27: Decomposition of the third level of the “Filtration according to preferences” process
Based on IDEF0, we will build a hierarchy of processes that affect the system’s performance. In the
All Fusion Process Modeller environment, we will generate a prototype of the already created order.
The result is shown in Figure 28:
Recommended system for monitoring the energy value of food products
Determining the
Formation of dietary
Product recognition energy value of Maintaining statistics
advice
products
1. Update history 1. Create a popularity
1. Recognize the object 1. Put in line with foods - rating
2. Break the dish into calories 2. Update the neural
network dataset 2. Generate advice on
ingredients 2. Get calories from the change
3. Get barcode data barcode 3. Filtration according to
4. Engage the user in preferences
recognition
Figure 28: Hierarchy of processes (functions, tasks)
To understand how external forces affect our system, we will conduct a PEST analysis: The results
are presented in the form of a matrix and are shown in table 2. We describe the information system as
an ordered sequence of events with the simultaneous placement of objects directly related to the
modelling process. The decomposition diagram is given in Figure 29.
Table 2
PEST – analysis
(P) Political (Е) Economic
Tax policy Exchange rate
Course on digitalisation of processes Unemployment rate
Settlement of hostilities Income level of the population
(S) Social (Т) Technical
The duration of the cult of a healthy lifestyle Development of innovations and technologies
Development of diet control habits Introducing a new generation of mobile communications
Population growth rates
“Standards” of beauty are accepted by society
Get the caloric content of
Sent photo Write information
the product
& to the database
Researched
photo Х Fill in unrecognized Update unrecognized
products items
Barcode sent
Display notifications
Update diary
about calories consumed
during the day
Investigate the diet for
rationality Return to the main
Х O page
Remind about missed
meals
Figure 29: Decomposition diagram of the system in IDEF notation
Thus, the topic’s relevance is confirmed by the massive demand for similar systems of similar scale
in the market of mobile applications [67-78]. The number approves of downloads on different
platforms. The subject area was carefully researched, and ontologies were created following the
standard of ontological research IDEF5 (Ontology Description Capture). The system’s primary goal is
established, and its decomposition is carried out at different hierarchical levels. The method based on
the IDEF0 hierarchy is specified. Within this structural methodology, there is a context diagram and its
decomposition into three groups of scale. Based on the order, the structural content of the existing
processes of the system is shown. The PEST analysis identified aspects of the external environment that
affect the developed system. In Fig. 30 shows a block diagram of the methodology used in this study.
First, the process of recognizing food images is based on a model for which the Food-500 dataset was
used, which has 500 categories of dishes and 399,726 images. The image of the dish is processed by
the method for determining the dish. To recognize a dish, it is necessary to decompose the image, which
includes the detection of descriptors of recognized foods. Once the food names and descriptors are
identified, the recognized products are ready for the next steps of the system algorithm. Thus, the end
result is a product image, standardized with unique descriptors, which will count calories and output
the energy content of the dish to the end user. To teach the deep learning model to recognize food, we
first needed to pre-process the image manually. The main purpose of the pre-processing stage is to
create an image of the product with the appropriate caption. The result of this step will be needed for
the controlled learning of the deep learning model. Usually the simplest approach to creating such
images is to label each image with one food class (food name) and teach the deep learning model so
that it returns the product name as a result of providing the image to the ingredient as an input parameter.
Because products often overlap on plates, and drinks can interfere with other items, we label each
product at the pixel level. This means that the result of this step is a new image with the same width
and height as the input image, with only one channel and not the three channels used in RGB images.
This label image contains the class prediction for each individual pixel. Because creating such images
without significant errors is a troublesome, non-trivial task, it is one of the main obstacles when trying
to develop a solution for pixel classification.
Machine learning
Food classes
model
Product ID Descriptive part Classification part
Product category Product category
Further
processing
Category of dish
Figure 30: Block diagram of segmentation and image recognition
We trained the model to recognize food using deep convolutional neural networks that work similar
to human vision. In particular, we used fully convolutional networks (FCN) for adaptive learning based
on research [79-93] and represent the current level of semantic segmentation. This process segments
the input image into individual parts and then classifies each part into an output class; the network does
this by performing a pixel-level classification. Therefore, the FCN outputs a pixel map instead of a class
text label, and this pixel map contains model predictions for each individual pixel of the input image,
not just one prediction for the entire image. This is important because it is the most accurate way to
describe all the foods presented in one image. It took about 35 hours to learn the deep learning model.
However, the classification of a single image takes much less time and computing power, making it
possible to use deep learning models even in mobile applications. Once the training was completed the
model was run once on a subset of the test. The essence of the developed model is to identify and
recognize specific characteristics in images to apply them to a specific area. Visually, the process of
recognizing the products of the system can be seen in Fig. 31. The figure clearly shows how the model
distinguishes the object under study in the image from extraneous noise.
Figure 31: Classification of ingredients (original image - isolated ingredients - intended ingredients)
In order to measure the effectiveness of the learned model of deep learning, we used formulas that
often serve as a guide to quality for fully convoluted semantic segmentation networks. Measurements
are based on standard pixel accuracy and cross-sectional (IU) measurements, including the following:
∑𝑖 𝑛𝑖𝑖
𝑅𝑒𝑐𝑜𝑔𝑛𝑖𝑡𝑖𝑜𝑛 𝑎𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (1)
∑𝑖 𝑡𝑖
1 𝑛𝑖𝑖
𝐴𝑣𝑒𝑟𝑎𝑔𝑒 𝑎𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = ∑ (2)
𝑛𝑐𝑙 𝑡𝑖
𝑖
1 𝑛𝑖𝑖
𝐼𝑛𝑡𝑒𝑟𝑠𝑒𝑐𝑡𝑖𝑜𝑛 𝑡ℎ𝑟𝑜𝑢𝑔ℎ 𝑡ℎ𝑒 𝑢𝑛𝑖𝑜𝑛 = ∑ (3)
𝑛𝑐𝑙 (𝑡𝑖 + ∑𝑗 𝑛𝑗𝑖 − 𝑛𝑖𝑖 )
𝑖
𝑡𝑖 𝑛𝑖𝑖
𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑 𝑟𝑒𝑐𝑜𝑔𝑛𝑖𝑡𝑖𝑜𝑛 𝑓𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 = (∑ 𝑡𝑘 )−1 ∑ (4)
(𝑡𝑖 + ∑𝑗 𝑛𝑗𝑖 − 𝑛𝑖𝑖 )
𝑘 𝑖
where ncl is the number of different classes of the trained model, nlj is the number of pixels of the class
i for which the possibility of belonging to the class is provided j, ti is the total number of pixels of the
class і [79-93].
The learning results of the model are shown in Table 3:
Table 3
The learning results
Recognition Average Intersection through Weighted recognition
Name
accuracy,% accuracy,% union,% frequency,%
Training 93 81 72 89
Validation 90 65 55 84
Testing 92 70 61 87
As expected, the model's performance is better on the training subset than on the other two subsets.
However, the difference is not significant, which means that the model has mastered features that are
well generalized. It is important to note that this performance was measured for all classes - this includes
different times of the day (day and night) and different backgrounds on which the image of the dish is
found. Because the subset of testing contains images new to the deep learning model, we consider the
results of this subset to be the most representative of actual performance. From these results, we chose
pixel accuracy as the final quality measure, as this measure is similar to the accuracy of classification
in traditional convolutional neural networks, which classify the whole image into one class. The
difference is that instead of calculating the accuracy at the image level, it is calculated at the pixel level.
As can be seen from Table 3, the accuracy of the model of deep learning is at a high level, which
indicates a willingness to implement this model in real conditions.
The software product is designed based on three levels of architecture, which consist of the following
levels - presentation, business logic and database access. Figure 32 shows a visualisation of the system
architecture, the relationships between the layers and the related technologies for implementation.
View Level
Business logic level
Data access level
Database
Figure 32: System architecture
It is worth noting that the sequence of layers is appropriate and constant. The level of representation
“knows” only about the existence of business logic. In turn, business logic interacts with the level of
expression and access to the database. The database access level brings the results to specific objects
and sends the result to a higher level using the scripting language and the specified connection. The
advantages of this architectural solution are:
• Code maintenance and support is each level is independent, so changes in one class do not lead
to changes in another;
• Flexibility is each level can be managed and scaled separately;
• Reuse is independence allows you to use levels within different systems;
• Faster development process is various specialists can deal with specific areas: front-end
specialists is representation group. Back-end engineers - the introduction of business logic; database
administrators is develop a database model.
The work of architecture is the following way. The user performs a set of operations that lead to a
request to the server. Within the presentation level, projects are exchanged via the REST interface. A
request comes from the mobile device and goes to the appropriate controller. The proper method of the
controller calls the object from the level of business logic and instructs it to perform tasks. The business
logic layer calls an object from the database access level to connect to the database to retrieve data. The
last group forms the language of the scripts that are executed for a given database. After processing the
data at each level (top-down), the response is returned to the controller. The result of the processing is
sent using the HTTP protocol with the corresponding status code on the development of the operation.
The information is provided to the user to process the body of the response that the device received
from the server part. Let’s start with a description of the database directly related to the lowest level
and is the central place to store data within the system. The database satisfies the 3NF requirements and
consists of the following tables: Users, Roles, Genders, Meals, MealTimes, Ingredients and Measures.
The scheme is shown in Figure 33:
Figure 33: Database schema
The following relationships have been established between entities:
• between Roles and Users - 1: N, as many users can register with a shared role;
• between Genders and Users - 1: N, as one gender can belong to many users;
• between Users and Meals - 1: N, because one user can create many unique dishes that differ in
image, UPC, consumption time, etc.;
• between MealTimes and Meals - 1: N, because, for example, many dishes can be
breakfast/lunch/dinner;
• between Ingredients and Meals - N: N (with intermediate table MealIngredients), because one
ingredient can be in several dishes and vice versa - one dish can consist of many components;
• between Ingredients and Measures - 1: N, which is true: one ingredient can be measured in
grams, tablespoons, etc.
Database-level validation was created during development, which will not allow you to create a
tuple if the required field is empty or the length of the area exceeds the allowable limits. Figure 34
shows an example of conditions that ensure efficient use of the system. Validation measures of this type
help to reduce the size of the database and get rid of empty entries shown in the table at a distance.
Figure 34: An example of “User” table validation
The database is directly related to the level of access to the data. This level has a connection that
allows you to connect to a database server that will run scripts in the future.
Because this system is highly dependent on the database, Dapper was chosen as the main ORM. The
implementation of this package will allow you to directly write scripts in the code that the database
server will process. The business logic level performs the basic operations within the system: saving
files, bringing objects to a specific look, sending requests to open APIs, etc. There are two types of
interlayer objects within a system: transport objects and model representations. Transport objects are
identified with a specific database model, and model representation is a collection of many models and
is an alternative to using temporary tables in the SQL standard.
The presentation level consists of a project in which the code for the user interface in mobile
platforms is written and a project that receives requests from the previous one.
An example of a query with the return of a successful result is shown in Figure 35. Using such a
powerful tool as Redux has created a single repository for the entire application. Applying this approach
allows you not to send a request if we have done so before. First, we speed up the system and optimise
the mobile device’s resources because we do not create new variables. The basic cycle of Redux is
shown in Figure 36. Components are the building blocks of our system that send specific signals to the
repository. Manipulations within the components cause specific actions that change the state in the
storage and update it accordingly. A store is an object that contains all the states at a particular point in
time. Repository registration occurs in the root, the main component, the first to run in the system. You
can access the state using the getState () function. You can update the status using the dispatch ()
function. Figure 37 shows the system repository. Figure 38 shows the state of the history of
consumption. It features the unique values of each product. The general state tree is given in Figure 39.
Figure 35: An example of the successful response
signal Components
call
Store Actions
update
Figure 36: Redux cycle
Figure 37: System storage view
Figure 38System storage states
Figure 39: Tree of estates
5. Results
For full use of the system, it is necessary to get acquainted with the technical characteristics and
functionality of the program. The user manual helps in this context. The name of the program is
“PocketDietCoach”. The logo is shown in Figure 38:
Figure 40: “PocketDietCoach” logo
The program is written using the C # programming language and JavaScript. The SQL standard was
used to write scripts to interact with the database. The following development environments were used
during the development: Visual Studio 2019 and Visual Studio Code. Microsoft SQL Management
Studio 2017 was used during the development. The Git version control system was used using Git Bash.
The program is designed to control the daily diet and automatic calorie counting. This system’s focus
suggests that its implementation is possible in various spheres of life. As for the system’s capabilities,
you should make a list of needs that it can meet for the user. The main functions include recognition of
products and QR-codes, determination of energy value, automatic calculation of caloric content, control
over consumption, formation of a consumer diary, and tracking history. Additional features of the
program are sending notifications about missed meals and recommendations for improving the diet.
The system solves the problem of determining the dish and its content, highlighting the exposure
and recognising the dish in the image, calculating the energy value and recording processed information
in the user profile, notification of missed meals and recommendations on consumption time.
One of the initial methods that help solve the tasks is authentication and authorisation methods.
These procedures determine whether a user can log in and perform certain operations. Successful
authentication results are in a temporary token that is active for 40 minutes. It means that it quickly
becomes inactive, and the system updates it. It allows the system to make sure that requests are sent by
a specific user, not by an attacker who changed the encrypted part of the key. The following method is
to get statistics for the current date. A successful result allows you to process data and display
information on the user’s screen. This method is parameterised, so navigating the home screen causes
a new request to be sent with a variable date setting. An essential method for organising processes for
the system is the formation of statistics related to nutrition. The processed information is submitted to
the relevant sections, which can be further defined in the user profile.
In terms of temporal characteristics, it recognises a product such as pizza is approximately 5s.
However, this time dependence depends on the speed of the Internet so that it may differ on the devices.
Compared to analogues, this time is more than satisfactory for product recognition and calorie counting.
Currently, there is only one mode of operation in the program - custom. At this stage, the possibility
of maintaining global statistics and creating an administrative method for closed users is being
developed. The means of control is the freely available Clarifai API, which can also recognise dishes.
If the local neural network could not match the received image, the unrecognised element will be sent
along the described route. NutritionxAPI is used to recognise the barcode.
The limitation of the program is the presence of a device with Android / iOS version 8 and higher.
The device must be equipped with a camera module and access the Internet for the whole operation.
The system requires access to these device parts to comply with all usage rules during the first startup.
Information on functional limitations for use. A smartphone, camera module, and Internet access are
required to use the system in full release mode fully. Requests can be sent via Wi-Fi or cellular data.
The main condition is the optimal data rate. Also, an important condition is to use the Android or iOS
operating system on your mobile device. Features for operating system versions are as follows: for
Android, the Android SDK version must be at least 8.0. For iOS devices, the system will run
successfully on iOS version 11 and above. You need to compile the project and get its build DLL in
local development mode to run. Run the ASP.NET CORE project regardless of the launch port. The
next step is to run React Native on your mobile device. The mobile project is launched using npm
commands in the developer console. It must be installed on a user’s mobile device to access the
application. The installation process meets the conditions of the platform community, which is
confirmed by permission to use the camera and the Internet connection. Step-by-step instructions. To
start, you need to log in to the PocketDietCoach application, shown in Figure 39a. Logging in the system
in Figure 39b. Here you need to enter the login and password of the created account. The main page is
shown in Figure 39с. The navigation menu is shown in Figure 40a. The photo mode is shown in Figure
40b. Activity statistics are presented in Figure 40c:
a) b) c)
Figure 41: a) Image of the system on the home screen, b) Logging in, and b). Main page
a) b) c)
Figure 42: а) Navigation menu, b) photo mode and c) activity statistics
Consider the efficiency of the system and the compliance of the task to the results of work.
After the user downloads the application, he sees the page shown in Figure 41a. This page appears
when the token has expired or does not exist. After a successful login process, the next login will redirect
the user to the home page. The user can go to the registration page or login with a pre-created account
on this screen. The user must click on the “Create Account” button and go through the account creation
procedure to complete the registration process. Figure 41b user is shown to enter gender and body
parameters: age, height, and weight. These parameters in the system allow you to determine how many
calories the body needs. After entering the required parameters, the user must fill out the registration
form, which is shown in Figure 41c. You need to fill in all available fields and enter your login, e-mail
and password.
а) b) c)
Figure 43: а) Home page, b) input of body parameters and c) registration page
The order of registration is accompanied by two stages: introducing body parameters and creating a
profile. If the user has created a profile in the system, he goes to the Login page, shown in Figure 42.
The system prompts users to enter their e-mail and the corresponding password. If the response is
successful, the user will be able to go to the home page.
а) b) c)
Figure 44: a) Login page, b) validation message in case of blank fields and c) validation message in case
of inconsistency of entered data
In case when the user has not filled in all the fields or entered a login/password that does not
correspond to reality, the user will see the appropriate messages that will notify him of non-compliance
with the validation conditions. In Figure 42b and Figure 42c. Examples of such messages are shown
above. The main page is shown in Fig. 43a. The page is conditionally divided into two parts. The first
contains information about the calories consumed, and the second is the dishes that the user consumed.
The current date and the left and right navigation buttons are indicated at the top of the screen. They
are responsible for moving by date and downloading the necessary information to the home screen
accordingly. The starting point from which you can view the consumption history is the user’s
registration date. The calorie ring clearly shows how many calories are left to consume to maintain the
body’s energy balance.
а) b)
Figure 45: a) Home page, b) the process of photographing the object
More detailed information is given under the ring. It contains data on how many calories a user
should consume, how many he consumes during the day and how many are left. The formation of the
daily norm parameter is based on the user’s age, weight, and height. The home page also displays the
products the user consumed during the day. The ability to add a dish or product is made possible by a
special button. The “plus” sign switches the user to the mode of use of the camera in which you can
point to the product and take pictures. Access to the camera is shown in Fig. 43b. After taking a photo,
the system displays the image taken and asks us to wait a few seconds to process the information.
During this time, the system recognises the elements in the picture, determines the caloric content and
records the data in the database.
6. Discussions
In case of successful recognition, the main menu is updated, and the information about calories
consumed changes. The obtained results are added, and the development is displayed whether you have
finished insufficient or exceeded. The photo taken is attached to the specific time range you ate the food
and is supported by the dish’s name. The recognition result is shown in Figure 44a. Tips are also
integrated into this system based on previous experience. For example, Figure 44b shows a message
indicating that the user has not consumed food to date. Therefore, the plan encourages him to fix this
because the user usually eats breakfast at this point.
Another page is the user profile, which is shown in Figure 44c. Initially, the information in the profile
is filled with the data that were entered during registration. You can change them by clicking the
appropriate option in the upper right corner. The user’s page lists the main parameters: name, age,
country, weight and height. In addition, the profile shows user activity. The activity is represented by
an activity schedule that indicates the number of actions during a given day. A graph is an array that
consists of an object that contains the date and number of contributions for a given day. Accordingly,
the richer the square, the more activity the user has spent during a specific date. Particular attention is
paid to the analysis of consumed substances and dishes, so the information in diagrams is presented in
the user’s diary. Figure 45a shows a line graph showing the number of calories consumed in the last
seven days. The abscissa shows the days of the week, and the ordinate indicates the number of calories.
It allows you to track a trend and draw conclusions about the improvement of the diet.
а) b) c)
Figure 46: a) Recognition result, b) meal notification and c) user profile
а) b)
Figure 47: a) Graph of calories consumed and the percentage between the amount of food consumed
and time and b) Bar chart of the average number of calories consumed and a pie chart of the most
popular dishes
Also Figure 45a is a circle chart showing the relationship between the amount of food consumed
and time. The system has four-time ranges, so the circles are also four. For example, breakfast should
have the highest percentage under ideal conditions, and dinner before bedtime - the lowest. This chart
allows you to determine the meal that covers the most calories and draw conclusions. With the data
from the two charts above, you can choose the trend of calories consumed daily and over time.
In Figure 45b shows a bar chart showing the average number of calories consumed during the day
over the last four months. The bar chart allows you to monitor progress and understand whether the
user is moving in the direction of weight loss or increasing it. Figure 45b shows a pie chart showing the
most popular dishes or foods during the last week and their share in the user’s daily diet.
7. Conclusions
The current market situation is analysed, and the need to create a system that will help automate the
process of counting calories based on the recognition of ingredients contained in the dish’s image. The
relevance of the topic is confirmed by the massive demand for similar systems of similar scale in the
market of mobile applications. The number approves of downloads on different platforms. The
advantages and disadvantages of alternative solutions are summarised in the table and summarise the
result. Various architectural solutions and algorithms for neural network training methods are analysed
to design the system. The efficiency of their implementation in different areas is compared. The
appropriate form is selected, and the optimal initial data set is set. All information was analysed with
the help of scientific articles, thematic literature, etc. The result of the system analysis is the creation of
a goal tree based on an integrated approach. The system’s primary goal is established, and its
decomposition is carried out at different hierarchical levels. The method based on the IDEF0 hierarchy
is specified. Within this structural methodology, there is a context diagram and its decomposition into
three groups of order. Based on the hierarchy, the structural content of the existing processes of the
system is shown. The result of successful performance of work is the creation of a system that satisfies
all tasks set: recognition of products, automation of the process of counting calories, formation of
councils and sending of notifications. All results are supported in the software product description and
user manual. The user manual describes the appropriate use of the system given the necessary conditions
for a startup. The system’s functionality is supported by images that show the sequence of using the
system step by step.
8. References
[1] O. Kryvoruchko, K. Khorolska, V. Chubaievskyi, Usage of neural networks in image recognition,
Foreign trade: economics, finance, law 3 (2019) 83–92.
[2] Y. V. Bodyanskyy, A. O. Deyneko, Z. V. Deyneko, M. O. Shalamov, Adaptyvne navchannya
neyronnoyi merezhi opornykh vektoriv naymenshykh kvadrativ, Informatsiyno-keruyuchi
systemy na zaliznychnomu transporti [Adaptive learning of the neural network of support vectors
of least squares, Information and control systems in railway transport] 2 (2015) 71–74.
[3] S. I. Alpert, Suchasni kryteriyi otsinky tochnosti klasyfikatsiyi zobrazhen [Modern criteria for
assessing the accuracy of image classification], Matematychni mashyny i systemy [Mathematical
machines and systems] 4 (2013) 187–197.
[4] V. K. Berehovyy, Osnovy naukovoyi orhanizatsiyi zdorovoho kharchuvannya [The basis of
scientific orhanizatsii healthy eating], Efektyvna ekonomika [Efficient economy], 2011. URL:
http://nbuv.gov.ua/UJRN/efek_2011_11_19.
[5] S. V. Bilashenko, N. N. Shapovalova, O. H. Rybalchenko, Rozpiznavannya zobrazhen za
dopomohoyu z·hortkovykh neyronnykh merezh z vykorystannyam biblioteky Keras [Image
recognition with the help of cortical neural networks using the Keras library], Hirnychyy visnyk
[Mining Bulletin] 10 (2018) 148–154.
[6] Y. V. Bodyanskyy, N. O. Teslenko, A. O. Deyneko, Evolyutsiyna neyronna merezha z yadernymy
funktsiyamy aktyvatsiyi y adaptyvnyy alhorytm yiyi navchannya [Evolutionary neural network
with nuclear functions of activations and adaptive algorithm of its training], Naukovi pratsi
Chornomorsʹkoho derzhavnoho universytetu imeni Petra Mohyly: Kompʺyuterni tekhnolohiyi
[Scientific works of the Petro Mohyla Black Sea State University. Computer technology] 160(148)
(2011) 53–58.
[7] K. Y. Bortnyk, O. V. Olshevskyy, A. L. Kyrylyuk, Mashynne navchannya, yak osnova dlya
rozvytku tekhnolohiy maybutnoho [Machine learning as a basis for the development of
technologies of the future], Kompʺyuterno-intehrovani tekhnolohiyi: osvita, nauka, vyrobnytstvo
[Computer-integrated technologies: education, science, production] 27 (2017) 85–88.
[8] V. M. Brodkevych, V. Y. Remeslo, Alhorytmy mashynnoho navchannya ta hlybokoho navchannya
(HN) i yikh vykorystannya v prykladnykh dodatkakh [Algorithms of machine learning and deep
learning (H) and their use in application applications], Mizhnarodnyy naukovyy zhurnal
«Internauka» [International Scientific Journal "Internauka"] 11(1) (2018) 56–60.
[9] I. V. Butyrska, A. V. Manhul, Tekhnolohiya QR-kodu yak instrument pidvyshchennya
efektyvnosti funktsionuvannya servisnykh system [NR-code technology as a tool to increase the
efficiency of service systems], Visnyk Chernivetsʹkoho torhovelʹno-ekonomichnoho instytutu:
Ekonomichni nauky [Bulletin of the Chernivtsi Trade and Economic Institute: Economic sciences]
1 (2015) 165–171.
[10] V. V. Vitlinskyy, Shtuchnyy intelekt u systemi pryynyattya rishen [Artificial intelligence in the
decision-making system], Neyro-nechitki tekhnolohiyi modelyuvannya v ekonomitsi [Neuro-
fuzzy modeling technologies in economics] 1 (2012) 97–118.
[11] G. Y. Voloshin, Metody raspoznavaniya obrazov [Methods of pattern recognition], VGUES, 2000.
[12] S. Hadetska, V. Horokhovatskyy, Metody strukturnoyi klasyfikatsiyi zobrazhen na zasadakh
bayesovskoyi teoriyi pryynyattya rishen [Methods of structural classification of images on the
basis of Bayesian theory of decision making], Radio electronics, computer science, management
2(2018) 90–97.
[13] I. Hamanyuk, Variant zastosuvannya bayyesivskykh metodiv dlya mashynnoho navchannya
shtuchnoho intelektu systemy pidtrymky pryynyattya rishen u borotbi zi spamom [Option of
application of Bayesian methods for machine learning of artificial intelligence system support
decision-making in the fight against spam], Zvyazok [Communication] 6 (2018) 14–18.
[14] N. V. Hlukhova, L. A. Pisotska, Rozrobka metodu analizu kolorovykh zobrazhen hazorozryadnoho
vyprominyuvannya [Development of a method for analysis of color images of gas-discharge
radiation], Systemy upravlinnya, navihatsiyi ta zvyazku [Management, navigation and
communication systems] 2 (2018) 59–62.
[15] N. V. Hlukhova, L. A. Pisotska, N. H. Kuchuk, Rozrobka systemy ekspres-klasyfikatsiyi vody na
osnovi bazy danykh zobrazhen [Development of an express water classification system based on
an image database] Zbirnyk naukovykh prats Kharkivskoho universytetu Povitryanykh Syl
[Collection of scientific works of Kharkiv University of the Air Force] 3 (2015) 112–118.
[16] L. Hlushchenko, Perspektyvy vykorystannya shtuchnoho intelektu dlya rozrobky funktsionalnomu
kharchuvanni [Prospects for the use of artificial intelligence to develop functional nutrition]
Visnyk Lvivskoho universytetu: Seriya biolohichna [Bulletin of Lviv University: Biological
series] 73 (2016) 437.
[17] V. A. Gorokhovatskiy, D. V. Pupchenko, Klassifikatsiya izobrazheniy vizualnykh obyektov po
mnozhestvu deskriptorov osobennykh tochek na osnove neyronnoy seti Kokhonena [Classification
of imaged visual objects by many singular point descriptors based on the Kohonen neural network],
Sistemi upravlínnya, navigatsi ta zvyazku [Management systems, navigation and communication],
2 (2018) 68–72.
[18] O. Hryhorenko, Naukovi pidkhody do formuvannya ratsioniv kharchuvannya [Scientific
approaches to the formation of diets], Prohresyvni tekhnika ta tekhnolohiyi kharchovykh
vyrobnytstv restorannoho hospodarstva i torhivli [Advanced techniques and technologies of food
production, restaurant business and trade] 2 (2009) 210–218.
[19] D. V. Hrynov, Metod rozpiznavannya zobrazhen obyektiv zasobamy vydovoyi rozvidky [Method
of image recognition of objects by means of species reconnaissance], Systemy ozbroyennya i
viyskova tekhnika [Weapons systems and military equipment] 4 (2007) 72–75.
[20] V. V. Hrytsyk, Metod obrobky skladnykh zobrazhen ta yikh rozpiznavannya [Method of
processing complex images and their recognition], Dopovidi Natsionalnoyi akademiyi nauk
Ukrayiny [Reports of the National Academy of Sciences of Ukraine] 1 (2011) 28–32.
[21] V. Hrytsyk, Obrobka skladnykh zobrazhen ta yikh rozpiznavannya v informatsiyno-analitychnykh
systemakh kompyuternoho zoru [Processing of complex images and their recognition in
information-analytical systems of computer vision], Dopovidi Natsionalnoyi akademiyi nauk
Ukrayiny [Reports of the National Academy of Sciences of Ukraine] 7 (2009) 36–41.
[22] I. H. Danylyuk, Aktualni problemy metodu hlybynnoho navchannya [Actual problems of the
method of deep learning], Linhvistychni studiyi [Linguistic studies] 35 (2018) 155-158.
[23] A. S. Dovbysh, H. A. Stadnyk, Informatsiyno-ekstremalnyy alhorytm navchannya systemy
pidtrymky pryynyattya rishen z hipertsylindroyidnym klasyfikatorom [Informatsion-extreme
algorithm for learning a system supporting decision-making with a hypercylindroid classifier],
Radioelektronni i kompyuterni systemy [Radio electronic and computer systems] 3 (2015) 11–18.
[24] A. S. Dovbysh, I. V. Shelekhov, Osnovy teoriyi rozpiznavannya obraziv [Fundamentals of image
recognition theory], Sumy: Sumy State University, 2015.
[25] I. Domanetska, O. Fedusenko, V. Khrolenko, Neyromerezhevi tekhnolohyi opratsyuvannya
zobrazhen v adaptyvnykh systemakh navchannya [Neural network imaging technologies in
adaptive learning systems], Shtuchnyy intelekt [Artificial Intelligence], 3 (2017) 24–31.
[26] V. O. Drofa, T. M. Yefimenko, Informatsiyno-ekstremalnyy alhorytm rozpiznavannya
nestatsionarnykh za yaskravistyu zobrazhen [Informatsion-extreme algorithm for recognizing non-
stationary brightness images], Byonyka yntellekta [Bionics of intelligence] 2 (2015) 100–104.
[27] D. O. Klymenko, O. A. Rudenko, Veb-dodatok dlya servisu skladannya ratsionu zdorovoho
kharchuvannya ta dostavky produktiv [Web application for the service of compiling a healthy diet
and food delivery], Systemy upravlinnya, navihatsiyi ta zvyazku [Management, navigation and
communication systems] 2 (2019) 103–109.
[28] V. H. Krasylenko, R. O. Yatskovska, V. I. Yatskovskyy, Modelyuvannya metodiv rozpiznavannya
ta klasyfikatsiyi frahmentiv kolorovykh zobrazhen zemel silskohospodarskoho pryznachennya pry
yikh dystantsiynomu monitorynhu [Modeling methods of recognition and classification of
fragments of color images of agricultural lands in their remote monitoring], Systemy obrobky
informatsiyi [Information processing systems] 5 (2017) 55–61.
[29] T. V. Lendyuk, Modelyuvannya kompyuternoho adaptyvnoho navchannya i testuvannya
[Modeling of computer adaptive learning and testing], Pratsi Odeskoho politekhnichnoho
universytetu [Proceedings of Odessa Polytechnic University] 1 (2013) 110–115.
[30] O. B. Leshchenko, A. S. Khlyupina, D. O. Bohdan, Veb-dodatok dlya vedennya shchodennyka
kharchuvannya ta trenuvannya: vymohy, rozroblennya i vprovadzhennya [Web application for
keeping a diary of nutrition and training: requirements, development and implementation],
Radioelektronni i kompyuterni systemy [Radio electronic and computer systems] 3 (2018) 49–62.
[31] H. V. Makarova, Stvorennya mobilnoho dodatku dlya optymizatsiyi vahy ta kharchuvannya
lyudyny [Creating a mobile application for optimizing weight and nutrition], Systemy obrobky
informatsiyi [Information processing systems] 2 (2017) 187–191.
[32] L. M. Malyarets, I. Y. Ryzhykh, Zastosuvannya QR-rozkladu pryamokutnykh matryts
Khauskholderovymy vidobrazhennyamy v rehresiynomu analizi [Application of QR-
decomposition of rectangular matrices by Hausholder mappings in regression analysis],
Ekonomika rozvytku [Development economics] 1 (2009). 16–20.
[33] N. O. Matvyeyeva, Doslidzhennya zasobiv mashynnoho navchannya iz zaluchennyam suchasnykh
mov prohramuvannya [Research of machine learning tools with the involvement of modern
programming languages], Systemni tekhnolohiyi [System technologies] 1 (2018) 85–91.
[34] L. Mestetskiy, Matematicheskiye metody raspoznavaniya obrazov [Mathematical methods of
pattern recognition], MGU, 2004.
[35] O. P. Mintser, Obriyi rozvytku adaptyvnoho navchannya [Horizons of development of adaptive
learning], Medychna informatyka ta inzheneriya [Medical informatics and engineering] 1 (2017)
5–11.
[36] O. A. Mokrintsev, Poperednya obrobka zobrazhen dlya avtomatychnoho rozpiznavannya
odnovymirnykh shtrykhkodiv [Image pre-processing for automatic recognition of one-dimensional
barcodes], Systemy upravlinnya, navihatsiyi ta zvyazku [Management, navigation and
communication systems] 1 (2017) 111–113.
[37] O. Y. Moroz, Shtuchnyy intelekt versus pryrodnyy intelekt? (maybutnye lyudyny v konteksti
vyklykiv intelektualnykh supertekhnolohiy) [Artificial intelligence versus natural intelligence?
(the future of man in the context of the challenges of intelligent supertechnology)], Politolohichnyy
visnyk [Political Science Bulletin] 72 (2014) 18–35.
[38] T. L. Mostenska, H. O. Kundyeyeva, Kharchuvannya yak skladova prodovolchoyi bezpeky
[Nutrition as a component of food security], Naukovi pratsi Natsionalnoho universytetu
kharchovykh tekhnolohiy [Scientific works of the National University of Food Technologies]
22(3) (2016) 11- 3–122.
[39] Y. Motuzka, Kharchova ta enerhetychna tsinnist produktiv dlya spetsialnykh medychnykh tsiley
[Food and energy cyanide products for special medical purposes], Tovary i rynky [Goods and
markets] 2(1) (2017) 59–66.
[40] M. Oldenderfer, R. K. Bleshfild, Klasternyy analiz. Faktornyy, diskriminantnyy i klasternyy analiz
[Cluster analysis. Factor, discriminant and cluster analysis], Finance and statistics, 1989.
[41] V. O. Pishvanova, Pryntsypy adaptyvnoho navchannya [Principles of adaptive learning], Visnyk
Zaporizkoho natsionalnoho universytetu: Pedahohichni nauky [Bulletin of Zaporizhzhya National
University: Pedagogical sciences] 1 (2015) 178–183.
[42] V. S. Simakov, Y. V. Lutsenko, Adaptivnoye upravleniye slozhnymi sistemami na osnove teorii
raspoznavaniya obrazov [Adaptive control of complex systems based on the theory of pattern
recognition], KGTU, 1999.
[43] V. V. Slastin, Ye. S. Samuseva, L. V. Moskalchuk, Sbalansirovannyy ratsion pitaniya [A balanced
diet], Problemi kharchuvannya [Nutrition problems] 1 (2014) 33–39.
[44] I. Tereykovskyy, Neyromerezheva metodolohiya rozpiznavannya Internet-oriyentovanoho
shkidlyvoho prohramnoho zabezpechennya [Neural network methodology for recognizing
Internet-oriented malware], Bezpeka informatsiyi [Information security] 19(1) (2013) 24–28.
[45] T. V. Trachuk, T. P. Radishchuk, Metod matematychnoho modelyuvannya yak zasib realizatsiyi
shchodennoho ratsionu [The method of mathematical modeling as a means of implementing a daily
diet], Pedahohichnyy poshuk [Pedagogical search] 1 (2014) 34–36.
[46] Y. O. Tupalo, Vykorystannya metodiv mashynnoho navchannya na praktytsi [Use of machine
learning methods in practice], Kompyuterni zasoby, merezhi ta systemy [Computer tools, networks
and systems] 17 (2018) 101–110.
[47] O. V. Fedusenko, A. O. Fedusenko, I. M. Domanetska, Kontseptualna model adaptyvnoyi
informatsiynoyi systemy navchannya [Conceptual model of adaptive information learning
system], Upravlinnya rozvytkom skladnykh system [Management of complex systems
development] 32 (2017) 86–90.
[48] Y. Y. Yaninovych, H. V. Kachay, T. M. Shvets, Enerhetychna tsinnist produktiv [Energy value of
products], Rybohospodarska nauka Ukrayiny [Fisheries science of Ukraine] 2 (2011) 122–126.
[49] K. R. Muller, S. Mika, G. Ratsch, et al., An introduction to kernelbased learning algorithms, in:
Proceedings of the IEEE 108 Transactions on Neural Networks, 2001, pp. 181–202.
[50] . B. T. Al-Janabi, Computer vision system for froth flotation based on centroid, Системи обробки
інформації [Information processing systems] 9 (2014) 3–5.
[51] N. Bolohova, I. Ruban, Image processing models and methods research and ways of improving
marker recognition technologies in added reality systems, Suchasnyy stan naukovykh doslidzhenʹ
ta tekhnolohiy v promyslovosti [The current state of research and technology in industry] 1 (2019)
25–33.
[52] A. S. Dovbysh, D. V. Velykodnyi, O. B. Protsenko, V. I. Zimovets, Optimization of parameters of
machine learning of the system of functional diagnostics of the electric drive of a shaft lifting
machine, Radio electronics, computer science, management 2 (2018) 44–50.
[53] R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification. John Wiley & Sons, New York, 2001.
[54] J. Han, M. Kamber, J. Pei, Data mining: concepts and techniques. Elsevier, 2012.
[55] S. Melnychuk, V. Gubarev, N. Salnikov, Using Information Features in Computer Vision for 3d
Pose Estimation in Space, Kibernetika i vychislitel'naya tekhnika [Cybernetics and Computer
Science] 190 (2017) 33–55.
[56] I. Motuzka, D. Antiushko, Сlassification of products for enteral nutrition, Tovary i rynky [Goods
and markets] 2 (2015) 17–24.
[57] M. G. Pantelyat, Application of the finite element nethod to computer simulation of
electromagnetic and thermal processes in induction cookers and heated dishes, Visnyk
Cherkasʹkoho universytetu. Seriya: Fizyko-matematychni nauky [Bulletin of Cherkasy University.
Series: Physical and Mathematical Sciences] 1 (2017) 79–85.
[58] R. J. Shalkoff, J. Digital image processing and computer vision. New York; Chichester; Brisbane;
Toronto; Singapore: John Wiley & Sons, 1989.
[59] V. N. Vapnik, The nature of statistical learning theory. Springer, 1999.
[60] E. Ziglio, The WHO cross-national study of health behavior in school aged children from 35
countries. New York; Chichester; Brisbane; Toronto; Singapore: J. School Health, 2001.
[61] Y. Burov, V. Vysotska, P. Kravets, Ontological approach to plot analysis and modeling, CEUR
Workshop Proceedings Vol-2362 (2019) 22–31.
[62] Lytvyn Vasyl, Vysotska Victoria, Dosyn Dmytro, Holoschuk Roman, Rybchak Zoriana,
Application of Sentence Parsing for Determining Keywords in Ukrainian Texts, in: Proceedings
of the International Conference on Computer Sciences and Information Technologies, CSIT, 2017,
pp. 326–331. doi: 10.1109/STC-CSIT.2017.8098797.
[63] V. Lytvyn, P. Pukach, І. Bobyk, V. Vysotska, The method of formation of the status of personality
understanding based on the content analysis, Eastern-European Journal of Enterprise Technologies
5/2(83) (2016) 4-12. doi: 10.15587/1729-4061.2016.77174.
[64] V. Vysotska, Linguistic Analysis of Textual Commercial Content for Information Resources
Processing, in: Proceedings of the International Conference on Modern Problems of Radio
Engineering, Telecommunications and Computer Science, TCSET, 2016, pp. 709-713. doi:
10.1109/TCSET.2016.7452160.
[65] V. Lytvyn, V. Vysotska, P. Pukach, M. Vovk, D. Ugryn, Method of functioning of intelligent
agents, designed to solve action planning problems based on ontological approach, Eastern-
European Journal of Enterprise Technologies 3/2(87) (2017) 11-17. doi: 10.15587/1729-
4061.2017.103630.
[66] V. Lytvyn, N. Sharonova, T. Hamon, V. Vysotska, N. Grabar, A. Kowalska-Styczen,
Computational linguistics and intelligent systems, CEUR Workshop Proceedings 2136 (2018).
[67] N. Oleksiv, O. Oborska, K. Mykich, S. Mushasta, Y. Pukach, O. Tereshchuk, Information System
of Dynamic and Adaptive Control of Human Diet Based on Machine Learning Technology, CEUR
Workshop Proceedings Vol-2917 (2021) 373-406.
[68] T. Zohud, S. Zein. Cross-platform mobile app development in industry: A multiple case-study,
International Journal of Computing 20(1) (2021) 46–54.
[69] M. R. Baharon, M. F. Abdollah, N. A. Abu, Z. Z. Abidin, A. Idris, Secure video transcoding in
mobile cloud computing, International Journal of Computing 17(4) (2018) 208–218.
[70] V. Inzillo, F. D. Rango, A. A. Quintana. A low energy consumption smart antenna adaptive array
system for mobile ad hoc networks, International Journal of Computing 16(3) (2017) 124–132.
[71] B. Rusyn, L. Pohreliuk, A. Rzheuskyi, R. Kubik, Y. Ryshkovets, L. Chyrun, S. Chyrun, A.
Vysotskyi, V.B. Fernandes, The mobile application development based on online music library for
socializing in the world of bard songs and scouts’ bonfires, Advances in Intelligent Systems and
Computing 1080 (2020) 734–756. doi: 10.1007/978-3-030-33695-0_49.
[72] A. Rzheuskyi, A. Gozhyj, A. Stefanchuk, O. Oborska, L. Chyrun, O. Lozynska, K. Mykich, T.
Basyuk, Development of Mobile Application for Choreographic Productions Creation and
Visualization, CEUR Workshop ProceedingsVol-2386 (2019) 340-358.
[73] T. Teslyuk, V. Teslyuk, H. Lypak, N. Kunanets, N. Veretennikova, A Mobile Museum Guide
Application, CEUR Workshop Proceedings Vol-2631 (2020) 314-326.
[74] O. Artemenko, V. Pasichnyk, N. Kunanets, K. Shunevych, Using sentiment text analysis of user
reviews in social media for e-tourism mobile recommender systems, CEUR workshop proceedings
Vol-2604 (2020) 259-271.
[75] V. Lytvyn, V. Vysotska, A. Rzheuskyi, Technology for the Psychological Portraits Formation of
Social Networks Users for the IT Specialists Recruitment Based on Big Five, NLP and Big Data
Analysis, CEUR Workshop Proceedings Vol-2392 (2019) 147–171.
[76] N. Shakhovska, S. Fedushko, M. ml. Greguš, I. Shvorob, Yu. Syerov, Development of Mobile
System for Medical Recommendations, in: The 15th International Conference on Mobile Systems
and Pervasive Computing (MobiSPC), 155, 2019, pp. 43-50.
[77] H. Lipyanina, A. Sachenko, T. Lendyuk, S. Nadvynychny, S. Grodskyi, Decision tree based
targeting model of customer interaction with business page, CEUR Workshop Proceedings 2608,
1001–1012.
[78] N. Antonyuk, A. Vysotsky, V. Vysotska, V. Lytvyn, Y. Burov, A. Demchuk, I. Lyudkevych, L.
Chyrun, S. Chyrun, І. Bobyk, Consolidated Information Web Resource for Online Tourism Based
on Data Integration and Geolocation, in: Proceedings of the International Conference on Computer
Sciences and Information Technologies, 2019, pp. 15-20. doi: 10.1109/STC-CSIT.2019.8929790.
[79] J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in:
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015,
pp. 3431–3440, doi: 10.1109/CVPR.2015.7298965.
[80] B. Rusyn, R. Kosarevych, O. Lutsyk, V. Korniy, Segmentation of atmospheric cloud images by
remote sensing, in: Proceedings of the 14th international conference TCSET 2018, pp.213–216.
[81] V. Lytvyn, V. Vysotska, I. Peleshchak, I. Rishnyak, R. Peleshchak, Time Dependence of the
Output Signal Morphology for Nonlinear Oscillator Neuron Based on Van der Pol Model,
International Journal of Intelligent Systems and Applications 10 (2018) 8-17. doi:
10.5815/ijisa.2018.04.02.
[82] O. Klymovych, V. Hrabchak, O. Lavrut, T. Lavrut, V. Lytvyn, V. Vysotska, The Diagnostics
Methods for Modern Communication Tools in the Armed Forces of Ukraine Based on Neural
Network Approach, CEUR Workshop Proceedings Vol-2631 (2020) 198-208.
[83] V. Lytvyn, I. Peleshchak, R. Peleshchak, V. Vysotska, Information Encryption Based on the
Synthesis of a Neural Network and AES Algorithm, in: Proceedings of the 3rd International
Conference on Advanced Information and Communications Technologies, AICT, 2019, pp. 447-
450. doi: 10.1109/AIACT.2019.8847896.
[84] V. Danylyk, V. Vysotska, V. Lytvyn, S. Vyshemyrska, I. Lurie, M. Luchkevych, Detecting Items
with the Biggest Weight Based on Neural Network and Machine Learning Methods,
Communications in Computer and Information Science 1158 (2020) 383- 396. doi: 10.1007/978-
3-030-61656-4_26.
[85] V. Lytvynenko, W. Wojcik, A. Fefelov, I. Lurie, N. Savina, et al. Hybrid Methods of GMDH-
Neural Networks Synthesis and Training for Solving Problems of Time Series Forecasting, Lecture
Notes in Computational Intelligence and Decision Making 1020 (2020) 513-531.
[86] I. Tsmots, M. Medykovskyy, O. Skorokhoda, Synthesis of hardware components for verticalgroup
parallel neural networks, in: Proceedings of the International Conference on Computer Sciences
and Information Technologies, CSIT, 2015, pp. 1-4.
[87] M.O. Medykovskyi, I.G. Tsmots, O.V. Skorokhoda, Spectrum neural network filtration
technology for improving the forecast accuracy of dynamic processes in economics, Actual
Problems of Economics 162(12) (2014) 410-416.
[88] N. Shakhovska, V. Vysotska, L. Chyrun, Features of E-Learning Realization Using Virtual
Research Laboratory, in: Proceedings of the International Conference on Computer Sciences and
Information Technologies, CSIT, 2016, pp. 143–148. doi: 10.1109/STC-CSIT.2016.7589891.
[89] N. Shakhovska, V. Vysotska, L. Chyrun, Intelligent Systems Design of Distance Learning
Realization for Modern Youth Promotion and Involvement in Independent Scientific Researches,
Advances in Intelligent Systems and Computing 512 (2017) 175-198. doi: 10.1007/978-3-319-
45991-2_12.
[90] Z. Hu, Y. V. Bodyanskiy, N. Y. Kulishova, O. K. Tyshchenko, A Multidimensional Extended Neo-
Fuzzy Neuron for Facial Expression Recognition, International Journal of Intelligent Systems and
Applications 9(9) (2017) 29–36. doi: 10.5815/ijisa.2017.09.04.
[91] V. Hamolia, V. Melnyk, P. Zhezhnych, A. Shilinh. Intrusion detection in computer networks using
latent space representation and machine learning, International Journal of Computing 19(3) (2020)
442–448.
[92] Z. A. Kakarash, H. S. Ezat, S. A. Omar, N. F. Ahmed. Time series forecasting based on support
vector machine using particle swarm optimization, International Journal of Computing 21(1)
(2022) 76–88.
[93] M. Sirola, J. E. Hulsund, Machine-learning methods in prognosis of ageing phenomena in nuclear
power plant components, International Journal of Computing 20(1) (2021) 11–21.