=Paper=
{{Paper
|id=Vol-3013/20210428
|storemode=property
|title=The Methodology of Managed Functional Networks for Organizing Effective and Adaptive Human-Machine Dialogue in Automated Systems
|pdfUrl=https://ceur-ws.org/Vol-3013/20210428.pdf
|volume=Vol-3013
|authors=Evgeniy Lavrov,Olga Siryk,Igor Kirichenko,Nataliia Barchenko,Yana Chybiriak
|dblpUrl=https://dblp.org/rec/conf/icteri/LavrovSKBC21
}}
==The Methodology of Managed Functional Networks for Organizing Effective and Adaptive Human-Machine Dialogue in Automated Systems==
https://ceur-ws.org/Vol-3013/20210428.pdf
The Methodology of Managed Functional Networks for Organizing
Effective and Adaptive Human-Machine Dialogue in Automated
Systems
Evgeniy Lavrov1, Olga Siryk2, Igor Kirichenko3, Nataliia Barchenko1 and Yana Chybiriak1
1
Sumy State University, Sumy, Ukraine
2
Taras Shevchenko National University of Kyiv, Kyiv, Ukraine
3
National University of Civil Defence of Ukraine , Kharkov, Ukraine
Abstract
In the article the authors substantiate the necessity for improvement of adaptation mechanisms in man-
machine systems. They set a task of optimizing the man-machine interaction; describe the use of
functional networks for ergonomic design tasks’ solution. There are introduced the concepts of
“controlled functional network” and “neural functional network”, the principle of multi-stage optimization
of man-machine interaction. The offered method differs from the existing ones: by interconnecting
functional and neural networks; by the possibility of multiple (in the course of work) adaptation
(optimization) of the system "tailored to a man", which provides properties of adaptive man-machine
interaction.
Keywords 1
Automated system, information technology, ergonomics, human factor, man-machine, interaction,
dialogue, human-operator, adaptation, optimization
1. Introduction
In the context of smart manufacturing and the industrial revolution [1-3], unfortunately, the number of accidents
and threats to the environment and human health is increasing every year [1,4,5]. To completely eliminate a
person from the control loop requires very high costs. The reliability of control systems increasingly depends on
the "human factor"[6,7]. There are many problems related to ergonomics and the need to adapt automatics to the
peculiarities of a human-operator, depending on the state of the control object and environmental parameters [8,9].
2. Problem Statement
Modern ergonomic research is devoted to the issues of working conditions, anthropometry, risk minimization,
modeling and optimization of activities, ergonomic examination and other important issues [10-14]. However, the
central problem is the problem of adapting the information system to the characteristics of a person[15,16] . Many
scientific works are devoted to solving adaptation problems [17-19]. P. Brusilovsky defined the basic principles of
constructing adaptive systems [20,21] (Figure. 1).
Figure 1: The principle of adaptive control in the "man-computer" system[21].
ICTERI-2021, Vol I: Main Conference, PhD Symposium, Posters and Demonstrations, September 28 – October 2, 2021, Kherson, Ukraine
EMAIL: prof_lavrov@hotmail.com(E. Lavrov); lavrova_olia@ukr.net (O. Siryk); ikir238@gmail.com(I.Kirichenko)
ORCID: 0000-0001-9117-5727 (E. Lavrov); 0000-0001-9360-4388(O. Siryk); 0000-0001-7375-8275 (I.Kirichenko); 0000-0002-5439-8750 (N. Barchenko);
0000-0002-0634-7609 (Y.Chybiriak)
©️ 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
Changes in interface construction technologies require new solutions and taking into account new features of
human-machine interaction (Fig. 2) –“traditional human-machine interaction includes only human perception at
the output of the system (1), modern models should include human characteristics and functional state (2), as well
as communication with other operators (3)” [22].
Figure 2: Factors that must be taken into account when building adaptive systems "man-computer" [22].
Obviously, the model of a person and a model of human activity should be placed at the center of the adaptive
management model.
The most convenient model of activity is the "functional network" (FN) [23]. However, the capabilities of FN
have not yet been fully used to build adaptive systems.
Unfortunately, despite the huge number of articles, including on the methods of ergonomics [10-14,23, 24], the
creation of intelligent agents [18,20, 26, 27], the issues of the flexibility of the response of computer systems to
human parameters have not been completely resolved.
The task of the research is to develop an approach to the creation of models that allow the analysis of objective
quantitative characteristics of the human-operator and the technology of his interaction with means of automation:
• Predict the time-reliable results of activities;
• Carry out, in the process of all activities (at the control points), the choice of the most appropriate options for
man-machine interaction.
3. Results
3.1. Principle of description and assessment structures of human-machine dialogue
Simulation of elementary actions of operators and automatics is carried out using typical functional units
(TFU). The most common of these are the "work operation" with the designation "rectangle", "control operation"
with the designation "circle", and "alternative operation" with the designation "rectangle with several outputs". A
complete description of TFU models is given in [28, 29]. The FN that describes the algorithmic activity of the
human operator is built from those TFU. Examples of models (accuracy and run-time estimation) for some typical
functional structures (TFS) are shown in Table 1.
Table 1
Models for calculating reliability and dialogue time (fragment)
Content TFS diagram Index Formula for computation
of typical
functional
structure
n
1.Consiste Probability В = Bi
nt of error-free i =1
implementati operation
on of Expectation n
M (T ) = M (Ti )
operations value of the i =1
time of
operation
Dispersion n
D(T ) = D(Ti )
of the time of i =1
operation
Probability 1
2.Cyclic B = B1 * K 11 *
1 − ( B1 * K 10 + B 0 * K 00 )
functional of error-free
structure "An operation
operation Expectation M (T ) = (M (Tр ) + M (Tк )) * M ( L)
with action value of the M (L ) =
1
control time of 1 − ( B 1
* K 10
+ B 0 * K 00 )
without operation
restrictions Dispersion D(T ) = D(T ) * ( M (Tp ) + M (Tк ))2 +
on the of the time of ( D(Tp ) + D(Tк )) * M ( L)
number of operation B 1 * K 10 + B 0 * K 00
D( L ) =
cycles" (1 − ( B 1 * K 10 + B 0 * K 00 ))2
Expectation B = B11 * K 11 +
value of the ( B10 * K 00 + B11 * K 10 ) * B 21
3. time of
Functional operation
structure Expectation M (T ) = M (Tр1 ) + M (Tк ) +
"An value of the ( B10 * K 00 + B11 * K 10 ) * M (Tр 2 )
operation time of
with action operation
control and Dispersion D(T ) = D(Tр1 ) + D(Tк ) +
without of the time of ( B10 * K 00 + B11 * K 10 ) * D(Tр 2 ) +
restrictions operation
on the ( B10 * K 00 + B11 * K 10 ) *
number of ( B11 * K 11 + B10 * K 01 ) * M 2 (Tp 2 )
cycles"
*
- Subscripts in formulas correspond to the type (operating course – p; course of control – k) and / or to the
number of TFU.
Here:
B1 - the probability of error-free handling operation;
K11 - the probability of recognizing the correct operations performing;
K00 - the probability of detecting any errors;
M(T)- mathematical expectation of the operational run-time;
D(T) - the variance of the operational run-time.
These models are used to evaluate the entire FN that describes the man-machine interaction algorithm. The
reliability and runtime estimation is carried out by the method of “reduction” FN [30-31].
For the evaluation of human-computer interaction has developed special software systems [32-33].
3.2. Analysis of multivariance process of man-machine interaction
Man-machine interaction consists of a sequence of steps. At each step, there are many possible ways both to
implement the main technologically necessary operations and to monitor and correct errors[34-39]. A multiplicity
of options (excerpt) is shown in Figure 3.
Figure 3: Demonstration of multiple versions of man-machine interaction ; n- number of steps
3.3. Formulation of the optimization problem of man - machine interaction
The problem can be formulated as follows:
f k ( X ) → max (1)
( X ) → max (2)
P{T ( X ) To } (3)
X Xo (4)
Where
• X0 - a set of alternative options for the algorithm of activities;
• β(X) – the probability of error-free implementation of the algorithm of activities;
• T – random variable time implementation algorithm of activities;
• T0 – scheduled time of performance;
• α– minimum allowable probability of timely completion;
• fk(X) - the degree of functional comfort,
X– a vector characterizing the alternative structure of human-computer interaction
3.4. The method of controlled functional network
To implement procedures of the FN “control”, we offer the idea of interconnecting neural and functional
networks for modeling human-computer interaction, which the authors proposed in their work [35,40] before.
Probabilistic characteristics of human work elements, which are used as input data to the models of the operation
algorithms, can be represented as neural models, displaying these characteristics depending on various factors.
3.4.1. The general scheme. The principle of integration of functional and neural
networks
According to this approach and taking into account the changing characteristics of the human-operator and the
environment, a neural network (NN) D-network is created for each element of the functional network (Figure 4).
The purpose of D-network is to provide FN with relevant source data. D-network is constructed for each case
according to the requirements of the designer.
The following characteristics of the human-operator [40] may be input parameters for NN:
o training;
o type of nervous system;
o functional state;
o motivation;
o emotional stress level and others.
Output parameters of NN include:
o the probability of error-free operation (algorithm);
o expectation time of the operation (algorithm);
o intensity of operator’s activity and others.
Let us consider the example of a model, constructed for the problem of fore-casting the results of a computer-
aided instruction system. Suppose, for example, one must take into account the individual characteristics of a
person and to adapt the system, at each step, to him.
There are, for example, such options (alternatives) of learning algorithm:
a) The sequence of operations without control.
b) The sequence of operations controlled after each operation.
c) The sequence of operations with the final control in the end.
d) The sequence of operations with functional control.
We need to select the most suitable mode of learning, taking into account the individual characteristics of the
trainee, his goals and the importance of the criteria To solve this problem, we form and train the A-network. Let
A-network input be data from the model of the human-operator, then the network output falls into the
recommended mode.
3.4.2. Adaptation points. Incremental methods of decision-making
Parameters of a man and an environment are not permanent and dynamically change over time. Moreover, the
selected mode of interaction cannot be sufficiently effective in this case. Therefore, there is a need for periodic
assessment of the situation and making adjustments in the process of interaction. Under adjustments we
understand algorithm change interaction. To do this, we add point adaptation (Figure 2) into the process to
provide control at each point and, if necessary- the reconstruction of the network.
Figure. 4: The principle of controlled functional network (idea of prof. Anatoly Ilyich Gubinsky)
- step of the functional network;
- operation of functional control;
- neural network;
- initial data about the quality of the operation;
P.а.i - point of adaptation № i ( i = 1, n , where n -number of points of adaptation).
Depending on the availability of input data, a decision-making at points of adaptation may be carried
out, using:
o neural network (А-network) [35,40];
o models of fuzzy logic [35,40];
o mathematical programming models used in functional networks [31,33].
Thus, the problem is reduced to the multiple implementation of the optimization problem of (1) - (4)
3.5. Testing. Computer system for adaptation of man-machine interaction.
The complex of models has been tested in the technology of intelligent agents for e-learning [35, 40]
(figure 5).
Figure 5: The principle of the agent-manager for e-learning
The neural-functional network uses data from all university databases and manages the human-
computer dialogue. The computer program performs the following actions:
1. analyzes data on previous sessions of training (for this student and other students);
2. forms several neural networks to predict the time and accuracy of the electronic educational
module (for a given student and for given learning conditions);
3. selects a neural network that provides minimal error for the prediction results;
4. builds a model of the process of dialogue (in the form of a functional network);
5. enters initial data for a particular student into the dialogue process model;
6. assesses the accuracy and time of activity for this student, taking into account the motivation
and time that is available, as well as taking into account the functional state of the human
operator;
7. forms tips for the student (which electronic training module to choose, how to carry out
monitoring activities, etc.);
8. analyzes current progress and correcting recommendations for the student.
A computer program that is controlled by a neural-functional network performs all of these actions for
all possible control points by dialogue.
The results were embedded in:
- Moscow State University named by M.V. Lomonosov,
- Belgorod Agrarian Academy,
- Sumy National Agrarian University,
- Sumy State University;
- Vinnytsia State Agrarian University,
- Ukrainian Engineering and Pedagogical Academy (Kharkov),
- and other universities.
The use of the agent-manager allowed the experimental group (Sumy National Agrarian University,
the results were obtained in the dissertation of Barchenko N.L.) to raise the average score from 72.32
to 81.43 and reduce the percentage of refusal to work with e-learning from 24.78% to 7.29%.
4. Conclusion
Man-machine interaction in discrete automated systems can be well described using models, based on
functional networks. Adaptive changes in man-machine interaction can be reduced to the problem of
step-by-step choosing the optimal fragment of the functional network. The method adapts the system
to the peculiarities of the human-operator and environmental parameters. The combined model, which
consists of a neural network for forming initial data, a functional network for modeling a dialogue and
a neural network for managing the dialogue process provides a higher level of adaptation to a human
operator than the known models built on the basis of unmanaged functional networks. The computer
program was used in the design process for systems of various purposes and its effectiveness was
shown. Experimental studies have shown the constructiveness of the developed method.
Models will be useful for automated control in industry, agriculture and e-learning
5. Acknowledgements
The authors dedicate this article to the memory of their teacher, professor Anatoly Ilyich Gubinsky
(1931-1990), who first formulated the ideas that formed the basis of the study.
6. References
[1] Zhong, R. Y. et al., Intelligent manufacturing in the context of industry 4.0: a review., in:
Engineering, 3(5), 616–630, (2017).
[2] Dudukalov E.V. et al.: The use of artificial intelligence and information technology for
measurements in mechanical engineering and in process automation systems in Industry 4.0 J.
Phys.: Conf. Ser. 1889, 052011 (2021), doi:10.1088/1742-6596/1889/5/052011
[3] Ogurtsov, E.S. et al., Microcontroller navigation and motion control system of the underwater
robotic complex, in: ARPN Journal of Engineering and Applied Sciences, 11(9), 3110-3121,
(2016).
[4] Wong, C. K., Designs for Safer Signal-Controlled Intersections by Statistical Analysis of
Accident Data at Accident Blacksites, in: IEEE Access, 7, 111302-111314(2019), doi:
10.1109/ACCESS.2019.2928038.
[5] Sedov, V. A., Sedova, N. A. and Glushkov, S. V., The fuzzy model of ships collision risk rating
in a heavy traffic zone, in:Vibroengineering PROCEDIA, 8, 453–458, (2016)
[6] Radziwon, A. et al., The smart factory: exploring adaptive and flexible manufacturing solutions,
in: Procedia Engineering, 69, 1184–1190, (2014).
[7] Saarelainen, K. and Jäntti, M., Quality and human errors in IT service infrastructures — Human
error based root causes of incidents and their categorization, in: 2015 11th International
Conference on Innovations in Information Technology (IIT), 207-212, (2015), doi:
10.1109/INNOVATIONS.2015.7381541.
[8] Sawangsri, W. at al, Novel Approach of an Intelligent and Flexible Manufacturing System. A
Contribution to the Concept and Development of Smart Factory, in: 2018 International
Conference on System Science and Engineering (ICSSE), New Taipei, 1–4, (2018) , doi:
10.1109/ICSSE.2018.8520029
[9] Charalambous, G. and Stout, M., Optimising train axle inspection with the implementation of
human-robot collaboration: A human factors perspective, in: 2016 IEEE International
Conference on Intelligent Rail Transportation (ICIRT), 254-258, (2016), doi:
10.1109/ICIRT.2016.7588741.
[10] Rothmorea, P. et al., The implementation of ergonomics advice and the stage of change
approach, in: Applied Ergonomics, 51, 370–376, (2015), doi: 10.1016/j.apergo.2015.06.013
[11] Cacciabue, P.C., Human error risk management for engineering systems: a methodology for
design, safety assessment, accident investigation and training , in: Reliability Engineering &
System Safety. 83(2), 229 – 269, ( 2014) , doi: 10.1016/j.ress.2003.09.013
[12] Dul, R. et al., Strategy for human factors/ergonomics: developing the discipline and profession,
in: Ergonomics, 55(4), 377–395, (2012), doi: 10.1080/00140139.2012.661087
[13] Bentley, T. A. et al., The Role of organisational support in teleworker wellbeing: A socio-
technical systems approach, in: Applied Ergonomics. 52, 207–215, (2016), doi:
10.1016/j.apergo.2015.07.019
[14] Pasquale, V. Di. et al., Methodology for the analysis and quantification of human error
probability in manufacturing systems, in: 2016 IEEE Student Conference on Research and
Development (SCOReD), 1-5, (2016), doi: 10.1109/SCORED.2016.7810093.
[15] Kotova, E. E., Applying Educational Data Mining Tools to Learning Management Problems, in:
2019 III International Conference on Control in Technical Systems (CTS), 180-183, (2019), doi:
10.1109/CTS48763.2019.8973291.
[16] Pisarev, I.A. et.al , Assessment of the Efficiency of Operators Work in Solving Test Problems in
the Structure of Intelligent Interfaces, in: 2020 IEEE Conference of Russian Young Researchers
in Electrical and Electronic Engineering (EIConRus), 810-813, (2020), doi:
10.1109/EIConRus49466.2020.9039331.
[17] Petukhov, I. et al., Application of virtual reality technologies in training of man-machine system
operators, in: 2017 International Conference on Information Science and Communications
Technologies (ICISCT), 1-7, (2017), doi: 10.1109/ICISCT.2017.8188569
[18] Osadchyi, V.et al., Personalized and Adaptive ICT-Enhanced Learning: ABrief Review of
Research from 2010 to 2019, in: ICTERI Workshops 2020: Computer Science. (2020). URL:
http://ceur-ws.org/Vol-2732/20200559.pdf
[19] Burov O. et al. , Cybersecurity in educational networks, in: Advances in Intelligent Systems and
Computing. Springer, 1131 AISC, 359-364, (2020), doi:10.1007/978-3-030-39512-4_56
[20] Brusilovsky, et al., Open Social Student Modeling for Personalized Learning, in: IEEE
Transactions on Emerging Topics in Computing, 4(3), 450-461, (2016), doi:
10.1109/TETC.2015.2501243.
[21] Kaya, G., Altun, A. , Utilizing a smart cognitive support system for K-8 education, in : Smart
Learn. Environ. 5, 17 (2018), doi: 10.1186/s40561-018-0066-x
[22] Riener, A.: Perceptual Computer Science—human-centric and reality-based human-machine
interaction (Habilitation thesis). Johannes Kepler University Linz, Austria , 2014
[23] Lavrov E. et al, Functional Networks for Modeling and Optimization Human-Machine Systems,
in: Sumpor D., Jambrošić K., Jurčević Lulić T., Milčić D., Salopek Čubrić I., Šabarić I. (eds)
Proceedings of the 8th International Ergonomics Conference. ERGONOMICS 2020. Advances
in Intelligent Systems and Computing, 1313. Springer, Cham (2021), doi:10.1007/978-3-030-
66937-9_21
[24] Tortorella, G. L. et. al, Lean manufacturing implementation: an assessment method with regards
to socio-technical and ergonomics practices adoption, in: The Int. J. of Advanced Manufacturing
Technology, 1–12 (2016)
[25] Lavrov, E. et al., Analysis of Working Conditions and Modeling of Activity Algorithms for
Contact-Center Operators, in: Sumpor D., Jambrošić K., Jurčević Lulić T., Milčić D., Salopek
Čubrić I., Šabarić I. (eds) Proceedings of the 8th International Ergonomics Conference.
ERGONOMICS 2020. Advances in Intelligent Systems and Computing, 1313. Springer, Cham.
(2021) doi:10.1007/978-3-030-66937-9_14
[26] Sedova, N. et al., Automated Stationary Obstacle Avoidance When Navigating a Marine Craft,
in: 2019 Int. Multi-Conference on Engineering, Computer and Information Sciences
(SIBIRCON), 0491-0495 (2019) doi: 10.1109/SIBIRCON48586.2019.8958145.
[27] Verkhova, G. V., Akimov, S. V. , Electronic educational complex for training specialists in the
field of technical systems management, in: Proceedings of IEEE II International Con-ference on
Control in Technical Systems (CTS), 26–29 (2017).
[28] A.N.Adamenko et al., Information controlling human-machine systems: research, design, testing,
Reference book, Gubinsky, A.I. & Evgrafov, V.G., eds., Mechanical Engineering, Мoscow,
1993. (In Russian)
[29] P.R Popovich, A.I. Gubinskiy, G.M. Kolesnikov, Ergonomic support of astronauts' activities,
Mechanical Engineering, 1985. (In Russian)
[30] Lavrov, E. et al., Mathematical Models for Reducing Functional Networks to Ensure the
Reliability and Cybersecurity of Ergatic Control Systems, in: 2020 IEEE 15th International
Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer
Engineering (TCSET), Lviv-Slavske, Ukraine, 179-184, (2020), doi:
10.1109/TCSET49122.2020.235418.
[31] Lavrov, E. A. et al., Decision Support Method for Ensuring Ergonomic Quality in Polyergatic IT
Resource Management Centers , in: 2019 III International Conference on Control in Technical
Systems (CTS), St. Petersburg, Russia, 148-151, (2019) doi: 10.1109/CTS48763.2019.8973265
[32] Lavrov, E.A.et al ., Automation of Functional Reliability Evaluation for Critical Human-
Machine Control Systems, in: 2019 III International Conference on Control in Technical Systems
(CTS), St. Petersburg, Russia, 2019, 144-147, (2019), doi: 10.1109/CTS48763.2019.8973294.
[33] Lavrov, E. et al., Management for the Operators Activity in the Polyergatic System. Method of
Functions Distribution on the Basis of the Reliability Model of System States, in: 2018
International Scientific-Practical Conference Problems of Infocommunications. Science and
Technology (PIC S&T), Kharkiv, Ukraine, 1-6, (2018), doi:
10.1109/INFOCOMMST.2018.8632102
[34] Grif, M. G. et al., Automation of Human-Machine Systems Design Based on Functional-
Structural Theory, in: 2018 XIV International Scientific-Technical Conference on Actual
Problems of Electronics Instrument Engineering (APEIE), Novosibirsk, 396-399, (2018), doi:
10.1109/APEIE.2018.8545934.
[35] Lavrov, E. et al, Development of models for the formalized description of modular e-learning
systems for the problems on providing ergonomic quality of human-computer interaction, in:
Eastern-European Journal of Enterprise Technologies. Ser. “Information technology”, 2/2(86),
4–13, (2017), doi: 10.15587/1729-4061.2017.97718
[36] Yang, J., Online application of a risk management system for risk assessment and monitoring at
NPPs , in: Nuclear Engineering and Design, 305, 200-212, (2016.)
[37] Hassnain, A., Available recovery time prediction in case of an accident scenario for NPP
component, in: Progress in Nuclear Energy, 97, 115-122 (2017 )
[38] Xu, P. et al ., Analysis of operator support method based on intelligent dynamic interlock in lead-
cooled fast reactor simulator, in: Annals of Nuclear Energy, 99, 279-282(2017)
[39] Pinchuk, O et al ., Learning as a Systemic Activity, in: Advances in Intelligent Systems and
Computing, 963, 335-342, (2019), doi: 10.1007/978-3-030-20135- 7_33
[40] Lavrov, E. et al., A method to ensure the effectiveness and attractiveness of e-learning. Human-
oriented systemic ergonomic approach, in: CEUR Workshop Proceedings, 2732, 572-582,
(2020)