<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Enhancing personal financial management skills through a machine learning-powered business simulator</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Dmytro S. Antoniuk</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Tetiana A. Vakaliuk</string-name>
          <email>tetianavakaliuk@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Vladyslav V. Didkivskyi</string-name>
          <email>v.didkivskyi@sana-commerce.com</email>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Oleksandr Yu. Vizghalov</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Oksana V. Oliinyk</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Valentyn M. Yanchuk</string-name>
          <email>v.yanchuk@gmail.com</email>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="editor">
          <string-name>PCWrEooUrckResehdoinpgs ISSNc1e6u1r-3w-0s0.o7r3g</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Academy of Cognitive and Natural Sciences</institution>
          ,
          <addr-line>54 Universytetskyi Ave., Kryvyi Rih, 50086</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Institute for Digitalisation of Education of the NAES of Ukraine</institution>
          ,
          <addr-line>9 M. Berlynskoho Str., Kyiv, 04060</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Kryvyi Rih State Pedagogical University</institution>
          ,
          <addr-line>54 Universytetskyi Ave., Kryvyi Rih, 50086</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Zhytomyr Polytechnic State University</institution>
          ,
          <addr-line>103 Chudnivsyka Str., Zhytomyr, 10005</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
      </contrib-group>
      <fpage>198</fpage>
      <lpage>205</lpage>
      <abstract>
        <p>Efective personal financial management is a critical life skill for all individuals, irrespective of their profession. A survey conducted among university students in Ukraine revealed a lack of suficient financial literacy and highlighted the need to incorporate personal finance education into curricula. Business simulators present a promising solution to bridge this gap. This paper introduces a novel web-based business simulator equipped with machine learning capabilities to facilitate the development of personal financial management skills. The methodology for utilising the simulator, including its content, objectives, formats, methods, and tools, is elucidated in detail. The key features and sections of the simulator are described, along with the specific personal finance management skills it aims to cultivate. To enhance the simulator's efectiveness, elements of machine learning, particularly reinforcement learning, have been incorporated. The simulator is designed to cater to a wide audience, from school-aged children to adults, and can be integrated into economics courses at both secondary and tertiary education levels in Ukraine. The paper concludes with a discussion on the future prospects of using such simulators to develop managerial and financial competencies among students from diverse specialities.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;personal financial management</kwd>
        <kwd>business simulator</kwd>
        <kwd>machine learning</kwd>
        <kwd>reinforcement learning</kwd>
        <kwd>financial literacy</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Personal financial management is a vital skill that everyone should possess, regardless of their profession
or background. It involves the efective management of one’s financial resources, including income,
expenses, savings, investments, and debts.</p>
      <p>
        The importance of developing personal finance competency has been widely acknowledged in
international research. Lusardi [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] discussed the influence of financial literacy on the well-being of
people worldwide and the need to recognise it as a fundamental right and universal need. Lusardi et al.
[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] demonstrated that 30–40% of retirement wealth inequality in the USA is associated with the level of
ifnancial literacy alone. Urban et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] concluded that high school financial literacy courses have a
significant impact on lower default rates and better credit scores.
      </p>
      <p>
        The use of business simulations in education has become increasingly common, with applications
ranging from practical training, such as flight or combat simulations, to economic, managerial, and
ifnancial domains. Researchers have studied the pedagogical significance of this technology-enhanced
educational method. Hernández-Lara et al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] concluded that business simulations have a positive
impact on generic competencies, while Farashahi and Tajeddin [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] confirmed their efectiveness in a
comparative study. The evolution of technologies has enabled the development of immersive, attractive,
and highly functional business simulations for various use cases, from elementary schools to lifelong
learning establishments. Korgin [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] provided evidence of the efectiveness of simulation games for
improving basic arithmetical operations literacy in children, while Palan [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] studied the criteria and
approaches for selecting business simulation software for asset market experiments in higher education.
      </p>
      <p>The analysis provides evidence of the efectiveness of using business simulations in education and the
need for further development of business simulations for economic and financial literacy development,
assessment, and the study of their behavioural aspects.</p>
      <p>The purpose of this article is to describe the methodology for using a business simulator with machine
learning elements to develop personal finance management skills.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Results</title>
      <p>
        An analysis of existing simulators [
        <xref ref-type="bibr" rid="ref8 ref9">8, 9</xref>
        ] revealed that most simulation software is designed for specific
areas of finance in countries with developed financial markets. Based on these findings, this work
presents an experiment in developing a more generalised simulator for countries with less developed
ifnancial instrument markets.
      </p>
      <p>
        Consequently, the authors developed a web-based business simulator [
        <xref ref-type="bibr" rid="ref10 ref11 ref12">10, 11, 12</xref>
        ] to foster personal
ifnance management skills. The methodology for using this simulator, which incorporates machine
learning elements, is presented in the following sections.
      </p>
      <p>The proposed methodology encompasses the purpose and content of the application, as well as the
forms, methods, and tools employed. It is focused on the expected outcome: improved personal finance
management skills through the use of a business simulator with machine learning elements.</p>
      <p>The purpose of using the simulator is to develop personal finance management skills, while the
content involves enhancing the teaching process of normative disciplines through its integration.</p>
      <p>The methodology includes two key methods for utilising the simulator:
1. Adaptive learning. The simulator creates a game-like process, known as the examination cycle, in
which users encounter problems corresponding to their current competence level at each stage.
As users progress through the simulation, they gain access to new personal finance management
tools that could not be efectively used at previous competence levels. This approach requires
users to iteratively master increasingly complex knowledge, skills, and abilities, and apply them
in situations resembling real-life scenarios. Users can explore previously unfamiliar tools and
situations or those of additional interest to them, considering their life cycle stage, field of activity,
or current problems and interests.
2. Situational modelling. The simulator is based on realistic simulations of common and specific
situations in personal or family finance management. It begins by modelling the creation of a
diversified currency and financial basket storage forms, then progresses to more complex tools,
such as credit and deposit operations with varying time parameters, payment schedules, and
provision forms. The simulation’s realism is ensured by the presence of probabilistic events
with positive or negative impacts of diferent monetary values on the user’s personal finances.
Non-financial investment transactions are introduced as the next step in familiarising users and
developing their competence in personal finance management.</p>
      <p>The main forms of conducting training sessions using the simulator within the framework of this
methodology include:
• Introductory classes for teachers and facilitators on the simulator’s functionality, modes, features,
analytical capabilities, and means of scientific research on the efectiveness and adaptability of
the user competence development process;
• Autonomous and group independent work with the simulator, aimed at both independent in-depth
or convenient pace learning through the training and game plot, and joint learning activities of
user groups and facilitators, with or without competitive elements;
• In-depth analysis of simulator situations can take place in face-to-face or remote formats with
individual students or groups, focusing on additional elaboration of situations, behavioural and
psychological aspects, or financial instruments that cause dificulties, misunderstanding, or attract
additional user interest.</p>
      <p>The tools for developing personal finance management skills provided in the proposed methodology
include computers, smartphones, and tablets with Internet access, the business simulator with machine
learning elements, and teaching materials.</p>
      <p>The expected result of the proposed methodology is the formation of personal money management
skills at a high level and the acquisition of skills to successfully apply the business simulator with
machine learning elements to perform practical work.</p>
      <p>Within the framework of this methodology, diferent forms and methods of using the simulator are
ofered, such as:
• Organisation and development of the simulator to develop personal finance management skills;
• Sessions using the simulator to simulate socio-economic situations corresponding to the lesson
topic;
• Organisation of thematic economic training using the simulator;
• Visualisation of economic and behavioural concepts;
• Using the simulator as a means of targeted in-depth problem-based learning;
• Using the simulator as a means of organising assessment.</p>
      <p>The main features of the simulator, presented in diferent sections (figure 1), include:
1. Current account management options.
2. Savings management options.
3. Deposit management options.
4. Credit management options.
5. Non-financial investment management options.
6. Information on changes in current accounts that have occurred in the last week.
7. Analytical information on the dynamics of changes in current account funds, savings, and
investments.
8. Information on current exchange rates.
9. List of recent transactions made on current accounts.</p>
      <p>The simulator facilitates the formation of knowledge, skills, competencies, and personal attitudes
towards:
• Using a diversified list of currencies.
• Forming a widely-acknowledged standard of emergency savings and assessing readiness for
longer-term investment.
• Using basic saving and investment instruments available in a wide range of countries, such as
deposits (deposit certificates), real estate, and business investments, and understanding their
specific characteristics.
• Debt management and forming a personal behavioural attitude towards using debt as a
constructive investment leverage.</p>
      <p>
        To enhance the efectiveness of the simulator, machine learning elements, particularly reinforcement
learning, have been applied. Reinforcement learning (RL) is a type of machine learning in which an
agent directly examines environmental data, receives rewards, and sets policies for optimal action
(figure 2). The goal of RL is to find the optimal policy that maximises the expected amount of future
rewards [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
      </p>
      <p>One of the algorithms used in reinforcement learning is Q-learning, which aims to learn a strategy
that informs the agent about the best action to perform in a particular state . The algorithm contains
elements such as the reward received (), the learning pace ( ), and the depreciation ratio ( ) (figure 3).</p>
      <p>
        The open-source library SharpRL [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] was chosen to provide the basic functionality for developing
a reinforcement learning environment based on the Q-learning algorithm. The program for defining
personal financial strategies is a console application that involves setting certain parameters, such as
the number of simulation passes in training mode, the number of simulations in policy adherence mode
after training, the duration of the simulation in weeks, and the user’s monthly income and expenses in
a specific currency.
      </p>
      <p>Experiments were conducted to determine the optimal ratio of initial parameters, and it was found
that the best results were obtained when the number of simulation passes in both training and policy
adherence modes was close to 1000. Increasing these figures did not significantly improve the results
(figure 4).</p>
      <p>To test the trained model, the results of the system were compared with the average results of real
users in the developed simulator over a 54-week period. It was found that the system consistently
outperformed the average user results (figures 5, 6, 7, 8).</p>
      <p>The graphs depicting the reinforcement learning model results exhibit jumps, particularly when
higher incomes and a large diference between income and expenses are involved. This is due to the
specific rules described to train the model, ensuring that the virtual agent system follows the best
path. For example, if an agent opens a 3-month deposit, the deposit will be closed automatically after
this period, significantly afecting the user’s reward for that week. However, the system learns and
“understands” where to invest or save, leading to sharp increases in performance, as evident in the
graphs. Such behaviour is not observed in the graphs representing real user results, indicating users’
lack of awareness of efective personal finance management strategies within the developed simulator.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Conclusions</title>
      <p>Economic processes are a new topic for research into the possibilities of applying the full potential
of machine learning. Scientists worldwide are making initial attempts to reproduce such processes
programmatically to use artificial intelligence to find solutions and answer various economic questions.</p>
      <p>The proposed developed software package consists of two parts: a personal finance management
simulator and a system for determining efective financial strategies, which utilises reinforcement
learning opportunities.</p>
      <p>The simulator can be used in the future to teach elements of personal finance management to people
who are not suficiently knowledgeable in this field. Moreover, the web application can be useful even
for school-aged children, complementing the educational process within economic courses not only in
higher education institutions but also in secondary education institutions in Ukraine.</p>
      <p>When constructing a methodology for using a business simulator with machine learning elements to
develop personal finance management skills, it is advisable to consider various types, scopes, methods
of placement, and purposes of using business simulators. The use of a business simulator with machine
learning elements is expedient and contributes to an increase in the eficiency of the educational process,
the formation of personal finance management skills, and the development of a steady cognitive interest
in students’ educational activities. The application of the author’s methodology will improve and
supplement the educational process in higher education by including a business simulator with machine
learning elements.</p>
      <p>Additionally, prospects for further study include the selection of diferent simulators of this type,
the possibility of using such simulators to develop managerial and financial competencies of students
in various specialities, as well as the development of an appropriate methodology and testing its
efectiveness. In the future, the authors plan to develop guidelines for teachers on using a business
simulator with machine learning elements to develop personal finance management skills in the
educational process of higher education.</p>
      <p>Declaration on Generative AI: During the preparation of this work, the authors used Claude 3 Opus in order to: Improve
writing style, Abstract drafting. After using this service, the authors reviewed and edited the content as needed and takes full
responsibility for the publication’s content.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>A.</given-names>
            <surname>Lusardi</surname>
          </string-name>
          ,
          <article-title>Financial literacy and the need for financial education: evidence and implications</article-title>
          ,
          <source>Swiss Journal of Economics and Statistics</source>
          <volume>155</volume>
          (
          <year>2019</year>
          )
          <article-title>1</article-title>
          . doi:
          <volume>10</volume>
          .1186/s41937-019-0027-5.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>A.</given-names>
            <surname>Lusardi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.-C.</given-names>
            <surname>Michaud</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O. S.</given-names>
            <surname>Mitchell</surname>
          </string-name>
          ,
          <article-title>Optimal financial knowledge and wealth inequality</article-title>
          ,
          <source>Journal of Political Economy</source>
          <volume>125</volume>
          (
          <year>2017</year>
          )
          <fpage>431</fpage>
          -
          <lpage>477</lpage>
          . doi:
          <volume>10</volume>
          .1086/690950.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>C.</given-names>
            <surname>Urban</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Schmeiser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. M.</given-names>
            <surname>Collins</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. Brown,</surname>
          </string-name>
          <article-title>The efects of high school personal financial education policies on financial behavior</article-title>
          ,
          <source>Economics of Education Review</source>
          <volume>78</volume>
          (
          <year>2020</year>
          )
          <article-title>101786</article-title>
          . doi:
          <volume>10</volume>
          .1016/j.econedurev.
          <year>2018</year>
          .
          <volume>03</volume>
          .006.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>A. B.</given-names>
            <surname>Hernández-Lara</surname>
          </string-name>
          ,
          <string-name>
            <surname>E.</surname>
          </string-name>
          <article-title>Serradell-López, À</article-title>
          . Fitó-Bertran,
          <article-title>Students' perception of the impact of competences on learning: An analysis with business simulations</article-title>
          ,
          <source>Computers in Human Behavior</source>
          <volume>101</volume>
          (
          <year>2019</year>
          )
          <fpage>311</fpage>
          -
          <lpage>319</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.chb.
          <year>2019</year>
          .
          <volume>07</volume>
          .023.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M.</given-names>
            <surname>Farashahi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Tajeddin</surname>
          </string-name>
          ,
          <article-title>Efectiveness of teaching methods in business education: A comparison study on the learning outcomes of lectures, case studies and simulations</article-title>
          ,
          <source>The International Journal of Management Education</source>
          <volume>16</volume>
          (
          <year>2018</year>
          )
          <fpage>131</fpage>
          -
          <lpage>142</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.ijme.
          <year>2018</year>
          .
          <volume>01</volume>
          .003.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>N. A.</given-names>
            <surname>Korgin</surname>
          </string-name>
          ,
          <article-title>Introduction to theory of control in organizations for kids via interactive games</article-title>
          ,
          <source>IFAC-PapersOnLine</source>
          <volume>48</volume>
          (
          <year>2015</year>
          )
          <fpage>289</fpage>
          -
          <lpage>294</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.ifacol.
          <year>2015</year>
          .
          <volume>11</volume>
          .250.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>S.</given-names>
            <surname>Palan</surname>
          </string-name>
          ,
          <article-title>GIMS-Software for asset market experiments</article-title>
          ,
          <source>Journal of Behavioral and Experimental Finance</source>
          <volume>5</volume>
          (
          <year>2015</year>
          )
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.jbef.
          <year>2015</year>
          .
          <volume>02</volume>
          .001.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>D.</given-names>
            <surname>Antoniuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Vakaliuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Yanchuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Yakobchuk</surname>
          </string-name>
          ,
          <article-title>Review of software tools for planning personal finances, in: Collection of materials of the VII All-Ukrainian Scientific and Practical Conference of Young Scientists "Scientific Youth-2019"</article-title>
          , Comprint,
          <year>2019</year>
          , pp.
          <fpage>105</fpage>
          -
          <lpage>107</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>D.</given-names>
            <surname>Antoniuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Vakaliuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Didkivskyi</surname>
          </string-name>
          ,
          <string-name>
            <surname>O. Vizghalov,</surname>
          </string-name>
          <article-title>The need to develop a personal finance management simulator</article-title>
          ,
          <source>Innovative pedagogy 24</source>
          (
          <year>2020</year>
          )
          <fpage>208</fpage>
          -
          <lpage>212</lpage>
          . URL: http://www.innovpedagogy. od.ua/archives/2020/24/part_2/43.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Personal</surname>
            <given-names>finance</given-names>
          </string-name>
          ,
          <year>2020</year>
          . URL: https://financesimulator.azurewebsites.net.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>D. S.</given-names>
            <surname>Antoniuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. A.</given-names>
            <surname>Vakaliuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. V.</given-names>
            <surname>Didkivskyi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Vizghalov</surname>
          </string-name>
          ,
          <article-title>Development of a simulator to determine personal financial strategies using machine learning</article-title>
          , in: A. E. Kiv,
          <string-name>
            <given-names>S. O.</given-names>
            <surname>Semerikov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. N.</given-names>
            <surname>Soloviev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Striuk</surname>
          </string-name>
          (Eds.),
          <source>Proceedings of the 4th Workshop for Young Scientists in Computer Science &amp; Software Engineering</source>
          , volume
          <volume>3077</volume>
          <source>of CS&amp;SESW</source>
          <year>2021</year>
          , CEUR Workshop Proceedings, CEUR-WS.org,
          <year>2022</year>
          , pp.
          <fpage>12</fpage>
          -
          <lpage>26</lpage>
          . URL: https://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>3077</volume>
          /paper02.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>D. S.</given-names>
            <surname>Antoniuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. A.</given-names>
            <surname>Vakaliuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. V.</given-names>
            <surname>Didkivskyi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Vizghalov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O. V.</given-names>
            <surname>Oliinyk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. M.</given-names>
            <surname>Yanchuk</surname>
          </string-name>
          ,
          <article-title>Using a business simulator with elements of machine learning to develop personal finance management skills</article-title>
          , in: V. A.
          <string-name>
            <surname>Ermolayev</surname>
            ,
            <given-names>A. E.</given-names>
          </string-name>
          <string-name>
            <surname>Kiv</surname>
            ,
            <given-names>S. O.</given-names>
          </string-name>
          <string-name>
            <surname>Semerikov</surname>
            ,
            <given-names>V. N.</given-names>
          </string-name>
          <string-name>
            <surname>Soloviev</surname>
            ,
            <given-names>A. M.</given-names>
          </string-name>
          <string-name>
            <surname>Striuk</surname>
          </string-name>
          (Eds.),
          <source>Proceedings of the 9th Illia O. Teplytskyi Workshop on Computer Simulation in Education (CoSinE</source>
          <year>2021</year>
          )
          <article-title>co-located with 17th International Conference on</article-title>
          ICT in Education, Research, and Industrial Applications: Integration, Harmonization, and Knowledge Transfer, volume
          <volume>3083</volume>
          <source>of CoSinE</source>
          <year>2021</year>
          , CEUR Workshop Proceedings, CEUR-WS.org,
          <year>2022</year>
          , pp.
          <fpage>59</fpage>
          -
          <lpage>70</lpage>
          . URL: https://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>3083</volume>
          / paper131.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Sutton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. G.</given-names>
            <surname>Barto</surname>
          </string-name>
          ,
          <article-title>Reinforcement learning: an introduction</article-title>
          , 2nd ed., The MIT Press, Cambridge,
          <year>2015</year>
          . URL: https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd. pdf.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>A.</given-names>
            <surname>Jansson</surname>
          </string-name>
          ,
          <article-title>SharpRL: Reinforcement learning library for</article-title>
          .
          <source>NET</source>
          ,
          <year>2015</year>
          . URL: https://github.com/ svenslaggare/SharpRL.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>