<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Weight Based Algorithms to Increase the Playability in 2D Games</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Alicja Winnicka</string-name>
          <email>winnicka.alicja@10g.pl</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Karolina Ke˛sik</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Kalina Serwata</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Kamil Ksia˛z˙ek</string-name>
          <email>kamilksiazek95@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Institute of Mathematics Silesian University of Technology Kaszubska 23</institution>
          ,
          <addr-line>44-100 Gliwice</addr-line>
          ,
          <country country="PL">Poland</country>
        </aff>
      </contrib-group>
      <fpage>25</fpage>
      <lpage>29</lpage>
      <abstract>
        <p>-Increasing the level of playability in games depends on the engine's operation. The more accurate and predictable the game is, the greater the difficulty level will be. However, the level of playability may decrease due to the player's interest. To prevent this, we propose a combination of different techniques for universal operation on various two-dimensional games. The used methods have been modified in such a way to show that hybrid forms can be much more advantageous. The proposition has been tested on selected games, and the obtained results were analyzed and discussed depending on the introduced modifications. The aim of the discussion was to indicate the various advantages and disadvantages of these techniques to increase the playability.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>The game development is dependent on graphics card
producers and player requirements. The more efficient the
equipment will be available on the market, the more real and
complicated games should be. Particularly the second aspect is
important, because even the simplest game can attract human
attention. The problem is the lack of holding this attention. To
remedy this, games use more and more new algorithms that
allow a computer counterattack during the game. However, the
perfect operation of the algorithm can cause the player will
not have the slightest chance of winning, which will cause the
game to quickly be forgotten. This lack of interest should be
prevented by the addition of a certain, preferably controlled
randomness in his movements.</p>
      <p>
        Particularly the development of artificial intelligence
techniques finds its applications in the various aspects of our
lives, not only in entertainment, which games are an example.
Decision support systems are the most known application
of these techniques. One such example is medicine, where
computer can confirm or even detect some diseases based on
a given samples of X-RAY or CT images. Selected tools for
that problems can be heuristic algorithms, what is presented
in [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. Another example are energy networks [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
Along with the rapid development of artificial intelligence,
other branches also sharply advance. This is especially
visible in databases and warehousing because more and more
information needs to be stored, where specific queries or sorts
Copyright held by the author(s).
are performed [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Subsequently, authorization of these data
is important, as can be seen from the number of publications
and scientific papers in the field of biometrics [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
      </p>
      <p>
        Entertainment forces a rapid pace of development, which
can be seen after indicating new trends such as extended,
virtual or mixed realities. In [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], the current open
challenge in the field of games are described. In [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], monte
carlo tree search method with learning mechanism for video
game is presented. Authors of [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] discuss about the idea
of reinforcement learning method for the use in multiplayer
nonzerosum games. Again in [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], some smart technique for
agents in game is described. Important part of artificial
intelligence are neural networks, which also have found their place
in games [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>In this paper, we propose a hybrid solution connected with
the classic techniques like q-learning, a* and tabu search. Our
technique is based on the fitness function which depends on
the selection of a specific metric.</p>
    </sec>
    <sec id="sec-2">
      <title>II. SIMPLIFIED A* WITH TABU TABLE</title>
      <p>Proposed algorithm is based on path search algorithms.
However it does not check whole path to the target, but only
the value of points, which are the neighbours of chasing player
(or enemy, depending on the point of view). Firstly, it needs
to find all possible fields on the board that can be visited.
After this action, it is necessary to compute the value of each
field ( ). The evaluation is made according to the following
formula
(A; B) = d(A; B) + wij ;
(1)
where A and B are the points in two-dimensional space
understood as (xA; yA) and (xB; yB). First point is the possible next
field of chasing player and the second one is the localization of
target. Function ( ) is the sum of two components, the value
of the field calculated as a specific metric and the weight wij ,
where i and j are given positions on the board. The weight is
selected in random way in the range [ 1; 1]. Let us assume
that d is such a function that
d : X</p>
      <p>X ! [0; 1];
(2)
(a) The pacman game
(b) A console game
where X is non-empty set. d is called a metric. For any two
points A; B 2 X, the function must satisfy the following
properties:
1) distance between two points is equal to 0 if and only if
points have the same coordinates
d(A; B) = 0 ()</p>
      <p>A = B;
d(A; B) = d(B; A);
(3)
(4)
2) symmetry rule a distance between A and B is the
same like a distance between B and A
3) triangle inequality a distance between A and B is less
or equal the sum of distances between A and C (C is
an intermediate point) and between C and B
d(A; B)
d(A; C) + d(C; B):
(5)
assigned to each empty field at the position (i; j). Additionally,
player will be marked as 10 and the enemies (moved by the
computer) as 20. Visualization of such a board is shown on
Fig. 1.</p>
      <p>The player moves towards the user by selecting the field,
in which a value of the fitness function described by Eq.
(1) is the smallest. However, it is possible a situation where
the algorithm gets stuck. An example of such a setting is a
corner, where a character on three sides has a wall (further
walk is prevented). Starting from this position, there is a very
high probability of returning to the same field. In order to
avoid described situation, we introduce a tabu table, where
movements that were made and did not bring any benefits
(i.e. preventing further walk in the direction of a user) are
saved. The proposed algorithm is presented in Alg. 1.</p>
    </sec>
    <sec id="sec-3">
      <title>III. EXPERIMENTS</title>
      <p>The most known metric is the Euclidean one defined as:</p>
      <p>The proposed solution has been implemented in C# and
vu n tested in terms of the number of won games, the number
dE (A; B) = tuX(xi yi)2; (6) of necessary moves that the algorithm needs to caught the
i=1 player, and various metrics which have been described earlier.
where n is the number of coordinates of points. Another In addition, all tests were carried out depending on the number
example is the taxicab metric (called also the Manhattan of fields on the board, the amount of obstacles or even the
metric), defined as follows: points which must be collected by the player. As points, we
mean in the case of the console the symbols ’#’, and in the
dM (A; B) = i=m1;a::x:;n jxi yij: (7) case of pacman – dots.</p>
      <p>In Fig. 2 is shown how the average number of moves
The last presented case is the jungle river metric understood performed by the algorithm increases depending on the size
as of the board. The initial size of board was 400 and during
dR(A; B) = dE (A; C1) + dE (C1; C2) + dE (C2; B); (8) subsequent steps 20 fields were added (up to 760). In each
case the growth of moves was linear (what is shown thanks to
where C1 and C2 are orthogonal projections of A and B, linear regression). However, the slope of the line is relatively
respectively, on the line r (r is called a river). small, which indicates the advantage of the proposed solution.</p>
      <p>Each game takes place in a certain area or board. Let us Even in case of a large board, presented method was effective.
assume that the board size is w h. In the case when all In addition, almost perfect linear growth in the average number
fields are empty, and the computer players are approaching of movements was obtained for the jungle river metric. A
to the user, the game would be too simple. It is necessary to bit worse results were achieved for the Manhattan metric
put some obstacles on the board, what will be marked as a and the worst metric was the Euclidean one. This is caused
large number, for example 100. A random value wij will be by randomly located obstacles at the game board. Again in</p>
      <p>Fig. 3, the average number of wins during 100 experiments
is presented. The most effective was the proposed algorithm
with the jungle river metric. This combination was the most
succesful during 32% out of the total number of games. The
Manhattan metric was slightly less effective (the winner of
26% games). What is interesting, the weakest results out of
presented metrics were achieved by the Euclidean one (24 %).</p>
      <p>The smallest number of victories had a player – only 18%. It
can be concluded that beating the algorithm was not easy but
still possible.</p>
      <p>The number of calculations is small in comparison to other
graphic algorithms which have to search the entire board.</p>
      <p>Only the position of the player is the necessary knowledge
for applying of this technique. The path to the given object
is determined by a dedicated function. In addition, various
metrics have different effectiveness, so it allows us to propose
a game with varied difficulty level by application of a specific
metric. This type of distinction not only diversifies the game,
but also does not allow the player to adapt to one level of
computer intelligence.</p>
    </sec>
    <sec id="sec-4">
      <title>IV. CONCLUSIONS</title>
      <p>A hybrid solution based on known algorithms has been
shown in this paper. Three different metrics were selected:
the jungle river, the Manhattan and the Euclidean, which were
used interchangeably in the assessment of possible movements
to be performed by a computer-controlled player. The obtained
results showed that this type of solution has the right to be used
as a computational intelligence. Especially, if the difficulty
level of the game would be understood as applying a different
metric. Measurements for each of presented metrics showed
that the method’s effectiveness depends on the right choice
of them. In addition, they are stable in terms of growth of
the calculation, which increases slightly when the size of the
board grows.</p>
      <p>During further research it is possible to apply other, less
known metrics or use described model in other, a bit more
complicated games and check efficiency of the method.
Measurements prove that such approach can be successfully
implemented in similar types of games.</p>
      <p>The algorithm can be used in different games when player
need to catch moving element in the game, which doesn’t need
to be player. This model of playing can be applied in most
arcade games and many another ones. Presented ideas has a
wide range of applications in this brand of science.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>G.</given-names>
            <surname>Capizzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. L.</given-names>
            <surname>Sciuto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Napoli</surname>
          </string-name>
          , and
          <string-name>
            <given-names>E.</given-names>
            <surname>Tramontana</surname>
          </string-name>
          .
          <article-title>Advanced and adaptive dispatch for smart grids by means of predictive models</article-title>
          .
          <source>IEEE Transactions on Smart Grid</source>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>G.</given-names>
            <surname>Capizzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. L.</given-names>
            <surname>Sciuto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Napoli</surname>
          </string-name>
          , and
          <string-name>
            <surname>E. Tramontana.</surname>
          </string-name>
          <article-title>An advanced neural network based solution to enforce dispatch continuity in smart grids</article-title>
          .
          <source>Applied Soft Computing</source>
          ,
          <volume>62</volume>
          :
          <fpage>768</fpage>
          -
          <lpage>775</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A.</given-names>
            <surname>Dobrovsky</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. W.</given-names>
            <surname>Wilczak</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Hahn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hofmann</surname>
          </string-name>
          , and
          <string-name>
            <given-names>U. M.</given-names>
            <surname>Borghoff</surname>
          </string-name>
          .
          <article-title>Deep reinforcement learning in serious games: Analysis and design of deep neural network architectures</article-title>
          .
          <source>In International Conference on Computer Aided Systems Theory</source>
          , pages
          <fpage>314</fpage>
          -
          <lpage>321</lpage>
          . Springer,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>E.</given-names>
            <surname>Ilhan</surname>
          </string-name>
          and A. S¸ . Etaner-Uyar.
          <article-title>Monte carlo tree search with temporaldifference learning for general video game playing</article-title>
          .
          <source>In Computational Intelligence and Games (CIG)</source>
          ,
          <source>2017 IEEE Conference on</source>
          , pages
          <fpage>317</fpage>
          -
          <lpage>324</lpage>
          . IEEE,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>J. Kapocˇiu</given-names>
            ¯te
            <surname>˙</surname>
            -Dzikiene˙
          </string-name>
          , A. Vencˇkauskas, and
          <string-name>
            <given-names>R.</given-names>
            <surname>Damaševicˇius</surname>
          </string-name>
          .
          <article-title>A comparison of authorship attribution approaches applied on the lithuanian language</article-title>
          .
          <source>In Computer Science and Information Systems (FedCSIS)</source>
          ,
          <source>2017 Federated Conference on</source>
          , pages
          <fpage>347</fpage>
          -
          <lpage>351</lpage>
          . IEEE,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>T.</given-names>
            <surname>Kapus</surname>
          </string-name>
          <article-title>´cin´ski</article-title>
          ,
          <string-name>
            <given-names>R. K.</given-names>
            <surname>Nowicki</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Napoli</surname>
          </string-name>
          .
          <article-title>Comparison of effectiveness of multi-objective genetic algorithms in optimization of invertible s-boxes</article-title>
          .
          <source>In International Conference on Artificial Intelligence and Soft Computing</source>
          , pages
          <fpage>466</fpage>
          -
          <lpage>476</lpage>
          . Springer,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.</given-names>
            <surname>Khalifa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Preuss</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Togelius</surname>
          </string-name>
          <article-title>. Multi-objective adaptation of a parameterized gvgai agent towards several games</article-title>
          .
          <source>In International Conference on Evolutionary Multi-Criterion Optimization</source>
          , pages
          <fpage>359</fpage>
          -
          <lpage>374</lpage>
          . Springer,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Marszałek</surname>
          </string-name>
          .
          <article-title>Performance tests on merge sort and recursive merge sort for big data processing</article-title>
          .
          <source>Technical Sciences</source>
          ,
          <volume>21</volume>
          (
          <issue>1</issue>
          ):
          <fpage>19</fpage>
          -
          <lpage>35</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>F.</given-names>
            <surname>Milani</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. C. B. De Marchi</surname>
            , and
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Rieder</surname>
          </string-name>
          .
          <article-title>Usability guidelines to develop gesture-based serious games for health: A systematic review</article-title>
          .
          <source>In Virtual and Augmented Reality (SVR)</source>
          ,
          <year>2017</year>
          19th Symposium on, pages
          <fpage>188</fpage>
          -
          <lpage>194</lpage>
          . IEEE,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>E.</given-names>
            <surname>Okewu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Misra</surname>
          </string-name>
          , R. Maskeliu¯nas, R. Damaševicˇius, and L.
          <string-name>
            <surname>Fernandez-Sanz</surname>
          </string-name>
          .
          <article-title>Optimizing green computing awareness for environmental sustainability and economic security as a stochastic optimization problem</article-title>
          .
          <source>Sustainability</source>
          ,
          <volume>9</volume>
          (
          <issue>10</issue>
          ):
          <year>1857</year>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>D.</given-names>
            <surname>Połap</surname>
          </string-name>
          .
          <article-title>Extraction of specific data from a sound sample by removing additional distortion</article-title>
          .
          <source>In Computer Science and Information Systems (FedCSIS)</source>
          ,
          <source>2017 Federated Conference on</source>
          , pages
          <fpage>353</fpage>
          -
          <lpage>356</lpage>
          . IEEE,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>D.</given-names>
            <surname>Połap</surname>
          </string-name>
          , M. Woz´niak, C. Napoli, and
          <string-name>
            <given-names>E.</given-names>
            <surname>Tramontana</surname>
          </string-name>
          .
          <article-title>Real-time cloudbased game management system via cuckoo search algorithm</article-title>
          .
          <source>International Journal of Electronics and Telecommunications</source>
          ,
          <volume>61</volume>
          (
          <issue>4</issue>
          ):
          <fpage>333</fpage>
          -
          <lpage>338</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>S.</given-names>
            <surname>Risi</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Togelius</surname>
          </string-name>
          .
          <article-title>Neuroevolution in games: State of the art and open challenges</article-title>
          .
          <source>IEEE Transactions on Computational Intelligence and AI</source>
          in Games,
          <volume>9</volume>
          (
          <issue>1</issue>
          ):
          <fpage>25</fpage>
          -
          <lpage>41</lpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>R.</given-names>
            <surname>Song</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. L.</given-names>
            <surname>Lewis</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Q.</given-names>
            <surname>Wei</surname>
          </string-name>
          .
          <article-title>Off-policy integral reinforcement learning method to solve nonlinear continuous-time multiplayer nonzerosum games</article-title>
          .
          <source>IEEE transactions on neural networks and learning systems</source>
          ,
          <volume>28</volume>
          (
          <issue>3</issue>
          ):
          <fpage>704</fpage>
          -
          <lpage>713</lpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>M.</given-names>
            <surname>Woz</surname>
          </string-name>
          <article-title>´niak and</article-title>
          <string-name>
            <given-names>D.</given-names>
            <surname>Połap</surname>
          </string-name>
          .
          <article-title>Bio-inspired methods modeled for respiratory disease detection from medical images</article-title>
          .
          <source>Swarm and Evolutionary Computation</source>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>M.</given-names>
            <surname>Woz</surname>
          </string-name>
          ´niak,
          <string-name>
            <given-names>D.</given-names>
            <surname>Połap</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gabryel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. K.</given-names>
            <surname>Nowicki</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Napoli</surname>
          </string-name>
          , and
          <string-name>
            <given-names>E.</given-names>
            <surname>Tramontana</surname>
          </string-name>
          .
          <article-title>Can we process 2d images using artificial bee colony?</article-title>
          <source>In International Conference on Artificial Intelligence and Soft Computing</source>
          , pages
          <fpage>660</fpage>
          -
          <lpage>671</lpage>
          . Springer,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>M.</given-names>
            <surname>Woz</surname>
          </string-name>
          ´niak,
          <string-name>
            <given-names>D.</given-names>
            <surname>Połap</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Napoli</surname>
          </string-name>
          , and
          <string-name>
            <given-names>E.</given-names>
            <surname>Tramontana</surname>
          </string-name>
          .
          <article-title>Application of bio-inspired methods in distributed gaming systems</article-title>
          .
          <source>Information Technology And Control</source>
          ,
          <volume>46</volume>
          (
          <issue>1</issue>
          ):
          <fpage>150</fpage>
          -
          <lpage>164</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>M.</given-names>
            <surname>Wózniak</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Połap</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. K.</given-names>
            <surname>Nowicki</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Napoli</surname>
          </string-name>
          , G. Pappalardo, and
          <string-name>
            <given-names>E.</given-names>
            <surname>Tramontana</surname>
          </string-name>
          .
          <article-title>Novel approach toward medical signals classifier</article-title>
          .
          <source>In International Joint Conference on Neural Networks (IJCNN)</source>
          , pages
          <fpage>1</fpage>
          -
          <lpage>7</lpage>
          . IEEE,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>