<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Creating driving behavior for arti cial agents in a social augmented micro-world</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>M.C. Schneider</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>H-D. Burkhard</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Humboldt-Universitat zu Berlin</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>This paper presents rst results on the implementation of driving agents in the socially augmented micro-world (SAM). Time series analysis are used for comparing the agent behavior with human behavior. We will discuss the results from time series analysis/clustering and compare them with classi cations done by hand. Further approaches for future research are presented and discussed.</p>
      </abstract>
      <kwd-group>
        <kwd>arti cial agents</kwd>
        <kwd>driving behavior</kwd>
        <kwd>micro-world</kwd>
        <kwd>time series</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>providing the data of their series analysis. The last chapter presents the results
of the comparison of both approaches and the rst results of arti cial agents and
outlines future work.</p>
    </sec>
    <sec id="sec-2">
      <title>Earlier Work</title>
      <p>
        Agent architecture
In the last year we nished the proposed architecture [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. The architecture
focuses on usability, e ciency, maintainability and portability. Usability can be
understood as clarity of the used concepts and the e ort needed to use the agents.
E ciency concerns the real time behavior of each component. The next point
is the maintainability, which refers to the cost of nding and correcting errors.
Modi ability is also part of maintainability and is the cost of changing parts of
the agent architecture. The last part is portability and describes the di culty
to port the agent from one environment to an other.
      </p>
      <p>
        One important aspect of the usability is the e ort which is needed to implement
new agent behavior. To simplify the architecture we use a design pattern called
strategy [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. This design pattern gives a lot of possibilities, new agents can easily
be added and the behavior can be changed during run time.
2.2
      </p>
      <p>
        Classi cation by hand
As mentioned in [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], the behavior of human MWI need to be analyzed, as a
prerequisite for the development of arti cial agents. There are 26 data sets from
former studies with human probands. One data set contains 11 tracks. For each
track there exists one le in the csv data format, which contains information
about the state of SAM, taken every 39ms.
      </p>
      <p>The rst approach focused on only one track, because this track exhibits
characteristics like forks or obstacles for the rst time and has no intervention from the
operator. We only analysed the critical situations (forks, the avoiding of
obstacles). Data gained in the situations provides more variance and hence supports
the identi cation of selective types of interaction behavior. Six di erent steering
behavior were found:
1. Adapted navigator: MWI adjusts his steering to characteristics of the track.
2. Extreme steering navigator: steers with maximum de ection alternating
extremely from one direction to the other within a short time.
3. No steering navigator: this MWI hardly shows own steering, the partner
steers.
4. Parallel navigator: produces comparable navigation lane to an adapted driver.</p>
      <p>The parallel navigator exists only in combination with a second parallel
navigator.
5. Indecisive navigator: makes no decision or decides too late.
6. Residue: remaining data sets which did not t in any other classi cation or
does not show any characteristic steering behavior.</p>
      <p>Fig. 2 shows an example for the visualization that support the classi cation
by hand. The driver types were classi ed by similarities of the curves.
2.3</p>
      <p>
        Classi cation with time series
Instead of analysis by hand as described above, the analysis of time series could
lead to automatized classi cation. Such an approach was done with the help of
FAP [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. FAP is a system for time-series anlysis, developed at Department of
Mathematics and Informatics, University of Novi Sad [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Three time series have
been investigated which described the path, the acceleration and the deviation
from the ideal path. The data have been derived from the CVS le. Three
different similarity measures were chosen: Dynamic Time Warping (DTW), Edit
Distance on Real sequence (EDR) and Edit Distance with Real penalty (ERP).
For each of the three time series a distance matrix was calculated. A hierarchical
clustering algorithm was used for clustering and creating dendrograms. This can
be seen in Fig. 3.
      </p>
      <p>As each of the three similarity measures follows a di erent methodology, they
lead to di erent clusters. Two distinct clusters were to be identi ed
distinguishing navigators focusing on speed from those focusing on accuracy. To do this, the
appropriatness of the three similarity measures was investigated. The di erent
instruction should result in di erent navigation styles. This di erence should be
observable by related clusters derived from the time series. As result the ERP
measure is the most suitable to group the MWI. The ERP measure is de ned
as:
8&gt;P1n jsi gj
&gt;&gt;&gt;&gt;&gt;&lt;P1m jri gj
ERP (R; S) = minfERP (Rest(R); Rest(S)) + distE RP (r1; s1);
ERP (Rest(R); S) + distERP (r1; gap);
if m = 0
if n = 0
otherwise</p>
      <p>ERP (R; Rest(S)) + distERP (s1; gap)g
distE RP (ri; si) =
8&gt;jri
&lt;</p>
      <p>jri
&gt;:jsi
sij if ri; sinot gaps
gj if siis a gap
gj if riis a gap
3</p>
    </sec>
    <sec id="sec-3">
      <title>Results and Discussion</title>
      <p>Comparison between classi cation by hand and classi cation
with time series
As described in the introduction, the goal is to create arti cal agents which can
be substituted human MWI. Those arti cal agents should show characteristic
human driving behavior.</p>
      <p>The ERP similarity measure is used to compare the human and agent
behavior. We looked to con rm the hypothesis that MWI in the same classi cation,
which was done by hand, are more similar to each other by the ERP similarity
measure. In the rst approach, the MWIs were compared using ERP by tracks
were they have interacted separatledly with SAM (track 1-4). The classi cation
by hand was done by another track (track 7) where steering were done
cooperatively. The tracks 1-4 were used since there was no interdependence between the
drivers just that their characteristic behavior could be observed more clearly. As
a negativ result, the MWIs in one of the classes found by hand did not show
more ERP similarity than the others. The question stands, if other results could
be achieved by comparing the same tracks.
3.2</p>
      <p>
        Arti cal agents rst results
We decided to use P-controller for our agents because of its simplicity. We want
to start from a simple and easy to understand agent and add more complex
concepts step by step. Ehlert et. al. [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] shows that with a reactive agent and
simple rules it is possible to create human-like driving behavior.
P-controller, or proportional controller, is well know as a control system and can
mathematically described.[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]
at = Kp(yt xt)
at is the resulting controller signal
Kp is the propotional gain
yt is the desired position of the vehicle at time t
xt is the actual position at time t
      </p>
      <p>In general, the agent behavior can be called seeking behavior, since it tries
to reach a desired point, in this case, the middle of the street. We take the
difference from the actual position of the vehicle to the desired position as input
for a p-controller. So we get the following equation:
steering = Kps (positiondesired
acceleration = Kpa (1024</p>
      <p>positioncurrent)
2 jsteeringj)
Both functions have a value range from -1024 till 1024. We tested two di erent
value pairs.
In this paper, we presented the rst attempts to create arti cal agents which
show human-like driving behavior. To compare the performances, we used the
ERP similarity measure. In the end, we showed a rst result for the arti cial
agents compared to human MWI. To improve the performance of the arti cial
agents foresight will be used. In this way, we come from a pure reactive agent to
more deliberative agent. A further point is the step from P-controller to
PIDcontroller. We will add more parameters which have to be tuned, but in the end
we should get a more human like steering behavior and more parameters to create
di erent navigation behavior. With human-like we mean a lower ERP similarity
value. Further work is necessary for the usage of time series for analysis.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Wandke</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Nachtwei</surname>
            ,
            <given-names>J.:</given-names>
          </string-name>
          <article-title>The di erent human factor in automation: the developer behind vs. the operator in action</article-title>
          . In: Waard, D. de (Hrsg.) ; Flemisch,
          <string-name>
            <surname>F. O.</surname>
          </string-name>
          (Hrsg.) ; Lorenz,
          <string-name>
            <surname>B.</surname>
          </string-name>
          (Hrsg.) ; Oberheid,
          <string-name>
            <surname>H.</surname>
          </string-name>
          (Hrsg.) ; Brookhuis,
          <string-name>
            <surname>K. A.</surname>
          </string-name>
          (Hrsg.):
          <article-title>Human factors for assistance and automation</article-title>
          . Maastricht, the Netherlands: Shaker Publishing,
          <year>2008</year>
          , S.
          <volume>493</volume>
          {
          <fpage>502</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Burkhard</surname>
          </string-name>
          ,
          <string-name>
            <surname>H-D.</surname>
          </string-name>
          ;
          <string-name>
            <surname>Jahn</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Kain</surname>
          </string-name>
          , S. ; Meyer, C. ;
          <string-name>
            <surname>Muetterlein</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Nachtwei</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ; Niestroj,
          <string-name>
            <given-names>N.</given-names>
            ;
            <surname>Rougk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            ;
            <surname>Schneider</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          :
          <article-title>Arti cial Subjects in the Psychological Experiment Socially Augmented Microworld (SAM)</article-title>
          .
          <source>In: Proceedings of International Workshop CS&amp;P</source>
          <year>2011</year>
          ,
          <year>2011</year>
          , S.
          <volume>55</volume>
          {
          <fpage>66</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Jahn</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Schneider</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Dokumentation der Entwicklung von Agenten zur Ersetzung von Mikroweltbewohner on der Socially Augmented Microworld</article-title>
          .
          <source>Studienarbeit an der Humboldt</source>
          University Berlin,
          <year>2012</year>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Gamma</surname>
          </string-name>
          , Erich ; Riehle, Dirk ; Helm, Richard ; Johnson, Ralph ; Vlissides,
          <source>John: Entwurfsmuster: Elemente wiederverwendbarer objektorientierter Software. [6</source>
          . Au .].
          <source>Munchen and Boston</source>
          and Mass. [u.a.] :
          <string-name>
            <surname>Addison-Wesley</surname>
          </string-name>
          ,
          <year>2011</year>
          http://www.worldcat.org/oclc/693880985
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Kurbalija</surname>
          </string-name>
          , Vladimir ; Burkhard, Hans-Dieter ; Ivanovic, Mirijana ; Meyer, Charlotte ; Nachtwei, Jens ; Fodor,
          <source>Lidija: Time Series Mining in a Prschological Domain</source>
          .
          <year>2012</year>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Kurbalija</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Radovanovic</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Geler</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Ivanovic</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>A framework for time-series analysis</article-title>
          .
          <source>In: Proceedings of the 14th international conference on Arti cial intelligence: methodology, systems and applications</source>
          , 2010, S.
          <volume>42</volume>
          {
          <fpage>51</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Patrick</surname>
            <given-names>A.M.</given-names>
          </string-name>
          <string-name>
            <surname>Ehlert</surname>
          </string-name>
          and
          <string-name>
            <surname>Leon J.M. Rothkrantz ; Patrick A.M.</surname>
          </string-name>
          <article-title>Ehlert (Hrsg.): A REACTIVE DRIVING AGENT FOR MICROSCOPIC TRAFFIC SIMULATION</article-title>
          . http://www.kbs.twi.tudelft.nl/docs/conference/2001/Ehlert.P.A.MESM2001.pdf. Version:
          <year>2001</year>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Russell</surname>
          </string-name>
          , Stuart ; Canny, John F.:
          <article-title>Kunstliche Intelligenz: Ein moderner Ansatz. 1. Munchen and</article-title>
          and Boston [u.a.]
          <source>: Pearson Studium</source>
          ,
          <year>2004</year>
          .
          <source>{ ISBN 978{3{8273{ 7089{1</source>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>