<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Declarative Solutions for the the Manipulation of Articulated Objects Using Dual-Arm Robots</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>DIBRIS, University of Genova</institution>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Riccardo Bertolucci</institution>
        </aff>
      </contrib-group>
      <abstract>
        <p>The manipulation of flexible object is of primary importance in industry 4.0 and in home environments scenarios. Traditionally, this problem has been tackled by developing ad-hoc approaches, that lack of flexibility and portability. We propose an approach in which a flexible object is modelled as an articulated object, or rather, a set of links connect via joints In this paper we present an extended analysis of the framework based on Answer Set Programming (ASP) for the automated manipulation of articulated objects in a robot architecture. In detail, we modeled the same scenario with different grades of precision: a simple model it is used to describe the scenario with an high level of abstraction, while an extended model it used to include more detail and therefore to increase the represented knowledge of such scenario. With respect to the simple reference scenario we analyse the behaviours of our strategy for the action planning module, while, for the extended scenario we introduce the concept of macro action and we then we analyse their performances w.r.t our problem. Our aim is to have an understanding of the performances of these approaches with respect to planning time and execution time as well.1</p>
      </abstract>
      <kwd-group>
        <kwd>Answer Set Programming</kwd>
        <kwd>Robots Manipulation</kwd>
        <kwd>Macro actions</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        The manipulation of articulated objects is of primary importance in robotics, and is
one of the most complex robotics tasks [
        <xref ref-type="bibr" rid="ref1 ref2">1,2</xref>
        ]. Traditionally, this problem has been
tackled by developing ad-hoc approaches, that lack of flexibility and portability. The
development of new software, algorithm and strategies, together with the
improvements in the mechanical design for grippers and robotic hands, for autonomous robots
with robust manipulation skills, can lead to breakthroughs in various applications, such
as humanoid robots, horticulture harvesting grasping, human robot interaction,
planetary exploration, flexible manufacturing and much more. This gives the possibility
of addressing some of the issues related to robotized work, such as mechanical
design issues, control issues, modelling achievements and issues, applications in
industrial field and non-conventional applications (including, for example, service robotics
and agriculture)[
        <xref ref-type="bibr" rid="ref1 ref3">3,1</xref>
        ]. In the past years attention has been paid to the development of
1 Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0)
approaches and algorithms for generating the sequence of movements a robot has to
perform in order to manipulate an articulated object but the issue is still not being fully
addressed. In the literature, the problem of determining the two-dimensional (2D)
configuration of articulated or flexible objects has received much attention in the past few
years [
        <xref ref-type="bibr" rid="ref4 ref5 ref6 ref7">4,5,6,7</xref>
        ], whereas the problem of obtaining a target configuration via
manipulation has been explored in motion planning [
        <xref ref-type="bibr" rid="ref10 ref8 ref9">8,9,10</xref>
        ]. A limitation of such manipulation
strategies is that they are often crafted specifically for the problem at hand, with the
relevant characteristics of the object and robot capabilities being either hard coded or
assumed; thus, in these contexts generalisation property and scalability are somehow
limited.
      </p>
      <p>
        In this paper we present the analysis of a framework based on Answer Set
Programming (ASP) [
        <xref ref-type="bibr" rid="ref11 ref12 ref13 ref14 ref15 ref16 ref17 ref18">11,12,13,14,15,16,17,18</xref>
        ] for the automated manipulation of articulated
objects in a robot 2D work-space. ASP is a general, prominent knowledge
representation and reasoning language with roots in logic programming and non-monotonic
reasoning [
        <xref ref-type="bibr" rid="ref19 ref20">19,20</xref>
        ], with readable syntax and clear semantics [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. In particular, ASP is
employed for representing the configuration of the articulated object, for checking the
consistency of the knowledge base, as well as for generating the sequence of
manipulation actions, i.e. the plan.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Problem Statement and the Simple Reference Scenario</title>
      <p>In this section we define the problem addressed, and we present the considered first
reference scenario.
2.1</p>
      <sec id="sec-2-1">
        <title>Problem Statement</title>
        <p>Our goal is to present (i) an overview of the ASP-based architecture for the
manipulation of articulated objects in terms of the representation of the desired scenario and the
planning strategies developed for the selection of manipulation actions aimed to
maximises the reliability of the robot execution, and (ii) give a simple overview on how we
modeled macro actions inside our ASP-based architecture.</p>
        <p>An articulated object is defined as a pair = (L; J ), where L is the ordered set of
its jLj links and J is the ordered set of its jJ j joints. Each link l 2 L is characterised by
two parameters, namely a length l and an orientation l. We allow only for a limited
number of possible orientations, which induces a finite set of allowed angle orientations
for each link. The configuration of an articulated object is modeled as jLj-ple:
C ;a =
1a; : : : ; jaLj :
(1)
2.2</p>
      </sec>
      <sec id="sec-2-2">
        <title>Modeled Scenarios</title>
        <p>Simple Model We relay on QR codes to compute and provide to the architecture an
overall link pose, which directly maps to an absolute link orientation la.
Extended model This scenario it is developed in order to describe with an higher
accuracy the robot and its work-space. However, this model does not modify any physical
characteristics with respect to the setup introduced in the previous paragraph.
Herewith we briefly describe such modelling, and whenever relevant we highlight the main
modifications we introduced to the initial scenario. Firstly, The robot grippers are now
explicitly modelled. Each gripper is now considered as a resource that can be occupied
(i.e., keeping a link firmly, or rotating a link) or free. This open the possibility of
representing which gripper will manipulate a given link. Then, each time a manipulation
action is carried out on a given link, it is assured that the link is centred in the robot
workspace. This is due to the fact that, due to the physical property of the object, some
of the link can be positioned outside the reach of the robot arms. Finally, grasping and
release actions by the two grippers are now explicitly modelled.</p>
        <p>All the above mentioned features allow for mainly two improvements: on the one
hand, the encoding is expected to be able to better manage the explicitly modelled robot
resources (i.e., the grippers); on the other hand, manipulation actions are characterised
by a more precise semantics, which does not make any implicit assumption about actual
robot behaviour.
3</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>The Robot Architecture</title>
      <p>The architecture of the Baxter from Rethink Robotics is shown in Figure 2. In the
current implementation, perception is managed using a camera sensor located on top of
the robot’s head and pointing downward, which provides 6D poses for each link, and in
turn update corresponding ASP-based representation structures in the Knowledge Base
module. The Consistency Checking module performs a check for knowledge base
validation. In case the check succeeds, the Goal Checker module is notified and it process
the information given by the Knowledge Base in order to compute which requirements
are already fulfilled. In fact, usually due to human intervention, some constrains
included in the goal can be already achieved. Tacking this in consideration the problem is
then generated. The Action Planner module receives such problem instance and
generates a plan in the form of a suitable sequence of actions to be performed. Once a plan
is generated, its actions are processed sequentially to drive the overall behaviour of the
robot Motion Planner module, which is responsible of the execution.</p>
      <p>
        For the sake of brevity we will focus only on the Action Planning module and how
we modified it in order to implement different approaches.
In this section we describe how ASP is used to implement the Action Planning Module
depicted. In the following, we assume the reader is familiar with ASP and ASP-Core-2
input language specification [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ].
      </p>
      <p>
        ASP is not a planning-specific language, but it can be also used to specify encoding
for planning domains [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ], like our target problem. We have defined several encoding
variants, for what concerns either the manipulation modes and the strategy for
computing plans.
4.1
      </p>
      <sec id="sec-3-1">
        <title>Simple Model</title>
        <p>
          The encoding described in this section is embedded into a classical iterative deepening
approach in the spirit of SAT-based planning [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], in which the maximum number of
step allowed, called timemax, is initially set to 1 and then increased by 1 if a plan
is not found. This guarantees the computation of an optimal plan, w.r.t. the number of
action to perform, that is the shortest possible plans for a sequential encoding, i.e., when
the robot performs only one action for each step.
4.2
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>Extended Model</title>
        <p>Inside the context of the extended scenario (see Section 2.2), we explored two different
strategies in order to investigate the pros and drawbacks of each approach. Moreover,
in this scenario, we investigate the propriety of the macros. The two strategies are:
– Simple Actions Extended Scenario (SAES): This encoding models the same
actions as the Standard Strategy in the simple scenario but it includes also the robot
resources that can be occupied at each time step (i.e. the robot gripper);
– Macro Actions Extended Scenario (MAES): Here we have modelled the same
scenario as in the SAES. The difference consists of the modelled actions: sets of simple
actions are gathered inside just one atoms.</p>
        <p>Representing Macros in ASP Given a rule ri representing an action, pre(ri) denotes
the body of the rule. Intuitively, it represents the conditions that must hold in order to
activate the action represented by the rule. Moreover, del(ri) (resp. add(ri)) represent
all the atoms that are set as false (resp. true) whenever the conditions denoted by pre(ri)
hold. We encoded a macro action as a single choice rules composed by a fresh atom
in its head containing all the variables appearing in its body. Furthermore, the choice
rule body it is composed as follows: a macro ri;j is constructed by assembling the
rules representing single actions and by generating pre(ri;j ), del(ri;j ), and add(ri;j ),
as follows:
– pre(ri;j ) = pre(ri) [ (pre(rj ) n add(ri))
– del(ri;j ) = (del(ri) n add(rj )) [ del(rj )
– add(ri;j ) = (add(ri) n del(rj )) [ add(rj )
where ri and rj are two distinct rules. Then, for a macro ri;j , the body of the choice
rule is represented by pre(ri;j ). The macro composed as such represent multiple simple
action and their effect that are modeled as a set of several simple rules.</p>
        <p>Macros for the Extended Scenario. The following macros have been considered:
– linkToCentral take: it is the composition of two actions move link to central,
that moves the articulated object so that the joint in between the links that have to
be manipulated is in the centre of the workspace, and takes links to move, that
grasps the links to be manipulated. As links cannot be grasped by the robot if they
are not in the centre of the workspace, this macro aims at providing a single rule
for cases where links are not in the right position.
– changeAngle release: it is the composition of the choice rule changeAngle,
that changes the angle of a link, and release links, that releases the links currently
grasped. This macro aims at providing a single rule for cases where it is necessary
to act on a link and then releasing it.
– take changeAngle release: represents the composition of takes links to move,
changeAngle, and release links. This macro aims at providing a single action
for cases where it is necessary to act on a link that was already in the center of the
workspace.
In order to obtain an overview of the capabilities of the considered encodings we used as
test problems the same problems, with the due adaptations for each model. Eventually,
we had 320 instances with 4 and 6, and granularity values of 4, 6, 8 or 12 possible
angles, 10 instances for each pair (number of links, granularity). For each testing instance
time limit of 300 seconds and memory limit of 16 GB was applied. Clingo was used to
solve the ASP-encoded instances. All the experiments were conducted on Intel i7-4790
CPU and Linux OS. We compared the performance of the considered encodings
using coverage (percentage of solved instances) and PAR10. Penalised Average Runtime
(PAR10) score is a metric usually exploited in machine learning and algorithm
configuration techniques. This metric trades off coverage and runtime for solved problems: if
an encoding e allows the solver to solve an instance in time t T (T = 300s in
our case), then P AR10(e; ) = t, otherwise P AR10(e; ) = 10 T (i.e., 3000s in
our case). The above tables summarises the results achieved by Clingo for solving
instances encoded in the Standard Strategy, SAES, and MAES. It is worth reminding that
the Standard Strategy encoding is much more simplistic then the others, as it ignores
the position of the links to be manipulated, and considers high level actions that have to
be broken down into a large number of low-level primitives. On the contrary, SAES and
MAES encodings provide a more detailed and rich description of the problem, that
allows to generate plans that are easier to be put in place by the manipulator. So, a direct
comparison between the Standard Strategy and SAES/MAES is not possible, but it is
nonetheless interesting to have also the results obtained by the Standard Strategy. The
comparison of the performance achieved by Clingo when using the SAES and MAES
encodings can shed some light on the usefulness of the macros. It is easy to notice that
the use of macros allows Clingo to solve a larger number of instances, and that macros
are generally helpful in improving the runtime.</p>
        <p>
          In the future we would like to test with other ASP solvers, e.g. WASP [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ], and
to inject in these solvers heuristics and algorithms, e.g. [
          <xref ref-type="bibr" rid="ref26 ref27 ref28">26,27,28</xref>
          ], that proved to be
effective in the planning domain.
        </p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1. J. Kr u¨ger, T. K. Lien,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Verl</surname>
          </string-name>
          , “
          <article-title>Cooperation of human and machines in assembly lines,”</article-title>
          <source>CIRP Annals</source>
          , vol.
          <volume>58</volume>
          , no.
          <issue>2</issue>
          , pp.
          <fpage>628</fpage>
          -
          <lpage>646</lpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>C.</given-names>
            <surname>Heyer</surname>
          </string-name>
          , “
          <article-title>Human-robot interaction and future industrial robotics applications,”</article-title>
          <source>in Proc. of IEEE International Conference on Intelligent Robots and Systems (IROS</source>
          <year>2010</year>
          ), pp.
          <fpage>4749</fpage>
          -
          <lpage>4754</lpage>
          , IEEE,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>C.</given-names>
            <surname>Heyer</surname>
          </string-name>
          , “
          <article-title>Human-robot interaction and future industrial robotics applications</article-title>
          ,”
          <source>in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)</source>
          , pp.
          <fpage>4749</fpage>
          -
          <lpage>4754</lpage>
          , IEEE,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>A.</given-names>
            <surname>Capitanelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Mastrogiovanni</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Vallati</surname>
          </string-name>
          , “
          <article-title>Automated planning techniques for robot manipulation tasks involving articulated objects</article-title>
          ,”
          <source>in Proceedings of the International Conference of the Italian Association for Artificial Intelligence (AI*IA)</source>
          , pp.
          <fpage>483</fpage>
          -
          <lpage>497</lpage>
          , Springer,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>A.</given-names>
            <surname>Capitanelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Mastrogiovanni</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Vallati</surname>
          </string-name>
          , “
          <article-title>On the manipulation of articulated objects in human-robot cooperation scenarios</article-title>
          ,
          <source>” Robotics and Autonomous Systems</source>
          , vol.
          <volume>109</volume>
          , pp.
          <fpage>139</fpage>
          -
          <lpage>155</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>A.</given-names>
            <surname>Nair</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Agrawal</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Isola</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Abbeel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Malik</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Levine</surname>
          </string-name>
          , “
          <article-title>Combining selfsupervised learning and imitation for vision-based rope manipulation</article-title>
          ,”
          <source>in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)</source>
          , pp.
          <fpage>2146</fpage>
          -
          <lpage>2153</lpage>
          , IEEE,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>H.</given-names>
            <surname>Wakamatsu</surname>
          </string-name>
          , E. Arai, and
          <string-name>
            <given-names>S.</given-names>
            <surname>Hirai</surname>
          </string-name>
          , “
          <article-title>Knotting/unknotting manipulation of deformable linear objects</article-title>
          ,”
          <source>International Journal of Robotic Research</source>
          , vol.
          <volume>25</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>371</fpage>
          -
          <lpage>395</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>L.</given-names>
            <surname>Bodenhagen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. R.</given-names>
            <surname>Fugl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Jordt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Willatzen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. A.</given-names>
            <surname>Andersen</surname>
          </string-name>
          ,
          <string-name>
            <surname>M. M. Olsen</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Koch</surname>
            ,
            <given-names>H. G.</given-names>
          </string-name>
          <string-name>
            <surname>Petersen</surname>
          </string-name>
          , and N. Kr u¨ger, “
          <article-title>An adaptable robot vision system performing manipulation actions with flexible objects</article-title>
          ,
          <source>” IEEE Transactions on Automation Science and Engineering</source>
          , vol.
          <volume>11</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>749</fpage>
          -
          <lpage>765</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>J.</given-names>
            <surname>Schulman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Ho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Lee</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P.</given-names>
            <surname>Abbeel</surname>
          </string-name>
          , “
          <article-title>Learning from demonstrations through the use of non-rigid registration</article-title>
          ,”
          <source>in Proceedings of the International Symposium on Robotics Research (ISRR)</source>
          , pp.
          <fpage>339</fpage>
          -
          <lpage>354</lpage>
          , Springer,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <given-names>Y.</given-names>
            <surname>Yamakawa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Namiki</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Ishikawa</surname>
          </string-name>
          , “
          <article-title>Dynamic high-speed knotting of a rope by a manipulator,” IJARS</article-title>
          , vol.
          <volume>10</volume>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>12</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>M. Gebser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maratea</surname>
            , and
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Ricca</surname>
          </string-name>
          , “
          <article-title>The Design of the Sixth Answer Set Programming Competition,” in LPNMR</article-title>
          , vol.
          <volume>9345</volume>
          of LNCS, pp.
          <fpage>531</fpage>
          -
          <lpage>544</lpage>
          , Springer,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>M. Gebser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maratea</surname>
            , and
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Ricca</surname>
          </string-name>
          , “
          <article-title>What's hot in the answer set programming competition</article-title>
          ,” in AAAI, pp.
          <fpage>4327</fpage>
          -
          <lpage>4329</lpage>
          , AAAI Press,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <given-names>F.</given-names>
            <surname>Calimeri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gebser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricca</surname>
          </string-name>
          , “
          <article-title>Design and results of the Fifth Answer Set Programming Competition,”</article-title>
          <source>Artificial Intelligence</source>
          , vol.
          <volume>231</volume>
          , pp.
          <fpage>151</fpage>
          -
          <lpage>181</lpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>M. Gebser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maratea</surname>
            , and
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Ricca</surname>
          </string-name>
          , “
          <article-title>The sixth answer set programming competition</article-title>
          ,
          <source>” Journal of Artificial Intelligence Research</source>
          , vol.
          <volume>60</volume>
          , pp.
          <fpage>41</fpage>
          -
          <lpage>95</lpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>M. Gebser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maratea</surname>
            , and
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Ricca</surname>
          </string-name>
          , “
          <article-title>The design of the seventh answer set programming competition,”</article-title>
          <source>in Proc. of the 14th International Conference Logic Programming and Nonmonotonic Reasoning</source>
          , LPNMR 2017
          <string-name>
            <surname>(M. Balduccini</surname>
          </string-name>
          and T. Janhunen, eds.), vol.
          <volume>10377</volume>
          of Lecture Notes in Computer Science, pp.
          <fpage>3</fpage>
          -
          <lpage>9</lpage>
          , Springer,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <given-names>Y.</given-names>
            <surname>Lierler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricca</surname>
          </string-name>
          , “Systems, engineering environments, and competitions,”
          <source>AI Magazine</source>
          , vol.
          <volume>37</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>45</fpage>
          -
          <lpage>52</lpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>M. Gebser</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Leone</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maratea</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Perri</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Ricca</surname>
          </string-name>
          , and T. Schaub, “
          <article-title>Evaluation techniques and systems for answer set programming: a survey,”</article-title>
          <source>in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence</source>
          ,
          <article-title>IJCAI 2018 (J</article-title>
          . Lang, ed.), pp.
          <fpage>5450</fpage>
          -
          <lpage>5456</lpage>
          , ijcai.org,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>M. Gebser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maratea</surname>
            , and
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Ricca</surname>
          </string-name>
          , “
          <article-title>The seventh answer set programming competition: Design and results</article-title>
          ,
          <source>” Theory and Practice of Logic Programming</source>
          ,
          <year>2020</year>
          . To appear.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <given-names>M.</given-names>
            <surname>Gelfond</surname>
          </string-name>
          and
          <string-name>
            <given-names>V.</given-names>
            <surname>Lifschitz</surname>
          </string-name>
          , “
          <article-title>The stable model semantics for logic programming</article-title>
          ,
          <source>” in Proceedings of the International Conference on Logic Programming (ICLP)</source>
          , pp.
          <fpage>1070</fpage>
          -
          <lpage>1080</lpage>
          , MIT Press,
          <year>1988</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <given-names>M.</given-names>
            <surname>Gelfond</surname>
          </string-name>
          and
          <string-name>
            <given-names>V.</given-names>
            <surname>Lifschitz</surname>
          </string-name>
          , “
          <article-title>Classical negation in logic programs</article-title>
          and disjunctive databases,
          <source>” New Generation Computing</source>
          , vol.
          <volume>9</volume>
          , pp.
          <fpage>365</fpage>
          -
          <lpage>385</lpage>
          ,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <given-names>F.</given-names>
            <surname>Calimeri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Faber</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gebser</surname>
          </string-name>
          , G. Ianni,
          <string-name>
            <given-names>R.</given-names>
            <surname>Kaminski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Krennwallner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Leone</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricca</surname>
          </string-name>
          , and T. Schaub, “
          <article-title>Asp-core-2 input language format</article-title>
          ,
          <source>” Theory and Practice of Logic Programming</source>
          ,
          <year>2020</year>
          . To appear.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <given-names>F.</given-names>
            <surname>Calimeri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Faber</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gebser</surname>
          </string-name>
          , G. Ianni,
          <string-name>
            <given-names>R.</given-names>
            <surname>Kaminski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Krennwallner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Leone</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricca</surname>
          </string-name>
          , and T. Schaub, “ASP-Core-2
          <string-name>
            <given-names>Input</given-names>
            <surname>Language</surname>
          </string-name>
          <string-name>
            <surname>Format</surname>
          </string-name>
          ,”
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23. V. Lifschitz, “
          <article-title>Answer set programming and plan generation</article-title>
          ,
          <source>” Artificial Intelligence Journal</source>
          , vol.
          <volume>138</volume>
          , no.
          <issue>1-2</issue>
          , pp.
          <fpage>39</fpage>
          -
          <lpage>54</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <given-names>H. A.</given-names>
            <surname>Kautz</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Selman</surname>
          </string-name>
          , “
          <article-title>Planning as satisfiability</article-title>
          ,”
          <source>in Proceedings of the European Conference on Artificial Intelligence (ECAI)</source>
          , pp.
          <fpage>359</fpage>
          -
          <lpage>363</lpage>
          ,
          <year>1992</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>M. Alviano</surname>
            , G. Amendola,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Dodaro</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Leone</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Maratea</surname>
            , and
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Ricca</surname>
          </string-name>
          , “
          <article-title>Evaluation of disjunctive programs in WASP,”</article-title>
          <source>in Proc. of the 15th International Conference on Logic Programming and Nonmonotonic Reasoning</source>
          , LPNMR 2019
          <string-name>
            <surname>(M. Balduccini</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <string-name>
            <surname>Lierler</surname>
          </string-name>
          , and S. Woltran, eds.), vol.
          <volume>11481</volume>
          of Lecture Notes in Computer Science, pp.
          <fpage>241</fpage>
          -
          <lpage>255</lpage>
          , Springer,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26. E. Giunchiglia,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Tacchella</surname>
          </string-name>
          , “
          <article-title>Dependent and independent variables in propositional satisfiability</article-title>
          ,”
          <source>in Proceedings of the European Conference on Logics in Artificial Intelligence</source>
          , vol.
          <volume>2424</volume>
          of Lecture Notes in Computer Science, pp.
          <fpage>296</fpage>
          -
          <lpage>307</lpage>
          , Springer,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27. E. Giunchiglia,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Tacchella</surname>
          </string-name>
          , “
          <article-title>(In)effectiveness of look-ahead techniques in a modern SAT solver</article-title>
          ,”
          <source>in Proceedings of 9th International Conference on the Principles and Practice of Constraint Programming</source>
          , vol.
          <volume>2833</volume>
          of Lecture Notes in Computer Science, pp.
          <fpage>842</fpage>
          -
          <lpage>846</lpage>
          , Springer,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28. E. DiRosa, E. Giunchiglia, and
          <string-name>
            <given-names>M.</given-names>
            <surname>Maratea</surname>
          </string-name>
          , “
          <article-title>A new approach for solving satisfiability problems with qualitative preferences,”</article-title>
          <source>in Proc. of the 18th European Conference on Artificial Intelligence</source>
          , Patras,
          <string-name>
            <given-names>ECAI</given-names>
            2008
            <surname>(M. Ghallab</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. D.</given-names>
            <surname>Spyropoulos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Fakotakis</surname>
          </string-name>
          , and N. M. Avouris, eds.), vol.
          <volume>178</volume>
          of Frontiers in
          <source>Artificial Intelligence and Applications</source>
          , pp.
          <fpage>510</fpage>
          -
          <lpage>514</lpage>
          , IOS Press,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>