<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Simple User Assistance by Data Posting (Discussion Paper)</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Elio Masciari</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Domenico Sacca</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Irina Trubitsyna elio.masciari@icar.cnr.it</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>sacca</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>trubitsynag@dimes.unical.it</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>ICAR-CNR, DIMES-Universita della Calabria</institution>
          ,
          <addr-line>87036 Rende (CS)</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Big Data deeply changed both researchers and industries approaches to data management. This impressive amount of data contains a lot of information regarding user suggestions and searches. In this paper, we focus on the e ect of users suggestions on their social environment. Suggestions are provided to user by means of simple rules that are dened on the basis of the data being analyzed. We address the reaction of users in a competitive environment when they were invited to judge each other choices. The above mentioned activity is crucial to perform e cient identi cation of users able to spread their in uence across the network, which is a key activity in several scenarios like tourism promotion, personalized marketing, and entertainment suggestion.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Big Data paradigm is currently driving both research and industrial interests.
Many proposals for innovative data analysis for managing impressive amounts of
data have been developed. These approaches share the common needs to rethink
storage and analysis techniques in order to deal with massive, heterogeneous
data, that are fast varying [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Indeed, the Big Data revolution has been guided
by the continuous advances of the web-based services that have millions of users
generating new data everyday. As an example, applications such as Uber,
Facebook or Twitter attract millions of people posting information about their habits
and activities. Quick analysis of these data is a key activity in several scenarios,
such as tourism promotion, personalized marketing and entertainment o ers.
      </p>
      <p>
        To this end in [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] a collaborative network that allows users to cooperate each
others for high performance task execution (by partitioning complex tasks in
smaller and easier sub-tasks) is described. Each user can operate as client and/or
executor. As the tasks assigned to executors may consist in processes de ning
speci c features for a context (e.g., deciding whether a restaurant is better than
another), it may happen to deal with inconsistent or incomplete data. Tasks
requiring details of all possible solutions obtainable with data exchange process can
Copyright © 2019 for the individual papers by the papers' authors. Copying
permitted for private and academic purposes. This volume is published and copyrighted
by its editors. SEBD 2019, June 16-19, 2019, Castiglione della Pescaia, Italy.
be solved applying approximate certain query answering algorithms [8{10, 12].
To extract concise data chosen among the possible alternatives on the basis of
analyst's indications the data posting framework can be used [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. The latter
formalism allows us to model di erent situations. However, it requires the ability
to manage count constraints and domain relations (we formally describe these
notions in the background section). To simplify the modelling process, we
propose to integrate the standard mapping with some parameters, avoiding complex
speci cations. We call the obtained rules smart mapping rules (more details can
be found in [
        <xref ref-type="bibr" rid="ref13 ref14">13, 14</xref>
        ]. As an example, in Section 3 we consider the task of
generating user suggestions on the basis of information posted by other peoples and
available from the Web. We show how this process can be modelled with smart
mapping rules.
      </p>
      <p>Since the sub-tasks are solved by independent executors, payed for their
activities, an important problem regards the quality evaluation of the obtained
results. In our context the clients can ask for search directions. Once the result
is obtained, we can ask the client to evaluate the obtained suggestion, i.e. give
a rank for the executor performing this task. Moreover, some credits to
executors suggesting useful search directions to clients are assigned. We analyze what
happens with users' evaluations when they are aware of the other ones. In
particular, we consider a group of users involved in a research project and we let
them know mutual preferences on some topics.</p>
      <p>
        We use Exponential Graph Random Models (EGRM) [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] to analyze the
network dynamics and to model the revenue ow generated by user interactions.
Since the maximum likelihood evaluation is computationally hard, we use two
approaches for sampling data: Metropolis Hastings sampling and Clustering based
sampling. Moreover, since the data are continuously updated, we use two type
of statistical analysis: cross-sectional and dynamic.
      </p>
      <p>
        It has been shown in literature, that in many settings the possibility to
observe the other user rankings causes mutual adjustment among raters. In our
framework, higher rankings are associated with positive evaluation (thus higher
rewards), such that being ranked below others is aversive. To properly
measure the above mentioned network dynamics, we compute three values proposed
in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]: (i) Global Nonconformity (GNC) - indicates how user conforms with
others'; (ii) Local Nonconformity (LNC) - indicates how user conforms with
neighbours; (iii) Deference Aversion (DA) - measures the following phenomenon: if
user A ranks B above C (B &gt; C), and user C would like rank A above B
(A &gt; B), since the transitivity of ranking would state that A is above C, user C
is in uenced negatively to express his ranking as above de ned. Based on these
measures we performed a deep experimental analysis.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Preliminaries</title>
      <p>
        The data posting framework [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] adapts the well-known data exchange techniques
to the new Big Data management and analysis challenges we nd in real world
scenarios. The data posting setting (S; D; T; st; t) consists of a nite source
database schema S, a nite domain database scheme D, a target schema T, a set
st of non-deterministic source-to-target TGDs and a set t of target count
constraints. A non-deterministic source-to-target TGD (NdTGD) is a dependency
over hS; D; Ti of the form 8x [ S (x[y~) ! T (z) ]; where x and z are lists of
universally quanti ed variables; y~ is a (possibly empty) list of variables, called non
deterministic, these variables can occur in S only in relations from D; x \ y~ = ;
and z x [ y~; the formula S and T are conjunctions of atoms with predicate
symbols in S [ D and in T, respectively. The structure of NdTGDs ensures that
any target database on T is nite. The NdTGD can be seen as the standard TGD,
where existentially quanti ed variables are replaced with non-deterministic
variables, whose values can be chosen from the nite domains de ned by domain
relations, i.e. relations from D. The mapping process is performed as usual but
presumes that for every assignment of x a subset of all admissible values for y~
can be chosen in an arbitrary way. A count constraint is a dependency over
T of the form 8x [ T (x) ! #(fY : 9 z (x; Y; z)g) &lt;op&gt; (x) ]; where T is a
conjunction of atoms with predicate symbol in T, &lt;op&gt; is any of the
comparison operators (=, &gt;, , &lt; and ), H = fY : 9 z (x; Y; z)g is a set term, #
is an interpreted function symbol that computes the cardinality of the (possibly
empty) set corresponding to H, #(H) is a count term, and (x) is an integer
or a variable in x or another count term with universally quanti ed variables in
x. The lists x, Y and z do not share variables, (x; Y; z) is a conjunction of
atoms Ti(x; Y; z) with Ti 2 T. Let IT be the instance of T. The active domain
ADI is the set of all values occurring in IT . Given a substitution x=v assigning
values in ADI to universally quanti ed variables, Kv = fY : 9 z (v; Y; z)g
de nes the set of values in ADI assigned to the free variables in Y for which
9 z (v; Y; z) is satis ed by IT and #(Kv) is the cardinality of this set. We say
that IT satis es the count constraint if each substitution x=v that makes true its
body expression, makes also true its head expression. The data posting problem
is de ned as follows: given nite source instance IS and nite domain instance
ID , nd a nite target instance IT such that hIS ; ID; IT i satis es both st and
t. This problem is N P-complete under the data complexity. When st does
not contain non-deterministic variables, the problem becomes polynomial.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Setting up user suggestions</title>
      <p>In this section we consider the task requiring generation user suggestions on the
basis of information posted in the Web. In the following example we show how
this process can be modelled with smart mapping rules.</p>
      <p>Example 1. Suppose to have the description of the user ratings (that we call
comments) stored in the relation C(U; N; V) with attributes U (user identi er),
N (argument of rating) and V (rating value). Suppose also to have a relation
T(U1; U2; L) that represents the trust level L of user U2 for the user U1.</p>
      <p>In order to suggest to the user some \relevant" arguments we can set the
following criterion: \An argument n is suggested to the user u if the following
conditions hold: 1) it is su ciently supported, i.e. it is \supported" by at least 20
users whose trust level towards u is greater than 70%; 2) if di erent comments
corresponding to the same argument are su ciently supported, only one can be
selected, with the greater level of support". In this case, the choice of the comment
has to take into account two di erent needs: 1) deciding if the argument has to
be suggested to the user; 2) selecting the rating value for this argument.</p>
      <p>This scenario can be modeled as follows:
{ C and T are source relations;
{ Add(U; N; V) is target relation, specifying the comments to be suggested to U ;
{ the smart mapping rule is reported below:</p>
      <p>C(u2; n; v) ^ T(u1; u2; l) ^ l &gt; 0; 70
u2;20;hv;unique^maxi
! Add(u1; n; v)
Intuitively, the body of the rule allows us to restrict the attention to the users
whose trust level towards u1 is greater than 70%, the selection criterion has been
synthesized on the arrow, indicating 1) the support variable u2, 2) the minimum
quantity 20 of support instances to be able to map, and 3) the variable v whose
value should be chosen from the set of candidate values following the indication
unique ^ max.</p>
      <p>More formally, a smart mapping rule has the form 8z[ S (z) y;k;hv;f!i r(x; v) ];
where x; y; z; v are vectors of variables, such that x [ y [ v z and x, y and
v do not share the variables; S is the conjunction of source relations and
expressions involving comparison operators (&gt;; &lt;; ; ; =; 6=) and variables in z or
constants; r is a target relation; y is called a support vector ; k is a natural
number (greater than 0) which indicates the support value; the pair hv; f i indicates
how the choice for the values of v should be performed: f can be max, unique,
the conjunction unique ^ max, or empty.</p>
      <p>We assume that each target relation can be de ned by only one smart
mapping rule. The smart mapping rule speci es that the tuple hX; Vi is added to
r only if it is supported by at least k (di erent) initializations fY1; :::Ykg of y
, i.e. for each j 2 [1::k] there exists an initialization Zj of z, that maps x, y e
v in X, Yj and V respectively, and that makes true S (Zj). In the case than
no further indications of choice are speci ed (the third arrow label is hv; i) all
the tuples supported by at least k (di erent) initializations of y are added to r.
Otherwise, the set of tuples to be added is further reduced using f content for
the selection of values in v.</p>
      <p>The hv; uniquei constraints speci es that the r relation must obey the
functional dependency x ! v, i.e. for each assignment of values in x the assignment
of values in v must be unique. In the case that several tuples are supported by
at least k (di erent) initializations of y and they have the same values in x, the
choice can be made arbitrarily.</p>
      <p>The indication hv; maxi speci es that, for each X only tuples supported by
a maximum number of initializations of y must be selected. It is easy to see that
this indication does not guarantee the uniqueness of the choice. For example, it
is possible that two tuples hX; V1i and hX; V2i have the same degree of support
corresponding to the maximum value.</p>
      <p>The indication hv; unique ^ maxi speci es that, xed X, only one tuple
hX; Vi can be chosen among those supported by a maximum number of (di
erent) initializations of y.</p>
      <p>The set of smart mapping rule can be translated into the standard data
posting constructs by translating every mapping rule as follows. We introduce
the unary domain relation D . If f is empty D contains values 1 and 1,
otherwise D contains values 1, 0 and 1. We also introduce the target relations
A (X; Y; V) and Add (X; V; F lag), where X, Y and V represent vectors of
attributes corresponding to the vectors of variables x, y, and v, respectively,
while the decision weather to select the pair hx; vi in the target relation r is
stored by the attribute F lag: 1 or 0 (not added) and 1 (added). The mapping
rules are:</p>
      <p>S (z) ! A (x; y; v)</p>
      <p>S (z) ^ D (flag) ! Add (x; y; flag)
where all the variables are universally quanti ed. Using the domain relation D
ensures that only a value among 1, 0 and 1 can be chosen for each initialization
of (x; y) in the relation Add .</p>
      <p>The set of target count constraints is constructed as follows.
1. We start by adding support constraints, that ensure that the value 1 is
assigned to the variable flag i the degree of support of the combination
hx; vi does not reach k. First, we add constraint for values 1 and 1.
If f is not empty, we also add constraint for value 0:</p>
      <p>Add (x; v; 1) ! #(f Y : A (x; Y; v)g) k
Add (x; v; 1) ! #(f Y : A (x; Y; v)g) &lt; k
Add (x; v; 0) ! #(f Y : A (x; Y; v)g)
k
2. When f = unique the uniqueness choice constraint is added:</p>
      <sec id="sec-3-1">
        <title>Add (x; ; flag) ^ flag</title>
        <p>0 ! #(f V : Add (x; V; 1)g) = 1
This constraint ensures that exactly one initialization of v for each X is
selected.
3. When f = max we add (i) the optimization constraint</p>
        <p>Add (x; v2; 1); Add (x; v; 0) ! #(fY : A (x; Y; v2)g)
#(fY : A (x; Y; v)g)
(ii) the choice constraint, ensuring that at least one initialization of v for
each X is selected</p>
      </sec>
      <sec id="sec-3-2">
        <title>Add (x; ; flag) ^ flag</title>
        <p>0 ! #(f V : Add (x; V; 1)g)
1
4. When f = unique ^ max we add the uniqueness choice constraint and the
optimization constraint.</p>
        <p>
          The use of the smart mapping rules o ers a notable simpli cation of the
data posting framework. The user and/or designer can construct the selection
criterion focusing only on the a small set of parameters. The selection criteria
modeled by smart mapping rules are the most widespread in practice. The
standardization of their representation allows us also to optimize the implementation
of the data posting process. Smart mapping rules, initially developed for our
application scenario, can be pro table used in di erent contexts. In particular, we
are investigating their application in P2P Deductive Databases [
          <xref ref-type="bibr" rid="ref4 ref5">4, 5</xref>
          ].
4
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Experimental Evaluation</title>
      <p>
        In this section we describe the evaluation of the user ranking behaviours. We
report the results obtained on a test set of user searches collected for four months
during a research project. We devised for the experimental evaluation an
approach similar to the one presented in [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] where the authors leverage ERGM
and clustering analysis to partition the user interaction graph in order to tackle
the complexity issues due to graph structures.
      </p>
      <p>We provided each user with two di erent options: 1) they can ask for search
suggestion or 2) they can suggest search directions on request. At xed time
intervals, users may evaluate obtained results (both as consumer and provider
of information) by assigning a rank ranging from 0 to 10 to their experience.
As we implemented a rewarding strategy, i.e. users pay for targeted suggestions
and are paid when they suggest a proper search, the e ectiveness evaluation is
crucial as we do not want user to pay for wrong information.</p>
      <p>We model our small network interactions by a random graph that is
continuously updated as new links among users appear (i.e. new interactions between
a pair of users). In order to better understand the information relevant to our
goal we leverage two types of analysis: the rst one, referred as cross-sectional
(CS), considers only the variation occurring when data are observed while the
second one considers each observation as an independent unit, thus it is referred
as dynamic (Dyn). In order to be fair and to evaluate the in uence excerpted
by users each other, every participant is aware of the other's scores.</p>
      <p>The latter results in the need to analyze factors that are endogenous, i.e.,
those phenomenon that may arise when users know each the rankings of other
thus causing the rankings given by user i to be in uenced by other user rankings
except i. To this end we measure the values of GNC, LNC and DA.
Evaluation. In order to analyze the evolution of user search e ects, we use the
following statistics: overall searches number performed by user; keyword total
number; mean assigned keyword rank. For the sake of completeness, we use both
CS and Dyn analysis combined with clustering (CL) and Metropolis-Hastings
(M-H) sampling. The results are shown in Figure 1. For Dyn analysis we compute
fteen graphs (one for each week except the rst one) for the CS analysis we
have 16 graphs. It is easy to observe that in tables a and b the values obtained
for DA are always high since the beginning. It is worth noting that, as we start
rewarding users (at week 4), the DA values further increase. More in detail,
both for DYn and CS analysis global nonconformity is not a signi cant factor.
The values reported for the other two factors are, on the whole, signi cant, but
are uniformly smaller in magnitude for dynamic analysis compared to those of
the corresponding weeks in the CS analysis and are less precisely estimated (as
represented by uniformly greater standard errors). This is because rather than
embodying the structure of the whole network, they embody only the structure
of changes in the network over the week, thus the only important value to rely
on is the weekly changes of values. This means that \instant" social e ects (e.g.,
friendship reciprocation) have been absorbed into the Week 0 observation, which
is not modeled in the dynamic analysis.</p>
      <p>Moreover, above mentioned phenomena can be considered as a kind of social
\envy": users tend to decrease the scores of other users in order to get more
requests for themselves and thus getting more rewards. We note that also LNC
value is high while GNC is not impressive: in a sense, users tend to agree on the
general topics but not on the speci c ones. Similar observations can be made for
all the analysis reported in Tables c and d. As a nal note, we can observe that
the results obtained when sampling data by clustering are slightly better. This
behaviour can be explained considering that when leveraging cluster approaches,
it is more likely to obtain more homogeneous groups (i.e. group of user sharing
common interests and features).
5</p>
    </sec>
    <sec id="sec-5">
      <title>Conclusion and Future Work</title>
      <p>The impressive amount of data shared on the WEB contains a lot of
information regarding user suggestions and searches. In this paper we focus on the e ect
of users suggestions on their social environment. We showed how personalized
suggestions can be extracted by using smart mapping rules. This kind of rules,
initially developed for our application scenario, can be pro table used in di
erent contexts. For instance, they allow us to de ne the fragment of data posting
framework, where the data posting problem becomes polynomial. We leveraged
many interesting mathematical tools to model several psychological and social
mechanisms that proved to be e ective in our scenario. We are aware that, with
respect to population-scale networks, the networks considered in our experiments
are fairly small. However, this aspect does not decrease the validity of our
approach as ranking data of the form analyzed here is typically of interest only
in speci c networks, that are by nature fairly small. Thus, highly scalable
techniques are less compelling in our setting. Nevertheless, computationally scalable
estimation is an interesting challenge for future research in this area and we want
to address it in next future.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>D.</given-names>
            <surname>Agrawal</surname>
          </string-name>
          et al.
          <article-title>Challenges and opportunities with big data. A community white paper developed by leading researchers across the United States</article-title>
          .
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>E.</given-names>
            <surname>Bazzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Cassavia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Chiggiato</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Masciari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Sacca</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Spada</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and I.</given-names>
            <surname>Trubitsyna</surname>
          </string-name>
          .
          <article-title>Evaluating user behaviour in a cooperative environment</article-title>
          .
          <source>Information</source>
          ,
          <volume>9</volume>
          (
          <issue>12</issue>
          ):
          <fpage>303</fpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>L. D.</given-names>
            <surname>Brown</surname>
          </string-name>
          .
          <article-title>Fundamentals of statistical exponential families with applications in statistical decision theory</article-title>
          .
          <source>Lecture Notes Monograph Series</source>
          ,
          <year>1986</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>L.</given-names>
            <surname>Caroprese</surname>
          </string-name>
          and
          <string-name>
            <given-names>E.</given-names>
            <surname>Zumpano</surname>
          </string-name>
          .
          <article-title>Aggregates and priorities in P2P data management systems</article-title>
          .
          <source>In Proc. of (IDEAS</source>
          <year>2011</year>
          ), Lisbon, Portugal, pages
          <fpage>1</fpage>
          <issue>{7</issue>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>L.</given-names>
            <surname>Caroprese</surname>
          </string-name>
          and
          <string-name>
            <given-names>E.</given-names>
            <surname>Zumpano</surname>
          </string-name>
          .
          <article-title>Computing a deterministic semantics for P2P deductive databases</article-title>
          .
          <source>In Proc. of IDEAS</source>
          <year>2017</year>
          ,
          <article-title>Bristol</article-title>
          , UK, pages
          <volume>184</volume>
          {
          <fpage>191</fpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>N.</given-names>
            <surname>Cassavia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Flesca</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ianni</surname>
          </string-name>
          , E. Masciari, and
          <string-name>
            <given-names>C.</given-names>
            <surname>Pulice</surname>
          </string-name>
          .
          <article-title>Distributed computing by leveraging and rewarding idling user resources from P2P networks</article-title>
          .
          <source>J. Parallel Distrib. Comput.</source>
          ,
          <volume>122</volume>
          :
          <fpage>81</fpage>
          {
          <fpage>94</fpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>N.</given-names>
            <surname>Cassavia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Masciari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Pulice</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Sacca</surname>
          </string-name>
          .
          <article-title>Discovering user behavioral features to enhance information search on big data</article-title>
          .
          <source>TiiS</source>
          ,
          <volume>7</volume>
          (
          <issue>2</issue>
          ):7:
          <issue>1</issue>
          {7:
          <fpage>33</fpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>N.</given-names>
            <surname>Fiorentino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Greco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Molinaro</surname>
          </string-name>
          ,
          <string-name>
            <surname>and I. Trubitsyna.</surname>
          </string-name>
          <article-title>ACID: A system for computing approximate certain query answers over incomplete databases</article-title>
          .
          <source>In Proc. of SIGMOD Conference</source>
          <year>2018</year>
          , Houston, TX, USA, pages
          <volume>1685</volume>
          {
          <fpage>1688</fpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>S.</given-names>
            <surname>Greco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Molinaro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and I.</given-names>
            <surname>Trubitsyna</surname>
          </string-name>
          .
          <article-title>Computing approximate query answers over inconsistent knowledge bases</article-title>
          .
          <source>In Proc. of IJCAI</source>
          <year>2018</year>
          , Stockholm, Sweden., pages
          <year>1838</year>
          {
          <year>1846</year>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <given-names>S.</given-names>
            <surname>Greco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Molinaro</surname>
          </string-name>
          ,
          <string-name>
            <surname>and I. Trubitsyna.</surname>
          </string-name>
          <article-title>Approximation algorithms for querying incomplete databases</article-title>
          .
          <source>Information Systems</source>
          (
          <year>2019</year>
          ), https://doi.org/10.1016/j.is.
          <year>2019</year>
          .
          <volume>03</volume>
          .010,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>P. N.</given-names>
            <surname>Krivitsky</surname>
          </string-name>
          and
          <string-name>
            <given-names>C. T.</given-names>
            <surname>Butts</surname>
          </string-name>
          .
          <article-title>Exponential-family random graph models for rank-order relational data</article-title>
          . http://arxiv.org/abs/1210.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <given-names>T.</given-names>
            <surname>Lukasiewicz</surname>
          </string-name>
          , E. Malizia, and
          <string-name>
            <given-names>C.</given-names>
            <surname>Molinaro</surname>
          </string-name>
          .
          <article-title>Complexity of approximate query answering under inconsistency in datalog+/-</article-title>
          .
          <source>In Proc. of IJCAI</source>
          <year>2018</year>
          , Stockholm, Sweden., pages
          <year>1921</year>
          {
          <year>1927</year>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13. E.
          <string-name>
            <surname>Masciari</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Sacca</surname>
            ,
            <given-names>and I.</given-names>
          </string-name>
          <string-name>
            <surname>Trubitsyna</surname>
          </string-name>
          .
          <article-title>Simple user assistance by data posting</article-title>
          .
          <source>In Second IEEE International Conference on Arti cial Intelligence and Knowledge Engineering</source>
          ,
          <string-name>
            <surname>AIKE</surname>
          </string-name>
          <year>2019</year>
          , Cagliari, Sardinia, Italy, June 3-5,
          <year>2019</year>
          , pages
          <issue>1{8</issue>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14. E.
          <string-name>
            <surname>Masciari</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Sacca</surname>
            ,
            <given-names>and I. Trubitsyna.</given-names>
          </string-name>
          <article-title>Simpli ed data posting in practice</article-title>
          .
          <source>In Proc. of the 23rd International Database Engineering and Applications Symposium (IDEAS</source>
          <year>2019</year>
          ), June 10 - 12,
          <year>2019</year>
          , Athens, Greece, pages
          <fpage>1</fpage>
          <issue>{6</issue>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>