<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Proposal for Anonymous Crowdsourced Online Arbitration</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Dusko Martic</string-name>
          <email>dusko.martic@gmail.com</email>
          <email>dusko.martic@unibo.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Law, Science and Technology Joint Degree EM program, IDT-Universitat Autonoma Barcelona</institution>
          ,
          <addr-line>Bellaterra, Barcelona</addr-line>
        </aff>
      </contrib-group>
      <abstract>
        <p>Shifting the academic discourse from general term of ODR to more specific modalities like online arbitration is a clear sign of research advancements in the area of online dispute resolution. In this paper we explore the online arbitration in relation to crowdsourcing trends. The goal of this paper is to present new approach to online arbitration, based on several different priniciples and technologies. The proposal of anonymous/ privacy preserving online arbitration or “blind arbitration” is built upon technologies for online arbitration, crowdsourcing, blind bidding negotiation and founded upon privacy-by-design principles. We aim to propose higher-level of confidentiality, secrecy and privacy preservation along with leveraging “the wisdom of the crowds”.</p>
      </abstract>
      <kwd-group>
        <kwd>Online dispute resolution</kwd>
        <kwd>ODR</kwd>
        <kwd>online arbitration</kwd>
        <kwd>privacy-bydesing</kwd>
        <kwd>crowdsourced arbitraton</kwd>
        <kwd>anonymous arbitration</kwd>
        <kwd>privacy preserving arbitration</kwd>
        <kwd>blind arbitration</kwd>
        <kwd>confindentiality</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        While arbitration, in traditional ADR1 terms, has been long developed and accepted
as a suitable means to resolves conflicts, especially in international business arena,
online arbitration[
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] has merely embraced online communication as extension of its
long established practices with appropriated case management systems and scheduling
tools for automation and ease of processes. Full potential of online arbitration, through
bigger role of technology still remains to be attained.
      </p>
      <p>
        Online arbitration compared to traditional ADR approach bears promise of cheaper
and efficient proceeding. However, technology also brings new risks to the protection
of integrity, confidentiality and security of e-proceedings[
        <xref ref-type="bibr" rid="ref15">15</xref>
        ][
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. Almost every
arbitration institution guarantees communication security, confidentiality (if agreed),
authenticity, and procedural integrity[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. But even with all safeguards in place, some
parties could feel reserved and seek for higher level of confidentiality, privacy or even
anonymity.
      </p>
      <p>
        At the same time we are witnessing the development of new online applications
using “crowds” to facilitate faster, cheaper, collective work to respond to the demands of
the markets or address some public or private need. Utility of “crowdsourcing“[
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] in
the field of dispute resolution has only been recently discussed and few initiatives have
already emerged. This paper aims to further the discussion by proposing new approach
to crowdsourced arbitration which provides with more private or anonymous model of
adjudicative dispute resolution.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Crowdsourcing in online dispute resolution</title>
      <p>
        The idea of using crowdsourcing for online dispute resolution has started with online
juries. Cyberjuries appeared as online version or imitation of traditional juries, where
they have similar role in representing values of a community. Marder points to their
evolution from opinion polls to online mock juries[
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Opinion polls model2 may allow
parties to express feeling about dispute without using legal language, anonymously and
for free. Anonymity is achieved by representing parties with codes or numbers. Online
mock trials are more specialized ODR tools design to give evaluation of the cases,
usually used by lawyer for testing their argumentation and strategies before court.
      </p>
      <p>
        Building on Marder’s work, van der Herik and Dimov give more comprehensive
overview of crowdsourced online dispute resolution [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] by offering three types: online
opinion polls, online mock trials and crowdsourced ODR procedures rendering
decisions that are enforced by private authorities. Adapting the ideas of Malone and
Dellarocas[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], they also offer four building blocks for crowdsourced online dispute
resolution (CODR): The crowd, incentives for motivating the crowd to participate in CODR,
types of disputes which can be solved through CODR and CODR procedure.
      </p>
      <p>
        Van der Herik and Dimov, display EBay’s Community Court as most prominent
example (up to now) of crowdsourced ODR procedures rendering decisions that are
enforced by private authorities. It has been built to deal with negative review disputes
between buyers and sellers on EBay. EBay’s Community Court[
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] had fairly
straightforward procedure: after submissions from buyer and seller, a case is put before
randomly selected panel of jurors. Jurors were experienced EBay community members
fulfilling certain criteria. Upon reviewing submission, a juror needs to decide with
2 Examples: iCourthouse (www.i-courthouse.com) SideTaker (www.sidetaker.com),
People’sCourtRaw(www.peoplescourtraw.com), Truveli (www.truveli.org).
which party he/she agrees or feels that cannot make decision. Each case is reviewed
by 21 jurors, on a voluntary basis. Rule and Nagarajan explained the motivation of
jurors as a sense of service to the community. This system allowed fast resolution and
private enforcement within EBay market.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Do we need privacy in online conflict resolution?</title>
      <p>
        Since data about disputes could be sensitive, informational privacy[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] is of highest
importance to the ODR system designers. However, as opposed to face-to-face ADR
where we can trust that no information is being recorded, online interaction always
leaves a trail. It is even more difficult to control or enforce professional and ethical
standards of arbitrators in online arbitration. What the online arbitrator is doing with
sensitive data and who can physically access his computer are just some of the issues
that make parties question the integrity and security of online arbitration.
      </p>
      <p>
        On the other hand, since dispute involve more than one party, sociologists[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] have
identified three phases in the dispute: “naming” (internally recognizing that one has
been harmed), “blaming” (confronting the wrongdoer) and “claiming” (pursuing legal
remedies). Blaming and claiming are in front of the third party or public. Orna
Rabinovich-Einy suggests that the more public a dispute, the less control over the
information regarding the dispute the parties have, and, accordingly, the less room there is
for secrecy and anonymity[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ].
      </p>
      <p>EBay’s Community Court was not designed to maintain secrecy, but to effectively
engage crowd in scrutinizing inappropriate reviews. Some providers, like iCourthouse,
offer anonymity but disclose the facts and submissions of the case, which in the age of
Google search could lead to easy discovery of identity. Jurors are also aware of the
result of the dispute. In some cases, even the knowledge that dispute exists can lead to
bad reputation to a party or a business. Spreading the information about disputes and
their trail online can have counter-effect to attracting parties to crowdsourced online
dispute resolution.</p>
      <p>Hence the question: how can we leverage the wisdom of the crowds, but keep the
high level of anonymity of parties and secrecy of the case?
4</p>
      <p>“Blind” proposal</p>
      <p>We propose “blind arbitration” or privacy preserving arbitration which could be
simplified with following formula of principles and technologies that were combined in
proposal for the conceptual model:</p>
      <p>
        Blind arbitration = crowdsourced arbitration + arbitration management software +
privacy by design[
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] + (blind bidding negotiation).
      </p>
      <p>The method for this anonymous crowdsourced arbitration can be described as
dividing the integral, bigger text submissions of parties to small questions for arbitrators to
answer. The principle idea that is being proposed is anonymous crowdsourced
arbitration, through obfuscation of the general picture of dispute by focusing on small tasks
or questions. In a way it is putting into practice famous big picture phrase “you can’t
see the forest for the trees”.</p>
      <sec id="sec-3-1">
        <title>In the following figure we illustrate broadly the model:</title>
        <p>
          This proposal assumes that both parties agree to maintain the confidentiality/secrecy
of the dispute and that it is in their best interest to keep it private, with as little
information to give away about the dispute or about facts related or connected to the parties.
Having in mind van der Herik and Dimov’s four building blocks for crowdsourced
online dispute resolution[
          <xref ref-type="bibr" rid="ref2">2</xref>
          ], we will mostly focus on the fourth block- the CODR
procedure. We will give only brief proposals for previous three building blocks.
5
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Crowd, incentives and types of disputes in blind arbitration</title>
      <p>Arbitrators will be selected by parties or software automatically (if decided by
parties or consensus could not be reached) by certain criteria that are most relevant to the
dispute, i.e. expertise in certain matter. Preferably, database of arbitrators on a global
level should be in place as arbitration will be held online and the location of persons is
not relevant.</p>
      <p>Most importantly, arbitrators will be informed on need-to-know basis. They will not
know who are the parties, what the dispute is about (except the question at their hand),
what is the result of the process, are there any other dispute questions posed and the
result of those issues. Arbitrators will be chosen to answer only one question and will
not be connected to any other issue in the arbitration. This means that the person who
deals with question of facts do not know what the dispute is about or what are the legal
issues within case, as well as who are other arbitrators. They are all communicating
only by online platform ignorant to who are other arbitrators in the database.</p>
      <p>We propose incentives to be similar to the model of incentive/payment for
microtasks offered at Amazon’s Mechanical Turk3, especially for answering simple
questions with right-wrong answers. For more complicated answers with written
explanations and reasoning, the payment would correspond to a price previously set by
arbitrator (per question).</p>
      <p>The range of types of disputes that theoretically could be solved by this method is
wide, but for the moment (until fully tested) we would suggest focusing the disputes
around single issue. It seems that the most appropriate would be e-commerce disputes
or labor/contract disputes about the quality of produced work4. However, the model is
flexible enough to be extended to more complicated disputes with several issues which
could be handled simultaneously or subsequently.
6</p>
    </sec>
    <sec id="sec-5">
      <title>The procedure</title>
      <p>We will present the concept through three separated phases: confidentially
agreement, submission and resolution. The third phase consists of four stages.
3 https://www.mturk.com/mturk/
4 i.e. disputes about the quality of outsourced work/results from the contract concluded on sites
like Elance.com, Guru.com, Odesk.com. One party would claim that the work is not done
according to the specifications and the other party disagrees
6.1</p>
      <sec id="sec-5-1">
        <title>Phase 1 – confidentiality agreement.</title>
        <p>From the beginning we need parties’ agreement to this kind of arbitration, specifying
it in more formal way, by arbitration agreement that clearly states the will of parties to
be the part of the blind arbitration and the commitment to confidentiality agreement
among themselves, with specific terms and high penalties in case of breach. If parties
do not agree on these terms, the use of this model of arbitration would be rendered
pointless since the other party would not care to keep the opposing party’s privacy in
check.
6.2</p>
      </sec>
      <sec id="sec-5-2">
        <title>Phase 2 – submission of claims.</title>
        <p>After registering for the platform, parties will be faced against each other during
dispute with open communication channel through any messaging technology. This
way they could directly negotiate and in any moment stop or stay the dispute
proceeding, in order to have an agreement putting stop to a dispute.</p>
        <p>
          For the same reason, but less revealing, we suggest simultaneous use of blind bidding
technology[
          <xref ref-type="bibr" rid="ref7">7</xref>
          ][
          <xref ref-type="bibr" rid="ref5">5</xref>
          ][
          <xref ref-type="bibr" rid="ref6">6</xref>
          ], especially for the possible monetary disputes or about potential
award issues. Blind bidding allows parties to hide their willingness to compromise, but
to actively engage in blind offering with hope that the opposing party will be willing to
compromise close to their offer. If at any moment offers overlap, the arbitration should
halt automatically by software intervention.
        </p>
        <p>The communication area should be also open to exchange of arguments of parties,
submission of queries and for discovery purposes, but also to post question and submit
issues and questions to arbitrators and to receive and respond on communications from
arbitrators. However the communication area should not be revealed to arbitrators.</p>
        <p>The conceptual model is founded upon idea that every dispute could be dissolved to
its parts and disputed issues. We divide possible disputes in 4 general types: applicable
law, disputes about facts (and connected to it-evidence evaluation procedure),
interpretation of law disputes (about the rights and duties, interpretation of contract…), dispute
about (appropriate) awards.</p>
        <p>The electronic (fill-in) form, for posing disputed questions to arbitrators,
corresponds to this division and organized in different brackets. When certain aspect is not
disputed, it is stated in the bracket for that part of general dispute areas. For example,
if applicable law is specified and not disputed by any party, it will be placed in the first
bracket and clarified if any other regulation should be consulted.</p>
        <p>If facts are not disputed, the recount of relevant facts has to be filled and confirmed
by both parties (or just one party gives factual information and other agrees). If one
party disputes any part then they are not in agreement and it should be decided by
arbitrage. If facts are disputed, the parties will be given opportunity to submit evidence that
should be properly anonymised when containing certain information connecting parties
to it (if this is not possible because of the nature of the facts, evidence or question we
will offer possible solution in later phase). Pictures of persons or any image or
information that could identify parties should be scrambled or obfuscated at least.</p>
        <p>If a dispute is about legal issues parties should support arguments citing legal
references. Parties should be clear about their interpretations and about the legal question
that they are submitting to the arbitrage. Arbitrators will not be giving decisions on the
whole case at once but on each question separately. The questions, therefore, should be
about essential issues and questions should be clear enough.</p>
        <p>An award proposal should be clearly stated as well. If rules of procedure (and
applicable law considers this lawful) insist that the award can be granted only based on
claims of parties, software could by itself recognize and grant an award to a party who
wins in dispute, so human involvement could be minimized in this aspect (in last phase).</p>
        <p>This phase should be open so both parties can see final statements, claims, arguments
of other party as it will be presented to the arbitrators. If they insist on their side of the
story and after at least one party finalizes its claim by pressing the submit option, the
software will react by giving reasonable timeframe for other party to finish its
argumentation, after which whatever is written in the form will be submitted.</p>
        <p>The most important factor of their cooperation will be formulation of their
submissions that do not reveal personal information, as previously agreed by their
confidentiality agreement. Both parties will clearly state their arguments and ask appropriate
question (legal or factual) to arbitrators to resolve a dispute. The whole phase will be
structured so the argumentation of the parties could be separated in different brackets,
and each bracket will end with specific question for arbitrators (about which there is a
misunderstanding). The parties will be advised to formulate (collaboratively) single
question for one issue. However, if they disagree on the question, the default question
should be posed, appropriate to the stage of dispute. It would be also possible to ask the
question in simplest manner: which party is right? If they do not reach consent about
questions the default option should be viable so arbitrators could always choose one
option. Possibilities of these questions will soon be further explained.</p>
        <p>This phase is characterized also by collaboration in discovery phase and in
anonymisation of data while submitting claims and questions. Not achieving any of these
elements would lead to breach of agreement and parties should resort to some other
form of dispute resolution like regular arbitration or judicial process. The agreement
will clearly state that in case of an obstruction of any party, or simply by the will of one
party, blind arbitration will seize and parties can resort to some other form of dispute
resolution.</p>
        <p>Submissions of claims and question will be handled by software built upon PET
principles and cryptography for assigning either false name to parties or code name or
any other type of hiding proper names of parties involved. Parties themselves will
control submissions to prevent indirect discovery of their identities or any confidential
information.
6.3</p>
      </sec>
      <sec id="sec-5-3">
        <title>Phase 3 – answering questions.</title>
        <sec id="sec-5-3-1">
          <title>The answering of individual question will be sequenced in stages:</title>
        </sec>
      </sec>
      <sec id="sec-5-4">
        <title>6.3.1. Applicable law.</title>
        <p>Applicable law is usually specified in contracts. Nevertheless, the issues sometimes
occur and there is a need for clarification, especially in some predispute agreements. It
would also be the first question raised in a proceeding both before an arbitration or in a
judicial proceeding, therefore it is only natural to be the first answered. Depending on
the answer, the following stages will be decided upon. Even for the factual dispute or
even more likely in the case of evaluation of evidence, the applicable law could be
sometimes essential precondition, especially if some evidences are to be evaluated in
connection with certain standards set by specific law. Default question could be: what
law should apply to the dispute?</p>
      </sec>
      <sec id="sec-5-5">
        <title>6.3.2. Facts and evaluation of disputed evidence.</title>
        <p>A dispute about facts and evidence evaluations is the most problematic since the
general idea is to preserve confidentiality/privacy by withholding information about
identity or any indirect information that could reveal the same. Arbitrators should give
answer to which facts have occurred based on provided evidence or recounts by parties.
The difficult question is how to keep an evaluator of facts and evidence in dark about
the parties if they are pointing to parties and indicating them clearly.</p>
        <p>At any moment at this phase, an arbitrator could pose questions to the parties, which
they will answer, always having in mind not to breach confidentiality agreement.
However, if there is more than one arbitrator answering the same question, answers of parties
should be available to all of them, as it will serve as a basis for their decision.</p>
        <p>If it is necessary to have someone’s expertise or to attest that certain disputed facts
have happened, an arbitrator would be limited to that aspect, not knowing a reason of
one’s testimony or deciding upon credibility. Arbitrator would not know the context of
one’s testimony or significance of his/hers decision for the rest of the process.</p>
        <p>If facts are particularly revealing of some elements that parties would want to keep
hidden there would still be options: the question could be broken down in several
different aspects, the question could be misleading to the nature and answer that is sought,
there could be possibility to hide certain elements of facts or to replace them with
interchangeable things, at the same time together with these facts there could be offered
additional false facts… We can imagine that parties offer 4 different set of supporting
facts or stories, but only parties and software know accurate version. It would mean
that arbitrators should decide on all of them and maybe their involvement would be 4
times higher, but that is again tradeoff between ultimate goal of confidentiality and
costs of the process.</p>
        <p>We could also imagine the situation where both parties have previously hired their
expert to validate their claims. An arbitrator would give opinion based on their expert
reports having in mind the reliability of their methods</p>
        <p>In the end arbitrator(s) of facts would give the opinion/ answer to a question which
would be taken as a ground on which the later phases would be decided, just as the facts
were not disputed at all. If in his answer there would be some revealing element for the
identity, it would be removed by parties before transitioning into the next stage. If a
dispute involves larger number of facts to be verified, it is even easier to imagine
involving bigger number of arbitrators to deal with this task simultaneously.</p>
        <p>Default question in case of disagreement about question would be: which party’s
interpretation of facts is correct?</p>
      </sec>
      <sec id="sec-5-6">
        <title>6.3.3. Legal expertise.</title>
        <p>Many times facts will be undisputed and the only issues that parties will have is a
legal interpretation of some situation. In the stage where different legal interpretations
are resolved (if there are any), we emphasize two possibilities in previous phases: first,
that there were no disagreements on facts of the case or they have been negotiated and
settled upon; second, the disputed facts have been settled by the previous arbitrators
and we can proceed with third stage.</p>
        <p>The problem here is of similar nature to the previous stage: if the question is too
revealing we must find a way for maintain the anonymity. One way would be for parties
to ask general legal question based on false facts. Second way would be to break down
legal question to the smallest possible unit, where answers could later be assembled
into plausible complete legal answer (by award arbitrator). Third way would be, in
addition to masking of names and identifiable information, to offer 4 different set of facts
of similar but slightly different nature that would make it impossible to know the real
facts of the case. Arbitrator would be required to give decision and reasoning on all 4
set of facts, not knowing which could be true.</p>
        <p>Default question in case of disagreement would be: whose legal interpretation is
correct?</p>
        <p>Finally, if many arbitrators are used and give different decisions about same
question, they could be compiled into one decision in favor of one party by majority. The
party with decisions that have more than 50 percent of votes wins that question/mini
case.</p>
        <p>6.3.4. Awards.</p>
        <p>The parties could specify the key question in one of the phases, where by answering
in favor of one party, that party wins automatically the case. If there is more than one
key question, they could set odd number of keys questions and agree that the party,
whose interpretation is correct in more answers, wins. Previous questions would
establish facts and build arguments for answering key question. However, if the parties do
not agree about the key question(s), it remains for arbitrators in the last stage to decide
who has won based on question and accepted argumentation of wining parties in
previous stage.</p>
        <p>If parties agree, they could state that according to the final decision in a case an
award is made in accordance with the claim of the winning party(automatically), or
they can pose question about which award is appropriate for the dispute.
7</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Future research</title>
      <p>
        The aim of the paper is the initial presentation of the idea, with the hopes of stirring
further discussion. Even though we tried to answer some question, inevitably there will
be a lot more, relating to the legal, technical and economic aspects of such proposal.
All of them will be subject of the future research. Some of the questions that instantly
come in mind are: what would be the quality of justice offered by this model, what is
the quality vs. secrecy relation, what are the average costs of the arbitration… As regard
to some concerns about fair trial/hearing or ethical issues, we do not propose this model
to be part of predispute mandatory arbitration clause, which could lead to abuse, but
merely to serve as a possible tool if need ever occurs. We would argue that it is
appropriate for certain disputes[
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] therefore the need might occur in some cases.
8
      </p>
    </sec>
    <sec id="sec-7">
      <title>Conclusion</title>
      <p>In this paper we state our belief that it is possible to leverage the wisdom of the
crowds for more efficient and cheaper dispute resolution but at the same time to keep
the dispute a secret. For these reasons we proposed a counterintuitive idea to solve
hidden disputes by opening the issues to unknown persons to decide about, but keeping
them in “dark” as much as possible.</p>
      <p>We have presented initial idea for the conceptual model that provides with certain
confidentiality/privacy improvements in crowdsourced arbitration proceedings. Parties
themselves will be bind by confidentiality set in the agreement on the start of the
process, specifying their commitment to guard privacy of opposing party. To prevent
arbitrators in online arbitration to know the parties or the subject of the dispute, we have
proposed the way to break the dispute in smaller bits in order to distribute those bits to
wider group on individual arbitrators (the crowd arbitrators), who work independently
only on one bit. Thus, we achieve the effect that one single arbitrator cannot grasp the
whole picture. We could say it puts into effect famous saying: “you cannot see the forest
for the trees”.</p>
      <p>
        Our proposal is focused on enhancing impartiality of arbitrators, crowd engagement
in dispute resolution and developing cheaper and more pleasant adjudicative form of
ODR. Significant potential of blind arbitration could also lay on developing such model
for mobile platforms[
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], extending the possibility of massive, anonymous dispute
resolution based on microworks. It could be one more tool in ODR arsenal giving a new
option to disputants who wish to remain anonymous, don’t want to be subject of
prejudice or receive bad reputation.
      </p>
      <sec id="sec-7-1">
        <title>After all, lady Justice (Iustitia) is blindfolded for a reason.</title>
        <p>9</p>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgements</title>
      <p>I would like express gratitude to Erasmus Mundus Law, Science and Technology PhD
(LAST-JD) program that allowed me to pursue my research.
10</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Felstiner</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          et al.:
          <source>Emergence and Transformation of Disputes: Naming</source>
          , Blaming, Claiming...,
          <source>The. Law Soc'y Rev</source>
          .
          <article-title>(</article-title>
          <year>1980</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Herik</surname>
            ,
            <given-names>J. Van Den</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dimov</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Towards Crowdsourced Online Dispute Resolution</article-title>
          .
          <source>J. Int. Commer. Law Technol</source>
          .
          <year>2006</year>
          , (
          <year>2012</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Hörnle</surname>
          </string-name>
          , J.:
          <article-title>Cross-border Internet Dispute Resolution</article-title>
          . Cambridge University Press (
          <year>2009</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Langheinrich</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Privacy by design-principles of privacy-aware ubiquitous systems</article-title>
          .
          <source>Ubicomp 2001 Ubiquitous Comput</source>
          .
          <article-title>(</article-title>
          <year>2001</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Larson</surname>
            ,
            <given-names>D.A.</given-names>
          </string-name>
          :
          <article-title>Artificial Intelligence: Robots, Avatars and the Demise of the Human Mediator</article-title>
          . (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Lodder</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>The Third Party and Beyond. An analysis of the different parties, in particular The Fifth, involved in online dispute resolution</article-title>
          .
          <source>Inf. Commun. Technol. Law</source>
          .
          <volume>15</volume>
          ,
          <issue>2</issue>
          ,
          <fpage>143</fpage>
          -
          <lpage>155</lpage>
          (
          <year>2006</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Lodder</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zeleznikow</surname>
          </string-name>
          , J.:
          <source>Artificial Intelligence and Online Dispute Resolution. Online Dispute Resolution: Theory and Practice: A Treatise on Technology and Dispute Resolution</source>
          . pp.
          <fpage>73</fpage>
          -
          <lpage>94</lpage>
          Eleven International Publishing (
          <year>2012</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Malone</surname>
          </string-name>
          , T.W. et al.:
          <article-title>Harnessing Crowds: Mapping the Genome of Collective Intelligence</article-title>
          . SSRN Electron. J. (
          <year>2009</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Marder</surname>
          </string-name>
          , N.:
          <article-title>Cyberjuries: A new role as online mock juries</article-title>
          .
          <source>Univ. Toledo Law Rev</source>
          . (
          <year>2006</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Martic</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Online Dispute Resolution for Cloud Computing Services. ceur-ws.org. ceur-ws (</article-title>
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Poblet</surname>
          </string-name>
          , M. ed: Mobile Technologies for Conflict Management. Springer Netherlands, Dordrecht (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Rabinovich-Einy</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>Going public: Diminishing privacy in dispute resolution in the Internet age</article-title>
          .
          <source>Va. JL Tech. 4</source>
          ,
          <fpage>1</fpage>
          -
          <lpage>55</lpage>
          (
          <year>2002</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Rule</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nagarajan</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Leveraging the Wisdom of the Crowds: the Ebay Community Court and the Future of online Dispute Resolution</article-title>
          .
          <source>ACResolution 2 4</source>
          . (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Surowiecki</surname>
          </string-name>
          , J.:
          <source>The Wisdom of Crowds. Anchor</source>
          (
          <year>2005</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Wahab</surname>
            ,
            <given-names>M.S.A.</given-names>
          </string-name>
          : ODR and e-Abritation.
          <source>Online Dispute Resolution: Theory and Practice: A Treatise on Technology and Dispute Resolution</source>
          . pp.
          <fpage>399</fpage>
          -
          <lpage>442</lpage>
          Eleven International Publishing (
          <year>2012</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Wahab</surname>
            ,
            <given-names>M.S.A.</given-names>
          </string-name>
          et al.:
          <article-title>Online Dispute Resolution: Theory and Practice: A Treatise on Technology and Dispute Resolution</article-title>
          . Eleven International Publishing (
          <year>2012</year>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>