<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>November</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>LeQua @ CLEF 2022: A Shared Task for Evaluating Quantification Systems</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Andrea Esuli</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alejandro Moreo</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Fabrizio Sebastiani</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Istituto di Scienza e Tecnologie dell'Informazione, Consiglio Nazionale delle Ricerche</institution>
          ,
          <addr-line>56124 Pisa</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2021</year>
      </pub-date>
      <volume>5</volume>
      <issue>2021</issue>
      <abstract>
        <p>LeQua 2022 (https://lequa2022.github.io/) is a new shared task (a.k.a., “challenge”, or “competition”) for comparatively evaluating systems for learning to quantify in a tightly controlled environment. LeQua 2022 is one of several shared tasks organized under the umbrella of the CLEF 2022 conference (https://clef2022.clef-initiative.eu/), which will take place in Bologna, Italy, in September 2022. LeQua 2022 participants will be provided with a training set of labelled documents and a development (validation) set of samples of labelled documents, and will be asked to provide class prevalence estimates for a test set consisting of samples of unlabelled documents. The evaluation will use a dataset obtained from a large crawl of product reviews. The training set will be obtained via stratified sampling from the dataset, while the validation samples and the test samples will be obtained by sampling from the dataset using predetermined class prevalence distributions, where these distributions will be obtained by sampling uniformly at random from the set of all legitimate class prevalence distributions. LeQua 2022 will comprise two tasks, the vector task and the raw-documents task. In the vector task, participants will be provided with data already in vector form; this task will thus purely measure the efectiveness of (data-agnostic) quantification algorithms. In the raw-documents task, participants will instead be provided with the original textual documents, and will thus be able to evaluate end-to-end systems. Both tasks will allow two subtasks, a binary task of quantification by sentiment and a multiclass task of quantification by topic. A one-day workshop during CLEF 2022 will be held in order to allow participants to discuss their results and participating systems.</p>
      </abstract>
    </article-meta>
  </front>
  <body />
  <back>
    <ref-list />
  </back>
</article>