<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>M. Stevenson, R. Bin-Hezam, Stopping Methods for Technology-assisted Reviews Based on Point
Processes, ACM Trans. Inf. Syst.</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="doi">10.1145/3631990</article-id>
      <title-group>
        <article-title>Stopping Methods Based on Point Processes: Recent Developments</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Mark Stevenson</string-name>
          <email>mark.stevenson@shefield.ac.uk</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Reem Bin-Hezam</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Information Systems, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University</institution>
          ,
          <country country="SA">Saudi Arabia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>School of Computer Science, University of Shefield</institution>
          ,
          <addr-line>Regent Court, 211 Portobello, Shefield</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2023</year>
      </pub-date>
      <volume>42</volume>
      <issue>2023</issue>
      <abstract>
        <p>Technology Assisted Review (TAR) develops methods to assist high-recall retrieval, the process of identifying a large proportion (or all) of the relevant documents within a collection. Stopping methods aim to determine when enough relevant documents have been identified to achieve a desired level of recall (the target recall), potentially leading to a substantial reduction in the efort required to screen a collection. We summarise two recent publications describing a TAR stopping method based on point processes [1, 2]. Point processes are statistical models that can represent the occurrence of events, such as encountering a relevant document when examining a ranked list. We estimate the total number of relevant documents in a ranked collection by obtaining relevance judgements for the highest ranked documents and then applying a point process to estimate the number of relevant documents in the remainder of the collection. This approach was evaluated using standard collections within TAR research and compared against eight baseline methods employing a range of stopping strategies. Results indicated that the proposed approach performs well in comparison with the baselines and is Pareto optimal in almost all configurations. The approach was extended by training a text classifier using existing relevance judgements which is then applied to the remaining documents and the information produced used to refine the point process's estimate of the number of relevant documents. Experiments showed that the approach substantially reduced the cost (often by as much as 50%) of the original approach with only minimal (statistically insignificant) reduction in reliability.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Technology Assisted Review</kwd>
        <kwd>TAR</kwd>
        <kwd>Stopping methods</kwd>
        <kwd>Point processes</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body />
  <back>
    <ref-list />
  </back>
</article>