<?xml version="1.0" encoding="UTF-8"?>
<TEI xml:space="preserve" xmlns="http://www.tei-c.org/ns/1.0" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.tei-c.org/ns/1.0 https://raw.githubusercontent.com/kermitt2/grobid/master/grobid-home/schemas/xsd/Grobid.xsd"
 xmlns:xlink="http://www.w3.org/1999/xlink">
	<teiHeader xml:lang="en">
		<fileDesc>
			<titleStmt>
				<title level="a" type="main">Modeling and Evaluation for Perspectivist NLP</title>
			</titleStmt>
			<publicationStmt>
				<publisher/>
				<availability status="unknown"><licence/></availability>
			</publicationStmt>
			<sourceDesc>
				<biblStruct>
					<analytic>
						<author>
							<persName><forename type="first">Valerio</forename><surname>Basile</surname></persName>
							<affiliation key="aff0">
								<orgName type="institution">University of Turin</orgName>
								<address>
									<country key="IT">Italy</country>
								</address>
							</affiliation>
						</author>
						<author>
							<affiliation key="aff1">
								<orgName type="department">Computer Science Department</orgName>
								<orgName type="institution">University of Turin</orgName>
								<address>
									<country key="IT">Italy</country>
								</address>
							</affiliation>
						</author>
						<title level="a" type="main">Modeling and Evaluation for Perspectivist NLP</title>
					</analytic>
					<monogr>
						<idno type="ISSN">1613-0073</idno>
					</monogr>
					<idno type="MD5">7098F22CB519919A0B9094D4C299B792</idno>
				</biblStruct>
			</sourceDesc>
		</fileDesc>
		<encodingDesc>
			<appInfo>
				<application version="0.7.2" ident="GROBID" when="2025-04-23T16:37+0000">
					<desc>GROBID - A machine learning software for extracting information from scholarly documents</desc>
					<ref target="https://github.com/kermitt2/grobid"/>
				</application>
			</appInfo>
		</encodingDesc>
		<profileDesc>
			<abstract>
<div xmlns="http://www.tei-c.org/ns/1.0"><p>A recent line of research in NLP proposes to never aggregate human annotations 1 , but rather to leverage the worth of knowledge found in label variation for building models [1] and evaluating them [2]. This approach is particularly relevant when dealing with highly subjective aspects of natural language such as irony or undesirable language. In this talk, I will present the perspectivist paradigm in NLP, and the results of recent and ongoing research focusing on building perspective-aware predictive models and automatically extract human perspectives from annotated data. Particular emphasis will be given to language resources, for training or few-shotting models, but also for benchmarking in the perspectivist framework.</p></div>
			</abstract>
		</profileDesc>
	</teiHeader>
	<text xml:lang="en">
		<body/>
		<back>
			<div type="references">

				<listBibl>

<biblStruct xml:id="b0">
	<analytic>
		<title level="a" type="main">Toward a perspectivist turn in ground truthing for predictive computing</title>
		<author>
			<persName><forename type="first">F</forename><surname>Cabitza</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Campagner</surname></persName>
		</author>
		<author>
			<persName><forename type="first">V</forename><surname>Basile</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proceedings of the AAAI Conference on Artificial Intelligence</title>
				<meeting>the AAAI Conference on Artificial Intelligence</meeting>
		<imprint>
			<date type="published" when="2023">2023</date>
			<biblScope unit="volume">37</biblScope>
			<biblScope unit="page" from="6860" to="6868" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b1">
	<analytic>
		<title level="a" type="main">We need to consider disagreement in evaluation</title>
		<author>
			<persName><forename type="first">V</forename><surname>Basile</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Fell</surname></persName>
		</author>
		<author>
			<persName><forename type="first">T</forename><surname>Fornaciari</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Hovy</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Paun</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Plank</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Poesio</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Uma</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proceedings of the 1st workshop on benchmarking: past, present and future, Association for Computational Linguistics</title>
				<editor>
			<persName><forename type="first">V</forename><surname>Basile</surname></persName>
		</editor>
		<meeting>the 1st workshop on benchmarking: past, present and future, Association for Computational Linguistics<address><addrLine>Bolzano, Italy valerio</addrLine></address></meeting>
		<imprint>
			<date type="published" when="2021">2021. NL4AI 2024. November 26-27th, 2024</date>
			<biblScope unit="page" from="15" to="21" />
		</imprint>
	</monogr>
	<note>Eighth Workshop on Natural Language for Artificial Intelligence. basile@unito.it</note>
</biblStruct>

				</listBibl>
			</div>
		</back>
	</text>
</TEI>
