=Paper=
{{Paper
|id=Vol-3671/paper2
|storemode=property
|title=Evaluating the Ability of Computationally Extracted Narrative Maps to Encode Media Framing
|pdfUrl=https://ceur-ws.org/Vol-3671/paper2.pdf
|volume=Vol-3671
|authors=Sebastián Concha Macías,Brian Keith Norambuena
|dblpUrl=https://dblp.org/rec/conf/ecir/MaciasN24
}}
==Evaluating the Ability of Computationally Extracted Narrative Maps to Encode Media Framing==
https://ceur-ws.org/Vol-3671/paper2.pdf
&WBMVBUJOH UIF "CJMJUZ PG $PNQVUBUJPOBMMZ &YUSBDUFE
/BSSBUJWF .BQT UP &ODPEF .FEJB 'SBNJOH
4FCBTUJBO $PODIB .BDBT #SJBO ,FJUI /PSBNCVFOB ࠭
6OJWFSTJEBE $BU³MJDB EFM /PSUF "W "OHBNPT "OUPGBHBTUB $IJMF
"CTUSBDU
/BSSBUJWFT TFSWF BT GVOEBNFOUBM GSBNFXPSLT JO PVS VOEFSTUBOEJOH PG UIF XPSME BOE QMBZ B DSVDJBM
SPMF JO DPMMBCPSBUJWF TFOTFNBLJOH QSPWJEJOH B WFSTBUJMF GPVOEBUJPO GPS TFOTFNBLJOH 'SBNJOH JT B
TVCUMF ZFU QPUFOU NFDIBOJTN UIBU JOऺVFODFT QVCMJD QFSDFQUJPO UISPVHI TQFDJहD XPSE DIPJDFT TIBQJOH
JOUFSQSFUBUJPOT PG SFQPSUFE OFXT FWFOUT %FTQJUF UIF SFDPHOJ[FE JNQPSUBODF PG OBSSBUJWFT BOE GSBNJOH B
TJHOJहDBOU HBQ FYJTUT JO UIF MJUFSBUVSF XJUI SFHBSE UP UIF FYQMJDJU DPOTJEFSBUJPO PG GSBNJOH XJUIJO UIF
DPOUFYU PG DPNQVUBUJPOBM FYUSBDUJPO BOE SFQSFTFOUBUJPO 5IJT BSUJDMF FYQMPSFT UIF DBQBCJMJUJFT PG B TQFDJहD
OBSSBUJWF FYUSBDUJPO BOE SFQSFTFOUBUJPO BQQSPBDIۚOBSSBUJWF NBQTۚUP DBQUVSF GSBNJOH JOGPSNBUJPO GSPN
OFXT EBUB 5IF SFTFBSDI BEESFTTFT UXP LFZ RVFTUJPOT %PFT UIF OBSSBUJWF FYUSBDUJPO NFUIPE DBQUVSF
UIF GSBNJOH EJTUSJCVUJPO PG UIF EBUB TFU %PFT JU QSPEVDF B SFQSFTFOUBUJPO XJUI DPOTJTUFOU GSBNJOH
0VS SFTVMUT JOEJDBUF UIBU XIJMF UIF BMHPSJUIN DBQUVSFT GSBNJOH EJTUSJCVUJPOT BDIJFWJOH DPOTJTUFOU GSBNJOH
BDSPTT WBSJPVT TUBSUJOH BOE FOEJOH FWFOUT QPTFT DIBMMFOHFT 0VS SFTVMUT IJHIMJHIU UIF QPUFOUJBM PG OBSSBUJWF
NBQT UP QSPWJEF VTFST XJUI JOTJHIUT JOUP UIF JOUSJDBUF GSBNJOH EZOBNJDT XJUIJO OFXT OBSSBUJWFT )PXFWFS
XF OPUF UIBU EJSFDUMZ MFWFSBHJOH GSBNJOH JOGPSNBUJPO JO UIF DPNQVUBUJPOBM OBSSBUJWF FYUSBDUJPO QSPDFTT
SFNBJOT BO PQFO DIBMMFOHF
,FZXPSET
/BSSBUJWF &YUSBDUJPO .FEJB 'SBNJOH $PNQVUBUJPOBM /BSSBUJWFT /BSSBUJWF .BQT
*OUSPEVDUJPO
/BSSBUJWFT QSPWJEF B GVOEBNFOUBM GSBNFXPSL UP PVS VOEFSTUBOEJOH PG UIF XPSME <> BOE BSF
JOUFHSBM UP IVNBO SFMBUJPOT <> *O QBSUJDVMBS OBSSBUJWFT TFSWF BT DPHOJUJWF UPPMT FOBCMJOH
JOEJWJEVBMT UP GPSHF DPOOFDUJPOT CFUXFFO TFFNJOHMZ EJTQBSBUF FWFOUT QMBZJOH B DSVDJBM SPMF
JO DPMMBCPSBUJWF TFOTFNBLJOH XJUIJO TPDJFUZ <> /BSSBUJWFT BMTP QSPWJEF UIF GPVOEBUJPO GPS
WFSTBUJMF TFOTFNBLJOH UPPMT JO WBSJPVT EPNBJOT JODMVEJOH OFXT BOE JOUFMMJHFODF BOBMZTJT <> *O
UIJT DPOUFYU DPOTUSVDUJOH NFBOJOHGVM SFQSFTFOUBUJPOT GSPN TPVSDFT TVDI BT OFXT EBUB PS JOUFMMJ
HFODF SFQPSUT GBDJMJUBUFT B EFFQFS VOEFSTUBOEJOH PG UIF JOUSJDBUF TUPSZMJOFT BOE UIF PWFSBSDIJOH
OBSSBUJWF DPOUBJOFE JO EBUB <>
*O DPOKVODUJPO XJUI OBSSBUJWFT GSBNJOH FNFSHFT BT B QPUFOU NFDIBOJTN GPS TIBQJOH QVCMJD
PQJOJPO BOE QFSDFQUJPO *O HFOFSBM GSBNJOH SFGFST UP UIF QSPDFTT CZ XIJDI JOGPSNBUJPO JT
TFMFDUJWFMZ QSFTFOUFE UP JOऺVFODF UIF QFSDFQUJPO BOE PQJOJPO PG UIF BVEJFODF *U JOWPMWFT
*O 3 $BNQPT " +PSHF " +BUPXU 4 #IBUJB . -JUWBL FET 1SPDFFEJOHT PG UIF 5FYU4UPSZ ڼ8PSLTIPQ (MBTHPX
4DPUMBOE .BSDI
࠭
$PSSFTQPOEJOH BVUIPS
Envelope-Open TFCBTUJBODPODIB!BMVNOPTVDODM 4 $PODIB .BDBT CSJBOLFJUI!VDODM # ,FJUI /PSBNCVFOB
GLOBE IUUQTCSJBOLFJUIOHJUIVCJP # ,FJUI /PSBNCVFOB
Orcid # ,FJUI /PSBNCVFOB
i $PQZSJHIU GPS UIJT QBQFS CZ JUT BVUIPST 6TF QFSNJUUFE VOEFS $SFBUJWF $PNNPOT -JDFOTF "UUSJCVUJPO *OUFSOBUJPOBM $$ #:
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
17
GPDVTJOH PO DFSUBJO BTQFDUT PG B UPQJD UISPVHI UIF VTF PG QBSUJDVMBS XPSET JNBHFT PS BQQSPBDIFT
UP IJHIMJHIU DFSUBJO BTQFDUT BOE EPXOQMBZ PUIFST <>
5ISPVHIPVU UIJT XPSL XF GPDVT PO NFEJB GSBNJOH JO UFYUVBM OFXT EBUB <> BT PQQPTFE UP
PUIFS UZQFT PG GSBNJOH VTFE JO UIF MJUFSBUVSF BOE OPOXSJUUFO NFEJB 0QFSBUJOH TVCUMZ UISPVHI
TQFDJहD XPSE DIPJDFT NFEJB GSBNJOH JOऺVFODFT SFBEFST CZ IJHIMJHIUJOH QBSUJDVMBS GBDFUT PG BO
FWFOU UIFSFCZ NPMEJOH UIF BVEJFODF۟T JOUFSQSFUBUJPO PG SFQPSUFE OFXT <> 'PS FYBNQMF UIF
DIPJDF PG UFSNT TVDI BT ۢJMMFHBM BMJFOۣ WFSTVT ۢVOEPDVNFOUFE JNNJHSBOUۣ TIPXT UIJT JOऺVFODF
JO EJTDVTTJPOT BSPVOE JNNJHSBUJPO 7BSJPVT BQQSPBDIFT UP TDSVUJOJ[JOH NFEJB GSBNJOH IBWF
CFFO EFWFMPQFE CZ DPNNVOJDBUJPO TDIPMBST XJUI B DPNNPO RVBOUJUBUJWF NFUIPE JOWPMWJOH
UIF QSFWJPVT EFहOJUJPO PG GSBNFT BOE TVCTFRVFOU NBOVBM DMBTTJहDBUJPO PG OFXT BSUJDMFT JOUP
JEFOUJहFE GSBNFT UISPVHI NBDIJOF MFBSOJOH UFDIOJRVFT <>
" OPUBCMF HBQ JO UIF MJUFSBUVSF JOWPMWFT UIF BCTFODF PG TUVEJFT FYQMJDJUMZ BEESFTTJOH GSBNJOH
XJUIJO UIF EPNBJO PG DPNQVUBUJPOBM FYUSBDUJPO BOE SFQSFTFOUBUJPO PG OBSSBUJWFT GSPN OFXT EBUB
<> 8IJMF OVNFSPVT SFTFBSDI XPSLT FYQMPSF OBSSBUJWF DPOTUSVDUJPO BOE SFQSFTFOUBUJPO JO UIF
OFXT EPNBJO B DSJUJDBM BTQFDU P॑FO PWFSMPPLFE JT UIF FYQMJDJU DPOTJEFSBUJPO PG NFEJB GSBNJOH
FMFNFOUT 5IF JOUFSQMBZ CFUXFFO DPNQVUBUJPOBM NFUIPET BOE UIF OVBODFE JOऺVFODF PG GSBNJOH
PO JOGPSNBUJPO JOUFSQSFUBUJPO SFNBJOT MBSHFMZ VOFYQMPSFE
*O UIJT DPOUFYU UIF QVSQPTF PG UIJT XPSL JT UP FWBMVBUF XIFUIFS B TQFDJहD DPNQVUBUJPOBM
OBSSBUJWF SFQSFTFOUBUJPOۚOBSSBUJWF NBQT <>ۚBOE UIF BTTPDJBUFE FYUSBDUJPO NFUIPE DBO QSPQFSMZ
DBQUVSF GSBNJOH JOGPSNBUJPO GSPN OFXT EBUB 5IVT PVS SFTFBSDI BTLT UIF GPMMPXJOH SFTFBSDI
RVFTUJPOT
ۨ RQ1: %PFT UIF OBSSBUJWF NBQT FYUSBDUJPO BMHPSJUIN QSPQFSMZ DBQUVSF UIF GSBNJOH EJTUSJ
CVUJPO PG UIF EBUB TFU
ۨ RQ2: %PFT UIF OBSSBUJWF NBQT FYUSBDUJPO BMHPSJUIN QSPEVDF B SFQSFTFOUBUJPO XJUI
DPOTJTUFOU GSBNJOH
8F BOTXFS UIFTF RVFTUJPOT CZ UFTUJOH UIF FYUSBDUJPO BMHPSJUIN PO B OFXT EBUB TFU XJUI GSBNJOH
MBCFMT *O QBSUJDVMBS XF FYQMPSF IPX UIF FYUSBDUJPO BMHPSJUIN CFIBWFT VOEFS EJसFSFOU DPN
CJOBUJPOT PG GSBNJOH GPS UIF TUBSUJOH BOE FOEJOH FWFOUT XIJDI BDU BT TFFET GPS UIF FYUSBDUJPO
NFUIPE 0VS SFTVMUT TVHHFTU UIBU UIF FYUSBDUJPO NFUIPE JT DBQBCMF PG DBQUVSJOH UIF TBNF GSBN
JOH EJTUSJCVUJPO PG UIF EBUB TFU CVU UIBU JU EPFT OPU OFDFTTBSJMZ QSPEVDF B SFQSFTFOUBUJPO XJUI
DPOTJTUFOU GSBNJOH XIFO DPOTJEFSJOH UIF TUBSUJOH BOE FOEJOH FWFOUT
#BDLHSPVOE BOE 3FMBUFE 8PSL
/BSSBUJWF .BQT BOE &YUSBDUJPO
/BSSBUJWF NBQT BSF B WJTVBM SFQSFTFOUBUJPO PG TUPSZMJOFT BOE FWFOUT CBTFE PO B DBSUPHSBQIJD
NFUBQIPS UP NPEFM UIF JOGPSNBUJPO MBOETDBQF < > 5IFTF NBQT BSF EFTJHOFE UP JMMVTUSBUF UIF
SFMBUJPOTIJQ CFUXFFO EJसFSFOU FWFOUT DIBSBDUFST UIFNFT BOE LFZ QPJOUT UIBU NBLF VQ B TUPSZ
PS B TFSJFT PG FWFOUT JO B HSBQIJDBM GPSNBU $PNQVUBUJPOBMMZ OBSSBUJWF NBQT BSF SFQSFTFOUFE BT
EJSFDUFE BDZDMJD HSBQIT XIFSF FBDI OPEF SFQSFTFOUT BO FWFOU
*O UIF DPOUFYU PG OBSSBUJWF BOBMZTJT BQQMJFE UP DPNQVUBUJPOBM KPVSOBMJTN PS PUIFS हFMET
OBSSBUJWF NBQT TFSWF BT UPPMT UP CSFBL EPXO UIF OBSSBUJWF TUSVDUVSF PG OFXT PS FWFOUT <>
18
5IFZ VTF HSBQIJDBM FMFNFOUT TVDI BT OPEFT BOE DPOOFDUJPOT UP TIPX IPX UIF TUPSZ VOGPMET
IPX FWFOUT BSF JOUFSDPOOFDUFE BOE IPX UIFZ DPOUSJCVUF UP UIF PWFSBMM VOEFSTUBOEJOH PG UIF
OBSSBUJWF
5IFTF NBQT BSF SFQSFTFOUFE BT EJSFDUFE BDZDMJD HSBQIT <> XIFSF OPEFT SFQSFTFOU LFZ FWFOUT
PS NPNFOUT BOE UIF DPOOFDUJPOT CFUXFFO UIFN EFQJDU UIF TFRVFODF PS SFMBUJPOTIJQ CFUXFFO
UIPTF FWFOUT "EEJUJPOBMMZ UIFZ NBZ JODPSQPSBUF JOUFSBDUJWF GFBUVSFT PS MBZFST PG BSUJहDJBM
JOUFMMJHFODF UP BMMPX UIF NPEFM UP BEBQU UP VTFS OFFET BOE QSPWJEF FYQMBOBUJPOT BCPVU UIF
OBSSBUJWF TUSVDUVSF <>
$PNQVUBUJPOBM 'SBNF .PEFMJOH
'SBNF FYUSBDUJPO NFUIPET UFOE UP GPDVT PO TFNBOUJD GSBNFT XIFSF FYUFOTJWF SFTFBSDI IBT MFE UP
UIF EFWFMPQNFOU PG NFUIPET TVDI BT 'SBNF/FU <> BOE FYUSBDUJPO UPPMT MJLF 0QFO4&4".& <>
XIJDI VTF EFFQ MFBSOJOH NPEFMT BMPOHTJEF TZOUBDUJD JOGPSNBUJPO UP FYUSBDU TFNBOUJD GSBNFT GSPN
UIF UFYU )PXFWFS UIFTF BQQSPBDIFT BSF UPP HSBOVMBS JO TDPQF BT TFNBOUJD GSBNFT GPDVT NPSF
PO MJOHVJTUJD BTQFDUT SBUIFS UIBO BQQMJDBCMF DPODFQUT JO UIF OFXT EPNBJO 5IVT SFTFBSDIFST
IBWF QSPQPTFE UIF DPODFQU PG NFEJB GSBNFT <>
3FTFBSDIFST IBWF VTFE WBSJPVT LJOET PG NPEFMT UP BOBMZ[F NFEJB GSBNFT < > 4PNF
BVUIPST GPMMPX BO VOTVQFSWJTFE BQQSPBDI XIFSF GSBNFT BSF JEFOUJहFE GSPN BO VOMBCFMFE TFU PG
EBUB 0UIFST IBWF DBSFGVMMZ DPOTUSVDUFE MBCFMFE EBUB TFUT UP USBJO NBDIJOF MFBSOJOH DMBTTJहFST
"OPUIFS BTQFDU PG UIJT UBTL UIBU NVTU CF DPOTJEFSFE JT UIF MFWFM PG HSBOVMBSJUZ PG UIF GSBNJOH
BOBMZTJT 4PNF XPSLT GPDVT PO BSUJDMFXJEF IJHIMFWFM GSBNFT < > XIJMF PUIFST FYQMPSF
TFOUFODFMFWFM GSBNJOH PS FWFO CPUI <>
)PXFWFS BT NFOUJPOFE CFGPSF EFTQJUF UIF TJHOJहDBOU QSPHSFTT NBEF JO EFWFMPQJOH NFUIPET
GPS OBSSBUJWF DPOTUSVDUJPO BOE SFQSFTFOUBUJPO JO UIF OFXT EPNBJO UIF FYQMJDJU DPOTJEFSBUJPO PG
NFEJB GSBNJOH FMFNFOUT JO UIF DPOUFYU PG DPNQVUBUJPOBM OBSSBUJWF FYUSBDUJPO BOE SFQSFTFOUBUJPO
GSPN OFXT EBUB SFNBJOT MBSHFMZ VOFYQMPSFE <>
'PS UIF QVSQPTFT PG UIJT BSUJDMF XF VTF B QSFMBCFMFE EBUB TFU <> UP DIFDL XIFUIFS UIF OBSSBUJWF
FYUSBDUJPO BMHPSJUIN JT DBQBCMF PG DBQUVSJOH GSBNJOH JOGPSNBUJPO 8F EP OPU BUUFNQU UP FYUSBDU
GSBNFT BVUPNBUJDBMMZ BU UIJT TUBHF BT UIF NBJO HPBM JT UP हSTU JEFOUJGZ UIF DBQBCJMJUJFT PG UIF
CBTFMJOF NPEFM
.BUFSJBMT BOE .FUIPET
%BUB 4FU %FTDSJQUJPO
8F VTF UIF (VO 7JPMFODF 'SBNF $PSQVT (7'$ GSPN -JV FU BM <> 5IF EBUB TFU DPOTJTUT PG
OFXT BSUJDMFT JO &OHMJTI GSPN NVMUJQMF 64 CBTFE TPVSDFT FYUSBDUFE EVSJOH UIF ZFBS 5IF
(7'$ GPDVTFT PO NFEJB GSBNFT DPNNPOMZ VTFE XIFO SFQPSUJOH UIF JTTVF PG (VO 7JPMFODF *O
QBSUJDVMBS JU IBT UZQFT PG GSBNFT JODMVEJOH CPUI JTTVFTQFDJहD BOE HFOFSJD GSBNFT <> 5BCMF
TIPXT UIF EJTUSJCVUJPO PG GSBNF MBCFMT JO UIF EBUB
8F OPUF UIBU UIF OBSSBUJWF FYUSBDUJPO NFUIPE IBT B IJHI DPNQVUBUJPOBM DPTU BT UIF EBUB TFU TJ[F
JODSFBTFT 5IVT XF EFDJEFE UP EFDSFBTF UIF EBUB TFU TJ[F GSPN BSUJDMFT XIJMF NBJOUBJOJOH
19
5BCMF
Distribution of each frame label (main media frame) in the Gun Violence Frame Corpus [9].
*% 'SBNF -BCFM <3FGFSFODF> 'SFR &YBNQMF )FBEMJOF
1: 2nd Amendment [18] 38 ”Breaking down the 27 words of the Second Amendment“
2: Gun Control/Regulation [18] 215 ”How gun background checks work“
3: Politics [19] 373 ”Romney open to new gun measures“
4: Mental Health [20] 65 ”Florida shooter a troubled loner with white supremacist ties“
5: School/Public Space Safety [21] 137 7 ways to help prevent school shootings
6: Race/Ethnicity [22] 114 ”Lawyers call US gun charges for Mexican man ’vindictive’“
7: Public Opinion [23] 237 “Students call for action after Florida school mass shooting”
8: Society/Culture [24] 41 “White House weighs video game link to gun violence”
9: Economic Consequences [23] 80 “Calstrs to engage with assault-weapon sellers first, divest last”
UIF PSJHJOBM EJTUSJCVUJPO PG UIF GSBNF MBCFMT 4JODF UIJT EBUB TFU JT VOCBMBODFE XJUI SFTQFDU UP UIF
GSBNF MBCFMT XF QFSGPSNFE TUSBUJहFE TBNQMJOH <> BOE PCUBJOFE B OFX EBUB TFU PG BSUJDMFT
'VSUIFSNPSF XF BSF GBDFE XJUI UIF BCTFODF PG FTTFOUJBM WBSJBCMFT GPS UIF OBSSBUJWF FYUSBDUJPO
NFUIPE 4QFDJहDBMMZ UIF BMHPSJUIN SFRVJSFT UFNQPSBM JOGPSNBUJPO CVU UIF PSJHJOBM EBUB TFU
EPFT OPU DPOUBJO BOZ UJNFTUBNQT UP HVJEF UIF FYUSBDUJPO QSPDFTT 'VSUIFSNPSF UIF EBUB TFU
EJE OPU QSPWJEF UIF TPVSDF PG UIF OFXT XIJDI JT BMTP SFRVJSFE CZ UIF FYUSBDUJPO BMHPSJUIN UP
HFOFSBUF UIF हOBM WJTVBMJ[BUJPO 5IVT XF IBE UP NBOVBMMZ TFBSDI GPS UIF PSJHJOBM TPVSDFT PG
FBDI BSUJDMF CBTFE PO JUT IFBEMJOF BOE BEE UIF NJTTJOH JOGPSNBUJPO UP UIF EBUB TFU
.PSFPWFS CFGPSF FYUSBDUJOH UIF OBSSBUJWF SFQSFTFOUBUJPO GSPN PVS OFX TVCTFU PG UIF PSJHJOBM
EBUB TFU XF EFDJEFE UP SFEVDF UIF DPNQMFYJUZ PG UIF GSBNJOH NPEFM UP BDDPVOU GPS PVS TNBMMFS
EBUB TFU TJ[F 5IVT XF SFEVDFE UIF OVNCFS PG GSBNFT JOUP UISFF IJHIFSMFWFM GSBNFT CZ HSPVQJOH
SFMBUFE GSBNFT JOUP B TJOHMF BCTUSBDU DBUFHPSZ 5IJT TJNQMJहDBUJPO JT OFDFTTBSZ UP FOTVSF UIBU
UIF GSBNJOH NPEFM SFNBJOT FसFDUJWF BOE BDDVSBUF EFTQJUF UIF SFEVDFE EBUB TJ[F #Z HSPVQJOH
SFMBUFE GSBNFT JOUP CSPBEFS NPSF BCTUSBDU DBUFHPSJFT XF NBJOUBJO UIF FTTFODF PG UIF PSJHJOBM
GSBNJOH TUSVDUVSF XIJMF NBLJOH JU NPSF NBOBHFBCMF BOE TVJUBCMF GPS UIF MJNJUFE EBUB TFU
*O QBSUJDVMBS GPS UIF हSTU TFDPOE BOE UIJSE GSBNFT PG UIF EBUB TFU XF OPUF UIBU UIFJS NBJO
GPDVT JT PO QPMJUJDBM JTTVFT NPTUMZ SFMBUFE UP HVO DPOUSPM 5IFSFGPSF XF VOJUFE UIFN VOEFS UIF
OBNF ۢ'SBNF 1PMJUJDBM *TTVFTۣ -JLFXJTF UIF GPVSUI BOE ह॑I GSBNFT GPDVTFE PO BTQFDUT SFMBUFE
UP NFOUBM IFBMUIDBSF JTTVFT BT XFMM BT TDIPPM BOE QVCMJD TBGFUZ #PUI PG UIFTF FMFNFOUT DPVME CF
BEESFTTFE CZ QVCMJD TFSWJDFT )FODF XF VOJUFE UIFN VOEFS UIF OBNF ۢ'SBNF 1VCMJD 4FSWJDFTۣ
'JOBMMZ UIF MBTU GPVS GSBNFT XFSF PSJFOUFE UPXBSET FJUIFS DVMUVSBM JTTVFT PS TPDJFUBM JTTVFT
JODMVEJOH EJTDVTTJPOT BSPVOE SBDF BOE FUIOJDJUZ QVCMJD PQJOJPO BOE FDPOPNJD DPOTFRVFODFT
5IVT XF VOJUFE UIFN VOEFS UIF OBNF ۢ'SBNF $VMUVSBM BOE 4PDJFUBM *TTVFTۣ 'JOBMMZ UIJT MFE
VT UP TJNQMJGZ UIF TFU PG OJOF QPTTJCMF GSBNFT UP KVTU UISFF GSBNFT
/BSSBUJWF &YUSBDUJPO .FUIPE
*O UIJT TUVEZ XF FNQMPZ B OBSSBUJWF FYUSBDUJPO NFUIPE EFWFMPQFE CZ ,FJUI BOE .JUSB <>
UP HFOFSBUF OBSSBUJWF NBQT GSPN UFYUVBM OFXT BSUJDMFT " OBSSBUJWF NBQ JT B DPNQVUBUJPOBM
5IF EBUB TFU JT BWBJMBCMF IFSF IUUQTEPJPSH[FOPEP
20
'JHVSF Technical methodological diagram illustrating the key steps in the narrative extraction and
analysis process, starting with the preprocessed final data set and ending with the quantitative and
qualitative evaluation of the resulting narrative maps.
SFQSFTFOUBUJPO PG B OBSSBUJWF JO UIF GPSN PG B EJSFDUFE BDZDMJD HSBQI XIFSF OPEFT SFQSFTFOU
FWFOUT BOE FEHFT SFQSFTFOU UIF DISPOPMPHJDBM PSEFS BOE DBVTBM SFMBUJPOTIJQT CFUXFFO FWFOUT
'JHVSF TIPXT B SFQSFTFOUBUJPO PG UIF HFOFSBM FYUSBDUJPO BOE TVCTFRVFOU BOBMZTJT QSPDFTT
'SPN B DPNQVUBUJPOBM TUBOEQPJOU UIF FYUSBDUJPO NFUIPE UBLFT B MJTU PG OFXT BSUJDMFT XIFSF
FBDI OFXT BSUJDMF JT SFQSFTFOUFE BT B QBJS PG UFYU FNCFEEJOHT BOE UIF QVCMJDBUJPO EBUF BT B
UJNFTUBNQ 'PMMPXJOH BO PQUJNJ[BUJPO QSPDFEVSF UIF NFUIPE HFOFSBUFT B OBSSBUJWF NBQ UIBU
DBQUVSFT UIF LFZ FWFOUT BOE UIFJS DPOOFDUJPOT XJUIJO UIF OBSSBUJWF 'PS UIF QVSQPTFT PG PVS
FYQFSJNFOU XF POMZ VTF UIF IFBEMJOF UP SFQSFTFOU UIF OFXT BSUJDMF BT JU DPOUBJOT UIF NPTU
SFMFWBOU JOGPSNBUJPO GPS GSBNJOH QVSQPTFT VOEFS UIF BTTVNQUJPO UIBU CSFBLJOH OFXT BSUJDMFT
UFOE UP GPMMPX UIF JOWFSUFE QZSBNJE TUSVDUVSF <> 'PS UIF FNCFEEJOHT XF VTF UIF 6OJWFSTBM
4FOUFODF &ODPEFS W GSPN 5FOTPS'MPX <> XIJDI JT BQQSPQSJBUF GPS TIPSU UFYUT TVDI BT OFXT
IFBEMJOFT CBTFE PO UIF TBNF JNQMFNFOUBUJPO PG UIF PSJHJOBM FYUSBDUJPO NFUIPE <>
3FHBSEJOH UIF FYUSBDUJPO QSPDFTT JUTFMG UIJT NFUIPE MFWFSBHFT BO PQUJNJ[BUJPO BQQSPBDI
CBTFE JO NBYJNJ[JOH DPIFSFODFۚBTTFTTJOH UIF MPHJDBM ऺPX BOE FOTVSJOH UIBU UIF TUPSZMJOFT
NBLF TFOTF *O QBSUJDVMBS XF JNQPTF TUSVDUVSBM BOE UPQJD DPWFSBHF DPOTUSBJOUT UISPVHI MJOFBS
QSPHSBNNJOH BEPQUJOH UIF NFUIPEPMPHZ PVUMJOFE CZ ,FJUI BOE .JUSB <> 5IF GPSNVMBUJPO PG
UIF MJOFBS QSPHSBN HPWFSOJOH UIF FYUSBDUJPO NFUIPE JT QSFTFOUFE JO 'JHVSF
0VS JOUFSQSFUBUJPO PG DPIFSFODF JT CBTFE PO UIF DPODFQU PG TJNJMBSJUZ HSPVOEFE JO UIF SBUJPOBMF
UIBU DPOOFDUFE FWFOUT TIPVME FYIJCJU NJOJNBM ऺVDUVBUJPOT JO UPQJDT PS DPOUFOU UISPVHIPVU UIF
OBSSBUJWF .PSF QSFDJTFMZ UIF DPIFSFODF WBMVF JOEJDBUJOH UIF MPHJDBM DPOUJOVJUZ PG KPJOJOH UXP
FWFOUT JT DPNQVUFE CZ FWBMVBUJOH CPUI UFYU TJNJMBSJUZۚCBTFE PO BO FNCFEEJOH SFQSFTFOUBUJPOۚ
21
Coherence
Chain
Constraints
Clustering
'JHVSF Linear programming formulation of the extraction method of Keith and Mitra [3].
BOE UPQJDBM TJNJMBSJUZۚEFSJWFE GSPN UPQJDBM DMVTUFST DPNQVUFE JO B % QSPKFDUJPO TQBDF HFOFSBUFE
CZ UIF 6."1 BMHPSJUIN <> XJUI UIF )%#4$"/ NFUIPE <>
5IF TUSVDUVSBM DPOTUSBJOUT TIPXO JO 'JHVSF QMBZ B DSVDJBM SPMF JO FOTVSJOH UIF HFOFSBUJPO
PG B EJSFDUFE BDZDMJD HSBQI GFBUVSJOH B TJOHVMBS TPVSDF BOE B TJOHMF TJOL JOUFSDPOOFDUFE JO
DISPOPMPHJDBM PSEFS UISPVHI NVMUJQMF TUPSZMJOFT 'VSUIFSNPSF UIF FYUSBDUJPO BMHPSJUIN SFMJFT
PO B VTFSEFहOFE QBSBNFUFS GPS UIF FYQFDUFE MFOHUI PG UIF NBJO TUPSZMJOF 𝐾 5IJT QBSBNFUFS
BMTP EJSFDUMZ JOऺVFODFT UIF PWFSBMM TJ[F PG UIF NBQ 5ISPVHIPVU PVS FYQFSJNFOUT XF VTFE B
QBSBNFUFS 𝐾 = 6 BT JU QSPWJEFT NBQT PG BQQSPQSJBUF DPNQMFYJUZ GPS PVS EBUB TFU TJ[F 5IJT WBMVF
JT CBTFE PO FNQJSJDBM PCTFSWBUJPOT GSPN QSFWJPVT TUVEJFT BOE PVS PXO QSFMJNJOBSZ FYQFSJNFOUT
TUSJLJOH B CBMBODF CFUXFFO DBQUVSJOH LFZ FWFOUT BOE NBJOUBJOJOH B NBOBHFBCMF OBSSBUJWF TJ[F
5IF UPQJD DPWFSBHF DPOTUSBJOUT FOTVSF UIBU UIF FYUSBDUFE OBSSBUJWF FODPNQBTTFT B NJOJNVN
QFSDFOUBHF PG UIF UPQJDT QSFTFOU JO UIF EBUB EFUFSNJOFE CZ B QSFEFहOFE DPWFSBHF UISFTIPME
5IF UPQJD DPWFSBHF DPOTUSBJOUT BSF CBTFE PO UPQJD TJNJMBSJUZ JOGPSNBUJPO <> CBTFE PO UIF TBNF
JOGPSNBUJPO VTFE UP DPNQVUF DPIFSFODF 5ISPVHIPVU UIFTF FYQFSJNFOUT XF VTFE B DPWFSBHF
UISFTIPME PG XIJDI BMMPXT GPS B TVऻDJFOU MFWFM PG BCTUSBDUJPO XIJMF QSFTFSWJOH UIF FTTFOUJBM
UPQJDBM JOGPSNBUJPO PG UIF EBUB TFU
&WBMVBUJPO .FUSJDT
5P RVBOUJUBUJWFMZ FWBMVBUF PVS SFTVMUT XF VTF UIF +FOTFO4IBOOPO EJWFSHFODF UP NFBTVSF IPX
XFMM UIF GSBNJOH EJTUSJCVUJPO PG UIF FYUSBDUFE OBSSBUJWF NBQ DBQUVSFT UIF GSBNJOH EJTUSJCVUJPO
PG UIF EBUB TFU *O HFOFSBM UIF +FOTFOۙ4IBOOPO EJWFSHFODF BMMPXT VT UP NFBTVSF UIF TJNJMBSJUZ
CFUXFFO UXP QSPCBCJMJUZ EJTUSJCVUJPOT *O UIJT DBTF XF DPOTJEFS UIF OPSNBMJ[FE GSFRVFODJFT PG
UIF EJसFSFOU GSBNFT JO UIF EBUB TFU BOE UIF NBQT BT QSPCBCJMJUZ EJTUSJCVUJPOT UP QFSGPSN UIF
DPNQBSJTPOT 8F OPUF UIBU UIJT NFBTVSF PG TJNJMBSJUZ UBLFT WBMVFT CFUXFFO BOE XIFSF
SFQSFTFOUT TJNJMBS EJTUSJCVUJPOT BOE SFQSFTFOUT DPNQMFUFMZ EJTTJNJMBS EJTUSJCVUJPOT
22
3FTVMUT BOE %JTDVTTJPO
3FTVMUJOH .BQT
8F HFOFSBUFE NBQT BDDPVOUJOH GPS BMM UIF QPTTJCMF DPNCJOBUJPOT PG TUBSUJOH BOE FOEJOH GSBNFT
5BCMF TIPXT UIF TJ[FT PG UIF NBQT CBTFE PO UIF OVNCFS PG FWFOUT DPOUBJOFE JO UIFN 5IF
NBQT IBWF B NFBO TJ[F PG OPEFT BOE NFEJBO TJ[F PG OPEFT 5IF TUBOEBSE EFWJBUJPO JT
OPEFT "MM NBQT IBWF HFOFSBMMZ TJNJMBS TJ[FT JO UFSNT PG DPWFSFE FWFOUT CVU UIFZ NBZ
DPWFS EJसFSFOU FWFOUT CBTFE PO UIF DIPTFO TUBSUJOH BOE FOEJOH FWFOUT 8F OPUF UIBU EFTQJUF
EJसFSFODFT JO TJ[F BOE TQFDJहD DPOUFOUT UIF GSBNJOH EJTUSJCVUJPO PG BMM UIFTF NBQT BSF RVJUF
TJNJMBS BT TIPXO JO UIF OFYU TFDUJPO
5BCMF
Map sizes of each combination starting and ending frames based on number of events in the map.
.BQ 4J[F End: Frame 1 End: Frame 2 End: Frame 3
Start: Frame 1 24 21 19
Start: Frame 2 16 17 22
Start: Frame 3 22 22 15
ࣅVBOUJUBUJWF 3FTVMUT
8F TIPX UIF RVBOUJUBUJWF SFTVMUT PG PVS FWBMVBUJPO JO 5BCMF 0O BWFSBHF UIF OBSSBUJWF
FYUSBDUJPO BMHPSJUIN DBQUVSFT B WFSZ TJNJMBS GSBNJOH EJTUSJCVUJPO UP UIF GVMM EBUB TFU 5IF
DPOTJTUFOUMZ MPX BWFSBHF +FOTFO4IBOOPO EJWFSHFODF PG BDSPTT BMM UFTUFE DPNCJOBUJPOT
TJHOBMT B OPUBCMF BMJHONFOU CFUXFFO UIF GSBNJOH EJTUSJCVUJPOT PG UIF HFOFSBUFE OBSSBUJWF NBQT
BOE UIF EBUB TFU " +FOTFO4IBOOPO EJWFSHFODF PG XPVME JOEJDBUF JEFOUJDBM EJTUSJCVUJPOT TP
WBMVFT DMPTFS UP TVHHFTU IJHI TJNJMBSJUZ 8JUI BMM EJWFSHFODF TDPSFT CFMPX UIF SFTVMUT
TVQQPSU UIF DPODMVTJPO UIBU UIF OBSSBUJWF NBQT DMPTFMZ DBQUVSF UIF PWFSBMM GSBNJOH EJTUSJCVUJPO
PG UIF PSJHJOBM EBUB
5BCMF
Jensen-Shannon divergence values obtained by comparing the framing distribution of each combination
starting and ending frames with the distribution of the data set.
+4 %JWFSHFODF End: Frame 1 End: Frame 2 End: Frame 3 Average
Start: Frame 1 0.0764 0.0009 0.0050 0.0274
Start: Frame 2 0.0270 0.0396 0.0172 0.0280
Start: Frame 3 0.0069 0.0171 0.0063 0.0101
Average 0.0368 0.0192 0.0095 0.0218
5IF NPTU EJTTJNJMBS EJTUSJCVUJPOT PDDVS XIFO XF TUBSU BOE FOE XJUI 'SBNF .BQ 5IJT
EJसFSFODF DPVME CF FYQMBJOFE CZ UIF GBDU UIBU XF BSF TFMFDUJOH CPUI UIF TUBSUJOH BOE FOEJOH
FWFOU XJUI UIF NPTU GSFRVFOU GSBNF JO UIF EBUB TFU XIJDI DPVME MFBE UP B CJBTFE OBSSBUJWF NBQ
UIBU NPTUMZ GPDVTFT PO UIBU GSBNF HVJEFE CPUI CZ UIF IJHIFS GSFRVFODZ PO UIF XIPMF EBUB BOE
UIF VTFS TFMFDUFE FWFOUT /PUF UIBU BSPVOE PG UIF FWFOUT BSF BTTJHOFE UP 'SBNF JO UIF
EBUB TFU XIJMF PG UIF FWFOUT PG UIF BGPSFNFOUJPOFE NBQ BSF BTTJHOFE UP UIJT GSBNF
23
/FWFSUIFMFTT PVS BOBMZTJT PG SFTVMUT TVHHFTUT UIBU UIF OBSSBUJWF NBQT FYUSBDUJPO BMHPSJUIN
QSPQFSMZ DBQUVSFT UIF GSBNJOH EJTUSJCVUJPO PG UIF EBUB TFU 5IVT CBTFE PO UIFTF SFTVMUT UIF
FYUSBDUFE OBSSBUJWF NBQT DPVME QPUFOUJBMMZ QSPWJEF VTFST PG UIF SFQSFTFOUBUJPO XJUI JOTJHIUT
SFHBSEJOH UIF GSBNJOH PG UIF OFXT OBSSBUJWF CZ UIF NFEJB
ࣅVBMJUBUJWF &YBNQMF
5P FWBMVBUF XIFUIFS UIF OBSSBUJWF NBQQJOH BMHPSJUIN QSPEVDFT SFQSFTFOUBUJPOT XJUI DPOTJTUFOU
GSBNJOH XF FYBNJOFE TBNQMF OBSSBUJWF NBQT FYUSBDUFE GSPN OFXT FWFOUT TQBOOJOH TFWFSBM
NPOUIT 8F TIPX BO FYBNQMF OBSSBUJWF NBQ JO 'JHVSF UIBU DPOUBJOT FWFOUT DPWFSJOH TDIPPM
TBGFUZ QPMJUJDBM EFCBUFT BOE DVMUVSBM JNQBDUT SFMBUFE UP HVO WJPMFODF JO UIF 6OJUFE 4UBUFT
*O QBSUJDVMBS UIF NBQ IBE FOEQPJOUT SFMBUFE UP 'SBNF PG UIF EBUB TFU 5IVT JG GSBNJOH
XBT DPOTJTUFOU UIF BMHPSJUIN TIPVME TFMFDU NPTUMZ EPDVNFOUT XJUI UIBU QBSUJDVMBS GSBNJOH
)PXFWFS BOBMZTJT PG UIF OBSSBUJWF NBQ TIPXO SFWFBMFE JODPOTJTUFODJFT JO IPX FWFOUT XFSF
GSBNFE UISPVHIPVU UIF OBSSBUJWF
'PS JOTUBODF XIJMF UIF JOJUJBM FWFOUT EFBMJOH XJUI EJTDPWFSZ PG B IBOEHVO JO B TDIPPM BOE
QSFWFOUJPO PG TDIPPM TIPPUJOHT XFSF GSBNFE BSPVOE QVCMJD TFSWJDF JTTVFT MBUFS FWFOUT JO UIF NBQ
TIJ॑FE GPDVT UP QPMJUJDBM EFCBUFT BSPVOE HVO DPOUSPM BOE DVMUVSBMTPDJFUBM JNQBDUT PG TIPPUJOHT
5IF BMHPSJUIN DPOOFDUFE UIF FBSMZ TDIPPM TBGFUZ FWFOUT UP UIFTF MBUFS FWFOUT FWFO UIPVHI UIFZ
DPOUBJOFE EJसFSFOU GSBNFT 5IJT TVHHFTUT UIF OBSSBUJWF NBQQJOH QSPDFTT GPDVTFE PO DPOUFOU
TJNJMBSJUJFT CFUXFFO FWFOUT SBUIFS UIBO NBJOUBJOJOH DPOTJTUFOU GSBNJOH BDSPTT UIF GVMM NBQ
"EEJUJPOBMMZ XF PCTFSWFE BCSVQU GSBNJOH TIJ॑T CFUXFFO DPOOFDUFE FWFOUT BU UJNFT TVDI BT
CFUXFFO B TUPSZ PO UIF 1BSLMBOE TIPPUFS۟T TXBTUJLBFUDIFE NBHB[JOFT BOE POF PO B 1BSLMBOE
UFBDIFS MFBWJOH B MPBEFE HVO JO B CBUISPPN 8IJMF DPOUBJOJOH TJNJMBS DPOUFOU UIFTF FWFOUT IBE
EJसFSFOU GSBNJOH DVMUVSBMTPDJFUBM JTTVFT WT QVCMJD TFSWJDFTTBGFUZ 5IFTF GSBNJOH JODPOTJTUFO
DJFT DBO VOEFSNJOF UIF DPIFSFODF PG UIF FYUSBDUFE OBSSBUJWF NBLJOH JU DIBMMFOHJOH UP EJTDFSO
B DMFBS GSBNJOH USBKFDUPSZ BDSPTT UIF NBQ "CSVQU GSBNF DIBOHFT CFUXFFO FWFOUT EJTSVQU UIF
MPHJDBM ऺPX BOE DPNQMJDBUF JOUFSQSFUBUJPO PG UIF OBSSBUJWF۟T PWFSBSDIJOH TUBODF PS QFSTQFDUJWF
5IF QSPCMFN JT OPU JTPMBUFE UP UIJT TJOHMF FYBNQMF TJNJMBS GSBNJOH EJTDSFQBODJFT BQQFBS BDSPTT
PUIFS TBNQMF NBQT JOEJDBUJOH B CSPBEFS MJNJUBUJPO PG UIF DVSSFOU BQQSPBDI
#BTFE PO BOBMZTJT PG UIJT FYBNQMF BOE PUIFS TBNQMF OBSSBUJWF NBQT XF GPVOE UIF BMHPSJUIN
EPFT OPU PQUJNJ[F GPS DPOTJTUFOU FWFOU GSBNJOH XIFO FYUSBDUJOH UIF OBSSBUJWFT 8IJMF UIF
UFDIOJRVF JT BCMF UP DBQUVSF UIF GSBNJOH EJTUSJCVUJPO PG UIF CSPBEFS EBUB TFU NBJOUBJOJOH
DPIFSFOU GSBNFT BDSPTT BO FOUJSF OBSSBUJWF SFNBJOT B DIBMMFOHF GPS UIF BQQSPBDI
*NQMJDBUJPOT
5IF हOEJOHT PG UIJT TUVEZ IBWF JNQPSUBOU JNQMJDBUJPOT GPS CPUI UIF SFTFBSDI DPNNVOJUZ BOE
QSBDUJUJPOFST TFFLJOH UP MFWFSBHF DPNQVUBUJPOBM OBSSBUJWF FYUSBDUJPO UFDIOJRVFT UP VOEFSTUBOE
NFEJB GSBNJOH 'SPN B SFTFBSDI QFSTQFDUJWF PVS XPSL IJHIMJHIUT UIF OFFE GPS HSFBUFS BUUFOUJPO
UP UIF SPMF PG GSBNJOH JO TIBQJOH UIF TUSVDUVSF BOE DPIFSFODF PG FYUSBDUFE OBSSBUJWFT
8IJMF UIF OBSSBUJWF NBQT BQQSPBDI EFNPOTUSBUFT UIF BCJMJUZ UP DBQUVSF IJHIMFWFM GSBNJOH
EJTUSJCVUJPOT UIF JODPOTJTUFODJFT SFWFBMFE JO PVS RVBMJUBUJWF BOBMZTJT TVHHFTU UIBU DVSSFOU
UFDIOJRVFT NBZ CF JOTVऻDJFOU GPS HFOFSBUJOH GVMMZ DPIFSFOU OBSSBUJWF SFQSFTFOUBUJPOT 'VUVSF
24
'JHVSF Sample narrative map extracted from news events spanning January to June 2018. Events are
represented as nodes labeled with headlines and frames. Connections indicate narrative relationships
between events. Inconsistencies in framing across the map highlight challenges in producing coherent
framing narratives through the extraction process.
25
SFTFBSDI TIPVME QSJPSJUJ[F UIF EFWFMPQNFOU PG OPWFM BQQSPBDIFT UIBU DBO NPSF FसFDUJWFMZ
NBJOUBJO DPOTJTUFOU GSBNJOH BDSPTT OBSSBUJWF DIBJOT
'PS QSBDUJUJPOFST TVDI BT KPVSOBMJTUT QPMJDZ NBLFST PS BOBMZTUT PVS हOEJOHT VOEFSTDPSF CPUI
UIF QPUFOUJBM CFOFहUT BOE QJUGBMMT PG SFMZJOH PO DPNQVUBUJPOBM OBSSBUJWF FYUSBDUJPO UP BTTFTT
NFEJB GSBNJOH 0O POF IBOE UIF UFDIOJRVF۟T BCJMJUZ UP QSFTFSWF UIF PWFSBMM GSBNJOH EJTUSJCVUJPO
PG B MBSHF DPSQVT PG OFXT BSUJDMFT QPJOUT UP JUT WBMVF BT B UPPM GPS RVJDLMZ JEFOUJGZJOH EPNJOBOU
GSBNFT BOE HBVHJOH UIFJS SFMBUJWF QSPNJOFODF 5IJT IJHIMFWFM QFSTQFDUJWF DPVME HVJEF GVSUIFS
JOWFTUJHBUJPO PS QSPWJEF B TUBSUJOH QPJOU GPS FYQMPSJOH GSBNJOH QBUUFSOT
)PXFWFS UIF GSBNJOH JODPOTJTUFODJFT XJUIJO JOEJWJEVBM OBSSBUJWF NBQT TVHHFTU UIBU DBVUJPO
TIPVME CF FYFSDJTFE XIFO JOUFSQSFUJOH UIF TQFDJहD TUSVDUVSF BOE DPOOFDUJPOT PG UIFTF SFQSF
TFOUBUJPOT 6TFST TIPVME CF BXBSF UIBU UIF OBSSBUJWF DIBJO NBZ OPU BMXBZT SFऺFDU B DPIFSFOU
GSBNJOH USBKFDUPSZ BOE UIBU BCSVQU TIJ॑T JO QFSTQFDUJWF DBO PCTDVSF UIF VOEFSMZJOH TUPSZ
$PODMVTJPOT
5IJT XPSL JOWFTUJHBUFE UIF DBQBCJMJUJFT PG OBSSBUJWF NBQT B DPNQVUBUJPOBM UFDIOJRVF GPS FYUSBDU
JOH OBSSBUJWFT GSPN UFYU EBUB UP DBQUVSF GSBNJOH JOGPSNBUJPO 'SBNJOH SFGFST UP UIF QSPNPUJPO
PG B QBSUJDVMBS QFSTQFDUJWF JO B OBSSBUJWF UISPVHI TFMFDUJWF XPSEJOH 0VS BOBMZTJT GPDVTFE PO
UXP LFZ RVFTUJPOT XIFUIFS UIF BMHPSJUIN BDDVSBUFMZ DBQUVSFT GSBNJOH EJTUSJCVUJPOT BOE JG JU
QSPEVDFT OBSSBUJWFT XJUI DPOTJTUFOU GSBNJOH 8IJMF UIF UFDIOJRVF TIPXFE QSPNJTF JO DBQUVSJOH
UIF GSBNJOH EJTUSJCVUJPO PG UIF EBUB TFU XF GPVOE MJNJUBUJPOT JO BDIJFWJOH DPOTJTUFOU GSBNJOH
GPS UIF FYUSBDUFE OBSSBUJWFT
*O QBSUJDVMBS PVS BOBMZTJT PG UIF QFSGPSNBODF PG UIF OBSSBUJWF FYUSBDUJPO BMHPSJUIN TIPXT
JUT BCJMJUZ UP DBQUVSF GSBNJOH EJTUSJCVUJPOT 5IF DPOTJTUFOUMZ MPX BWFSBHF +FOTFO4IBOOPO
EJWFSHFODF PG BDSPTT BMM UFTUFE DPNCJOBUJPOT TJHOBMT B OPUBCMF BMJHONFOU CFUXFFO UIF
GSBNJOH EJTUSJCVUJPOT PG UIF HFOFSBUFE OBSSBUJWF NBQT BOE UIF EBUB TFU 0VS BOBMZTJT TVQQPSUT UIF
PWFSBSDIJOH DPODMVTJPO UIBU UIF OBSSBUJWF NBQT FYUSBDUJPO BMHPSJUIN FसFDUJWFMZ FODBQTVMBUFT UIF
GSBNJOH EJTUSJCVUJPO JOIFSFOU JO UIF CSPBEFS EBUB TFU 5IFTF हOEJOHT TVHHFTU UIBU UIF OBSSBUJWF
NBQT BT FYUSBDUFE IBWF UIF QPUFOUJBM UP PसFS VTFST WBMVBCMF JOTJHIUT JOUP UIF JOUSJDBUF GSBNJOH
EZOBNJDT FNCFEEFE XJUIJO UIF OFXT OBSSBUJWF
)PXFWFS XIFO XF FYBNJOFE XIFUIFS UIF BMHPSJUIN QSPEVDFT OBSSBUJWF NBQT XJUI DPOTJTUFOU
GSBNJOH PVS BOBMZTJT SFWFBMFE JODPOTJTUFODJFT XJUIJO JOEJWJEVBM NBQT EFTQJUF DBQUVSJOH UIF
CSPBEFS EJTUSJCVUJPO 'PS JOTUBODF POF NBQ TIJ॑FE GPDVT GSPN QVCMJD TFSWJDF JTTVFT UP QPMJUJDBM
BOE DVMUVSBM JTTVFT BDSPTT DPOOFDUFE FWFOUT 5IJT JOEJDBUFT UIBU UIF BMHPSJUIN DVSSFOUMZ GPDVTFT
PO DPOUFOU TJNJMBSJUJFT SBUIFS UIBO DPIFSFOU GSBNJOH XIFO FYUSBDUJOH OBSSBUJWFT
'JOBMMZ XF OPUF UIBU CZ FYQMJDJUMZ FWBMVBUJOH GSBNJOH JO UIF DPOUFYU PG DPNQVUBUJPOBM OBSSB
UJWF FYUSBDUJPO BOE SFQSFTFOUBUJPO QSPDFTT PVS TUVEZ BEESFTTFT UIF MBDL PG TUVEJFT SFHBSEJOH
GSBNJOH JO UIF DPOUFYU PG DPNQVUBUJPOBM OBSSBUJWF FYUSBDUJPO 5IF DVSSFOU NPEFM DBO JOEJSFDUMZ
DBQUVSF GSBNJOH JOGPSNBUJPO UISPVHI DVFT JO UIF VOEFSMZJOH UFYU SFQSFTFOUBUJPO )PXFWFS
EJSFDUMZ DBQUVSJOH BOE VTJOH GSBNJOH JOGPSNBUJPO EVSJOH UIF FYUSBDUJPO QSPDFTT SFNBJOT BO PQFO
DIBMMFOHF 'VUVSF XPSL TIPVME BEESFTT UIJT JTTVF CZ FYQMPSJOH OPWFM BQQSPBDIFT BOE UFDIOJRVFT
UIBU DBO FYQMJDJUMZ JODPSQPSBUF GSBNJOH JOGPSNBUJPO JOUP UIF FYUSBDUJPO QJQFMJOF QPUFOUJBMMZ
JNQSPWJOH UIF BDDVSBDZ BOE DPOUFYUVBM VOEFSTUBOEJOH PG UIF FYUSBDUFE EBUB
26
3FGFSFODFT
<> ) 1 "CCPUU 5IF $BNCSJEHF JOUSPEVDUJPO UP OBSSBUJWF $BNCSJEHF 6OJWFSTJUZ 1SFTT 0OF
-JCFSUZ 1MB[B /FX :PSL /: 64"
<> " .JTLJNNPO # 0۟MPVHIMJO - 3PTFMMF 4USBUFHJD OBSSBUJWFT $PNNVOJDBUJPO QPXFS BOE
UIF OFX XPSME PSEFS 3PVUMFEHF SE "WF /FX :PSL /: 64"
<> # ,FJUI /PSBNCVFOB 5 .JUSB /BSSBUJWF NBQT "O BMHPSJUINJD BQQSPBDI UP SFQSFTFOU
BOE FYUSBDU JOGPSNBUJPO OBSSBUJWFT JO 1SPD "$. )VN$PNQVU *OUFSBDU WPMVNF
"$. /FX :PSL /: 64" Q QBHFT
<> + 4UBTLP $ (¶SH ; -JV 4FOTFNBLJOH BDSPTT UFYU EPDVNFOUT IVNBODFOUFSFE WJTVBM
FYQMPSBUJPO XJUI KJHTBX JO 4FOTFNBLJOH 8PSLTIPQ! $)* QQ ۙ
<> # ' ,FJUI /PSBNCVFOB 5 .JUSB $ /PSUI " TVSWFZ PO FWFOUCBTFE OFXT OBSSBUJWF
FYUSBDUJPO "$. $PNQVU 4VSW 63- IUUQTEPJPSH EPJ10.
1145/3584741
<> 3 . &OUNBO 'SBNJOH 5PXBSE DMBSJहDBUJPO PG B GSBDUVSFE QBSBEJHN +PVSOBM PG DPNNV
OJDBUJPO ۙ
<> % " 4DIFVGFMF 'SBNJOH BT B UIFPSZ PG NFEJB FसFDUT +PVSOBM PG DPNNVOJDBUJPO
ۙ
<> ' )BNCPSH " ;IVLPWB # (JQQ *MMFHBM BMJFOT PS VOEPDVNFOUFE JNNJHSBOUT UPXBSET UIF
BVUPNBUFE JEFOUJहDBUJPO PG CJBT CZ XPSE DIPJDF BOE MBCFMJOH JO *OUFSOBUJPOBM $POGFSFODF
PO *OGPSNBUJPO 4QSJOHFS QQ ۙ
<> 4 -JV - (VP , .BZT . #FULF % 5 8JKBZB %FUFDUJOH GSBNFT JO OFXT IFBEMJOFT
BOE JUT BQQMJDBUJPO UP BOBMZ[JOH OFXT GSBNJOH USFOET TVSSPVOEJOH 64 HVO WJPMFODF
JO DFFE1SPJOHT PG UIF SE $POGFSFODF PO $PNQVUBUJPOBM /BUVSBM -BOHVBHF -FBSOJOH
$P/-- "TTPDJBUJPO GPS $PNQVUBUJPOBM -JOHVJTUJDT )POH ,POH $IJOB QQ ۙ
63- IUUQTBDMBOUIPMPHZPSH, EPJ10.18653/v1/K19-1047
<> % 4IBIBG $ (VFTUSJO & )PSWJU[ + -FTLPWFD *OGPSNBUJPO DBSUPHSBQIZ $PNNVOJDBUJPOT
PG UIF "$. ۙ
<> # ' ,FJUI /PSBNCVFOB 5 .JUSB $ /PSUI .JYFE NVMUJNPEFM TFNBOUJD JOUFSBDUJPO GPS
HSBQICBTFE OBSSBUJWF WJTVBMJ[BUJPOT JO 1SPDFFEJOHT PG UIF UI *OUFSOBUJPOBM $POGFSFODF
PO *OUFMMJHFOU 6TFS *OUFSGBDFT *6* ۟ "TTPDJBUJPO GPS $PNQVUJOH .BDIJOFSZ /FX :PSL
/: 64" Q ۙ 63- IUUQTEPJPSH EPJ10.1145/
3581641.3584076
<> $ ' #BLFS $ + 'JMMNPSF + # -PXF 5IF CFSLFMFZ GSBNFOFU QSPKFDU JO 1SPD PG UIF UI
*OU $POG PO $PNQVUBUJPOBM MJOHVJTUJDT7PMVNF "$- QQ ۙ
<> 4 4XBZBNEJQUB 4 5IPNTPO $ %ZFS / " 4NJUI 'SBNFTFNBOUJD QBSTJOH XJUI TP॑NBY
NBSHJO TFHNFOUBM SOOT BOE B TZOUBDUJD TDBसPME BS9JW QSFQSJOU BS9JW
<> % $BSE " #PZETUVO + ) (SPTT 1 3FTOJL / " 4NJUI 5IF NFEJB GSBNFT DPSQVT
"OOPUBUJPOT PG GSBNFT BDSPTT JTTVFT JO 1SPD PG UIF SE "OOVBM .FFUJOH PG UIF "$- BOE
UIF UI *OUM +PJOU $POG PO /BUVSBM -BOHVBHF 1SPDFTTJOH 7PMVNF 4IPSU 1BQFST
QQ ۙ
<> / /BEFSJ ( )JSTU $MBTTJGZJOH GSBNFT BU UIF TFOUFODF MFWFM JO OFXT BSUJDMFT 1PMJDZ
ۙ
<> ' )BNCPSH , %POOBZ # (JQQ "VUPNBUFE JEFOUJहDBUJPO PG NFEJB CJBT JO OFXT BSUJDMFT
27
BO JOUFSEJTDJQMJOBSZ MJUFSBUVSF SFWJFX *OUFSOBUJPOBM +PVSOBM PO %JHJUBM -JCSBSJFT
ۙ
<> - (VP $ 4V 4 1BJL 7 #IBUJB 7 1 "LBWPPS ( (BP . #FULF % 8JKBZB 1SPQPTJOH
BO PQFOTPVSDFE UPPM GPS DPNQVUBUJPOBM GSBNJOH BOBMZTJT PG NVMUJMJOHVBM EBUB %JHJUBM
KPVSOBMJTN ۙ
<> 5 " #JSLMBOE 3 ( -BXSFODF .FEJB GSBNJOH BOE QPMJDZ DIBOHF B॑FS DPMVNCJOF "NFSJDBO
#FIBWJPSBM 4DJFOUJTU ۙ
<> ' 4DIOFMM ,BSFO $BMMBHIBO "TTFTTJOH UIF EFNPDSBUJD EFCBUF )PX UIF OFXT NFEJB GSBNF
FMJUF QPMJDZ EJTDPVSTF 1PMJUJDBM DPNNVOJDBUJPO ۙ
<> 3 %F'PTUFS / 4XBMWF (VOT DVMUVSF PS NFOUBM IFBMUI GSBNJOH NBTT TIPPUJOHT BT B
QVCMJD IFBMUI DSJTJT )FBMUI DPNNVOJDBUJPO ۙ
<> % 1 )BJEFS.BSLFM . 3 +PTMZO (VO QPMJDZ PQJOJPO USBHFEZ BOE CMBNF BUUSJCVUJPO
5IF DPOEJUJPOBM JOऺVFODF PG JTTVF GSBNFT +PVSOBM PG 1PMJUJDT ۙ
<> 1 -FBWZ , 1 .BMPOFZ "NFSJDBO SFQPSUJOH PG TDIPPM WJPMFODF BOEQFPQMF MJLF VT۟ "
DPNQBSJTPO PG OFXTQBQFS DPWFSBHF PG UIF DPMVNCJOF BOE SFE MBLF TDIPPM TIPPUJOHT $SJUJDBM
4PDJPMPHZ ۙ
<> 8 3 /FVNBO 3 8 /FVNBO . 3 +VTU " / $SJHMFS $PNNPO LOPXMFEHF /FXT BOE
UIF DPOTUSVDUJPO PG QPMJUJDBM NFBOJOH 6OJWFSTJUZ PG $IJDBHP 1SFTT
<> 3 ( -BXSFODF 5 " #JSLMBOE (VOT IPMMZXPPE BOE TDIPPM TBGFUZ %FहOJOH UIF TDIPPM
TIPPUJOH QSPCMFN BDSPTT QVCMJD BSFOBT 4PDJBM 4DJFODF ॕVBSUFSMZ ۙ
<> 7 - 1BSTPOT 4USBUJहFE TBNQMJOH 8JMFZ 4UBUT3FG 4UBUJTUJDT 3FGFSFODF 0OMJOF ۙ
<> # , /PSBNCVFOB . )PSOJOH 5 .JUSB &WBMVBUJOH UIF JOWFSUFE QZSBNJE TUSVDUVSF
UISPVHI BVUPNBUJD XI FYUSBDUJPO BOE TVNNBSJ[BUJPO JO 1SPD PG UIF $PNQVUBUJPO
+PVSOBMJTN 4ZNQPTJVN $PNQVUBUJPO +PVSOBMJTN #PTUPO ." 64" QQ
ۙ
<> % $FS : :BOH 4 ,POH / )VB / -JNUJBDP 3 4 +PIO / $POTUBOU . (VBKBSEP
$FTQFEFT 4 :VBO $ 5BS : 4VOH # 4USPQF 3 ,VS[XFJM 6OJWFSTBM TFOUFODF FODPEFS
$P33 BCT ۙ arXiv:1803.11175
<> - .D*OOFT + )FBMZ + .FMWJMMF 6NBQ 6OJGPSN NBOJGPME BQQSPYJNBUJPO BOE QSPKFDUJPO
GPS EJNFOTJPO SFEVDUJPO BS9JW QSFQSJOU BS9JW
<> - .D*OOFT + )FBMZ 4 "TUFMT IECTDBO )JFSBSDIJDBM EFOTJUZ CBTFE DMVTUFSJOH +PVSOBM PG
0QFO 4PVSDF 4P॑XBSF
28