<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>From keywords to discursive legitimation: Representing 'kuffar' in jihadist propaganda magazines</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Anina L. Kinzel</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>College of Arts and Humanities Swansea University Singleton Park Swansea</institution>
          ,
          <addr-line>Wales SA2 8PP</addr-line>
        </aff>
      </contrib-group>
      <fpage>26</fpage>
      <lpage>33</lpage>
      <abstract>
        <p>This paper explores the integration of Corpus Linguistics and Discourse Analysis methodologies, specifically the use of keyword analysis to pinpoint salient discourse representations of given social identities. The particular identity examined here is that of 'kuffar', which is a derogatory term used by jihadist-ideology groups to describe people who do not share their faith. Two measures (Log Likelihood and %DIFF) are used to determine the keyness value of the term kuffar in the propaganda magazines produced by two such groups: Al Qaeda (Inspire) and Islamic State (Dabiq). Although they yield different keyness values, both confirm the salience of this social identity within jihadist propaganda. The results of a Key Word in Context concordance analysis of the term kuffar show how negative representations thereof are mainly legitimized on impersonal authority grounds (Van Leeuwen, 2007). The results also reveal a trend towards reminding Muslims of their “duty” to fight and kill kuffar individuals. This is supported by positive expressions to describe Muslims who perform this duty, and negative expressions and traits associated with the non-believers. This case study supports the view that CL and Discourse Analysis can offer a 'useful synergy' (Baker et al. 2008, 2015) to research into, amongst other areas, representation and ideology in language.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        The research in this paper was carried out as part
of a research project funded by Swansea
University and directed by professors Stuart
Macdonald and Nuria Lorenzo-Dus. The project
examined jihadist propaganda from different
disciplinary perspectives, including Terrorism
Studies and Linguistics. All available online
publications of five jihadist magazines released
between January 2009 and the end of June 2015
were collected. These publications are Jihad
Recollections, Gaidi Mtaani, Inspire, Dabiq and
Azan. To be included in the data set, publications
had to be written in English and meet the project’s
definition of a magazine (Macdonald et al., 2015).
The reason this paper focuses on Dabiq and
Inspire, produced by IS and Al-Qaeda,
respectively, is that these two groups currently
pose the biggest terrorist threat to the Western
world. Terrorist groups are also often regarded as
acting the same, especially if they follow a jihadist
ideology. However, they might target different
social groups and compete for recruits in fighting
a common enemy – usually an out-group who
does not share their ideology and faith.
Comparative research within these groups is
therefore most useful to understand their
similarities and differences. The aim of this paper
is twofold: (1) to contribute to the current
academic debate about one of the main analytic
tools in Corpus Assisted Discourse Studies
(CADS): keywords; and (2) to show how IS and
Al-Qaeda discursively construct “kuffar” in their
propaganda magazines and legitimize this as a
negative social identity. “Kuffar” is a derogatory
term, which describes individuals on religious
grounds, or lack thereof. Analyzing this term
provides insights into the role of religion in
jihadist ideologies, which is still heavily debated.
1.1 CADS
Corpus-Assisted Discourse Studies (CADS) is the
application of Corpus Linguistics (CL) tools such
as keywords, wordlist, and concordance line
analysis in Discourse Studies. The field has been
evolving since the 1990s, when new Corpus
Linguistics tools were developed. Until 2004 not
many studies used a CADS approach
        <xref ref-type="bibr" rid="ref1">(Partington,
2004)</xref>
        . Partington describes CADS as ‘the
uncovering, in the discourse type under study, of
what we might call non-obvious meaning’
        <xref ref-type="bibr" rid="ref2">(Partington, 2008, p. 191)</xref>
        . It combines the
statistical techniques of CL, such as keyword and
frequency lists and concordancing, with the
qualitative tools of Discourse Analysis, namely
close-readings and reflection
        <xref ref-type="bibr" rid="ref2">(Partington, 2008)</xref>
        to
uncover non-obvious meaning. Baker 2015 argues
that using a CADS approach, especially with a
large corpus, removes some of the bias a DA
researcher may have. It also gives DA researchers
a starting point: a CADS methodology may drive
the analysis in ‘unexpected directions’ (Baker,
2015, p. 144). In a 2008 journal article, Baker et
al. conclude that using Corpus Linguistics tool
and Critical Discourse Analysis is ‘a useful
methodological synergy’ (Baker, et al., 2008, p.
273). Some examples of work that uses a CADS
approach to examine how Muslims are
constructed by various media are as follows:
Baker et al. use Sketch Engine, a corpus analysis
tool for grammatically tagging items, and other
CL techniques to examine how collocates of the
word ‘Muslim’ is constructed in British
Newspapers articles between 1998 and 2008
(Baker et al., 2013). In their analysis they focus on
the two most frequent immediate right-hand
collocates “world” and “community” (Baker et
al., 2013). Their findings include that Muslims are
often associated with negative aspects, they are
also portrayed as being easily offended and in
conflict with non-Muslim communities (Baker et
al., 2013). Muslim world was often referring to
different branches of Islam as one while being
constructed on language or geographical grounds,
rather than religious grounds (Baker et al., 2013).
McEnery et al. also look at the word “Muslim” as
part of their analysis of media reactions to the
murder of Private Lee Rigby by Michael
Adebolajo and Michael Adebowale, who
converted to Islam (McEnery et al., 2015). The
findings showed that the word is associated with
the murderers and the victim, but is used to
distance this action from other Muslims. Islam as
a keyword is associated with negative aspects
such as betrayal, radicalization and extremism
(McEnery et al., 2015).
      </p>
    </sec>
    <sec id="sec-2">
      <title>1.2 Keyword analysis</title>
      <p>
        Keyword analysis is a much used technique of CL
        <xref ref-type="bibr" rid="ref4">(Gabrielatos and Marchi, 2011)</xref>
        . Bondi notes that
‘[T]he study of keywords has become central in
corpus linguistics, especially through the
development of techniques for the analysis of the
meaning of words in context’
        <xref ref-type="bibr" rid="ref8">(Bondi, 2010, p. 3)</xref>
        .
In the broadest sense, keywords are words that are
important in a given text (Stubbs, 2010). They
mirror what the text is about (Scott and Tribble,
2006), which is why they are an important tool of
CL to help identify a subset of textual items to
analyze (Baker, 2006a). Keyword lists in CL
tools such as AntConc
        <xref ref-type="bibr" rid="ref7">(Anthony, 2016)</xref>
        and
WordSmith
        <xref ref-type="bibr" rid="ref10">(Scott, 2016a)</xref>
        are based on statistical
significance compared to a usually larger
reference corpus. This is done by first compiling
two wordlists of the two corpora. Words with a
high ‘keyness’ appear in the given text more
frequently than expected. Although there are
different statistical techniques to determine the
‘keyness’ of words, log-likelihood (Baker et al
2006b) is arguably the standard one. It is also a
technique that has received some criticism,
leading to the CL / CADS research community
seeking to develop alternative techniques. One
such technique, which is compared in this paper
to Log Likelihood, is called %DIFF. Introduced
by Gabrielatos and Marchi (2011, 2012), its main
difference lies in that it is based on effect size
      </p>
    </sec>
    <sec id="sec-3">
      <title>1.2.1 Log-Likelihood</title>
      <p>
        Log-likelihood is a test that shows statistical
significance (Baker et al., 2006b). It assigns a
pvalue to every word and measures how significant
the word is in comparison to a reference corpus.
According to Biber et al. the p-value ‘represents
the probability that this keyness is accidental’
        <xref ref-type="bibr" rid="ref6">(Biber, et al., 2007, p. 138)</xref>
        . Log-Likelihood does
not take into consideration the size of the study
corpus, which means it does not reflect the
definition of keywords
        <xref ref-type="bibr" rid="ref4">(Gabrielatos and Marchi,
2011)</xref>
        . If an English language magazine, which
also features some Arabic words was compared to
an English language corpus, such as the British
National Corpus (BNC)
        <xref ref-type="bibr" rid="ref3">(British National Corpus,
2007)</xref>
        , all Arabic words would be assigned a low
p –value, because they appear much more
frequently in this magazine. However, this does
not mean that all of these words are especially key
in this corpus or indicate aboutness, these words
are just different compared to the BNC.
      </p>
    </sec>
    <sec id="sec-4">
      <title>1.2.2 Effect size (%DIFF)</title>
      <p>
        Effect size measures ‘the difference of the
frequency of a word in a study corpus when
compared to that in the reference corpus’
        <xref ref-type="bibr" rid="ref4">(Gabrielatos and Marchi, 2011)</xref>
        . Contrasting to
log-likelihood, the size of the two corpora is also
taken into account, which is especially useful
when comparing corpora of different sizes. In this
case, the reference corpus does not have to be
larger than the study corpus.
      </p>
    </sec>
    <sec id="sec-5">
      <title>1.3 Structure</title>
      <p>This paper will first compare the results of a
keyword analysis of the corpus (see 3.1), focusing
on the term ‘kuffar’1, by applying log-likelihood
and %DIFF measures. It will then report the
results of a KWIC analysis for “kuffar” that shows
how its negative discourse representation is
legitimized by the magazine authors.</p>
    </sec>
    <sec id="sec-6">
      <title>3. Methodology</title>
      <p>3.1 Data
The corpus for this study consist of 22 issues of
Dabiq (185,951 words) and Inspire (304, 347
words). 13 of these issues are from Inspire and
were published between the summer of 2010 and
the spring of 2015. The remaining nine issues
were published by IS between the summer of 2014
and the spring of 2015. The digital versions of the
issues were collected in the summer of 2015,
coinciding with the onset of the project.</p>
    </sec>
    <sec id="sec-7">
      <title>3.2 Procedure / Framework</title>
      <p>
        Aim (1) entailed calculating keywords using
Loglikelihood and %DIFF (see section 4 and
Appendix A and B for details). In order to further
our understanding of how “kuffar” is used and
constructed as a social identity for aim (2), it was
necessary to carry out a Key Word in Context
(KWIC) analysis, which requires manual
examination of concordance lines generated by
AntConc
        <xref ref-type="bibr" rid="ref7">(Anthony, 2016)</xref>
        containing those
KWIC in Dabiq and Inspire. As aim (2) was
specifically concerned with the discursive
justification for the negative construction of
‘kuffar’ identity in jihadist ideology, an
established discourse analytic framework for the
analysis of legitimation was followed, namely
Van Leeuwen’s (2007) legitimation framework.
This framework consists of four key categories:
Authorization, moral evaluation, rationalization
and mythopoesis. Authorization, which this
analysis will focus on for space reasons, is
subdivided into six self-explanatory
subcategories: These are personal authority, expert
authority, role model authority, impersonal
authority, the authority of tradition and the
authority of conformity. The reason this analysis
focuses on impersonal authority is that it is the
most frequently used strategy to legitimize the
term “kuffar” in Dabiq and Inspire2. Two coders,
the author of this paper and one of the principal
investigators of the project, individually coded
each of the concordance lines. Differences in
coding were resolved through discussion. 12% of
the concordance lines in both Dabiq and Inspire
1 This term is the Arabic equivalent to “disbelievers” and is
used to describe people in the West who do not share the
same faith as IS and Al-Qaeda
could not be assigned to any legitimation
category: they were general instances of “kuffar”
in which the derogatory term was not used
alongside any legitimating behavior, such as
rulings on fighting against and killing “kuffar”.
      </p>
      <p>The corpus was not lemmatized because
one of the project’s aims was to examine stylistic
variation, including at the spelling level, between
the different magazines, especially regarding use
of Arabic terms.</p>
    </sec>
    <sec id="sec-8">
      <title>4. Results</title>
      <p>In CADS, the analysis usually starts with a
keyword analysis. As can be seen in Tables 1 and
2, the log-likelihood for “kuffār” is the same in
both tables, even though in the first Dabiq is the
study corpus, and in the second Inspire is the
study corpus.</p>
      <p>word
kuffār
kuffar
frequencies
SC
28
47</p>
      <p>RC</p>
      <p>LL
125
%DIFF</p>
      <p>-86.31
1335.81
The highest %DIFF with Inspire as the study
corpus was the word inspire with a %DIFF of
28738.42, however since this is the title of the
magazine as well as a verb, this is not surprising
and does not reflect what the text is about. It
appeared once in Dabiq probably as a verb. The
word “kuffar” is 30th in the list with a %DIFF of
1335.81. The word is only used twice in Dabiq,
which suggests that this is the preferred spelling
in Inspire. This is an interesting finding,
considering kuffār is only used 28 times in Inspire
and 125 times in Dabiq. Its %DIFF is also not very
high, compared to the 100th highest, which still
has a %DIFF of 342.96.</p>
      <p>Note that if two or more words have the
same %DIFF, they are all the top Xth word, for
example the words “road”, “verily”, “al-bukhari”,
“depend”, “dry”, “perspective”, “proceed”, “race”
and “reveal” all have the same %DIFF of 358.24
and are therefore all the top 95th word. These
words also happen to have a Log-likelihood of
5.87. According to this measure, they are not very
relevant to the corpus. Similarly, “kuffar” would
2 For further details see a forthcoming paper by the author
not have been in the top 100 keywords, if the
keywords had been measured by log-likelihood; it
has a LL of 31.99 as can be seen in Table 1.</p>
      <p>In table 2 the %DIFF and Log-Likelihood
of the words “kuffār” and “kāfir”3 are shown.
“kuffār” is not in the Top 100 keywords, it has a
%DIFF of 630.67, but it is not significant in
Dabiq, since it is also used in Inspire. The top
keyword in Dabiq is “shām”, which is a region in
Syria near the city of Damascus, with a %DIFF of
28542.34. This is logical, because IS are in Syria
fighting a war for territory among other things.
This word is only used once in Inspire, which
could be because it is not that important to
AlQaeda, or because there are other spellings they
use.</p>
      <p>frequencies
word
kuffār
kāfir
%DIFF
“kāfir” is the 30th keyword, with a %DIFF of
2791.51 and a log-likelihood of 82.23. It probably
would not have been in the top 30 keywords if
they had been compiled using log-likelihood.</p>
      <p>Two words with a very high
loglikelihood are “wa“ with 707.35 and “abū” with
641.25. Abū means “father of”, and is often used
as part of a name. “Wa” is part of the phrase “salla
llāhu ʿalayhi wa-alehe wa-sallam”, which
translates to “peace be upon him” and is used
immediately after mentioning the prophet
Mohammad. Both terms are therefore not what the
two texts are about, they are frequently used as
part of or following names. They cannot be
considered keywords, although they would be
likely at the top of the list according to
LogLikelihood.</p>
      <p>
        Keywords indicate what the text is about.
Despite the different keyness values, “kuffar”
emerges as a salient keyword using both methods.
Other keywords mentioned in this section did not
occur as keywords using both methods. It is worth
pursuing other case studies to determine
advantages and disadvantages of both methods.
This is especially relevant to CADS Researchers,
who usually only analyze the top keywords. It is
important to ensure that these words are indeed
the most important ones in the given corpus
        <xref ref-type="bibr" rid="ref4">(Gabrielatos and Marchi, 2011)</xref>
        .
      </p>
      <sec id="sec-8-1">
        <title>The paper will now move on to the</title>
        <p>aforementioned KWIC analysis. In the context of
terrorist magazines, impersonal authority is
defined as follows: Legitimation through laws,
rules and regulations (such as laws of Islam, the
Koran, scriptures) or unwritten guidelines made
by the terrorist groups. The following are two
typical examples of impersonal authority as
legitimation in Inspire:</p>
        <p>Every Muslim who carries the creed of pure
Tawheed, alliance to Muslims and disavowal from
kuffār and lives in the West has a duty to find a
suitable method to severely kill the kuffār and
waylay4 them at every place of ambush until they
cease killing Muslim women and their children.
(Inspire, issue 13)
We are proud that every muslim [sic] believes in
the Islamic unity and waging jihad against kuffar
who occupy our lands. (Inspire, issue 10)
This example is a general appeal to Muslims
living in the West, who identify with a jihadist
ideology such as the one Al-Qaeda puts forward,
to carry out their duty to kill non-believers
(kuffar). This example shows that Inspire uses the
words “duty” to remind Muslims what they have
to do. The example uses the oneness of Allah
(Tawheed) as a reminder and it might want to
inflict guilt on Muslims who do not follow this
“duty”. In fact, it attempts to do this by explicitly
stating that in order to belong to the terrorist
group, one must not associate with non-believers.
The impersonal authority in this case comes from
Al-Qaeda, who have ruled that Muslims living in
the West have to kill kuffar.</p>
        <p>The second example shows the unity and
sense of community one can expect when joining
Al-Qaeda. It expresses pride in waging war (jihad)
against the non-believers) and makes joining the
fight sound quite appealing to Muslims. This is a
softer approach to reminding Muslims of their
duty, playing on the fact that Muslims who share
Al-Qaeda’s ideology might feel isolated and
discriminated against and long to belong to a
group of like-minded individuals. This example
portrays the terrorist very positively and the kuffar
as negatively, especially by describing them as
“occupy[ing]” their lands. The impersonal
authority is again expressed by Al-Qaeda based on
their ideology.</p>
      </sec>
      <sec id="sec-8-2">
        <title>3 “kāfir” is another spelling for the term “kuffar”</title>
      </sec>
      <sec id="sec-8-3">
        <title>4 stop</title>
        <p>The analysis will now focus on Dabiq. These are
typical examples of legitimation through
impersonal authority in Dabiq:</p>
        <p>The Muslims in the West will quickly find
themselves between one of two choices, they either
apostatize and adopt the kufrī religion propagated
by Bush, Obama, Blair, Cameron, Sarkozy, and
Hollande in the name of Islam so as to live amongst
the kuffār without hardship, or they perform hijrah
to the Islamic State and thereby escape persecution
from the crusader governments and citizens.
(Dabiq, issue 7)
A halt of war between the Muslims and the kuffār
can never be permanent, as war against the kuffār
is the default obligation upon the Muslims only to
be temporarily halted by truce for a greater shar’ī
interest, as in the offer of truce from the Prophet
(sallallāhu ‘alayhi wa sallam) to the mushrikīn of
Makkah in Hudaybiyah. (Dabiq, issue 8)
But due to the deviance and arrogance of the
enemies of Allah, they plot against His religion and
His allies. Their plots almost cause the mountains
to collapse out of shock that the kuffār dare to
oppose the Lord of the heavens and the earth.
(Dabiq, issue 9)
The first example describes the two choices
Muslims living in the West have. They can adopt
Islam the way non-believers spread it and not face
any problems from the government. The other
option is to join the Islamic State and flee pursuit
from Western governments. The words
“apostatize” and “kufrī religion” imply that this is
not the option Muslims should choose. The
impersonal authority is expressed by IS, which is
explicitly mentioned in the text.</p>
        <p>The second example describes the fact
that according to Dabiq, Muslims are always at
war with the non-believers (kuffar). It emphasizes
their obligation to fight the kuffar. A cease-fire
can only be temporary, which happened in a
historic situation that is outlined in the example,
involving the Prophet, which indicates a second
layer of legitimation. This example supports the
call to join IS in their fight, which is outlined in
the previous example.</p>
        <p>In the third example, kuffar are associated
with being deviant and arrogant and it is explicitly
stated that they oppose Allah, which might inflict
fear in Muslims who are living in the West or have
not joined IS’s jihad. All three examples show that
IS portray their jihad as the only true way to obey
Allah and fight the non-believers, who oppose
him.</p>
      </sec>
    </sec>
    <sec id="sec-9">
      <title>4. Conclusion</title>
      <p>
        This paper has compared %DIFF with the
traditional technique Log-Likelihood and pointed
out that kuffar emerges as a salient term using
both methods. In this case, the method did not
change the salience of the keyword in question.
This paper did not seek to prescribe to use one
method over the other, it simply presented %DIFF
as an appropriate technique to select keywords for
further analysis, especially to compare corpora of
different sizes, since it takes into account the size
of both corpora, unlike Log-Likelihood.
LogLikelihood will likely continue to be the
predominantly used method. However, the newest
version of WordSmith (WordSmith 7.0) includes
%DIFF as an option of “statistical tests” to
calculate keywords
        <xref ref-type="bibr" rid="ref10">(Scott, 2016b)</xref>
        . Choosing
which statistics to use is an important decision a
researcher has to make in the process of an
analysis. More research using %DIFF as a method
of identifying keywords is needed as it is
important to validate new emerging methods.
      </p>
      <p>The paper then analysed the term “kuffar”
in the terrorist propaganda magazines Inspire and
Dabiq. The findings include a trend towards
reminding Muslims of their “duty” or “obligation”
to fight and kill non-believers (kuffar). This is
supported by positive expressions and notions to
describe Muslims who perform this duty, and
negative expressions and traits, such as being
deviant, arrogant and occupying lands, associated
with the non-believers. Al-Qaeda portrayed
making hijra (travelling to the territory of war)
and joining them as something to take pride in.
They also imply a sense of belonging in their
propaganda, which may speak to individuals who
feel isolated. Their propaganda seems welcoming.
Contrastingly, IS focus on the negative traits
associated with non-believers and emphasise that
kuffar and Muslims can never be at peace. They
draw on historical examples of the Messenger and
Allah to legitimise their discourse and evoke fear
and guilt in Muslims who have not joined their
jihad.</p>
      <p>To conclude, Corpus linguistics
techniques are useful tools to apply in discourse
analysis to narrow the text down to a few areas of
interest. The CADS approach combines
quantitative and qualitative analyses as outlined
above, which ensures that the findings are reliable.</p>
    </sec>
    <sec id="sec-10">
      <title>Acknowledgments</title>
      <p>Many thanks go to Nuria Lorenzo-Dus for the
support, expertise and comments that greatly
improved this paper. I’d also like to thank the
Cyberterrorism Project and Swansea University.
Thanks also go to Matteo Di Christofaro for
insights into effect size (%DIFF).</p>
      <p>Michael Stubbs. (2010). Three concepts of keywords.</p>
      <p>In Marina Bondi and Mike Scott (Eds.),</p>
      <p>Mike Scott and Christopher Tribble. (2006). Textual
Patterns - Key words and corpus analysis in
language education. Amsterdam: John
Benjamin Publishing Company.
Mike</p>
      <p>Scott. (2016b). Get started guides for
WordSmith. Retrieved from WordSmith
Tools:
http://www.lexically.net/downloads/version7
/HTML/index.html?keywords_calculate_info
.htm
Paul Baker. (2006a). Usind Corpora in Discourse</p>
      <p>Analysis. London: Continuum.</p>
      <p>Paul Baker, Andrew Hardie and Tony McEnery.
(2006b). A Glossary of Corpus Linguistics.</p>
      <p>Edinburgh: Edinburgh University Press.</p>
      <p>Paul Baker, Costas Gabrielatos, Majid Khosravinik,
Michal Krzyzanowsk, Tony McEnery and
Ruth Wodak. (2008). A useful
methodological synergy? Combining critical
discourse analysis and corpus linguistics to
examine discourses of refugees and asylum
seekers in the UK press. Discourse &amp; Society,
273-307.</p>
      <p>Paul Baker, Costas Gabrielatos and Tony McEnery.
(2013). Sketching Muslims: A Corpus Driven
Analysis of Representations Around the Word
‘Muslim’ in the British Press 1998–2009.</p>
      <p>Applied Linguistics, 34(3), 255–278.</p>
      <p>Paul Baker. (2015). Introduction to the Special Issue.</p>
      <p>Discourse &amp; Communication, 9(2), 143-147.
Stuart Macdonald, Nuria Lorenzo-Dus,
Matteo di Cristofaro, David Mair, Anina Kinzel,
Saffron Lee, Nyasha Maravanyika, David Nezri, Jodie
Parker, Elliot Parry, Kate Thomas and Luke Walker.
(2015). Online Terrorist Magazines. Cyberterrorism
Project Research Report (No. 5). Available via:
www.cyberterrorism-project.org
Theo Van Leeuwen. (2007). Legitimation in discourse
and communication. Discourse &amp;
Communication, 1(1), 91-112.</p>
      <p>Tony McEnery, Mark McGlashan and Robbie Love.
(2015). Press and social media reaction to ideologically
inspired murder: The case of Lee Rigby.
Discourse &amp; Communication, 9 (2), 237–259.</p>
    </sec>
    <sec id="sec-11">
      <title>Appendix A: %DIFF in Inspire</title>
      <p>1 66.74997 109.2500316
1 43.9943 72.00570265
1 32.61646 53.38353817
1 32.61646 53.38353817
1 23.13493 37.86506778
1 20.85937 34.14063488
1 19.34232 31.65767962
4 76.99002 126.0099796
1 17.06675 27.93324672
2 31.09942 50.90058291
1 13.27414 21.72585856
2 25.03124 40.96876186
1 12.13636 19.86364211
1 11.7571 19.2429033
2 21.23863 34.7613737
2 20.4801 33.51989606
1 10.24005 16.75994803
1 9.48153 15.5184704
1 9.48153 15.5184704
1 9.48153 15.5184704
1 9.48153 15.5184704
1 8.723007 14.27699277
4 34.13351 55.86649344
2 16.68749 27.3125079
1 8.343746 13.65625395
8 64.85366 106.1463375
20 160.8067 263.193258
1 7.964485 13.03551514
1 7.585224 12.41477632
24 180.1491 294.8509376
2 14.41192 23.58807501
2 14.41192 23.58807501
2 14.41192 23.58807501
1 7.205962 11.7940375
1 7.205962 11.7940375
1 7.205962 11.7940375
3 21.23863 34.7613737
2 14.03266 22.96733619
2 13.6534 22.34659738
1 6.826701 11.17329869
1 6.826701 11.17329869
1 6.826701 11.17329869
1 6.826701 11.17329869
4 26.92754 44.07245593
3 20.10084 32.89915725</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <given-names>Alan</given-names>
            <surname>Partington</surname>
          </string-name>
          . (
          <year>2004</year>
          ).
          <article-title>Corpora and discourse, a most congruous beast</article-title>
          .
          <source>In Alan Partington, John Morley and Louann Haarman (Eds.)</source>
          ,
          <source>Corpora and Discourse</source>
          (pp.
          <fpage>9</fpage>
          -
          <lpage>18</lpage>
          ). Bern: Peter Lang.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Alan</surname>
            <given-names>Partington</given-names>
          </string-name>
          , (
          <year>2008</year>
          ).
          <article-title>The armchair and the machine: Corpus-Assisted Discourse Studies</article-title>
          . In Carol T. Torsello, Katherine Ackerley and Eric Castello (Eds.), Corpora for University Language Teachers (pp.
          <fpage>189</fpage>
          -
          <lpage>213</lpage>
          ). Oxford: Peter Lang.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>British National Corpus.</surname>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>What is the BNC? Retrieved from http://www</article-title>
          .natcorp.ox.ac.uk/: http://www.natcorp.ox.ac.uk/corpus/index.x ml
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <given-names>Costas</given-names>
            <surname>Gabrielatos</surname>
          </string-name>
          and
          <string-name>
            <given-names>Anna</given-names>
            <surname>Marchi</surname>
          </string-name>
          . (
          <year>2011</year>
          ).
          <article-title>Keyness: Matching metrics to definitions. Theoreticalmethodological challenges in corpus approaches to discourse studies - and some ways of addressing them</article-title>
          .
          <source>Portsmouth: 5th November</source>
          . [conference slides]
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <given-names>Costas</given-names>
            <surname>Gabrielatos</surname>
          </string-name>
          and
          <string-name>
            <given-names>Anna</given-names>
            <surname>Marchi</surname>
          </string-name>
          . (
          <year>2012</year>
          ).
          <article-title>Keyness: Appropriate metrics and practical issues</article-title>
          .
          <source>CADS International Conference</source>
          . Bologna, Italy. [conference slides]
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <given-names>Douglas</given-names>
            <surname>Biber</surname>
          </string-name>
          , Ulla Connor, Thomas A.
          <string-name>
            <surname>Upton</surname>
            , Molly Anthony and
            <given-names>Kostyantyn</given-names>
          </string-name>
          <string-name>
            <surname>Gladkov</surname>
          </string-name>
          . (
          <year>2007</year>
          ).
          <article-title>Rhetorical appeals in fundraising</article-title>
          . In Douglas Biber,
          <article-title>Ulla Connor and Anthony Upton, Discourse on the Move: Using corpus analysis to describe discourse structure</article-title>
          (pp.
          <fpage>121</fpage>
          -
          <lpage>151</lpage>
          ). Amsterdam: John Benjamin Publishing Company.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <given-names>Laurence</given-names>
            <surname>Anthony</surname>
          </string-name>
          . (
          <year>2016</year>
          ).
          <source>AntConc Homepage. Retrieved April 14</source>
          ,
          <year>2016</year>
          , from Laurence Anthony's website: http://www.laurenceanthony.net/software/ant conc/
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <given-names>Marina</given-names>
            <surname>Bondi</surname>
          </string-name>
          . (
          <year>2010</year>
          ).
          <article-title>Perspectives on keywords and keyness: an introduction</article-title>
          .
          <source>In Marina Bondi and Mike</source>
          Scott (Eds.), Keyness in texts (pp.
          <fpage>1</fpage>
          -
          <lpage>20</lpage>
          ). Amsterdam: John Benjamins Publishing Company.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          Keyness in Texts (pp.
          <fpage>21</fpage>
          -
          <lpage>42</lpage>
          ). Amsterdam :
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <given-names>Mike</given-names>
            <surname>Scott</surname>
          </string-name>
          . (
          <year>2016a</year>
          ).
          <source>Downloads. Retrieved April 14</source>
          ,
          <year>2016</year>
          , from WordSmith Tools: http://www.lexically.net/wordsmith/downloa ds/
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>