<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Feb</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>The dark side of vulnerability exploitation: a proposal for a research analysis. ?</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Universita degli studi di Trento Trento</institution>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2012</year>
      </pub-date>
      <volume>15</volume>
      <issue>2012</issue>
      <abstract>
        <p>Software security research has put much e ort in evaluating security as a function of the expected number of vulnerabilities and their criticality. As hackers become more sophisticated and economicallydriven, I argue that exploitation activities are a much more interesting index of risk than the number of vulnerabilities: the economics of the black market can shed light on attacking processes and trends, and can be very useful in better assessing security and re-thinking patching behavior and patches priority.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>that could better choose which software con guration is more secure in that
particular time-frame; (b) the vendor, that could better allocate human and
economic resources by means of a more knowledgeable understanding of actual
exploitation risks, thus increasing product security and monetization of e ort.</p>
      <p>In the next section I give some brief de nitions helpful in describing
vulnerability exploitation. In section 3 I introduce the di erent markets involved in
the process and describe the structure supporting it. Finally in section 4 I draw
my conclusions and Ph.D. proposals, formulating three hypothesis that, if hold
true, could help improve software security and patches scheduling.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Some quick de nitions</title>
      <p>
        Vulnerabilities. I use Ozment's de nition of vulnerability [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], according to
which vulnerabilities are mistakes in the code or in the con guration of a software
that can cause violations in its security policy. These mistakes can be exploited
by an attacker to get access to the vulnerable system.
      </p>
      <p>
        Exploits. One could identify di erent levels of maturity of an exploit as they
usually are born as simple proofs-of-concept, are then scripted and eventually
automated [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Frei et al. in [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] analyzed more than 14 thousand vulnerabilities.
Out of these only about 3400 were exploited, most of which within a month from
the disclosure of the vulnerability.
      </p>
      <p>
        Attacks. An attacker needs to exploit (at least) one vulnerability in the
system to reach his goal. The relation between exploitation time and vulnerability
disclosure date is shown in [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] by Arora et. al: attacks increase at the time of the
vulnerability disclosure. There also seems to be a correlation between
randomwide information scans and attack probes, evidencing that untargeted attacks
are common practice [
        <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
        ].
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>The Markets</title>
      <p>
        Vulnerability and exploit markets are distinct but related: while the former is
divided between legitimate and illegitimate markets [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], the latter is mostly an
underground activity usually labeled as `black market'. On the other hand, the
nancial consequences of vulnerability exploitation have been shown to go far
from solely their market value [13{15].
      </p>
      <p>
        The market of vulnerabilities. No extended study exists, to the best of
my knowledge, on the value of vulnerabilities in the black market. In [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] a very
interesting insight on the legitimate vulnerability market is given; there are many
di culties in the legitimate selling of vulnerabilities to vendors, because of the
`secretive' nature of the good. The relationship between the software vendor and
the security researcher, especially if independent and external to the company,
can be trouble1: the vendor may indeed not appreciate the bad publicity that
the disclosure of a vulnerability earns him [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
1 http://news.cnet.com/8301-27076 3-57320190-248/apple- boots-
security-guru-whoexposed-iphone-exploit/
      </p>
      <p>
        The market of attacks. The economical value for the attacker seems
significant: in [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] Franklin et al. investigate the amount of nancial and
economicalrelated information that circulate in the market; they calculate the value of the
market of the credit cards only to be about 37million USD; if one considers
bank data theft and identity theft, their estimation increases up to 93million
USD. The magnitude of these estimates is also con rmed in [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. Motoyama et
al. in [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] study the dynamics of underground forums, in which these data are
actually traded, and show the high interest the criminals put in online payment
accounts and stolen nancial information.
      </p>
      <p>
        The market of exploits. Lately spam has become a way to di use malicious
links that drive the user toward domains controlled by the attackers, that can
then try (and often succeed) to exploit their systems; very di use vectors for
such an activity are porn sites [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ], botnets [
        <xref ref-type="bibr" rid="ref15 ref19 ref20">15, 19, 20</xref>
        ], and social networks [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ].
Once the attacker gets access to the victim's machine, he can install keyloggers
or any kind of malware that will provide him with the victim's private data or
`permanent' access to the machine, at his will.
      </p>
      <p>The pro le of the coders that write exploits may vary a lot, ranging from
security enthusiasts to professionals. Some coders put a lot of e ort in e ciently
exploiting vulnerabilities; these `e cient' exploits are featured in web
applications with a MySql backend; the community calls them Exploit Kits.</p>
      <p>It is my opinion that the Exploit Kits phenomenon can shed some light on the
exploiting economics underlying the whole data-theft market, and due to some of
its peculiarities can perhaps be of great value in better evaluating e ective risk.
Moreover, it provides preliminary evidence that exploiting activities are governed
by an economical process not yet investigated by the scienti c community.</p>
      <p>There are many di erent Exploit Kits on the market, very often advertised
in underground forums such as exploit.in and vendors.pro. Examples of these
are Phoenix, Eleonore, Blackhole, Crimepack. Exploit kits are rented to the
interested attacker for di erent periods of time, usually up to an year; a
oneyear license would cost from 1000USD to 2500USD2. Perhaps the most popular
Exploit Kit around is now Blackhole3, but Phoenix and others have a signi cant
market share too. Their coders seem to put a lot of e ort in code obfuscation
and encryption4. Even more importantly and perhaps counter-intuitively, but
supporting the hypothesis that exploitation is driven by economical processes,
the number of exploited vulnerabilities in these packs is in the order of ten or
less, and many of them are very old ones.</p>
      <p>As an example, these are the softwares exploited in Eleonore v1.6.5, released
in March 20115 featuring only 10 exploits, most of which are at least 1-year old
and two of which are 5+ years old: MDAC(2006), WMI Object Broke (2006),
2 http://malwareint.blogspot.com/2010/01/state-of-art-in-eleonore-exploit-pack.html
3 http://dvlabs.tippingpoint.com/blog/2011/04/26/blackhole-exploit-kit
4 http://research.zscaler.com/2011/02/blackhole-exploits-kit-attack-growing.html
5 http://exploit.in/forum/index.php?showtopic=46653 (account required to access
the page; the reader might want to use a TOR network or a secure proxy to
access the page, depending to whom belongs the IP used)</p>
      <p>Snapshot (2008), IEpeers (2010), HCP (2010), PDF libti mod v1.0 (2010), Flash
&lt;10.2 (2011), Flash &lt; 10.2.159 (2011), Java Invoke (2010), Java trust (2010).
Analogous are Blackhole's6 and Phoenix's7 o erings, as many others' too8. The
vulnerabilities in those Exploit Kits concern a small set of widely di used and
exposed softwares such as Java, Flash, or Adobe Reader plugins; while at the
time of writing Java seems to be the main target in the most di used exploit
kits9 (Blackhole, Phoenix), in the past were mainly targeted O ce Plugins and
Flash10, suggesting there might be additional, software-related trends in the
process. Exploit kits are advertised by screenshots and exploiting success rates11.</p>
      <p>The actual exploitation takes place when the victim requests the, say,
`exploit.php' page on the attacker's domain12. The attacker must fool the user in
requesting that web-page: apart from social engineering and direct link spam
techniques, the attacker usually compromises one or more hosts (often by means
of SQL Injection) and insert an iframe in the domain's homepage that redirects
connections towards the attacker's `exploit.php' page; this is a very common
practice, as evidenced by sites such as Malware Domain List13 that serve as a
database of hosts that have been compromised. Once the victim reaches the
attacker's host, a set of exploiting scripts is run; as a consequence, the successful
attacker can often execute code on the target machine: install keyloggers, steal
data, download malware and/or make the machine part of his botnet. In order
to increase the hit rate, the compromised sites might be acquired by somebody
else; the attacker could (and this may not be an exhaustive list):
{ buy a set of hosts compromised by somebody else
{ rent connections to compromised hosts from whom acquired them
{ rent connections from tra c brokers14;15 that buy tra c from some third
party (2-6USD per 1k connections).</p>
      <p>
        In particular, the second approach is made easier by the existence of
trafc dispatchers (e.g. SimpleTDS16), and often augmented by botherders
themselves [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]; the third is widely di used in pay-per-click(-install) scenarios such
as porn networks [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] and others [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]: the tra c from a compromised host is
sold by the `compromiser' to the tra c broker, which will then receive a certain
amount of connections from victims that accessed the compromised host. These
6 http://exploit.in/forum/index.php?showtopic=41662
7 http://exploit.in/forum/index.php?showtopic=37627
8 http://vil.nai.com/images/FP BLOG 100527 1.jpg
9 http://www.kaspersky.com/about/news/virus/2011/Ja va the Target of
      </p>
      <p>
        Choice for Exploit Kits in 2011
10 https://threatpost.com/en us/blogs/carberp-and-black-hole-exploit-
kit-wreakinghavoc-120511
11 http://malwareint.blogspot.com/2010/09/black-hole-exploits-kit-another.html
12 http://blog.imperva.com/2011/12/deconstructing-the-black-hole- exploit-kit.html
13 http://www.malwaredomainlist.com/
14 http://www.tra cshop.com/
15 http://www.tra cholder.com/
16 http://www.simpletds.com/
very connections are redirected by the tra c broker to his clients, that in turn
have bought a certain amount of `tra c' that will be directed straight to the
`exploit.php' page under their control [
        <xref ref-type="bibr" rid="ref18 ref22">18, 22</xref>
        ].
4
      </p>
    </sec>
    <sec id="sec-4">
      <title>A research plan</title>
      <p>From this preliminary analysis, the exploitation market results far from being
simply driven by enthusiasts, unorganized hackers or groups of hackers: there
is a whole infrastructure supporting both the exploitation of vulnerabilities and
the economic investment that the attacker must (and apparently actually do)
sustain. This gives preliminary evidence and, to my opinion, a very good reason
to further investigate the dynamics of exploitation and the attackers' goals, in
order to provide insights about actual security and perhaps, eventually, to
better evaluate security metrics, countermeasures, risk assessment and to support
vendors' patching behavior.</p>
      <p>My research goal is to nd a novel, more precise way to describe vulnerability
exploitation, and thus to evaluate the e ective risk factor a ecting a system. In
order to accomplish that, I formulate the following three hypotheses:
{ Hypothesis (1). Attackers are economically rational.
{ Hypothesis (2). There is a substantial di erence in success rates between
public and commercial exploits.
{ Hypothesis (3). Commercial exploits are not redundant (i.e. not many
exploits exist in the same time-frame for the same system con guration).</p>
      <p>Therefore by (2) higher risk would come from those vulnerabilities for which
a commercial exploit exists; if (1) holds, then the most dangerous vulnerabilities
will be those that are e ciently exploitable, because those would optimize the
exploitation success rate and thus maximize attackers' return on investment.
Following (3) vulnerabilities that provide access to a certain system con guration
for which other, easier or more e ciently exploitable vulnerabilities exist would
represent a lower risk because of less interest to the attacker.</p>
      <p>I'm planning to investigate those hypothesis during my Ph.D. program here
at the University of Trento. Hypothesis 3 can be validated by analyzing
hackers' exploitation resources; I'm planning to further understand how much di used
those tools are as attack vectors. I'm also willing to understand who is behind
their development and how pro table this activity is. Hypothesis 2 will involve
testing the e cacy of publicly released exploits against the ones featured in
exploitation tools from (3). Dulcis in fundo, Hypothesis 1 will be the toughest
one to investigate: to collect evidence of the importance of the economic aspects
in the attacking process may not be su cient; I'm planning to conduct interviews
with (professional) hackers and to design and deploy a social experiment with
the purpose of better understanding how much e ort the attackers are willing
to put into the exploitation of a system.</p>
      <p>The validity of those hypotheses could smooth the way toward a more precise
and realistic risk assessment process, and signi cantly improve security metrics's
reliability, patching priorities, and system hardening e ciency and e cacy.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>M.</given-names>
            <surname>Howard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Pincus</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Wing</surname>
          </string-name>
          , \
          <article-title>Measuring relative attack surfaces,"</article-title>
          <source>Comp. Sec. in the 21st Century</source>
          , pp.
          <volume>109</volume>
          {
          <issue>137</issue>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>P. K.</given-names>
            <surname>Manadhata</surname>
          </string-name>
          and
          <string-name>
            <given-names>J. M.</given-names>
            <surname>Wing</surname>
          </string-name>
          , \
          <article-title>An attack surface metric,"</article-title>
          <source>TSE</source>
          , vol.
          <volume>37</volume>
          , pp.
          <volume>371</volume>
          {
          <issue>386</issue>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>I.</given-names>
            <surname>Kotenko</surname>
          </string-name>
          and
          <string-name>
            <given-names>M.</given-names>
            <surname>Stepashkin</surname>
          </string-name>
          , \
          <article-title>Attack graph based evaluation of network security," in Proc. of CMS'06, ser</article-title>
          .
          <source>LNCS</source>
          . Springer,
          <year>2006</year>
          , vol.
          <volume>4237</volume>
          , pp.
          <volume>216</volume>
          {
          <fpage>227</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>L.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Singhal</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Jajodia</surname>
          </string-name>
          , \
          <article-title>Measuring the overall security of network con gurations using attack graphs,"</article-title>
          <source>in Proc. of DAS'07</source>
          ,
          <year>2007</year>
          , pp.
          <volume>98</volume>
          {
          <fpage>112</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>P.</given-names>
            <surname>Mell</surname>
          </string-name>
          and
          <string-name>
            <given-names>K.</given-names>
            <surname>Scarfone</surname>
          </string-name>
          ,
          <string-name>
            <surname>A Complete</surname>
          </string-name>
          <article-title>Guide to the Common Vulnerability Scoring System Version 2.0</article-title>
          . CMU,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>A.</given-names>
            <surname>Ozment</surname>
          </string-name>
          , \
          <article-title>Improving vulnerability discovery models,"</article-title>
          <source>in Proc. of QoP'07</source>
          , ser.
          <source>QoP '07</source>
          . New York, NY, USA: ACM,
          <year>2007</year>
          , pp.
          <volume>6</volume>
          {
          <fpage>11</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>W.</given-names>
            <surname>Arbaugh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Fithen</surname>
          </string-name>
          , and
          <string-name>
            <surname>J. McHugh</surname>
          </string-name>
          , \
          <article-title>Windows of vulnerability: a case study analysis,"</article-title>
          <source>Computer</source>
          , vol.
          <volume>33</volume>
          , no.
          <issue>12</issue>
          , pp.
          <volume>52</volume>
          {
          <issue>59</issue>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>S.</given-names>
            <surname>Frei</surname>
          </string-name>
          , M. May, U. Fiedler, and
          <string-name>
            <given-names>B.</given-names>
            <surname>Plattner</surname>
          </string-name>
          , \
          <article-title>Large-scale vulnerability analysis,"</article-title>
          <source>in Proc. of LSAD'06. ACM</source>
          ,
          <year>2006</year>
          , pp.
          <volume>131</volume>
          {
          <fpage>138</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>A.</given-names>
            <surname>Arora</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Krishnan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Nandkumar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Telang</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Yang</surname>
          </string-name>
          , \
          <article-title>Impact of vulnerability disclosure and patch availability-an empirical analysis,"</article-title>
          <source>in Proc. of WEIS'04</source>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <given-names>S.</given-names>
            <surname>Ransbotham</surname>
          </string-name>
          and
          <string-name>
            <given-names>S.</given-names>
            <surname>Mitra</surname>
          </string-name>
          , \
          <article-title>Choice and chance: A conceptual model of paths to information security compromise,"</article-title>
          <source>ISR</source>
          , vol.
          <volume>20</volume>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>B. W.</surname>
          </string-name>
          , H. M.,
          <string-name>
            <surname>H. A.</surname>
          </string-name>
          , and
          <string-name>
            <given-names>H. C.</given-names>
            <surname>David</surname>
          </string-name>
          , \
          <source>2011 data breach investigation report," Verizon, Tech. Rep.</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>C. Miller</surname>
          </string-name>
          , \
          <article-title>The legitimate vulnerability market: Inside the secretive world of 0-day exploit sales,"</article-title>
          <source>in Proc. of WEIS'07</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>J. Franklin</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Paxson</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Perrig</surname>
            , and
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Savage</surname>
          </string-name>
          , \
          <article-title>An inquiry into the nature and causes of the wealth of internet miscreants," in Proc. of CCS'07, ser</article-title>
          .
          <source>CCS '07</source>
          ,
          <year>2007</year>
          , pp.
          <volume>375</volume>
          {
          <fpage>388</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>M. Motoyama</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>McCoy</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Savage</surname>
            , and
            <given-names>G. M.</given-names>
          </string-name>
          <string-name>
            <surname>Voelker</surname>
          </string-name>
          , \
          <article-title>An analysis of underground forums,"</article-title>
          <source>in Proc. of IMC'11</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>C. Kanich</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Kreibich</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Levchenko</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Enright</surname>
            ,
            <given-names>G. M.</given-names>
          </string-name>
          <string-name>
            <surname>Voelker</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Paxson</surname>
            , and
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Savage</surname>
          </string-name>
          , \
          <article-title>Spamalytics: an empirical analysis of spam marketing conversion," in Proc. of CCS'08, ser</article-title>
          .
          <source>CCS '08. ACM</source>
          ,
          <year>2008</year>
          , pp.
          <volume>3</volume>
          {
          <fpage>14</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>R. Anderson</surname>
            and
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Moore</surname>
          </string-name>
          , \
          <article-title>The economics of information security,"</article-title>
          <source>Science</source>
          , vol.
          <volume>314</volume>
          , p.
          <fpage>610</fpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <given-names>A.</given-names>
            <surname>Cardenas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Radosavac</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Grossklags</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Chuang</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Hoofnagle</surname>
          </string-name>
          , \
          <article-title>An economic map of cybercrime,"</article-title>
          <source>in Proc. of TPRC'09</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18. G. Wondracek,
          <string-name>
            <given-names>T.</given-names>
            <surname>Holz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Platzer</surname>
          </string-name>
          , E. Kirda, and
          <string-name>
            <given-names>C.</given-names>
            <surname>Kruegel</surname>
          </string-name>
          , \
          <article-title>Is the internet for porn? an insight into the online adult industry,"</article-title>
          <source>in Proc. of WEIS'10</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19. B.
          <string-name>
            <surname>Stone-Gross</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Cova</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Cavallaro</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Gilbert</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Szydlowski</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Kemmerer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Kruegel</surname>
          </string-name>
          , and G. Vigna, \
          <article-title>Your botnet is my botnet: analysis of a botnet takeover,"</article-title>
          <source>in Proc. of CCS'09. ACM</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>J. B. Grizzard</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Sharma</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Nunnery</surname>
            ,
            <given-names>B. B.</given-names>
          </string-name>
          <string-name>
            <surname>Kang</surname>
            , and
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Dagon</surname>
          </string-name>
          , \
          <article-title>Peer-to-peer botnets: overview and case study,"</article-title>
          <source>in Proc. of HOTBOTS'07</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21. T. Kurt, G. Chris,
          <string-name>
            <given-names>P.</given-names>
            <surname>Vern</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Dawn</surname>
          </string-name>
          , \
          <article-title>Suspended accounts in retrospect:an analysis of twitter spam,"</article-title>
          <source>in Proc. of IMC'11. ACM</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22. J. Baltazar, \
          <article-title>More tra c, more money: Koobface draws more blood," TrendLabs, Tech</article-title>
          . Rep.,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>