This paper surveys the server-side use of security-related options for protecting websites against cross-site scripting (XSS) attacks. By using data from a bug bounty platform, the use of these header-based options is approached with a case-control study that contrasts popular Internet domains against less popular domains that have explicitly been verified to have been vulnerable to XSS. According to the results based on the analysis of nearly 800 thousand domains, (a) the header-based security options are only infrequently used. However, (b) the domains known to have been vulnerable to XSS have been much less likely to use these options compared to popular domains. Furthermore, (c) the options surveyed tend to statistically form clear latent dimensions, which can be speculated to relate to the effort required to enforce strict security policies for websites.
Cross-site scripting vulnerabilities have continued to frequently appear in the top-rankings of the most common security bugs in web applications [OWASP 2018]. The essence behind these security bugs is simple: an attacker injects malicious executable code to a vulnerable website, and a client's web browser executes this code during rendering of the website. The consequences for clients may range from stealing of web cookies and browsing history to phishing and even key logging.
If time, resources, and talent are all available, the "best strategy for protecting webservers is to write secure Web applications from scratch" [Stritter et al. 2016]. Combined with the commonplace lack of time, resources, and talent, the sheer complexity and the chaotic nature of the current Web have prompted the introduction of many countermeasures against XSS. Most of these are bandages that merely patch the symptom-the only real remediation against cross-site scripting is arguably proper input validation. However, as it is deviously difficult to do this in practice [Weinberger et al. 2011], different remediation solutions have been considered practically on all sides of the current Web [Stritter et al. 2016]. On the infrastructure side, remediation is deep-rooted in the fundamental issues affecting web standards, encryption of hypertext transfer protocol (HTTP) traffic, the global public key infrastructure, the domain name system, the so-called same-origin policy, and ultimately the whole client-server paradigm. On the client-side, the mitigation solutions include sandboxes, blacklists, and different heuristics to profile the execution of JavaScript. On the server-side, the remediation solutions include web application firewalls, algorithmic solutions, and different options that can be set in HTTP header responses for instructing browsers about intended behavior. Although much of existing research has focused on the detection and prevention of XSS vulnerabilities with algorithms and heuristics [Hydara et al. 2015], the header options are noteworthy for a couple of important reasons.
First, many of these satisfy the desirable property of bandages: these are easy to apply to fix bleeding in existing implementations, and these are recognized by most browsers due to (implicit) stan-dardization [Ross et al. 2013;W3C 2016;2017]. Second, these have enlarged the scholarly intersection between web application security research and the increasingly popular large-scale Internet measurement research [Lekies et al. 2013;Tajalizadehkhoob et al. 2017;Vasek et al. 2016;Weissbacher et al. 2014]. This intersection is also the area to which this paper contributes. The contribution stems from the data used. While numerous studies have collected data based on blacklists and related collections about domains and websites associated with spam, phishing, and malware [Ruohonen et al. 2016;Tajalizadehkhoob et al. 2017;Vasek et al. 2016], these are only implicitly related to web vulnerabilities. In contrast, this paper uses data from a so-called bug bounty specialized to the discovery and disclosure of XSS vulnerabilities in particular. In other words, a part of the data used is explicitly about websites that have been verified to have been vulnerable to security bugs the headers surveyed are meant to partially address. Equipped with this improvement to the always difficult ground truth problem, the paper sets to answer to the following three research questions (RQs):
-RQ 1 : How common is the use of HTTP headers for XSS prevention? -RQ 2 : Are different HTTP header fields used in conjunction with other fields designed to mitigate XSS? -RQ 3 : Is there a difference in the use of XSS-related HTTP header fields between popular Internet domains and less popular domains that have been vulnerable to XSS according to data from a bug bounty?
To answer to these three questions, the paper proceeds by introducing the materials and methods in Section 2. The empirical results are presented in Section 3 and further discussed in the final Section 4.
The following discussion will outline the headers surveyed, the empirical materials, and the methods used.
The header fields surveyed are listed in Table I. The listing is not comprehensive; only well-known fields either explicitly or at least implicitly related to XSS are surveyed. Even with this narrowing of the scope, it is not easy to demarcate between fields related to cross-site scripting and those not related to it. In other words: in many attack scenarios, XSS can be used to facilitate other attacks, or cross-site scripting can be facilitated by other attacks. The header fields surveyed can be further elaborated as follows:
-The HttpOnly field can be set for cookies to prevent these from being accessible by scripts executed by a client's browser [Ruohonen and Lepp änen 2017b]. Setting this flag is generally recommended for session cookies in order to mitigate theft of session information via XSS. -A value nosniff can be set to prevent some browsers from deducing about media types and file formats (i.e., MIME types) that do not match the declared Content-Type. The rationale for sniffing MIME types relates to the lack of universally accepted meta-data for web content and the common occurrence of incorrectly defined types. Unfortunately, this sniffing exposes XSS attacks [Barth et al. 2009]. -X-Frame-Options are not directly related to XSS per se, although the so-called clickjacking can be used in conjunction with XSS, cross-site request forgery (CSRF), and related attacks [Kim and Kim 2015;Takamatsu and Kono 2014]. The values specified for X-Frame-Options can mitigate some of these attacks by either preventing or restricting a client's browser from displaying web content through <frame> and <iframe> elements [Ross et al. 2013]. The three available values forbid framing altogether, restrict it to the same origin than the content itself, or allow framing only from explicitly specified origins. -Referrer-Policy governs the information a client sends along with the Referer header to the landing page. A certain class of XSS attacks are known to be possible with the Referer information [Gremwell BVBA 2010], although privacy and other security concerns (including CSRF) have been more pressing [Barth et al. 2008;W3C 2017]. Some of these attacks and related concerns can be remedied by forbidding referrer information to be sent with a no-referrer value. In addition, more fine-grained control is possible with other values [W3C 2017]. The metric labeled REFERRER-ORIGIN groups values that tighten the default client-side policy, excluding the strictest no-referrer policy. -X-XSS-Protection is a HTTP response header field that can be set to instruct some web browsers to enable either sanitization or filtering of reflected XSS attacks [Mozilla Foundation et al. 2017]. The main targets of the field are inline resources, such as inline event handlers and inline <script> or <style> elements. In general, this field has been augmented by the more comprehensive Content-Security-Policy. -Content-Security-Policy (CSP) is currently the most comprehensive header-based mitigation solution against cross-site scripting. The basic idea behind CSP is to use a policy document for listing those domains that are allowed to execute scripts, load web resources, or embed cross-domain functionality [W3C 2016]. The explicitly defined policy directives are grouped into the metric CSP-SRC.
The metric CSP-URI, in turn, captures the presence of policy restrictions based on uniform resource identifiers. Reflecting CSP's complexity, there are numerous additional directives that can be set for further restrictions. These are grouped into CSP-MISC. Finally, it is worth remarking that the deprecated reflected-xss has similar functionality to X-XSS-Protection.
A value origin, same-origin, strict-origin, origin-whencross-origin, or strict-origin-when-cross-origin is defined.
X-XSS-FILTER X-XSS-Protection: 1 is present. X-XSS-BLOCK X-XSS-Protection: 1; mode=block is present.
CSP-SRC A value default-src, script-src, object-src, style-src, img-src, media-src, frame-src, child-src, font-src, connect-src, or manifest-src is defined.
A base-uri, form-action, or frame-ancestors is set.
A value sandbox, script-nonce, referrer, plugin-types, block-all-mixed-content, or upgrade-insecure-requests is present.
A CSP contains reflected-xss.
Much of the current XSS-related standardization work concentrates on CSP. Because the current (v.3) revision [W3C 2016] is detailed and complex, applying a strict CSP successfully requires careful fine-tuning on per-site basis. Due to the complexity, it should be also stressed that the four simple metrics in Table I only probe CSP's surface by counting the policy directives themselves; the actual 17:4
• J. Ruohonen and V. Lepp änen values specified for these directives differentiate a secure policy from an insecure one [Liu et al. 2016;Weissbacher et al. 2014].
The dataset is based on two sources. The first source is the Alexa's conventional list of top-million most popular Internet domains [Alexa Internet, Inc. 2017]. Although generalizability remains an open question, Alexa's popularity lists have been the de facto benchmark sources for different Internet measurement studies [Lekies et al. 2013;Liu et al. 2016;Ruohonen and Lepp änen 2017a;Tyson et al. 2017;Weissbacher et al. 2014]. The second source comes from the web vulnerabilities disseminated through the Open Bug Bounty (OBB) [2017] platform. This bug bounty is a community-based platform for dissemination of XSS and CSRF vulnerabilities in particular [Ruohonen and Allodi 2018]. Unlike some other bug bounties, OBB neither pays for vulnerabilities nor targets some particular vendors or websites.
Given the two data sources, the retrieval of the headers was implemented by following the simple scheme illustrated in Fig. 1. Before continuing further, it should be noted that a success is defined as a query that returned a status code starting either with a number 2 or with a number 3. A failure, in turn, includes all other status codes as well as queries failed for undefined or unknown reasons. Despite of these disqualification criteria, it should be emphasized that initial redirections (status code 301) were followed during the retrieval.
Each domain in both samples was queried three times in order to rule out timeouts, domain name resolution failures, and other temporary errors. Although OBB provides uniform resource locators (URLs) pointing to the vulnerable web applications, Alexa's list only contains domain names. To maintain a uniform retrieval scenario for both samples, also the entries in the OBB sample were queried based on the host field in the URLs (hence, a domain that has been exposed to multiple vulnerabilities is counted only once). The host field was further restricted to cover only fully qualified domain names. In other words, Internet protocol (IPv4) addresses were excluded during pre-processing of the OBB sample. Like in previous empirical research [Tyson et al. 2017], the actual retrieval was done with the conventional HTTP/1.1 via which the headers were retrieved based on GET requests. If an HTTP query failed, however, another GET request was attempted over the transport layer security protocol. If both requests failed for three times for a given domain during the retrieval in November 2017, the domain was finally excluded from the empirical analysis. This retrieval scenario could be further extended (for instance, by querying the domains also with www-prefixes; see Weissbacher14), but some limitations would still be present [Ying and Li 2016]. While acknowledging the imperfection of the retrieval routine, more than enough data was retrieved from both sources (see Fig. 2). Finally, a few words are required about the parsing required for the metrics listed in Table I. The header fields are delivered as case-insensitive key-value pairs. As the details for standards-compliant parsing are fairly detailed, the headers were parsed line-by-line with simple string searches after both the keys and the values had been transformed to lower-case letters. As an example: to construct the X-XSS-BLOCK metric, a character 1 and a string block were searched from the value delivered via the transformed x-xss-protection key. These heuristic searches avoid some ambiguities (such as those related to semicolons and white-space). Nevertheless, it should be remarked that some outlying domains observed may be interpreted as having set an option-even though actual web browsers may interpret the option as incorrectly specified.
The following three methods are used:
(1) To answer to RQ 1 , the first "method" simply plots the relative share of all metrics enumerated in Table I. For this plotting, a symmetric difference (i.e., a union without an intersection) of the two sources is used. To determine the intersection, Alexa's popularity ranks are used. A domain is classified as belonging to both sets in Fig. 2 when: (a) the domain is present in both samples, or (b) a popularity rank is provided for the domain indirectly through OBB. This twofold detection is necessary because OBB provides its own historical Alexa-ranks [Ruohonen and Allodi 2018]. In other words, some of the domains in the OBB sample may have appeared in some older ranking that is no longer publicly available from Alexa.
(2) To answer to RQ 2 , the principal component analysis (PCA) is used by examining whether the metrics can be reduced into smaller dimensions. As all metrics have the same scale, a sample covariance matrix is used as the input to PCA, and the number of components is approximated with a so-called scree plot (for the details refer to Zuur et al. [2007], i.a.). The symmetric difference is again used for the two samples.
(3) To answer to RQ 3 , a small case-control study is computed. Case-control studies are classical ways to conduct experimental studies in fields such as medicine and epidemiology [Rundle et al. 2012], but these have recently gained traction also in security research [Vasek et al. 2016]. By removing the intersection in Fig. 2, the case is composed of the vulnerable domains in the OBB subsample, while the domains in the Alexa subsample constitute the control. Due to the definition given for the intersection of the two samples, the case refers also to less popular domains, which are compared against the control group of popular domains. Given the 1 ≤ i ≤ 13 metrics, the resulting casecontrol setup is illustrated in Table II.
Table II. The Case-Control Setup (frequencies nij)
Odds ratio
Thus: if the nonintersecting OBB and Alexa subsamples do not differ in terms of the i:th metric, the corresponding odds ratio equals unity. A value smaller (larger) than one indicates that the vulnerable OBB domains are less (more) likely to have had a header-based XSS protection in place compared to the control domains. Given the unequal sample sizes (see Fig. 2), a simple resampling computation [Rundle et al. 2012] is used by drawing 1000 random subsets from the Alexa subsample, the size of each subset equaling n 11 + n 21 . Arithmetic mean is used to report the odds ratios from the thousand draws. 17:6
• J. Ruohonen and V. Lepp änen
The answer to RQ 1 is clear: the HTTP header fields related to security are only infrequently used. The most common use case is to restrict the accessibility of session cookies (see Fig. 3). Roughly around one tenth of the domains observed have also restricted MIME sniffing, enforced the same-origin policy for framing, and adopted X-XSS-Protection. The use of CSP is strikingly rare. The top-5 metrics in Fig. 3 explain as much as about 90% of the total variation in the sample (see Fig. 5). When looking at the relevant PCA loadings (see Figure 6), it seems that there are three latent dimensions: cookie restriction (PC2), XSS protection (PC1), and partial enforcement of the same-origin policy (PC3). The answer to RQ 2 is positive: the easily enforced XSS restrictions are used in conjunction with the other fields requiring little effort. The answer to RQ 3 is also clear: the less popular OBB domains (without Alexa's ranks) have been much less likely to use HTTP header fields for security improvements. The odds ratios are equal or greater than one only for two metrics (see Fig. 4), which are only infrequently used in general (see Fig. 3). The fields explicitly related to cross-site scriptingincluding NOSNIFF and the X-XSS-Protection options-attain odds ratios clearly smaller than unity. Thus, when compared to popular domains, the less popular domains that have explicitly been known to be vulnerable to XSS have failed to enforce basic protections-or, alternatively, these domains may have been vulnerable due to the lack of the protections.
This paper surveyed the use of HTTP header options designed to mitigate XSS and related attacks. According to the results, the use of these headers is still at a modest level even among popular Internet domains (RQ 1 ). This result supports recent empirical observations; even simple techniques such as JavaScript-restricted cookies and X-XSS-Protection are only infrequently used on the server-side of the current Web [Ruohonen and Lepp änen 2017b;Tajalizadehkhoob et al. 2017]. However, not all websites are equal.
The results clearly show that less popular domains that have been verified to have been vulnerable to XSS are much less likely to use header-based protections (RQ 3 ). Although the statistical computations are exceptionally clear in this regard, this result should be still taken only tentatively. The reason for this caution is well-known. The most challenging part in the design of a case-control study is the selection of a control group [Rundle et al. 2012;Vasek et al. 2016]. Even with the careful subsetting of the two subsamples, the Alexa's top-million list can hardly be taken as a bulletproof control group. Furthermore, the results are only approximations because it is impossible to deduce whether the domains in the OBB subsample used header-based mitigation at the time when the vulnerabilities were disclosed through the bug bounty [Ruohonen and Allodi 2018]. A third limitation relates to the sampling strategy (see Fig. 1), which only probes a domain's main page [Ying and Li 2016]. While acknowledging these three limitations-which largely apply also to many related Internet measurement studies in general, the case-control study presented demonstrates that data from bug bounties can sharpen the assumptions about vulnerable websites.
Finally, the results further show that the XSS-related header tend to statistically parcel into distinct latent dimensions (RQ 2 ). If these dimensions are interpreted to reflect effort [Tajalizadehkhoob et al. 2017], it seems that the dimensions that require little effort are more frequently enforced. A plausible explanation relates to the ever increasing amount of dependencies to external websites [Ruohonen et al. 2018]. As the empirical results also support the existing observations about CSP's adoption problems [Weissbacher et al. 2014], it seems fair to conclude with a remark that web developers may have difficulties to implement and enforce security policies when third-party domains or URLs must be explicitly stated.