We report on the first large-scale evaluation of author obfuscation approaches built to attack authorship verification approaches: the impact of 3 obfuscators on the performance of a total of 44 authorship verification approaches has been measured and analyzed. The best-performing obfuscator successfully impacts the decision-making process of the authorship verifiers on average in about 47% of the cases, causing them to misjudge a given pair of documents as having been written by “different authors” when in fact they would have decided otherwise if one of them had not been automatically obfuscated. The evaluated obfuscators have been submitted to a shared task on author obfuscation that we organized at the PAN 2016 lab on digital text forensics. We contribute further by surveying the literature on author obfuscation, by collecting and organizing evaluation methodology for this domain, and by introducing performance measures tailored to measuring the impact of author obfuscation on authorship verification.
The development of author identification technology has reached a point at which it can be carefully employed in the wild to resolve cases of unknown or disputed authorship. For a recent example, a state-of-the-art forensic software played a role in breaking the anonymity of J. K. Rowling, who published her book “The Cuckoo’s Calling” under the pseudonym Robert Gailbraith in order to “liberate” herself from the pressure of stardom, caused by her success with the Harry Potter series.1 Moreover, forensic author identification software is part of the toolbox of forensic linguists, who employ the technology on a regular basis to support their testimony in court as expert witnesses in cases where the authenticity of a piece of writing is important. Despite their successful application, none of the existing approaches has been shown to work flawless—a fact that is rooted in the complexity and the ill-posedness of the problem. All approaches have a likelihood of returning false decisions under certain circumstances, but the circumstances under which they do are barely understood. It is hence particularly interesting to analyze whether and how these circumstances can be controlled, since any form of control over the outcome of an author identification software bears the risk of misuse.
In fiction, a number of examples can be found where authors tried to remain anonymous, and where they, overtly or covertly, tried to imitate the writing style of others. In ? A summary of this report has been published as part of [92] 1 http://languagelog.ldc.upenn.edu/nll/?p=5315 is known to have written publishes automatically obfuscated text circumvents OR obstructs
is received by automatically verifies authorship Eve fact, style imitation is a well-known learning technique in writing courses. But the question of whether humans are ultimately capable of controlling their own writing style so as to fool experts into believing they have not written a given piece of text, or even that someone else has, is difficult to answer based on observation alone: are the known cases more or less all there is, or are they just the tip of the iceberg (i.e., examples of unskilled attempts)? And, if the “expert” to be fooled is not a human but an author identification software, the rules are changed entirely. The fact that software is used to assist author identification increases the attack surface of investigations to spoil the decision-making process of the software. This is troublesome since the human operator of such a software may be ignorant of its flaws, and biased toward taking the software’s output at face value instead of treating it with caution. After all, being convinced of the quality of a software is a necessary precondition to employing it to solve a problem.
At PAN 2016, we organized for the first time a shared task on author obfuscation
to begin exploring the potential vulnerabilities of author identification technology. A
number of interesting subtasks related to author obfuscation can be identified, from
which we have selected that of author masking. This task is built on top of the task
of authorship verification, a subtask of author identification, which was organized at
PAN 2013 through PAN 2015 [
Given two documents, decide whether they have been written by the same author. vs.
Given two documents by the same author, paraphrase the designated one so that the author cannot be verified anymore.
The two tasks are diametrically opposed to each other: the success of a certain approach for one of these tasks depends on its “immunity” against the most effective approaches for the other. The two tasks are also entangled, since the development of a new approach for one of them should build upon the capabilities of existing approaches for the other. However, compared to authorship verification, author obfuscation in general (and author masking in particular) received little attention to date. A reason for this may be rooted in the fact that author masking requires (automatic) paraphrasing as a subtask, which poses a high barrier of entry to newcomers. To facilitate future research on both tasks, we contribute the following analyses and building blocks: 1. First-time large-scale evaluation of 44 state-of-the-art authorship verification approaches attacked by 3 author obfuscation approaches in 4 evaluation settings. This evaluation allows for judging both the feasibility of author obfuscation as well as the vulnerability of the state of the art in authorship verification. To cut a long story short, it turns out that even basic author obfuscation approaches have significant impact on many authorship verification approaches. 2. Proposal of performance measures to quantify the impact that obfuscation has on authorship analysis technology. 3. Survey of related work on author obfuscation, and a systematic review and organization of evaluation methodology for author obfuscation. In particular, we identify the three performance dimensions safety, soundness, and sensibleness, wherein an author obfuscation approach should excel before being considered fit for practical use; we detail how obfuscation approaches can be assessed today, and what may be useful in the future. 4. Organization of a shared task at PAN 2016 on author obfuscation to which the three obfuscators evaluated have been submitted. Moreover, we experiment with peer evaluation in shared tasks by inviting participants as well as interested third parties to co-evaluate the obfuscators with regard to the three aforementioned performance dimensions.
In what follows, Section 2 surveys the related work on author obfuscation, and Section 3 systematically reviews and organizes the corresponding evaluation methodology; here, the obfuscation impact measures are introduced. Section 4 reviews the obfuscation approaches that have been submitted to our shared task, and Section 5 reports on their evaluation against the state of the art in authorship verification, including the results of the outlined peer evaluation initiative. 2
The literature on author obfuscation goes into three directions: (1) obfuscation generation, (2) obfuscation evaluation, and (3) obfuscation detection and reversal. This section reviews the contributions that have been made to date.
Obfuscation generation approaches divide into manual, computer-assisted, and
automatic ones. The manual approaches include a study by Brennan and Greenstadt [
Mechanical Turk to obfuscate the reviews of 40 Yelp users, where up to 5 reviews per user were obfuscated many times over on Mechanical Turk, and up to 99 reviews per user were used to check whether the original user could still be identified among the 40 candidate users. After obfuscation, the success rate at attributing obfuscated reviews to the correct user dropped from 95% to 55% under a standard authorship attribution model employing words, POS tags, POS bigrams, as well as character bigrams and trigrams as features.
Anonymouth is the name of a tool developed by McDonald et al. [72, 73] which
belongs to the computer-assisted approaches for obfuscation generation. It supports
its users to manually obfuscate a given document by analyzing whether it can still be
attributed to its original author after each revision, and by identifying which of the
underlying authorship attribution model’s features perform best to suggest parts of the
given document that should be changed in order to maximize the next revision’s impact
on classification performance. An important component of a computer-assisted author
obfuscation tools is its underlying analysis to determine the success of incremental text
revisions; Kacmarcik and Gamon [
Among the first to propose automatic obfuscation generation were Rao and
Rohatgi [91], who suggested the use of round-trip machine translation: the
to-beobfuscated document is translated to an intermediate language and the result then back
to its original language. More than one intermediate languages may be used in a row
before returning to the initial one. Supposedly, the translation round-trip distorts the
writing style of the original’s author sufficiently to confuse an authorship analysis. With the
rise of machine translation systems, this approach has been studied many times to date,
becoming a de-facto baseline for author obfuscation. However, the question whether
this approach works is still undecided; some find that it does not perform well in terms
of safety, soundness, and sensibleness of the obfuscated text [
Obfuscation evaluation is about assessing the performance of an obfuscation
approach. All of the aforementioned obfuscation approaches have been evaluated to some
extent by their authors. However, little has been said to date on how an obfuscator
should be evaluated; rather, evaluation setups have been created individually for each
paper, rendering the reported results incomparable across papers. This is one of our
primary contributions, and Section 3 introduces a comprehensive evaluation setup for
author obfuscation under the three performance dimensions safety, soundness, and
sensibleness. Nevertheless, our setup takes inspiration from the literature by collecting
and organizing the previously employed evaluation procedures. For example, the most
common evaluation approach under the safety dimension is to employ an existing
authorship analysis approach to verify whether an obfuscated text can still be attributed
to its original’s author [
Finally, obfuscation detection and reversal is the task of deciding whether a given
text has been obfuscated, and in that case, to undo the obfuscation in order to retrieve
as much of the original text as possible. The possibility of reversing the effects of an
(automatic) obfuscation threatens its safety. Afroz et al. [
This section collects and organizes the evaluation methodology for author obfuscation for the first time. We introduce three performance dimensions in which an author obfuscation approach should excel to be considered fit for practical use. Afterwards, we present an operationalization of each dimension based on both manual review as well as performance measurement. 3.1
The performance of an author obfuscation approach obviously rests with its capability to achieve the goal of fooling forensic experts, be they software or human. However, this disregards writers and their target audience whose primary goal is to communicate, albeit safe from deanonymization. For them, the quality of an obfuscated text as well as whether its semantics are preserved are also important. In our survey of related work, these performance dimensions are often neglected or mentioned only in passing. Altogether, we call an obfuscation software – safe, if its obfuscated texts can not be attributed to their original authors anymore, – sound, if its obfuscated texts are textually entailed by their originals, and – sensible, if its obfuscated texts are well-formed and inconspicuous. These dimensions are orthogonal; an obfuscation software may meet each of them to various degrees of perfection. Each dimension keeps the others in check since trivial or flawed approaches stand no chance of achieving perfection in all three dimensions at the same time. The operationalization of each dimension, however, poses significant challenges in terms of resource requirements as well as scalability. In what follows, we review in detail the challenges involved in making each dimension measurable as well as how they have been operationalized in related work. 3.2
The safety of an obfuscation approach depends on it withstanding three kinds of
attacks: (1) manual authorship analyses, (2) automatic authorship analyses, and (3)
deobfuscation attacks.
(1) Manual authorship analyses Manual authorship analyses can only be done by
trained forensic linguists, so that this kind of analysis is expensive and therefore does
not scale. Furthermore, it is unlikely that a forensic linguist would take part as human
subject in an experiment to evaluate an obfuscation approach (not even anonymously),
since the risks of suffering reputation damage from failing to beat it are too high,
whereas they have little to gain otherwise. At any rate, beating one forensic linguist
is insufficient to establish the safety of an obfuscation approach; the approach would
have to be tested against a number of experts to raise sufficient confidence. Given these
limitations, author obfuscation approaches are probably not going to be analyzed for
safety against manual authorship analyses any time soon.
(2) Automatic authorship analyses Automatic authorship analyses are by
comparison much more straightforward, practical, and scalable to be employed for obfuscation
evaluation. Dozens of approaches have been proposed for the two author identification
subtasks authorship attribution and authorship verification, so that evaluating an author
obfuscation approach boils down to running the existing, pre-trained authorship
analysis approaches against problem instances with and without obfuscated texts to observe
the difference in performance. To be considered safe against automatic authorship
analyses, an obfuscation approach should be able to defeat the best-performing authorship
analysis approaches. In this connection, the related work on author obfuscation has
employed a number of approaches for authorship attribution from the literature. Most
safety evaluations rely on two basic feature sets, namely the so-called Basic 9 features
(used in [
Measuring obfuscation impact. When using existing authorship analysis approaches to measure the performance of an obfuscation approach, the impact obfuscation has on their classification performance is of interest. More specifically, for all problem instances where authors are correctly identified, the question is how many of them are not correctly identified, anymore, after obfuscation.
Let da denote a document written by author a, and let DA denote a set of documents written by authors from the set of all authors A. We call D = hdu; DAi an instance of an authorship problem, where du is a document written by an unknown author u and DA is a set of documents of known authorship which may or may not contain a subset 3 http://www.philocomp.net/humanities/signature.htm 4 List of repositories: https://github.com/search?q=authorship+attribution+user:pan-webis-de of documents Du DA written by the author u 2 A of du. If DA comprises only documents written by a single author a so that DA = Da, D is called an authorship verification problem, and otherwise an authorship attribution problem. If DA comprises documents written by more than one author, and if it can be guaranteed that one of them is the author of du, D is a closed-class classification problem, and otherwise and openclass classification problem. Closed-class attribution problems can also be considered ranking problems, where the authors represented by disjunct subsets of DA are to be ranked so that the highest-ranking author u is that of du.
We denote the universe of all authorship problem instances by D, and : D ! A [ f;g denotes the true mapping from D to the set of known authors A for problems D 2 D whose true author u 2 A of du is among the candidates found in DA, and to ; otherwise. An authorship analysis approach y : D ! A [ f;g is an approximation of that has been trained on Dtrain D. The extent to which the learned approximation of y to has been successful is evaluated using Dtest D n Dtrain by checking whether y returns answers matching those of for the problem instances in Dtest. A basic measure that is frequently applied to measure the performance of a given authorship analysis approach y is its accuracy acc(y; Dtest) on a given test set Dtest: acc(y; Dtest) = jfD 2 Dtest : y(D) = (D)gj : jDtestj
In this setting, an author obfuscation approach o : D ! D maps the universe of authorship problems onto itself; here, o(D) = hdo; DAi, where do is the obfuscated version of du 2 D and DA 2 D is kept as is. The true author of an obfuscated problem o(D) is the same as without, say (o(D)) = (D). But if an obfuscator works, an authorship analysis approach y would return y(o(D)) 6= (o(D)). Let o(D) = fo(D) : D 2 Dg. A straightforward way to evaluate an author obfuscation approach o is to apply it to the problem instances in Dtest, measure the accuracy acc(y; o(Dtest)), and to calculate the performance delta: acc(o; y; Dtest) = acc(y; o(Dtest)) acc(y; Dtest): (1) However, in case Dtest comprises verification problems or open-class attribution problems, this measure takes into account problems where obfuscation need not be applied on the document of unknown authorship du, since its true author is not among the candidates DA. Therefore, we consider only the subset Dt+est Dtest, which comprises only problem instances D+ = hdu; DAi where the true author u of du has written at least one document found in DA. Measuring the accuracy of an authorship analysis + approach y on Dtest is equivalent to measuring recall, hence: + rec(y; Dtest) = acc(y; Dtest); and rec(o; y; Dtest) = rec(y; o(Dtest)) rec(y; Dtest): (2) The domain of this measure is [ 1; 1], where 1 indicates the best possible performance of an obfuscator (i.e., flipping all decisions of an authorship analysis approach y that makes no errors on unobfuscated texts), 0 indicates the obfuscator has no impact, and a score greater than 0 indicates the worst case, namely that the obfuscator somehow improves the classification performance of the given authorship analysis approach y instead of decreasing it. In practice, the range of possible scores of rec is governed by the a priori performance rec(y; Dtest) of the authorship analysis approach y: [ rec(y; Dtest); 1 rec(y; Dtest)]. This means that rec does not reveal whether an obfuscation approach has accomplished everything it can against y. When achieving a score in the interval ( 1; 0) it remains unclear whether the obfuscator has flipped all of y’s correct authorship attributions, or not. To get an idea of the relative impact an obfuscator has on y’s recall, we apply the following normalizations dependent of rec’s sign: imp(o; y; Dtest) =
The domain of this measure is, independent of the a priori performance of y, in the interval [ 1; 1], where 1 indicates the best performance an obfuscator o can reach by successfully obfuscating the problem instances where y made a correct attribution before, and where 1 indicates that an obfuscator supports y instead obstructing it by allowing it to correctly attribute problems it has not correctly attributed before. Note that we change the sign of the measure to emphasize that it captures obfuscator performance, and to allow for a more natural ordering. A potential drawback of measuring relative impact may be that the impact measured on authorship analysis approaches with a poor a priori performance may be overemphasized: for example, it may be much easier to flip the few correct attributions of an a priori poor-performing authorship analysis approach to earn a high relative impact than to flip the many correct attributions of a well-performing one. To mitigate this issue, a least-performance threshold under Dtest may be imposed that an authorship analysis approach y must exceed to be considered attack-worthy by an obfuscator o.
As discussed at the outset, the performance of an obfuscation approach o should not only be evaluated against a single authorship analysis approach y, but against as large a collection Y of approaches as possible. After all, author obfuscation approaches are supposed to protect authors across the board of forensic analyses, and not just against specific specimen. Therefore, for a given collection of authorship analysis approaches Y , and for a given obfuscation approach o, we compute its average impact under Dtest as follows: avg imp(o; Y; Dtest) = 1
X imp(o; y; Dtest):
jY j y2Y
The average impact of different obfuscation approaches on a large number of authorship
analysis approaches Y allows for ranking among the obfuscation approaches in order
to determine which of them performs best in terms of safety under Dtest.
(3) De-obfuscation attacks De-obfuscation attacks include attempts to undo the
effects an obfuscation approach has on a text, as well as analyses thereof that allow for a
(semi-)accurate attribution of authorship despite the text having been obfuscated. The
analytical nature of de-obfuscation attacks require a clear formulation of the
assumptions under which de-obfuscation becomes possible, just like any proof of the safety
(3)
(4)
of an obfuscation approach against de-obfuscation does. Such assumptions are
sometimes enumerated as “attacker capabilities” or referred to as “threat model.” We propose
to make the following general assumptions when analyzing an obfuscation approach’s
safety against de-obfuscation:
– Kerckhoffs’ principle: the obfuscation approach used is public
– Data used during obfuscation is public except for the original text
– Seeds used to initialize pseudorandom number generators are secret
– No available meta data links an obfuscated text to its author
This way, the safety of an author obfuscation approach against de-obfuscation depends
only on its merits at generating an irreversible obfuscation, and not on the fact that the
approach or data used during obfuscation are secret. At the same time, if an obfuscation
approach is deterministic and its text operations are easily recognizable and reversible
to the original state, the approach must be considered unsafe and unfit for practical use.
To date, the only systematic analysis of obfuscation approaches has been conducted by
Le et al. [65] who show that the obfuscation approach of Kacmarcik and Gamon [
The soundness of an obfuscation approach depends on its ability to transfer the semantics of an original text to its obfuscated version. While the author of a text may value safety pretty high, the goal of writing a text is still to get a message across, which should remain untarnished by automatic obfuscation. A version of a text that conveys the same meaning as its original is called a paraphrase, and the goal of author obfuscation is to generate one under the constraint that the style of the original’s author is not recognizable, anymore (i.e., so that it is safe against forensic authorship analyses). Consequently, an author obfuscation approach must be evaluated whether and to what extent it generates paraphrases.
At the time of writing, research and development on paraphrase generation is mostly carried out at the sentence level; hardly any approaches exist that paraphrase at the paragraph level or even at the discourse level. Nevertheless, a paraphrase of a whole text may not only be done sentence-by-sentence, but it may also involve rearrangement of sentences, paragraphs, and entire lines of argumentative discourse. When evaluating soundness, a key challenge therefore is to trace the changes made by an obfuscator and to compare the parts of an obfuscated text to their unobfuscated counterparts in the original text. Given the possible non-linear changes that can be made during paraphrasing, an a posteriori comparison and judgment of the obfuscated text compared to its original is rendered difficult, since the apparent relations between an original text and its obfuscation may be ambiguous. While automatic obfuscation approaches can output which part of the original test went into generating which part of its obfuscation, this may not be as straightforward for manual obfuscations, unless the manual text editing operations are traced minutely as they happen.
Concerning evaluation, research on paraphrase generation relies almost
unanimously on manual review. Nevertheless, inspired by the success of the well-known
BLEU metric for machine translation evaluation, several performance measures for
paraphrasing evaluation have been proposed, namely ParaMetric by Callison-Burch
et al. [
In the long run, the aforementioned measures may prove to be useful also for evaluating the soundness of author obfuscation approaches. At this time, however, we prefer manual review despite its disadvantages in terms of overhead, since the literature has not settled on a metric of choice, yet. Instead, to facilitate and scale manual reviews of obfuscated texts in comparison to their original texts, we develop a visual analytics tool for text comparison. The tool features various text comparison visualizations that assist manual soundness review: visualizations are applied to monitor the changes made by an author obfuscation approach. Figures 2, 3, and 4 show examples of the visualizations in action, contrasting the three approaches submitted to our shared task. As a brief explanation, the visualizations show the original text and the obfuscated text at the same time. The text is arranged in phrase, where each line either shows a large phrase the two compared texts have in common, or two stacked phrases where the two texts differ. This allows for quick comprehension of the effects an obfuscator has, as well as for quick judgments, thus significantly decreasing the time for manual review.
Relaxing Soundness Constraints. The constraint that both original and obfuscation
possess the exact same semantics may be relaxed to some extent: it may be sufficient if
the statements made in the obfuscated version of a text will be considered true under
the presumption that the corresponding statements in the original are true (i.e., if the
obfuscated statements follow logically from their respective originals). This is called
textual entailment, and an obfuscated text would be considered entailed under the
original text in such a situation. Relaxing the soundness constraint to allow for textual
entailment opens a much wider space of possible obfuscations compared to paraphrases.
A comprehensive survey of algorithms to recognize textual entailment as well as a
series of corresponding shared tasks on this subject has been given by Dagan et al. [
Finally, when relaxing the soundness constraints, one may also question how exact the original message has to be transferred into the obfuscated text. For example, in early machine translation systems as well as the ones deployed at scale today, not all translations are perfect, but the translation results are still useful to get a broad, and sometimes even a detailed understanding of a text in a foreign language. The same may apply to author obfuscation: to get a message across, it may not be necessary that every last detail gets transmitted correctly (as in properly paraphrased or textually entailed), but it may be sufficient for the reader to get a “readable” text whose message can be discerned with reasonable effort. While the goal of automatic author obfuscation should be to be perfectly sound and to generate actual paraphrases, early systems that do not come close to this requirement may still be useful in practice despite their deficiencies. 3.4
The sensibleness of an obfuscation approach depends on its ability (1) to create
readable, ideally grammatically well-formed text, and (2) to hide the fact that a given
obfuscated text has been obfuscated.
(1) Obfuscation grammaticality Next to being safe against forensic analyses and
sound, another desired property of an obfuscated text is that it is well-formed in terms of
grammar and that it fits into its genre. Automatic grammar checking has a long history in
natural language processing and computer linguistics. Almost all approaches developed
so far are designed to find specific error types in an ungrammatical text. For evaluation,
however, the specific errors made by an obfuscator may be of less interest as opposed
to deciding which parts of an obfuscated text are grammatical and which are not, for
whatever reason. This latter task of judging grammaticality of a given piece of text is by
far less often studied [
Again, just like for obfuscation soundness, relaxing the criteria for obfuscation grammaticality may be reasonable in certain situations: for example, an obfuscator may be allowed to return ungrammatical text as long as it can be understood sufficiently well, or even as a means to mislead readers into thinking that an obfuscated text has been written by a second-language speaker of a given language. In this connection, manual review cannot be entirely avoided when evaluating author obfuscation approaches, since the line between what is acceptable and what is not is much more blurry. (2) Hiding obfuscation style Although the safety of an obfuscated text and therefore the obfuscation approach used to generate it cannot rely on the fact that readers of the obfuscated text do not know that it has been obfuscated, hiding obfuscation is still an interesting side-goal of obfuscation generation. The inconspicuousness of an obfuscated text may serve as a first line of defense that forecloses detailed investigations. This is particularly important for automatic author obfuscation approaches that easily defeat automatic authorship analysis approaches but are vulnerable to manual authorship analyses by forensic experts. Conceivably, there are not enough forensic experts to analyze all texts that may be desirable to be analyzed, so that automatic authorship approaches will be applied to attain scale, whereas manual analyses will only be conducted in cases of doubt or suspicion. Avoiding to raise suspicion is therefore a worthwhile goal for an author obfuscation approach.
In the related work on author obfuscation, Afroz et al. [
Since there are typically many alternative ways to paraphrase a given statement in order
to obfuscate it, and even more so considering an entire text, the question arises which
alternatives are better than others. One possible way to decide this question is to search
for the alternative which is least different from the original but still achieves the goals
of safety, soundness, and sensibleness. In this connection, Kacmarcik and Gamon [
The three approaches submitted to our shared task follow three rather different strategies: round-trip translations, replacement of at most one frequent word per sentence, and style feature changes. While replacement of one word per sentence is a rather conservative strategy in that it changes the to-be-obfuscated text only slightly, the other two approaches change the text more substantially.
Keswani et al. The approach of Keswani et al. [
As for the resulting text, our evaluation showed that hardly any human-readable text is produced (not even on a sentence level). One reason might be that the Europarl corpus is not particularly suited as a training corpus for the different genres of our test datasets. In the current form of Keswani et al.’s translation obfuscation, especially the soundness and the sensibleness suffer from severe errors that render the texts non-readable and not understandable by a human. Figure 2 shows the output for the problem instance 429. Especially the “missing” grammaticality of the output substantially decreases sensibleness and soundness at the same time since the texts do not make sense at all. A couple of translation artifacts (remaining German or French words from the intermediate translations) would raise some suspicions even if the text was readable. However, do-ityourself statistical machine translation will improve over time such that the text quality will get better with new translation approaches and larger corpora. In this respect, employing some of the best public translation approaches in the form of the proprietary big commercial search engine APIs might be interesting for the future although potentially difficult with the current evaluation setting at PAN.
Mansoorizadeh et al. The approach of Mansoorizadeh et al. [70] focuses on
attacking the feature of word frequencies that are the basis of many verification approaches.
The idea is to exchange some occurrences of the 200 most frequent words in the
to-beobfuscated text with synonyms obtained from WordNet [76] and scored based on their
similarity to the original to-be-replaced word. For the most similar candidates, the
commonness in the context of the to-be-replaced word is measured using a 4-gram language
model trained on the Brown corpus [
Since not that many changes are introduced overall, the texts in general remain as readable and grammatically correct as the original text and usually are fully understandable. Still, we spotted some issues, an example of which is shown in Figure 3. In problem instance 5 (a text about Java programming), the word “machine” that refers to a computer is consistently changed to “car,” the word “Java” is sometimes changed to “coffee,” etc. This makes the obfuscations less sensible and sound than in case of most other problem instances. Although the text quality is not reduced in most cases, the conservative paraphrasing also is the reason for a suboptimal safety score since the slight changes on average cannot obfuscate authorship very well. Allowing for some more changes per sentence and taking other than the most frequent words into account might be good future options that could be tested.
Mihaylova et al. The approach of Mihaylova et al. [75] targets a wide range of
styleindicating features that are frequently used in author identification approaches: sentence
length, punctuation, stop words, parts of speech, all caps, and individual word
frequencies. As a background corpus (in the current approach formed by some books from
Project Gutenberg and the PAN training datasets), the expected values of the features
are computed. For a to-be-obfuscated text, the scores for the same feature types are
calculated and then tried to be transformed more to the background average, applying
a wide range of rule-based and random text operations. In particular, the operations
employed are merging/splitting sentences, removing/inserting commas, removing stop
words, improving/impairing spelling, exchanging words with synonyms, applying rules
from the paraphrase database [
In sum, the approach of Mihaylova et al. changes the text a lot since there are no restrictions on the number of changes per sentence. However, the context is usually not taken into account (i.e., whether a particular word or expression is frequent or common enough in the given context around the replacement). Thus, a lot of the changes look rather odd to a human—also the typos and randomly inserted words—and sometimes even change the meaning completely (e.g., “horrible night” changed to “good night” in problem instance 134, shown in Figure 4). Still, the soundness and sensibleness on average are slightly better than for the round-trip machine translation approach of Keswani et al. Also, the quality of the produced text might be improved a lot by not overdoing the spelling errors (i.e., not introducing a spelling error in every occurrence of some word) and probably even more by taking context into account for word replacements that can increase the chance of a more common formulation resulting in more “meaningful” text. 5
We automatically evaluate the safety of the three obfuscation approaches against 44 authorship verifiers which have been submitted to the previous three shared tasks on authorship identification at PAN 2013 through PAN 2015, and we manually assess sensibleness and soundness of the obfuscated texts of each obfuscator. Our safety evaluation is built with an eye on reproducibility, so that future evaluations of author obfuscators may be conducted with ease. In what follows, we detail our setup, the datasets used, and report on the results obtained.
Evaluation Setup The scale of our safety evaluation is made possible based on our
long-term evaluation-as-a-service initiative [
our evaluation.6 This collection of software represents the state of the art in authorship
verification, implementing many different paradigms of tackling this task as well as
hundreds of different features. The best-performing approaches of each year are those
of Seidman [95] submitted to PAN 2013, Fréry et al. [
The participants of the shared task on author obfuscation, too, have been invited to submit their obfuscation approaches in the form of software to TIRA. This way, any newly submitted authorship verification software can be immediately evaluated for vulnerabilities against the submitted obfuscators, and vice versa.
Evaluation Datasets The test datasets on which our evaluation is based correspond
to those used at the shared tasks on authorship verification at PAN 2013 to PAN 2015,
covering a selection of genres:
– PAN13. Collection of English computer science textbook excerpts. Formulas and
source code were removed. Problem instances comprise one document of unknown
authorship and average 4 documents of known authorship. The training dataset
comprises 10 problem instances at an average 1000 words per document, and the
test dataset 30 problem instances at the same average document length.
– PAN14 EE. Collection of English essays written by English-as-a-second-language
students at different language proficiency levels. Essays were divide into age-based
clusters before forming verification problems. Problem instances comprise one
document of unknown authorship and an average 2.6 documents of known authorship.
The training dataset comprises 200 problem instances at an average 848 words per
document, and the test dataset 200 problem instances at an average 833 words per
document.
– PAN14 EN. Collection of English horror fiction novel excerpts from H.P.
Lovecraft’s “Cthulhu Mythos”. The genre is selected very narrowly to strip away any
cross-topic or cross-genre effects typically found in other collections. Many
unfamiliar terms are found in these documents, creating strong indicators of shared topic
and style. Most texts’ themes have a strong negative coloring. Lovacraft’s original
6 Five approaches had to be excluded: the three approaches submitted by Halvani et al. [
All datasets have a balanced ratio of problem instances where the author of the documents of known authorship is the same as that of the document of unknown authorship to problem instances where this is not the case. The training datasets were release to participants so that they could develop their obfuscation approaches against it. The test datasets were kept private. Participants who made a successful software submission could run their software on TIRA against the test datasets, while TIRA prevents any direct access of participants to the test datasets hosted there, and takes precautions against data leaks. This way, any optimization of approaches against test datasets is rendered impossible.
Evaluation Results Table 1 shows the results of our safety evaluation of the three
submitted author obfuscation approaches against at total of 44 authorship verification
approaches on the aforementioned four PAN evaluation datasets. The best-performing
approach across all performance measures and across all datasets is the author
obfuscator of Mihaylova et al. [75]. In terms of average impact (avg imp), it manages to
flip between 46% and 49% of the correct authorship attributions of the verifiers on
the datasets PAN13, PAN14 EE, and PAN14 EN, but only about 36% on the PAN15
dataset. While the approach of Keswani et al. [
However, while the averaged performance deltas and the average impact all point into the direction that obfuscation approaches decrease the performance of authorship verification approaches, this is not unanimously the case. Tables 3, 4, and 5 show the performances when applying the individual authorship verification approaches on the four respective test datasets, each one obfuscated once by the approaches of Mihaylova et al. (Table 3), Keswani et al. (Table 4), and Mansoorizadeh et al. (Table 5). Notably, all obfuscators appear to improve a number of verifiers instead of obstructing them. Table 2 -0.29 -0.39 -0.38 -0.33 -0.54 -0.05 -0.01 -0.01 -0.11 -0.10 -0.29 -0.54 -0.04 -0.74 -0.04 -0.02 -0.01 -0.06 -0.10 -1.00 -0.14 -0.11 -0.25 -0.05 -0.02 -0.03 -0.83 -0.16 -0.22 -0.06 -0.02 -0.03 -0.01 collects all cases where an authorship verification approach benefits from obfuscation. However, not all of these cases spell out failure for the obfuscation approaches: – Reason 1. Negative impacts greater than -0.1 can be explained by random variation of the authorship verification approach. – Reason 2. Authorship verification approaches which have an a priori performance of less than 0:6 acc may perhaps not be trusted to make reasonable decisions in the face of text alterations. – Reason 3. High improvements on the PAN13 dataset are easily obtained by flipping only a few decisions, since the dataset is very small compared to the others. – Reason 4. In three cases, the high negative obfuscation impacts on Maitra’s [69] approach go along with a high negative accuracy performance delta ( acc), which indicates that obfuscation has caused this verifier to shift a high number of its decisions from “different author” to “same author,” but also on problem instances where no obfuscation has been applied (i.e., problems whose documents have indeed been written by different authors). This suggests that the decisions of Maitra’s approach are not independent of one another but relative to decisions on other problem instances, which must be considered erroneous behavior.
These reasons notwithstanding, some cases remain where a priori well-performing obfuscators significantly improve after obfuscation, and the question remains why this is the case. It remains unclear whether these are random fluctuations which are to be expected when applying many authorship verification approaches implemented by as many different people onto obfuscated datasets, or whether something like this can be expected in practice as well. After applying all these rules of the thumb, only three cases remain unexplained, namely the negative obfuscation impact of Mihaylova et al.’s obfuscator on Kocher’s verifier on the dataset PAN14 EN, and that of Mansoorizadeh et al.’s obfuscator on Mechti’s verifier and Modaresi’s verifier on the dataset PAN14 EN. Since in all three cases, the PAN14 EN dataset is involved, it’s characteristics may cause this behavior, but this is just a speculation.
Altogether, we draw three conclusions from these results: (1) even basic author obfuscation can already achieve some degree of safety against state-of-the-art automatic forensic authorship analyses, (2) the state of the art in authorship verification is extremely vulnerable to obfuscation, and (3) obfuscation approaches must be evaluated in as many different situations as possible to identify odd behavior of both the obfuscator as well as authorship verifiers. The latter conclusion is particularly important since it forces us to now take the effectiveness of authorship verification technology reported in the literature with a grain of salt, especially in adversarial scenarios: its application in court is rendered doubtful, since texts of disputed authorship may have been tampered with to influence the court’s decision. 5.2
A human assessor skimmed through a random subset of ten obfuscated texts for each approach using the aforementioned visual analytics tool outlining the differences of the original and obfuscated texts (some example screenshots are given in Section 4). The Verifier Team
[Reference]
Bagnall [
Section 3 final acc
Section 3 formances and performance deltas of various authorship verification approaches submitted to PAN 2013 through PAN 2015 when run on test datasets that have been obfuscated by this obfuscator. Verifiers that failed to process a dataset (e.g., for being incompatible or not scalable) have been omitted from the tables. Verifiers whose optimal classification threshold classification accuracy acc on the unobfuscated test dataset turned out to be negative or positive infinity (i.e., marking all problem instances “same author” or “different author”, respectively) were omitted from forming the average performances reported in Table 1. PAN 2014 EN test dataset final acc
Section 3 final acc
Section 3 formances and performance deltas of various authorship verification approaches submitted to PAN 2013 through PAN 2015 when run on test datasets that have been obfuscated by this obfuscator. Verifiers that failed to process a dataset (e.g., for being incompatible or not scalable) have been omitted from the tables. Verifiers whose optimal classification threshold classification accuracy acc on the unobfuscated test dataset turned out to be negative or positive infinity (i.e., marking all problem instances “same author” or “different author”, respectively) were omitted from forming the average performances reported in Table 1. PAN 2014 EN test dataset PAN measures AUC final acc
Section 3 final acc
Section 3 performances and performance deltas of various authorship verification approaches submitted to PAN 2013 through PAN 2015 when run on test datasets that have been obfuscated by this obfuscator. Verifiers that failed to process a dataset (e.g., for being incompatible or not scalable) have been omitted from the tables. Verifiers whose optimal classification threshold classification accuracy acc on the unobfuscated test dataset turned out to be negative or positive infinity (i.e., marking all problem instances “same author” or “different author”, respectively) were omitted from forming the average performances reported in Table 1. assessor’s observation on these sample cases was that most of the paraphrased texts of each particular approach have a very similar characteristic with respect to sensibleness and soundness. The final decision then was to base the manual assessment on just one random text from each year of PAN’s test datasets, excluding the original texts from language learners (their original text quality might already be suboptimal): in-depth manual assessment was performed on problem instances 5, 134, and 429.
The human assessor started by reading the obfuscated texts without knowing which was the output of what approach. During this reading phase, the assessor marked up errors (typos, grammar) and assigned school grades (on a scale from 1 (excellent) to 5 (fail)) for the sensibleness of each of the sample problem instances. As a result, the obfuscated texts of Mansoorizadeh et al.’s approach got a grade 2 for all three cases mainly due to the many punctuation problems where a white space was inserted before every punctuation mark. The texts of Mihaylova et al.’s obfuscator all got a grade 4 due to the many grammatical errors, capitalizing issues (lower-case sentence starts, capitalized words in the middle of sentences), punctuation problems (many missing or useless punctuations), and consistent typos (“tto” and “oof”). The assessor noted that the texts were difficult to read due to the many problems but that grade 4 was given to show the difference to the even worse texts of Keswani et al.’s obfuscator. For Keswani et al.’s approach, our assessor noted that the texts were impossible to read or understand with lots of grammatical errors, capitalization problems at sentence beginnings, etc. The assessor even wanted to stop reading before finishing the whole text due to the “painful” experience.
After grading the sensibleness of the obfuscated texts, the assessor read the original texts and used the visual analytics tool highlighting the textual differences in various ways to assess the soundness of the obfuscated texts on a three-point scale as correct, passable, or incorrect. The obfuscated texts of Mihaylova et al.’s and Keswani et al.’s approaches were both judged “incorrect” for all three cases since they are almost impossible to read. Mihaylova et al.’s obfuscator might produce slightly more sound texts than Keswani et al.’s, yet, the assessor did not want to assign a “passable” to any of these but suggested to further differentiate the point scale as a future evaluation improvement. Not that surprising, Mansoorizadeh et al.’s very conservative approach achieved “correct” as a score for the problem instances 134 and 429. However, some of the changes for problem instance 5 did change the meaning a lot (“car” instead of “machine” when a computer was meant, etc.) such that for problem instance 5, Mansoorizadeh et al.’s approach only got a “passable” score. Three groups participated in the peer evaluation; one external group (Liebeck et al. [67]) and two of the author obfuscation task participants (Mansoorizadeh et al. and Mihaylova et al.).
As for the safety evaluation, all three groups employ the GLAD authorship
verification approach [
As for soundness, Liebeck et al. suggest an automatic measure based on approaches submitted to the shared tasks on semantic textual similarity (STS) organized at SemEval, where an optimum soundness would achieve a score of 5 and the worst soundness a score of 0. In Liebeck et al.’s evaluation, all three obfuscation approaches achieve scores above 4, where Mansoorizadeh et al. performs almost perfect and Keswani et al. the worst. Still, given our own soundness evaluation, the scores for the approaches of Keswani et al. and Mihaylova et al. with 4+ out of 5 seem a too optimistic and should be much closer to 0. Similarly, also the peer review of Mansoorizadeh et al. scores soundness from 5 down to 0 but by a manual judgment. Here, their own obfuscation approach outperforms the others by far (4.86), while Mihaylova et al. achieve a 3.93 and Keswani et al. a 2.02; the same ordering as in Liebeck et al.’s evaluation but probably also rather high scores for the two worse approaches. The peer review of Mihaylova et al. is also based on manual scoring for soundness and seems to come to the conclusion that their own approach is the best, followed by Mansoorizadeh et al.’s obfuscations. However, this seems to be a rather biased outcome since our own manual soundness evaluation showed the soundness of Mihaylova et al.’s obfuscation approach to be inferior to Mansoorizadeh et al.’s approach.
As for sensibleness, all three peer evaluators opt for a manual analysis and rank the obfuscations of Mansoorizadeh et al. clearly more sensible than Mihaylova et al.’s that again are more sensible than Keswani et al.’s obfuscations. Interestingly, Mansoorizadeh et al. employ a scale from 0–5 with 5 as most sensible, Liebeck et al. employ a three point scale, while Mihalyova et al. only have a two point scale (sensible or not) and grade half of a sample of their own approach’s obfuscations as sensible. Just as is the case for Mihalyova et al.’s peer review of soundness, this somewhat contradicts our own manual sensibleness evaluation, but in sum the ordering of the approaches of all three peer reviews is consistent with our own sensibleness grading.
Not surprisingly, the approach of Mansoorizadeh et al. that hardly changes anything in a text, except for introducing many spaces before punctuation marks, achieves good and very good scores for sensibleness and soundness but is the least safe of the tested obfuscators. Although hardly being sound or sensible, the texts produced by the safest obfuscator of Mihaylova et al. have a slightly better quality compared to the roundtrip translations produced by Keswani et al.’s obfuscator. Most of the three external peer reviews agree with these evaluation results at least on the relative ordering of the individual obfuscators. 6
We have conducted the first large-scale evaluation of author obfuscation approaches in terms of their safety against the state of the art in authorship verification. A total of 44 verification approaches have been tested as to their vulnerability to obfuscation, and we found that many of them are indeed vulnerable to a greater or lesser extent. Moreover, for the first time, we have shown that author obfuscation technology can take on many authorship verification approach simultaneously, which is a must if this technology is supposed to be useful in practice. The best-performing obfuscator flips on average about 47% of an authorship verifier’s decisions towards choosing “different author” when the opposite decision would have been correct. The obfuscation approaches evaluated have been collected via a shared task on author obfuscation that we organized at PAN 2016; three obfuscators have been submitted which are now hosted on the TIRA evaluation-as-a-service platform, ready for re-evaluation against new authorship verification approaches. Furthermore, we have systematically reviewed the literature on author obfuscation and collected and organized for the first time its evaluation methodology, introducing the three main performance dimensions of an author obfuscator: safety, soundness, and sensibleness.
There are still many open challenges when it comes to evaluating author obfuscation approaches properly and at scale, some requiring original research into new technologies that are capable of recognizing paraphrases, textual entailment, grammaticality, and style deception. Conceivably, approaches to these problems can be devised which are tailored to the evaluation of author obfuscation approaches and therefore exploit certain aspect of this application domain to achieve better performance than in the general case. We leave a more detailed investigation in this direction for future work.
We thank the participating teams of this shared task. Our special thanks go to Adobe Systems Inc. for sponsoring the event. 72. McDonald, A., Afroz, S., Caliskan, A., Stolerman, A., Greenstadt, R.: Use Fewer Instances of the Letter "i": Toward Writing Style Anonymization. In: Fischer-Hübner, S., Wright, M. (eds.) Privacy Enhancing Technologies - 12th International Symposium, PETS 2012, Vigo, Spain, July 11-13, 2012. Proceedings. Lecture Notes in Computer Science, vol. 7384, pp. 299–318. Springer (2012), http://dx.doi.org/10.1007/978-3-642-31680-7_16 73. McDonald, A., Ulman, J., Barrowclift, M., Greenstadt, R.: Anonymouth Revamped: Getting Closer to Stylometric Anonymity. In: Kapadia, A., Caine, K., Camp, L., Lee, A., Patil, S., Reiter, M., Staddon, J. (eds.) Proceedings of the Workshop on Privacy Enhancing Tools PETools, Bloomington, Indiana, USA, July 9, 2013. (2013) 74. Mechti, S., Jaoua, M., Faiz, R., Belguith, L., Bsir, B.: On the Empirical Evaluation of
Author Identification Hybrid Method—Notebook for PAN at CLEF 2015. In: [
Clustering—Notebook for PAN at CLEF 2014. In: [
Parameters using a Genetic Algorithm—Notebook for PAN at CLEF 2014. In: [
Author Verification—Notebook for PAN at CLEF 2015. In: [
Features—Notebook for PAN at CLEF 2013. In: [
Identification—Notebook for PAN at CLEF 2015. In: [
Using Syntactic N-grams—Notebook for PAN at CLEF 2015. In: [
In: Gurrin, C., He, Y., Kazai, G., Kruschwitz, U., Little, S., Roelleke, T., Rüger, S., van Rijsbergen, K. (eds.) Advances in Information Retrieval. 32nd European Conference on Information Retrieval (ECIR 10). Lecture Notes in Computer Science, vol. 5993, p. 672.
Springer, Berlin Heidelberg New York (Mar 2010) 91. Rao, J., Rohatgi, P.: Can Pseudonymity Really Guarantee Privacy? In: Bellovin, S., Rose, G. (eds.) 9th USENIX Security Symposium, Denver, Colorado, USA, August 14-17, 2000. USENIX Association (2000), https://www.usenix.org/conference/9th-usenix-securitysymposium/can-pseudonymity-really-guarantee-privacy 92. Rosso, P., Rangel, F., Potthast, M., Stamatatos, E., Tschuggnall, M., Stein, B.: Overview of PAN’16—New Challenges for Authorship Analysis: Cross-genre Profiling, Clustering, Diarization, and Obfuscation. In: Fuhr, N., Quaresma, P., Larsen, B., Gonçalves, T., Balog, K., Macdonald, C., Cappellato, L., Ferro, N. (eds.) Experimental IR Meets Multilinguality, Multimodality, and Interaction. 7th International Conference of the CLEF Initiative (CLEF 16). Springer, Berlin Heidelberg New York (Sep 2016) 93. Sari, Y., Stevenson, M.: A Machine Learning-based Intrinsic Method for Cross-topic and
Cross-genre Authorship Verification—Notebook for PAN at CLEF 2015. In: [
CLEF 2013. In: [
B.: Overview of the Author Identification Task at PAN 2015. In: [
Author Identification—Notebook for PAN at CLEF 2013. In: [
Verification—Notebook for PAN at CLEF 2014. In: [
Cost in Author Verification—Notebook for PAN at CLEF 2015. In: [
In: Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09). pp. 176–179. Association for Computational Linguistics, Paris, France (October 2009), http://www.aclweb.org/anthology/W09-3827 109. Wagner, J., Foster, J., van Genabith, J.: Judging grammaticality: Experiments in sentence classification. CALICO Journal 26(3), 474–490 (2009) 110. Weese, J., Ganitkevitch, J., Callison-Burch, C.: Paradigm: Paraphrase diagnostics through grammar matching. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. pp. 192–201. Association for Computational Linguistics, Gothenburg, Sweden (April 2014), http://www.aclweb.org/anthology/E14-1021 111. Wong, S.M.J., Dras, M.: Parser features for sentence grammaticality classification. In: Proceedings of the Australasian Language Technology Association Workshop 2010. pp. 67–75. Melbourne, Australia (December 2010) 112. Xu, W., Ritter, A., Dolan, B., Grishman, R., Cherry, C.: Paraphrasing for style. In: Proceedings of COLING 2012. pp. 2899–2914. The COLING 2012 Organizing Committee, Mumbai, India (December 2012), http://www.aclweb.org/anthology/C12-1177 113. Zamani, H., Abnar, S., Dehghani, M., Forati, M., Babaei, P.: Submission to the Author Identification Task at PAN 2014. http://www.uni-weimar.de/medien/webis/events/pan-14 (2014), http://www.clef-initiative.eu/publication/working-notes, From the University of Tehran, Iran 114. Zheng, R., Li, J., Chen, H., Huang, Z.: A framework for authorship identification of online messages: Writing-style features and classification techniques. Journal of the American Society for Information Science and Technology 57(3), 378–393 (2006)