The participating approach predicts whether a tweet, which is accompanied by multimedia content (image/video), is trustworthy (real) or deceptive (fake). We combine two di erent methods a) one using a semi-supervised learning scheme that leverages the decisions of two independent classi ers to produce a decision and b) one using textual patterns to extract claims about whether a post is fake or real and attribution statements about the content source. The experiments, carried out on the Verifying Multimedia Use dataset, used di erent combinations of content quality and trust-oriented features, namely tweet-based, user-based and forensics.
After high-impact events, large amounts of unveri ed
information usually start spreading in social media. Often,
misleading information is getting viral a ecting public
opinion and sentiment [
To tackle this challenge, we present a method combining
two approaches. The rst approach is an extension of [
Being an extension of [
The classi ers are built using three types of features: a) tweet-based (TB), which use the post's metadata, b) userbased (UB), which use the user's metadata, c) multimedia forensics features (FOR), which are computed for the image that accompanies the post. Except for the ones shared by the task, we extract and use additional ones on each set. TB: Binary features such as the presence of a word, symbol or external link are added to the list. We also use languagespeci c binary features that correspond to the presence of speci c terms; for languages, in which we cannot manage to de ne such terms, we consider these values as missing. We perform language detection with a publicly available library1. We add a feature for the number of slang words in a text, using slang lists in English2 and Spanish3. For the number of nouns, we use the Stanford parser4 to assign parts of speech to each word (supported only in English) and for text readability, the Flesch Reading Ease method5, which computes the complexity of a piece of text as a score in the interval (0: hard-to-read, 100: easy-to-read). UB: We extract user-speci c features such as the number of media items, the account age and others that summarize information shared by the user. For example, we check whether the user shares a location and whether this can be matched to a city name from the Geonames dataset6.
For both TB and UB features, we adopt trust-oriented
features for the links shared, through the post itself (TB) or
the user pro le (UB). The WOT metric7 is a score
indicating how trustworthy a website is, using reputation ratings
by Web users. We also include the in-degree and harmonic
centralities, rankings computed based on the links of the
web forming a graph8. Trust analysis of the links is also
performed using Web metrics provided by the Alexa API.
FOR: Following the method in [
This approach is motivated by the human veri cation process employed by journalists, where attributed sources are key to trustworthiness of claims. A classic natural language processing pipeline is employed, involving text tokenization, Parts of Speech (POS) tagging9 and a permissive named entity recognition pattern focussing on noun phrases. A number of regex patterns were created to extract typical linguistic constructs around image and video content, such as debunking reports, claims of being real or attribution to a third party source such as a news provider.
Our approach is semi-automated, using a list of a priori known trusted and untrusted sources. We can either learn an 1https://code.google.com/p/language-detection/ 2http://onlineslangdictionary.com/word-list/0-a/ 3http://www.languagerealm.com/spanish/spanishslang.php 4http://nlp.stanford.edu/software/lex-parser.shtml 5http://simple.wikipedia.org/wiki/Flesch_Reading_Ease 6http://download.geonames.org/export/dump/cities1000.zip 7https://www.mywot.com/ 8http://wwwranking.webdatacommons.org/more.html 9http://www.cis.uni-muenchen.de/~schmid/tools/TreeTagger entity list automatically using information theoretic weightings (i.e., TF-IDF) or create a list manually (i.e., using a journalist's trusted source list). All news providers have long lists of trusted sources for di erent regions around the world so this information is readily available. For this task we created a list of candidate named entities by rst running the regex patterns on the dataset. We then manually checked each entity via Google search (e.g., looking at Twitter prole pages) to determine if they were obvious news providers or journalists.
We assign a con dence value to each matched pattern based on its source trustworthiness level. Evidence from trusted authors is more trusted than evidence attributed to other authors, which is more trusted than unattributed evidence. In a cross-check step we choose the most trustworthy claims to use for each image URI. If there is evidence for both a fake and genuine claim with an equal con dence we assume it is fake (i.e., any doubt = fake). Our approach provides a very high precision, low recall output.
We submitted ve runs that explore di erent combinations of features (TB, UB, FOR) and methods (ARM, ATT). Table 1 shows the speci c run con gurations and performance.
In RUN-1 and RUN-2, we apply the ARM in which we build
CL1 and CL2 (Figure 1) by using the sets of features
speci ed in Table 1. For example, in RUN-2, we use the
concatenation of TB + FOR for CL1 and UB for CL2. RUN-3 is
a combination of ARM and ATT methods, in which we
consider for each post the result of ATT as correct if available,
otherwise we use the output of ARM. Similarly, in RUN-4, we
consider the results of ATT as samples for retraining (step (b)
in Figure 1) along with the agreed ones of ARM. All models
built in ARM use a Random Forest WEKA implementation
[
This work is supported by the REVEAL project, partially funded by the European Commission (FP7-610928). 4.