An exploding Zeppelin; a Buddhist monk engulfed in flames; a portrait of a young Cuban revolutionary; an astronaut taking man's first steps on the moon; a protester blocking a tank; President Obama in the situation room. For many readers, these textual sketches conjure up a group of well-known iconic images: photographs that, in an often-quoted definition, are 'widely recognised and remembered, are understood to be representations of historically significant events, activate strong emotional identification or response, and are reproduced across a range of media, genres, or topics.' [8] Traditionally, the theoretical concept of the iconic photograph has mostly been tied to twentieth-century top-down mass media, such as the newspaper, the illustrated magazine, and television [23,8]. In recent years, scholars have started to debate the effects of digital media on the creation, selection, distribution, reception, and meaning of iconic images [3,21,6,9,17]. What happens to older iconic photographs when they are circulated online and how do digital media impact the formation of new iconic imagery?
While answers to these questions vary, most scholars agree that digital media diminish the power of the iconic image. The digital circulation of digitized and born-digital iconic pictures is described as 'trivializing' [3], 'decontextualizing' [21], 'eroding' [6], 'fracturing' [17] or 'collapsing' [16] the original context and meaning of iconic images. Meme-ification of photographic icons is seen as the most extreme manifestation of this process. Memes collapse the 'original historical and biographical contexts' of the iconic picture [16], they threaten to 'destroy' its original meaning and, as a result, its 'political and ethical significance' [3], and memes 'poach' the original meaning of the icon and 'supplement it with new interpretations that typically deviate from the main narrative behind the famous image' [17].
Most studies on iconic imagery, both off-and online, are based on a close reading of the iconic image itself or a limited sample -in number, period and geographic distribution -of circulations [7,11,24,3,10]. Introducing a combination of 'distant reading' [20] and 'distant viewing' [2,26] methodologies to the study of iconic images, our project uses the Google Cloud vision Application Programming Interface (GCV API) to retrieve 940K online circulations, remediations and appropriations of 26 iconic photographs between 1995 and 2020 (see table 1). Emphasizing how meaning is created by an interplay of images and texts, we apply several computational techniques to test if iconic images lose their original iconic meaning in the digital realm.
This short paper presents the first two steps of the project. First, we present the process of data gathering and harmonization. We use the GCV API to find circulations of 26 photographs that are widely described as being iconic in secondary literature [12]. By re-uploading identified circulations to the API, we partly mitigate the temporal bias of the GCV API and were able to find circulations that stretch back until the early days of the internet. The API returns metadata of the URLs where the images can be found, as well as the titles of the page and labels assigned to the images. Additionally, we scraped the webpages where the images can be found and extract the data, language and full-text on the page. Upon completion of the project, we will release a data-set that allows for further study of the online life of iconic imagery and comparison over time.
We see the meaning of an iconic image as the product of the reciprocal, or 'dialectical,' interplay between image and text [18]. We are currently applying several computer vision techniques to discover large-scale visual patterns in the 940K images in our corpus. In this paper, however, we operationalize the 'loss of meaning/context' hypothesis mentioned above by focusing on the text. Using document embedding clustering, we posit that 'loss of context' can be measured by the prominence of clusters that refer to the original context. For example, a contextual digital circulation of the famous 'accidental napalm' photograph will be surrounded by textual references to the Vietnam War, while a non-contextual and/or memeified version will lack these references.
Visual search engines, such as Google Image Search (since 2007) and TinyEye (since 2008) allow users to track where images are published online. Humanities scholars have used these services to map the online circulation of images, for example by tracking the reuse of paintings uploaded by the British National Gallery [13]. Others have used reverse image search to track the digital afterlife of an iconic photograph of a Swedish woman hitting a neo-Nazi with a handbag [16]. More recently, scholars have started to apply the GCV API instead of the interface to map the digital 'cross-platform circulation' of images of the 2018 FIFA World Cup Final Draw [5]. Building on this last study, we found that neither Google's interface nor the GCV API give a representative overview of where an image is published on the web. Being a search engine, the results that Google shows are especially biased towards the 'recent' internet. Because we are especially interested in developments over time, we developed a pipeline to circumnavigate this problem.
Despite the observation that most modern media, such as the newspaper, the television and the internet, are 'multimodal' -i.e. consist of combinations of text, images and audiodigital and computational humanities research has mostly been applied to discover large-scale patterns of meaning in text(s) [26]. Following the theoretical concept of image-text, Wevers e.a have applied jointly used computer vision and NLP techniques to study the patterns of meaning in a large corpus of advertisements [27]. We are currently applying computer vision techniques to study the 940K images in our corpus. This short paper, however, presents only the textual side of our project.
Researchers of iconic photographs have made a distinction between national and global icons, while also noting the possible overlap between these two sets of images [22,4]. Based on these kind of studies, Van der Hoeven (2019) set out to discover if some iconic photographs are part of global visual memory: 'a limited set of images that people all over the world have seen and remembered' [12]. Based on a literature review, he presents a list of twenty-six iconic photographs that are widely described as iconic. While there are other lists of photographs that are frequently described as 'iconic,' such as Wikipedia's "List of Photographs Considered the Most Important" [14], we decided to use Van der Hoeven's (2019) list as our corpus because it is derived from the academic discussion of iconic photographs (see Table 1).
Our data gathering pipeline consists of two parts. First, an image is uploaded to the GCV API, which enables users to apply all sorts of computer vision techniques in the cloud. Our pipeline relies on the basic functionality of the API to find full and partial circulations of the uploaded image on the web. The API returns a list of web addresses (URLs) that contain the iconic image. Because circulations of the iconic image on these URLs are often slightly different than the version we uploaded, they can be used as input for the second iteration. Using this iterative process, the pipeline not only finds more images but also less recent ones (see figures 1 and 2). The second part of the pipeline includes several methods to scrape the URLs returned in the first part and collects (meta)data, such as the HTML time-tags and the language of the webpage. Although the pipeline can be used to map the dissemination of images online, it also has some shortcomings. First, the algorithms behind the GCV API are proprietary, which makes it impossible to know what percentage of the images are indexed and if specific parts of the internet, social media for example, might be relatively under-or overrepresented. Furthermore, we also don't know which percentage of indexed URLs containing the uploaded image are returned by the API. Second, the pipeline only returns URLs that are online, meaning that many previous circulations of iconic photographs, for example from the early 2000s, will not be retrieved by it. As a result, our data-set carries a specific time stamp (the date on which our pipeline scraped the URL). By releasing our data-set (upon completion of the project), we hope to enable more sound historical comparisons in the future.
In their influential book No Caption Needed, Robert Hariman and John Louis Lucaites rely on the work of visual culture studies scholar W.J.T. Mitchell to describe how the meaning of iconic images is constructed [10]. Mitchell argued that the meaning of an image can only be deduced by looking at it, or reading it, in relation to its surrounding (con)text: "The interaction of pictures and text is constitutive of representation as such: All media are mixed media, and all representations are heterogeneous" [19]. In an overview of his work, Mitchell identified three main types of the intertwined 'dialectical constellations' between images and texts: "'imagetext' (if word and image are seamlessly united), image-text (if they are distinct but connected) and image/text (if they are in conflict or tension)" [18]. Our data-set provides a unique opportunity to study the relation between images and texts, because the meaning of (roughly) the same 26 images is constructed over and over again, 940K times in fact, by different texts. Using Mitchell's concepts, we see iconic "imagetexts" as those constellations where the text refers to original historical referent. For example, the 'accidental napalm' photograph is used to say something about the Vietnam War. Iconic "image-texts" are marked by texts that refers to concepts that fall outside what is shown on the image, but are still connected to it. For example, an iconic image is used to say something about iconicity itself, image manipulation, or the power of photography. Third, iconic "image/texts" display a tension between the image and the text. Memeified versions, where the text is entirely non-referential, would fall within this category.
We hypothesize that we can understand the three different iconic "image/-text" types by the presence or absence of specific clusters. Several techniques could be used to gain insight into the semantic context of the online circulations of the iconic images. We originally looked into topic modelling, as it is widely used and has shown its value in humanities research [25]. However, topic modelling has several downsides. For example, relying on bag-of-words representations of documents, LDA ignores the order of words. Second, determining the optimal number of topics remains, in most cases, educated guesswork. In order to circumvent these issues, we apply a modified version of the recently proposed top2vec method, which uses joint document and word semantic embedding and Hierarchical Density-Based Spatial Clustering to find topic vectors in a corpus [1]. For preprocessing, we applied lowercasing, removed all non-alphanumeric tokens and removed webpages with less than fifty tokens. This last step was especially important because it removes many of the "404 not found" and "you need javascript to view this page" .html files from our corpus. As a result of computational limits, we only trained document embeddings on English-language URLs, roughly round 70%) of our corpus, and on samples of 15.000 webpages per iconic photograph (if the number of URL's was larger than 15.000).
The original top2vec method clusters document embeddings (trained with the popular doc2vec algorithm) by first reducing the number of dimensions using UMAP, and then clustering the (reduced) embedding space using HDBSCAN, a density-based clustering algorithm [15]. Because doc2vec embeds both documents and words in a vector space, the most topical words for every cluster can be found by averaging the document vectors in every cluster and subsequently identifying the most similar words to the average document vector. In this way, the sets of webpages are clustered and the clusters are identified by their top terms, similar to topic modelling approaches. The advantage of the top2vec method lies in the data-driven identification of the number of clusters. The density-based clustering does not need intervention through manual setting the number of clusters. Also, the document (and word) embeddings offer more versatility compared to LDA topic modelling and can be used in other methods, for example the comparison of documents across subsets through model alignment.
However, the use of HDBSCAN by the orginal top2vec paper results in "hard clustering," which means that every document is assigned to one cluster. Because of the heterogeneous nature of our corpus, we decided to use a Gaussian Mixture Model (GMM) instead of HDB-SCAN for clustering. GMM clustering superpositions clusters as Gaussian distributions. For every document the probability of the document belonging to cluster k is calculated, which results in a probability distribution for every document. This is important for our research specifically, because we found that hard clustering obscures the "self-referential" language of iconicity. In GMM soft clustering, words such as "photograph", "iconic", and "famous" are grouped together, while hard clustering disperses these words over other clusters.
One issue with GMM (soft) clustering is that it involves the manual setting of the k. For setting the number of clusters, we initially used the number of clusters returned by HDBSCAN hard clustering. However, a more common an statistically sound way of determining the number of clusters in GMM clustering is the use of the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC). We trained GMM models with 3 to 40 clusters and looked for the point where the BIC and AIC score were lowest.
By comparing prominent clusters of all the 26 iconic photographs, we can discover several large-scale patterns in the relationship between the images and the surrounding text that play an important role in their online circulation. Most importantly, we find similar clusters for all the 26 photographs. First of all, almost all of the photographs have a imagetext topic that references the original historical event (see table 2). These clusters basically tells us what is on the picture. For example, cluster 3 of the Tank Man photograph contains the words 'crackdown, Tiananmen, massacre, protests, Beijing' and cluster 0 of the Kent State 'Kent, students, camus, guardsmen, shootings.'
Next to these topics on the historical event, most iconic photographs also contain a imagetext topic that is self-referential and contains words that refer to photography, visuality and/or iconicity. For example, cluster 5 of the Reichstag photograph contains the words 'iconic, Yevgeny, Khaldei (the first and last name of the photographer), photographer, camera' The 3). These self-referential clusters also often contain references to other iconic photographs. For example, next to the words photo, camera, image and other references to visuality, cluster 4 of the Abu Graib photograph contains the words 'moon, Neil, Armstrong, Capa (the famous photographer) and napalm,' which clearly reference other iconic pictures. Cluster 5 of the accidental napalm photograph entirely references the photograph of the drowned Syrian boy Aylan Kurdi, starting with the words 'Kurdi, Aylan, refugee, Syrian, boy, washed, drowned.' Thirdly, almost all photographs also include a image/text cluster that refers to memeified online circulations of the iconic photograph (see table 4). This kind of cluster not only contains words like 'meme, memes, funny and lol' but also names of specific memes that are apparently important for the online circulation of the iconic photograph. For example, the Kent State photograph is associated with the 'Strutting Leo' meme, while the Accidental Napalm photograph is connected to the 'Swiggity Swooty, I'm coming for that booty' meme.
Several photographs also have a cluster that points to a specific event that increased its online circulation. Cluster 6 of the Accidental Napalm photograph refers to the controversy surrounding its removal from Facebook, which cited rules concerning under-aged nudity, after a Danish newspaper posted it in 2016. Somewhat differently, cluster 0 of the Tank Man photograph clearly indicates that it was frequently circulated in connection to the Hong Hong protest of 2019/2020 (see table 5).
This paper has presented the first two steps of our project on the digital circulation of iconic photographs. We have shown how we can use the GCV API to retrieve 940K circulations of 26 images that are frequently described as iconic in academic debates. By taking an iterative approach of reuploading images to the API, we were able to retrieve less recent circulations of the iconic images. Second, we have shown how document embeddings can be used to study the relationship between the iconic photographs and the text surrounding them. We can use this method to operationalize the 'loss of context/meaning' hypothesis that was put forward by several recent studies on the digital circulation of iconic pictures. While more research is needed, our results do not suggest an increasingly pronounced link between digital media and decontextualized circulation, in the form of memes or otherwise, of iconic photographs. Rather, it shows how iconic photographs are circulated in different distributions of contextual imagetexts, self-referential image-texts and decontextualized image/text.
In the next phases of our project we will further explore how we can use computational methods to explore the relationship between images and text in the production of meaning. First, we want to improve the textual analysis by experimenting with different methods and seeing how the distribution of clusters changes over time. Second, we hope to combine computer vision and text analysis to study iconicity. This entails the detection and classification of variation in the images, through for example crop detection and object detection. Moreover, we hope to combine visual and textual features in embedding models that will hopefully shed more light on the online afterlife of iconic images.