Quantifying Iconicity in 940K Online Circulations of 26 Iconic Photographs

Abstract

What impact do digital media have on the creation, selection, distribution, reception and meaning of iconic photographs? Recent studies have suggested that digital circulation, especially in a memeified form, might lead to an 'erosion,' 'fracturing,' or 'collapsing' of the original context and meaning of iconic pictures. Using a close reading methodology, these studies are necessarily based on a limited sample - in number, period and geographic distribution - of online circulations. Introducing a distant reading methodology to the study of iconic photographs, this paper applies the Google Cloud Vision API to retrieve 940K online circulations of 26 iconic images between 1995 and 2020. We operationalize the 'loss of meaning/context' hypothesis by using document embeddings to study the relationship between the iconic photographs and the text surrounding them on the webpage. Based on this distant reading, we argue that the digital circulation of iconic photographs is comprised of similar contextual, self-referential and non-referential combinations of images and texts.

Keywords

• eol>iconic photographs
• data mining
• image-text analysis
• top2vec
• document embedding

1. Introduction

An exploding Zeppelin; a Buddhist monk engulfed in flames; a portrait of a young Cuban revolutionary; an astronaut taking man’s first steps on the moon; a protester blocking a tank; President Obama in the situation room. For many readers, these textual sketches conjure up a group of well-known iconic images: photographs that, in an often-quoted definition, are ‘widely recognised and remembered, are understood to be representations of historically significant events, activate strong emotional identification or response, and are reproduced across a range of media, genres, or topics.’ [ 8 ]

Traditionally, the theoretical concept of the iconic photograph has mostly been tied to twentieth-century top-down mass media, such as the newspaper, the illustrated magazine, and television [ 23, 8 ]. In recent years, scholars have started to debate the efects of digital media on the creation, selection, distribution, reception, and meaning of iconic images [ 3, 21, 6, 9, 17 ]. What happens to older iconic photographs when they are circulated online and how do digital media impact the formation of new iconic imagery?

While answers to these questions vary, most scholars agree that digital media diminish the power of the iconic image. The digital circulation of digitized and born-digital iconic pictures is described as ‘trivializing’ [ 3 ], ‘decontextualizing’ [ 21 ], ‘eroding’ [ 6 ], ‘fracturing’ [ 17 ] or ‘collapsing’ [ 16 ] the original context and meaning of iconic images. Meme-ification of photographic icons is seen as the most extreme manifestation of this process. Memes collapse the ‘original historical and biographical contexts’ of the iconic picture [ 16 ], they threaten to ‘destroy’ its original meaning and, as a result, its ‘political and ethical significance’ [ 3 ], and memes ‘poach’ the original meaning of the icon and ‘supplement it with new interpretations that typically deviate from the main narrative behind the famous image’ [ 17 ].

Most studies on iconic imagery, both of- and online, are based on a close reading of the iconic image itself or a limited sample – in number, period and geographic distribution – of circulations [ 7, 11, 24, 3, 10 ]. Introducing a combination of ‘distant reading’ [ 20 ] and ‘distant viewing’ [ 2, 26 ] methodologies to the study of iconic images, our project uses the Google Cloud vision Application Programming Interface (GCV API) to retrieve 940K online circulations, remediations and appropriations of 26 iconic photographs between 1995 and 2020 (see table 1). Emphasizing how meaning is created by an interplay of images and texts, we apply several computational techniques to test if iconic images lose their original iconic meaning in the digital realm.

This short paper presents the first two steps of the project. First, we present the process of data gathering and harmonization. We use the GCV API to find circulations of 26 photographs that are widely described as being iconic in secondary literature [ 12 ]. By re-uploading identified circulations to the API, we partly mitigate the temporal bias of the GCV API and were able to find circulations that stretch back until the early days of the internet. The API returns metadata of the URLs where the images can be found, as well as the titles of the page and labels assigned to the images. Additionally, we scraped the webpages where the images can be found and extract the data, language and full-text on the page. Upon completion of the project, we will release a data-set that allows for further study of the online life of iconic imagery and comparison over time.

We see the meaning of an iconic image as the product of the reciprocal, or ‘dialectical,’ interplay between image and text[ 18 ]. We are currently applying several computer vision techniques to discover large-scale visual patterns in the 940K images in our corpus. In this paper, however, we operationalize the ‘loss of meaning/context’ hypothesis mentioned above by focusing on the text. Using document embedding clustering, we posit that ‘loss of context’ can be measured by the prominence of clusters that refer to the original context. For example, a contextual digital circulation of the famous ‘accidental napalm’ photograph will be surrounded by textual references to the Vietnam War, while a non-contextual and/or memeified version will lack these references.

2. Related work

Visual search engines, such as Google Image Search (since 2007) and TinyEye (since 2008) allow users to track where images are published online. Humanities scholars have used these services to map the online circulation of images, for example by tracking the reuse of paintings uploaded by the British National Gallery [ 13 ]. Others have used reverse image search to track the digital afterlife of an iconic photograph of a Swedish woman hitting a neo-Nazi with a handbag [ 16 ]. More recently, scholars have started to apply the GCV API instead of the interface to map the digital ‘cross-platform circulation’ of images of the 2018 FIFA World Cup Final Draw [ 5 ]. Building on this last study, we found that neither Google’s interface nor the GCV API give a representative overview of where an image is published on the web. Being a search engine, the results that Google shows are especially biased towards the ‘recent’ internet. Because we are especially interested in developments over time, we developed a pipeline to circumnavigate this problem.

Despite the observation that most modern media, such as the newspaper, the television and the internet, are ‘multimodal’ – i.e. consist of combinations of text, images and audio – digital and computational humanities research has mostly been applied to discover large-scale patterns of meaning in text(s) [ 26 ]. Following the theoretical concept of image-text, Wevers e.a have applied jointly used computer vision and NLP techniques to study the patterns of meaning in a large corpus of advertisements [ 27 ]. We are currently applying computer vision techniques to study the 940K images in our corpus. This short paper, however, presents only the textual side of our project.

3. Corpus

Researchers of iconic photographs have made a distinction between national and global icons, while also noting the possible overlap between these two sets of images [ 22, 4 ]. Based on these kind of studies, Van der Hoeven (2019) set out to discover if some iconic photographs are part of global visual memory: ‘a limited set of images that people all over the world have seen and remembered’[ 12 ]. Based on a literature review, he presents a list of twenty-six iconic photographs that are widely described as iconic. While there are other lists of photographs that are frequently described as ‘iconic,’ such as Wikipedia’s “List of Photographs Considered the Most Important” [14], we decided to use Van der Hoeven’s (2019) list as our corpus because it is derived from the academic discussion of iconic photographs (see Table 1).

4. Data gathering and harmonization

Our data gathering pipeline consists of two parts. First, an image is uploaded to the GCV API, which enables users to apply all sorts of computer vision techniques in the cloud. Our pipeline relies on the basic functionality of the API to find full and partial circulations of the uploaded image on the web. The API returns a list of web addresses (URLs) that contain the iconic image. Because circulations of the iconic image on these URLs are often slightly diferent than the version we uploaded, they can be used as input for the second iteration. Using this iterative process, the pipeline not only finds more images but also less recent ones (see figures 1 and 2). The second part of the pipeline includes several methods to scrape the URLs returned in the first part and collects (meta)data, such as the HTML time-tags and the language of the webpage. Although the pipeline can be used to map the dissemination of images online, it also has some shortcomings. First, the algorithms behind the GCV API are proprietary, which makes it impossible to know what percentage of the images are indexed and if specific parts of the internet, social media for example, might be relatively under- or overrepresented. Furthermore, we also don’t know which percentage of indexed URLs containing the uploaded image are returned by the API. Second, the pipeline only returns URLs that are online, meaning that many previous circulations of iconic photographs, for example from the early 2000s, will not be retrieved by it. As a result, our data-set carries a specific time stamp (the date on which our pipeline scraped the URL). By releasing our data-set (upon completion of the project), we hope to enable more sound historical comparisons in the future.

5. Theoretical Background

In their influential book

No Caption Needed, Robert Hariman and John Louis Lucaites rely on the work of visual culture studies scholar W.J.T. Mitchell to describe how the meaning of iconic images is constructed [ 10 ]. Mitchell argued that the meaning of an image can only be deduced by looking at it, or reading it, in relation to its surrounding (con)text: “The interaction of pictures and text is constitutive of representation as such: All media are mixed media, and all representations are heterogeneous” [ 19 ]. In an overview of his work, Mitchell identified three main types of the intertwined ‘dialectical constellations’ between images and texts: “‘imagetext’ (if word and image are seamlessly united), image-text (if they are distinct but connected) and image/text (if they are in conflict or tension)” [ 18 ].

Our data-set provides a unique opportunity to study the relation between images and texts, because the meaning of (roughly) the same 26 images is constructed over and over again, 940K times in fact, by diferent texts. Using Mitchell’s concepts, we see iconic “imagetexts” as those constellations where the text refers to original historical referent. For example, the ‘accidental napalm’ photograph is used to say something about the Vietnam War. Iconic “image-texts” are marked by texts that refers to concepts that fall outside what is shown on the image, but are still connected to it. For example, an iconic image is used to say something about iconicity itself, image manipulation, or the power of photography. Third, iconic “image/texts” display a tension between the image and the text.

Memeified versions, where the text is entirely non-referential, would fall within this category.

6. Methodology

We hypothesize that we can understand the three diferent iconic “image/-text” types by the presence or absence of specific clusters. Several techniques could be used to gain insight into the semantic context of the online circulations of the iconic images. We originally looked into topic modelling, as it is widely used and has shown its value in humanities research [ 25 ]. However, topic modelling has several downsides. For example, relying on bag-of-words representations of documents, LDA ignores the order of words. Second, determining the optimal number of topics remains, in most cases, educated guesswork. In order to circumvent these issues, we apply a modified version of the recently proposed top2vec method, which uses joint document and word semantic embedding and Hierarchical Density-Based Spatial Clustering to find topic vectors in a corpus [1]. For preprocessing, we applied lowercasing, removed all non-alphanumeric tokens and removed webpages with less than fifty tokens. This last step was especially important because it removes many of the “404 not found” and “you need javascript to view this page” .html files from our corpus. As a result of computational limits, we only trained document embeddings on English-language URLs, roughly round 70%) of our corpus, and on samples of 15.000 webpages per iconic photograph (if the number of URL’s was larger than 15.000).

The original top2vec method clusters document embeddings (trained with the popular doc2vec algorithm) by first reducing the number of dimensions using UMAP, and then clustering the (reduced) embedding space using HDBSCAN, a density-based clustering algorithm [ 15 ]. Because doc2vec embeds both documents and words in a vector space, the most topical cluster words (first 10) 7 troops battle germans operation surrendered stalingrad allied soviets german surrender 0 kent students campus guardsmen shootings guard state ohio nine university 9 phuc ut her she kim scars pain screaming waibel bang 3 crackdown tiananmen massacre protests beijing student chinese suppression hundreds students words for every cluster can be found by averaging the document vectors in every cluster and subsequently identifying the most similar words to the average document vector. In this way, the sets of webpages are clustered and the clusters are identified by their top terms, similar to topic modelling approaches. The advantage of the top2vec method lies in the data-driven identification of the number of clusters. The density-based clustering does not need intervention through manual setting the number of clusters. Also, the document (and word) embeddings ofer more versatility compared to LDA topic modelling and can be used in other methods, for example the comparison of documents across subsets through model alignment.

However, the use of HDBSCAN by the orginal top2vec paper results in “hard clustering,” which means that every document is assigned to one cluster. Because of the heterogeneous nature of our corpus, we decided to use a Gaussian Mixture Model (GMM) instead of HDBSCAN for clustering. GMM clustering superpositions clusters as Gaussian distributions. For every document the probability of the document belonging to cluster k is calculated, which results in a probability distribution for every document. This is important for our research specifically, because we found that hard clustering obscures the “self-referential” language of iconicity. In GMM soft clustering, words such as “photograph”, “iconic”, and “famous” are grouped together, while hard clustering disperses these words over other clusters.

One issue with GMM (soft) clustering is that it involves the manual setting of the k. For setting the number of clusters, we initially used the number of clusters returned by HDBSCAN hard clustering. However, a more common an statistically sound way of determining the number of clusters in GMM clustering is the use of the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC). We trained GMM models with 3 to 40 clusters and looked for the point where the BIC and AIC score were lowest.

7. Results

By comparing prominent clusters of all the 26 iconic photographs, we can discover several large-scale patterns in the relationship between the images and the surrounding text that play an important role in their online circulation. Most importantly, we find similar clusters for all the 26 photographs. First of all, almost all of the photographs have a imagetext topic that references the original historical event (see table 2). These clusters basically tells us what is on the picture. For example, cluster 3 of the Tank Man photograph contains the words ‘crackdown, Tiananmen, massacre, protests, Beijing’ and cluster 0 of the Kent State ‘Kent, students, camus, guardsmen, shootings.’

Next to these topics on the historical event, most iconic photographs also contain a imagetext topic that is self-referential and contains words that refer to photography, visuality and/or iconicity. For example, cluster 5 of the Reichstag photograph contains the words ‘iconic, Yevgeny, Khaldei (the first and last name of the photographer), photographer, camera’ The inclusion of the words ‘iconic, pulitzer and famous’ in cluster 6 of the Kent State photograph makes clear that this is also self-referential. Clusters 3 and 8 of the Accidental Napalm and Tank Man photographs contain many of the same words (see table 3). These self-referential clusters also often contain references to other iconic photographs. For example, next to the words photo, camera, image and other references to visuality, cluster 4 of the Abu Graib photograph contains the words ‘moon, Neil, Armstrong, Capa (the famous photographer) and napalm,’ which clearly reference other iconic pictures. Cluster 5 of the accidental napalm photograph entirely references the photograph of the drowned Syrian boy Aylan Kurdi, starting with the words ‘Kurdi, Aylan, refugee, Syrian, boy, washed, drowned.’

Thirdly, almost all photographs also include a image/text cluster that refers to memeified online circulations of the iconic photograph (see table 4). This kind of cluster not only contains words like ‘meme, memes, funny and lol’ but also names of specific memes that are apparently important for the online circulation of the iconic photograph. For example, the Kent State photograph is associated with the ‘Strutting Leo’ meme, while the Accidental Napalm photograph is connected to the ‘Swiggity Swooty, I’m coming for that booty’ meme.

Several photographs also have a cluster that points to a specific event that increased its online circulation. Cluster 6 of the Accidental Napalm photograph refers to the controversy surrounding its removal from Facebook, which cited rules concerning under-aged nudity, after a Danish newspaper posted it in 2016. Somewhat diferently, cluster 0 of the Tank Man photograph clearly indicates that it was frequently circulated in connection to the Hong Hong protest of 2019/2020 (see table 5).

8. Conclusion

This paper has presented the first two steps of our project on the digital circulation of iconic photographs. We have shown how we can use the GCV API to retrieve 940K circulations of 26 images that are frequently described as iconic in academic debates. By taking an iterative approach of reuploading images to the API, we were able to retrieve less recent circulations of the iconic images. Second, we have shown how document embeddings can be used to study the relationship between the iconic photographs and the text surrounding them. We can use this method to operationalize the ‘loss of context/meaning’ hypothesis that was put forward by several recent studies on the digital circulation of iconic pictures. While more research is needed, our results do not suggest an increasingly pronounced link between digital media and decontextualized circulation, in the form of memes or otherwise, of iconic photographs. Rather, it shows how iconic photographs are circulated in diferent distributions of contextual imagetexts, self-referential image-texts and decontextualized image/text.

In the next phases of our project we will further explore how we can use computational methods to explore the relationship between images and text in the production of meaning. First, we want to improve the textual analysis by experimenting with diferent methods and seeing how the distribution of clusters changes over time. Second, we hope to combine computer vision and text analysis to study iconicity. This entails the detection and classification of variation in the images, through for example crop detection and object detection. Moreover, we hope to combine visual and textual features in embedding models that will hopefully shed more light on the online afterlife of iconic images. [1] D. Angelov. “Top2Vec: Distributed Representations of Topics”. In: arXiv:2008.09470 [cs, stat] (Aug. 2020). arXiv: 2008.09470. url: http://arxiv.org/abs/2008.09470 (visited on 09/17/2020). [14] List of photographs considered the most important. en. Page Version ID: 961271436. June 2020. url: https://en.wikipedia.org/w/index.php?title=List_of_photographs_consider ed_the_most_important&oldid=961271436 (visited on 07/08/2020).

References

[2] T. Arnold and L. Tilton . “ Distant viewing: analyzing large visual corpora”. en . In: Digital Scholarship in the Humanities 34.Supplement_1 (Dec . 2019 ), pp. i3 - i16 . issn: 2055 - 7671 . doi: 10 .1093/llc/fqz013. (Visited on 07/01/ 2020 ).

[3] S. Boudana , P. Frosh , and A. A. Cohen . “ Reviving icons to death: when historic photographs become digital memes”. en . In: Media, Culture & Society 39 .8 ( Nov . 2017 ), pp. 1210 - 1230 . issn: 0163 - 4437 . doi: 10 .1177/0163443717690818. url: https://doi.org /10.1177/0163443717690818 (visited on 02/06/ 2020 ).

[4] A. A. Cohen , S. Boudana , and P. Frosh . “ You Must Remember This: Iconic News Photographs and Collective Memory” . en. In: Journal of Communication 68.3 (June 2018 ), pp. 453 - 479 . issn: 0021 - 9916 . doi: 10 .1093/joc/jqy017. (Visited on 02/06/ 2020 ).

[5] C. D'Andrea and A. Mintz . “ Studying the Live Cross-Platform Circulation of Images With Computer Vision API: An Experiment Based on a Sports Media Event” . en. In: International Journal of Communication 13.0 ( Apr . 2019 ). Number: 0 , p. 21 . issn: 1932 - 8036 . (Visited on 04/21/ 2020 ).

[6] N. S. Dahmen , N. Mielczarek , and D. D. Perlmutter . “ The Influence-Network Model of the Photojournalistic Icon:” en . In: Journalism & Communication Monographs (Oct. 2018 ). doi: 10 .1177/1522637918803351. (Visited on 06/29/ 2020 ).

[7] P. Hagopian . “ Vietnam war photography as a locus of memory” . In: Locating memory: Photographic acts 4 ( 2006 ). Publisher: Berghahn Books New York, p. 201 .

[8] R. Hariman and J. L. Lucaites . No Caption Needed: Iconic Photographs, Public Culture, and Liberal Democracy. en. University of Chicago Press, June 2007 . isbn: 978 -0- 226 - 31606-2.

[9] R. Hariman and J. L. Lucaites . “ Predicting the Present: Iconic Photographs and Public Culture in the Digital Media Environment” . en. In: Journalism & Communication Monographs 20.4 ( Dec . 2018 ), pp. 318 - 324 . issn: 1522 - 6379 . doi: 10 .1177/1522637918803354. (Visited on 06/11/ 2020 ).

[10] R. Hariman and J. L. Lucaites . “ Public Identity and Collective Memory in U.S. Iconic Photography: The Image of ”Accidental Napalm”” . In: Critical Studies in Media Communication 20 .1 ( Jan . 2003 ). Publisher: Routledge _eprint: https://doi.org/10.1080/0739318032000067074, pp. 35 - 66 . issn: 1529 - 5036 . doi: 10 .1080/0739318032000067074. (Visited on 05/24/ 2020 ).

[11] J. M. Harris . “ Icons of Horror: Three Enduring Images from the Vietnam War” . en. In: The Palgrave Handbook of Artistic and Cultural Responses to War since 1914: The British Isles, the United States and Australasia. Ed. by M. Kerby , M. Baguley , and J. McDonald . Cham: Springer International Publishing, 2019 , pp. 181 - 198 . isbn: 978 -3- 319 -96986-2. doi: 10 .1007/978-3- 319 -96986-2_ 11 . (Visited on 06/18/ 2020 ).

[12] R. van der Hoeven. The Global Visual Memory: A Study of the Recognition and Interpretation of Iconic and Historical Photographs . English. Universiteit Utrecht , 2019 . isbn: 978 - 90 -393-7149-7.

[13] I. Kirton and M. Terras . “ Where do images of art go once they go online? A reverse image Lookup study to assess the dissemination of cultural heritage” . In: Selected papers from Museums and the Web North America , 2013 ( 2013 ), pp. 237 - 248 .

[15] L. McInnes , J. Healy , and S. Astels . “hdbscan: Hierarchical density based clustering” . In: Journal of Open Source Software 2 .11 ( 2017 ), p. 205 .

[16] S. Merrill . “Following The Woman with the Handbag: Mnemonic Context Collapse and the Anti-fascist Activist Appropriation of an Iconic Historical Photograph” . In: Social Movements, Cultural Memory and Digital Media . Ed. by S. Merrill , E. Keightley, and P. Daphi. Palgrave Macmillan , 2020 , pp. 111 - 139 .

[17] N. Mielczarek . “ The Situation Room icon and its Internet memes: Subversion of the Osama bin Laden raid and fragmentation of iconicity in remix culture.” en . In: First Monday (May 2020 ). issn: 1396 - 0466 . doi: 10 .5210/fm.v25i6. 10531 . (Visited on 06/29/ 2020 ).

[18] W. J. T. Mitchell . “ Pictorial Turn” . In: Visual Global Politics. Ed. by R. Bleiker . London: Routledge, 2018 , pp. 230 - 233 .

[19] W. J. T. Mitchell . Picture Theory. Essays on Verbal and Visual Representation. ENG . Chicago: Chicago University Press, 1994 .

[20] F. Moretti . Graphs, maps, trees: abstract models for literary history . English. London: Verso , 2007 . isbn: 978 -1- 84467 -185-4.

[21] M. Mortensen . “Constructing, confirming, and contesting icons: the Alan Kurdi imagery appropriated by #humanitywashedashore, Ai Weiwei, and Charlie Hebdo” . en. In: Media, Culture & Society 39 .8 ( Nov . 2017 ). Publisher: SAGE Publications Ltd , pp. 1142 - 1161 . issn: 0163 - 4437 . doi: 10 .1177/0163443717725572. (Visited on 04/02/ 2020 ).

[22] G. Paul. Das Jahrhundert der Bilder . German. Göttingen: Vandenhoeck & Ruprecht, 2009 . isbn: 978 -3- 525 -30011-4 978 - 3 - 525 -30012-1.

[23] D. D. Perlmutter . Photojournalism and foreign policy: icons of outrage in international crises . English. Westport, Conn.: Praeger , 1998 . isbn: 978 -0- 275 -96362-0.

[24] C. Rouquet . “ Creation and Afterlife of the Iconic Photographs of the Vietnam War” . In: Arts of War and Peace 2 .1 ( 2014 ), pp. 1 - 14 .

[25] M. Tolonen et al. “ A Quantitative Approach to Book-Printing in Sweden and Finland, 1640 - 1828 ”. en. In: Historical Methods: A Journal of Quantitative and Interdisciplinary History 52 .1 ( Jan . 2019 ), pp. 57 - 78 . issn: 0161 - 5440 , 1940 - 1906 . doi: 10 .1080/01615440 . 2018 . 1526657 . url: https://www.tandfonline.com/doi/full/10.1080/01615440. 2018 . 152 6657 (visited on 07/27/ 2020 ).

[26] M. Wevers and T. Smits . “ The Visual Digital Turn. Using Neural Networks to Study Historical Images” . In: Digital Scholarship in the Humanities 35.1 ( 2020 ), pp. 194 - 207 . doi: https://doi.org/10.1093/llc/fqy085.

[27] M. Wevers , T. Smits , and L. Impett . “ Modeling the Genealogy of Imagetexts: Studying Images and Texts in Conjunction using Computational Methods”. en-us . In: Mexico City, June 2018 , p. 1 . url: https://dh2018.adho. org/modeling-the-genealogy-of-imagetexts -studying-images-and-texts-in-conjunction-using-computational-methods/ (visited on 09/30/ 2020 ).