A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections Florian Windhager Paolo Federico Eva Mayr Danube University Krems University of Technology Danube University Krems Krems, Austria Vienna, Austria Krems, Austria florian.windhager@donau-uni.ac.at federico@ifs.tuwien.ac.at eva.mayr@donau-uni.ac.at Günther Schreder Michael Smuc Danube University Krems Danube University Krems Krems, Austria Krems, Austria guenther.schreder@donau-uni.ac.at michael.smuc@donau-uni.ac.at http://www.europeana.eu, http://trove.nla.gov.au/, or http://dp.la/) have emerged, which aggregate cultural Abstract heritage objects across institutions, domains, and countries, and make the web the largest museum ever After decades of digitization, the web hosts a around. Yet the situation is known to be rather bleak, large scale museum, consisting of millions of when it comes to actually accessing the collected riches digital cultural objects. To balance the – not only, but especially for non-expert users, who drawbacks of parsimonious search-centric often have no idea what to expect in the digital interfaces, various approaches have been collection. The rampant problems with the widely developed to enable also visual access to these dominant search box approach to cultural object collections, and to browse and explore the collections have been thoroughly exposed and cultural richness of existing archives. This discussed [BOP82, DCW11, THC12, Whi15]. paper reviews information visualization Whitelaw retells the typical search-based visit to online approaches to digital cultural heritage collections as a bizarre purchase order situation, where collections, reflects on prominent arrangement the widely dominant information retrieval paradigm principles and design choices for digital over-successfully reduces data complexity (which in collection interfaces, and points out options the CH context is often appreciated as its own reward), for future research. thus throwing the baby out with the bath water. Rather than throwing the collection doors open and offering 1. Introduction multiple ways of access, visitors have to enter a drab (search box) lobby, which asks them “yes, what?” – From things making them smart (like tools, and urges them to come up with demands towards the achievements, or information artifacts), to things lifting unknown [Whi15]. them up (art and entertainment) – cultures collect In contrast, more generous interfaces open up the things. To share and preserve them for future digital archives’ walls, tear down the drab lobbies, and generations, populations draw artful or useful objects offer multiple ways in, where they foster free-roaming, (like texts, images, material objects, concepts, music, browsing and exploring, and support rich, or films) together. These cultural heritage (CH) serendipitous discoveries [DCW11]. We build on the collections (libraries, galleries, museums, archives) multiply proven assumption, that information contain notable works and objects – as well as visualization (InfoVis) methods and techniques can associated knowledge and data. strongly support such generous approaches. Yet With developing media technologies and according to our best knowledge, no systematic collaborations, large digital meta collections (e.g. collection of InfoVis approaches to CH collections has been undertaken until now. To close this gap, we Copyright © by the paper’s authors. Copying permitted only for private and academic purposes. review related work and outline a possible classification of InfoVis approaches and interfaces for In: W. Aigner, G. Schmiedl, K. Blumenstein, M. Zeppelzauer (eds.): digital CH collections, which aims to consolidate the Proceedings of the 9th Forum Media Technology 2016, St. Pölten, Austria, 24-11-2016, published at http://ceur-ws.org growing research field and to inform future projects. 74 A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections Figure 1: Common cultural object types (left) and common dimensions of object metadata (right). 2 Design Patterns for Interfaces to Digital should not be underestimated as a major factor for the overall success of any arts and culture mediation Object Collections initiative.1 We focus on the question how to visualize If museums, libraries, or archives are the original collection overviews and assemble relevant design three-dimensional display spaces for cultural object patterns in the following sections, which will provide collections, their spatial arrangements are generated by the categories for a more systematic recollection of a minimum of standard layouts: parallel tableaus on InfoVis interfaces further down. museum floors or in showcases, and linear arrangements along walls or shelves, ordered mostly 2.1 Close-ups, Previews and Collection Overviews due to the metadata dimensions of date, style, artist, or place of origin. Procedures of digitization extend Cultural object collections commonly contain much cultural collections (complementing physical objects more objects than could be displayed in a parallel with digital ones) – and put their visual arrangement on close-up perspective on a screen. This challenge is digital display spaces up for renegotiation. For that commonly taken on by the design of macroscopic purpose, all available metadata dimensions could be collection overviews – and their connection to vertical utilized – and furthermore encoded into novel drill down and horizontal browsing options on demand collection representations. [DCW11, GMPS00]. Figure 1 illustrates the multitude of possible digital As a review of interfaces shows, collection object types (left), and a selection of prominent overviews are usually following one of three design metadata dimensions (right), with the latter being options: Whole object collections could be represented usually formatted due to a given documentation as i) multitudes of miniature previews (thumbnails), or standard [Bac02]. This two-sided representation also mirrors the common dual nature of digital ob jects, 1 duplicating an object into a realistic image of the object Well knowing that the remote exploration of cultural (provided by a spatial layout-preserving scientific collections on screens still “doesn’t compare to being visualization procedure), and a (semi)structured, multi- there“ [RHQ14], digital interfaces mostly strive to augment dimensional metadata entry. While the realistic image and enrich traditional in situ-interaction with collections. This includes the design of approaches i) to provide allows to study cultural objects in a close up- macroscopic perspectives on high-volume collections in perspective, their accessibility in a larger collection is which patterns and relations become visible, ii) to extend either provided by a search functionality – or by visitors’ working memory to grasp large, complex datasets alternate, more generous approaches to interface often for the first time, iii) to add to richer, contextualized design, including a wide variety of InfoVis images and observations through linked data dimensions, or iv) to methods. With interfaces thus taking over the role of reduce collectors’ and curators’ biases and to facilitate museums or exhibition halls, their design determines more inclusive representations, suited for a broader user an online collections’ accessibility and impact, and group [Sul13, GMD15]. 75 A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections as ii) multitudes of abstracted visual marks only (e.g. features traditional ways of (multi)linear aggregations, dots representing objects), whose arrangement the center right column lists methods for the visual principles are laid out in section 2.2. As a third option encoding of spatial (i.e. cross-sectional, non-temporal) (iii), overviews can abstract from displaying separated metadata aspects. Here “spatial” not only refers to objects, but encode selected object attributes into the geographic aspects of metadata, but also to their visual variables of various diagrams (cf. 2.2.5), which distributions in algebraic or vector spaces. opens up the field for the use of a wide spectrum of InfoVis methods, that can support further collection 2.2 Encoding of Spatial Data Dimensions exploration too. From a user and interaction perspective, overviews Following a distinction by Kerracher et al. [KKC14], feature as natural starting or entry points to a we distinguish methods of encoding spatial data collection. They provide initial orientation, and dimensions from encoding methods for temporal (i.e. commonly enable further operations of zooming, longitudinal) data aspects, which we consider to play a filtering, and browsing to study details and close-ups crucial role for the omnipresent time-orientation of CH on demand. While these transitions between micro and collection data. Distributed across both sides of this macro perspectives pose a central challenge for distinction, we refer to the most prominent traditional interaction design, we turn to prominent arrangements spatial arrangement principle of object collections as for macroscopic overviews first. As mentioned above, (multi)linear arrangements (2.2.1), which are also this is where various dimensions of object metadata frequently chosen for digital collection interfaces. (like place of origin, date of origin, artist, topics, or styles) come into play. 2.2.1 Lists, Slideshows, Grids and Mosaics Figure 2 shows prominent arrangement principles Mirroring the sequential arrangements in physical for collection overviews: While the center left column exhibitions along walls or shelves, vertical lists or Figure 2: Principles for the visualization of cultural collections, from close-ups (left), to (multi-)linear aggregations (center left) to spatial (center right) and temporal (right) visual encoding methods. 76 A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections horizontal slideshows arrange object collections in an charts [IF:SCE], ring charts [IF:DDBV], scatter plots unilinear sequence of previews on computer screens [Man09, ABO12, IF:CG], and many more. [IF:HTA]. As multilinear arrangements, grids and These different diagrams again could be integrated mosaics arrange previews in multiple rows, to raise the into multiple coordinated views by CH collection item-screen-ratio (Fig. 2, center left). In contrast to dashboards [UTA10]. As an interesting crossover physical hangings, the guiding aspect for (multi)linear approach, diagrams could also be synthesized from arrangements can often be freely chosen amongst object previews, allowing for seamless micro-macro existing metadata dimensions, so that either date of transitions [IF:PVWF]. origin, alphabetical sequence, or even user metrics (like item popularity) determine the visible sequence of 2.3 Encoding of Temporal Data Dimensions objects on screens [IF:GCI]. Furthermore, grids and mosaics can be dynamized, so that tiles represent While maps, networks, set and other diagrams provide object categories or subcollections and change their specific insights into spatial data aspects and content over time, to enable also passive contemplation distributions, they initially offer static images for without clicking and scrolling [Whi15, para 39]. Going aggregated data only. Yet with temporal aspects (like beyond (multi)linear arrangements, several InfoVis date of origin) playing a crucial role in the domain of methods support the visual encoding and exploration CH data, most interfaces have to encode temporal of spatial (non-temporal) data aspects for whole information too. collections. 2.3.1 (Linked) Timelines 2.2.2 Geographic Maps One prominent option is to represent time linearly, As place of origin counts among the most frequently which is done with linear timelines as singular views, documented data dimensions of cultural objects and or with linked timelines, usually implemented as artifacts, geographic maps often serve as a coordinated temporal view in addition to spatial visualization method to show the spatial distribution of representations [Kra16, IF:DGB, IF:HTA, IF:MOTW, artifacts’ origins [BGSvdB14, IF:DGB, TO:GBDE, TO:PAL, TO:VS]. IF:NL, IF:PAN, TO:VS]. 2.2.3 Network Diagrams 2.3.2 Animation As for relational data (e.g. influences, references, inter- Further options for encoding temporal data aspects artifact relations) network diagrams allow users to build on the abovementioned spatial visualizations and explore the proximities and distances of artifacts or add temporal information in a hybrid, spatiotemporal cultural actors in relational or topological spaces way. Among these, animation is frequently used, [HSC08, IF:DDBV, IF:ECB, IF:IA, IF:HG, IF:EDG, mapping time to time [IF:DGB, IF:PAN]. TO:PAL]. 2.3.3 Superimposition 2.2.4 Set Diagrams Superimposition approaches merge multiple temporal Given different thematic or stylistic classifications of layers or snapshots into one visualization, with cultural artifacts, set diagrams or treemaps offer temporal data aspects often being distinguished by insights into categorically and often also hierarchically different colors [BGSvdB14], or visualization of structured object metadata constellations [XEJJ14, movement trajectories [TO:NL]. UPM12, IF:PAN]. 2.3.4 Space-Time Cube 2.2.4 Other Diagrams Space-time cube representations build on 2D planes of When overviews abstract from single objects and focus encoded spatial data dimensions (like maps or on data distributions in different metadata dimensions, networks), and map time to an additional spatial a wide variety of further InfoVis diagrams can provide dimension, i.e. the orthogonal z-axis. Cultural object overview on selected collection aspects, including area collections thus arrange as characteristically shaped 3D 77 A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections point clouds, according to various spatio-temporal [Dru13]. Due to the relevance of this design principle, layouts [Kra05, WMS*16]. the following collection primarily takes approaches and interfaces into account which have been 2.4 Multi-Method Interfaces implementing a multi-method approach. As the assembly of approaches and interfaces in table 1 shows, multiple spatial encoding methods have already 3 Assembling Information Visualization been implemented in the CH data domain – often also as multi-method interfaces to enable the combination Approaches to Digital CH Collections of different exploratory views on the data. The same Table 1 provides an overview of prominent InfoVis holds true for different temporal encoding methods: It approaches to digital CH collections. Interfaces are is well known that different temporal encoding classified and specified according to four main methods show different strengths and weaknesses. Due categories. While the first two categories make the to this reason, advanced InfoVis interfaces increasingly chosen spatial and temporal encoding methods visible combine multiple temporal and spatial encoding (cf. 2.2 und 2.3), the third column specifies the focus techniques, to compensate their drawbacks and add up of interest, which predominantly is either a certain type of cultural objects, or a focus on cultural actors (FCA), their complementary benefits [KKC14]. This equals or a focus on cultural topics or styles. The fourth the provision of multiple access points and overviews column points out whether the approach is of [THC12], which form complementary composites, conceptual and prototypical nature, or whether it revealing different “parallax” views of a collection provides an open, web-based interface [IF:XYZ] or a Table 1: Information visualization approaches and interfaces to digital cultural object collections, ordered according to their chosen method of spatial encoding, temporal encoding, entity focus, and type of project. 78 A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections tool [TO:XYZ], with which external DH collection indispensable as a medium for knowledge data could be visually explored [cf. Pos16]. communication. Despite restricted budgets of local collectors and institutions, efforts for digitization and 3.1 Interpretation dissemination will continue, as will the development of While looking at single approaches helps to specify web-based interfaces. their implemented combination of methods, parsing of From an InfoVis perspective, we consider the field columns helps to explore the prominence of encoding of CH data, users and tasks, to be a specifically methods or object types. With regard to the overall productive one, revolving around grand design distribution, well-established InfoVis techniques can challenges. While CH data is often characterized by be identified, as well as structural holes, which might massively heterogeneous and time-oriented data deserve closer attention by future interface design and complexity, its audiences approach it with research. Exemplarily, the distribution of temporal heterogeneous, underspecified tasks [MFM*16]. encoding methods shows a dominant use of (linked) Besides the consideration of well-known principles of timelines, which again are known to evoke split graphical excellence, such casual users require also attention effects [AS05]. To reduce cognitive, load more aesthetics-oriented, entertaining approaches. In more spatio-temporally integrated encoding techniques contrast to principles of parsimonious design and like space time-cube representations could be tested. complexity minimization, the preservation of aesthetic complexity and diversity matter in the CH domain, and non-conclusive explorations provide their own reward. 3.2 Limitations Therefore, the value of methods supporting horizontal Aiming for the consolidation of the research field and browsing, multiple access points and serendipitous for orientation of future approaches, we are still aware insight creation is ranging high. This makes CH data a of two obvious limitations. As the interaction with challenging research field, expanding and enriching the ‘cultural object collections’ is investigated in multiple scope of consolidated playing fields for InfoVis academic domains, the current review is far from research far beyond expert-oriented professional exhaustive. Yet by highlighting and comparing recent applications. works and developments, we hope to lay ground for a Furthermore, we expect new options for interface more systematic and critical discussion – as well as for their future enrichment and refinement. design to emerge from the expansion and pervasion of Furthermore, we consider the chosen categories linked data in the CH realm [KAR15, IF:CS], as well of classification to be relevant from an InfoVis as the utilization of user data, which will open up new methods perspective, but are aware of possible other ways to weigh, highlight, recommend, and tailor foci of attention. As such we exemplarily consider interfaces for general audiences and specific user interaction and navigation techniques to provide groups alike. productive categories or further analysis, as well as a From a systematic point of view – which might be wide variety of ‘humanistic’ user experience and most relevant because of its didactic implications – we design principles [Dru13, DCW12, Whi15], which hope for a continued discussion and consolidation could help to shape the focus on relevant DH interface process to accompany the outlined developments. We functions and features with even more precision. consider such macroscopic reflections not only to be relevant for integrating the state of the art on academic 4. Conclusions and Outlook grounds (informing new directions and approaches), but also for introducing visitors to the workings of We presented a review of InfoVis approaches and their new online museums and archives. In contrast to interfaces to digital CH collections, and arranged traditional encounters with culture collections, their existing work by the means of a categorical experiences and learnings will also depend on their framework, which we submit for critical examination ability to comprehend and master the powerful (re-) and collective refinement. arrangement, encoding and interaction techniques, We expect the field of CH collection visualization which new interfaces are already providing us with. to further develop and diversify – not least due to the fact that the world wide web renders itself ever more 79 A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections Acknowledgements Approaches and Strategies of Inclusion in Digital CH Interfaces, in: C. Busch, J. Sieck This research has been supported by the Austrian (Eds.), Kultur Und Informatik (XIII) - Cross Science Fund (FWF), Project No. P28363 Media, Hülsbusch, Berlin, pp. 105–118. [GMPS00] Greene, S., Marchionini, G., Plaisant, C., & References Shneiderman, B. (2000). Previews and overviews in digital libraries: Designing [ABO12] Algee, L., Bailey, J. Owens, T. (2012). surrogates to support visual information Viewshare and the Kress Collection: Creating, seeking. Journal of the American Society for sharing, and rapidly prototyping visual Information Science, 51(4), 380–393. interfaces to cultural heritage collection data, [HSC08] Hinrichs, U., Schmidt, H., & Carpendale, S. D-Lib Magazine 18(3). (2008). EMDialog: Bringing information [Bac02] Baca, M. (2002). A picture is worth a visualization into the museum. IEEE thousand words: Metadata for art objects and Transactions on Visualization and Computer their visual surrogates. ALCTS Papers on Graphics, 14(6), 1181–1188. Library Technical Services and Collections, [KBM16] Kontiza, K, Bikakis, A., Miller, R. (2015). 131-138. Cognitive-based Visualization of Semantically [BGSvdB14] A. Betti, D. Gerrits, B. Speckmann, & H. Structured Cultural Heritage Data, Van den Berg. (2014). GlamMap: Visualising Proceedings of the International Workshop on Library Metadata. Proceedings of VALA. Visualizations and User Interfaces for Ontologies and Linked Data. URL: [BOB82] Belkin, N.J., Oddy, R.N. & Brooks, H.M. http://ceur-ws.org/Vol-1456/ (1982). ASK for information retrieval: Part I. Background and theory. Journal of [Kra05] Kraak, M. J. (2005). Timelines, temporal documentation 38(2): 61–71. resolution, temporal zoom and time geography. In Proceedings of the 22nd [AS05] Ayres, P., & Sweller, J. (2005). The split- International Cartographic Conference. A attention principle in multimedia learning. In Coruña Spain. R. E. Mayer (Ed.), Handbook of multimedia learning (pp. 135-146). New York: [Kra16] Kräutli, F. (2016). Visualising cultural data: Cambridge University Press. exploring digital collections through timeline visualisations. Doctoral dissertation, Royal [DCW11] Dörk, M., Carpendale, S., Williamson, C. College of Art. (2011). The information flaneur: A fresh look at information seeking. In Proceedings of the [KKC14] Kerracher, N. , Kennedy, J., & Chalmers, K. SIGCHI Conference on Human Factors in (2014). The design space of temporal graph Computing Systems (pp. 1215–1224), ACM. visualisation. In Proceedings of the 18th EuroVis (Vol. Short). Swansea: Eurographics. [Dru13] Drucker, J. (2013). Performative Materiality and Theoretical Approaches to [Man09] Manovich, L. (2009). Cultural Analytics: Interface. Digital Humanities Quarterly 7(1). Visualizaing cultural patterns in the era of “more media”. Domus. [DWC15] Düring, M., Wieneke, L., & Croce, V. (2015). Interactive Networks for Digital [MFM16] Mayr, E., Federico, P., Miksch, S., Schreder, Cultural Heritage Collections - Scoping the G., Smuc, M., & Windhager, F. (2016). Future of HistoGraph. In P. Cimiano, et al. Visualization of Cultural Heritage Data for (Eds.), Engineering the Web in the Big Data Casual Users. Proceedings of the 1st Era (pp. 613–616). Springer International. Workshop for Visualization for the Digital Humanities, Baltimore, MD. [GMD15] Glinka, K. Meier, S., Dörk, M. (2015). Visualising the »Un-seen«: Towards Critical 80 A Review of Information Visualization Approaches and Interfaces to Digital Cultural Heritage Collections [Pos16] Posner, M. (2016). The Digital Art Historian’s [IF:DDBV] Deutsche Digitale Bibliothek Visualisiert Toolkit.URL:http://program.dh.ucla.edu/getty/ URL: http://infovis.fh-potsdam.de/ddb/ index.php/the-digital-art-historians-toolkit/. [IF:DGB] DARIAH-DE Geo-Browser / Europeana4D [RHQ14] Rogers, K., Hinrichs, U., & Quigley, A. URL: https://geobrowser.de.dariah.eu/ (2014). It doesn’t compare to being there: in- [IF:ECB] eclap-Browser / Social Graph situ vs. remote exploration of museum URL: http://www.eclap.eu/portal/ collections. The Search Is Over! Exploring [IF:EDG] Edgemaps [FCA] Cultural Collections with Visualization, URL: http://mariandoerk.de/edgemaps/demo/ London, UK. http://searchisover.org/ [IF:GCI] Google Cultural Institute [Sul13] Sula, C. A. (2013). Quantifying Culture: Four URL:https://www.google.com/culturalinstitute Types of Value in Visualisation. In J. Bowen, [IF:HG] histograph S. Keene, & K. Ng (Eds.), Electronic URL: http://histograph.cvce.eu/ Visualisation in Arts and Culture (pp. 25–37), Springer. [IF:HTA] Heilbrunn Timeline of Art History | The MET URL: [THC12] Thudt, A., Hinrichs, U., & Carpendale, S. http://www.metmuseum.org/toah/chronology/ (2012). The bohemian bookshelf: supporting [IF:KB] Kindred Britain [FCA] serendipitous book discoveries through information visualization. In Proceedings of URL: http://kindred.stanford.edu/ the SIGCHI Conference on Human Factors in [IF:IA] Inventing Abstraction 1910-1925 [FCA] Computing Systems (pp. 1461–1470). ACM. URL: www.moma.org/inventingabstraction [UTA10] Urban, R.J., Twidale, M.B., Adamczyk, [IF:MOTW] Museum of the World P.D. (2010). Cultural Heritage Information URL: https://britishmuseum.withgoogle.com/ Dashboards.URL:http://www.ideals.illinois.e [IF:PAN] Pantheon [FCA] du/handle/2142/14936 URL: http://pantheon.media.mit.edu/ [Whi15] Whitelaw, M. (2015). Generous Interfaces for [IF:PVFW] Past Visions by Frederick William IV. Digital Cultural Collections, Digital URL: https://uclab.fh-potsdam.de/fw4/ Humanities Quarterly, 9. [IF:ROL] Republic of Letters [FCA] [WMS*16] Windhager, F. Mayr, E., Schreder, G., URL: http://ink.designhumanities.org/dalembert/ Smuc, M., Federico, P., & Miksch, S. (2016). Reframing Cultural Heritage Collections in a [IF:SCE] SelfieExploratory | SelfieCity Visualization Framework of Space-Time URL: http://selfiecity.net/selfiexploratory/ Cubes. In M. Düring et al.(eds.) Proceedings of the 3rd HistoInformatics Workshop, (pp. 20–24), Krakow. http://ceur-ws.org/Vol-1632/ CH InfoVis Tools [XEJJ14] Xu, W., Esteva, M., Jain, S.D., & Jain, V. [TO:GBDE] Geo-Browser Datasheet Editor (2014). Interactive visualization for curatorial URL: https://geobrowser.de.dariah.eu/edit/ analysis of large digital collection, [TO:IPS] ImagePlot Suite Information Visualization 13: 159–183. URL: http://lab.softwarestudies.com/p/software- for-digital-humanities.html [TO:NL] Neatline | Omeka CH InfoVis Web-Interfaces URL: http://neatline.org/ [IF: CG] Culturegraphy [TO:PAL] Palladio URL: http://www.culturegraphy.com/ URL: http://hdlab.stanford.edu/palladio/#/ [IF:CS] CultureSampo [TO:VS] ViewShare URL: http://www.kulttuurisampo.fi/?lang=en URL: http://viewshare.org/ 81