The Digital Online Museum

Abstract

This paper describes a novel approach to satisfy the needs of museum's website visitors with a unique experience that cannot be reproduced in the museum itself. We aim at providing a continuous and lasting experience, without the emphasis of a single, nal result - a process we call digital strolling. The view supports this process by displaying results as a path on which the user strolls. To enable the user to nd new and unexpected inspiration, recommendations to related exhibits are proposed in di erent dimensions to vary the user's path. The common approach of image retrieval as the sole method to generate recommendations of related exhibits is not su cient. Authored tagging is still the better but more costly solution. The proposed approach claims to ll the gap between current digital museums and the needs of the digital museums' visitors.

Keywords

• Digital Museum
• Digital Strolling
• Semantic Search and Indexing
• Semantic Tagging
• Virtual Heritage

-

In the federally funded project Mediaplatform, we are researching new and enhanced ways of searching and displaying the online collections of galleries, libraries, archives and museums (GLAMs) together with the German Stadel Museum [ 23 ]. Within this context, we have built a standard information retrieval system, that supports the proposed browsing-based usage paradigm by hosting various kinds of media meta data, recommender logics and media stocks. The application, developed within this research has to meet high requirements in feasibility, usability and performance, as it will represent the Stadel Museum [ 23 ] in their 200 year anniversary in 2015.

Modern museums, like the Stadel Museum [ 23 ] can only exhibit nearly 2% of their art stock. Therefore the objective is not to attract more visitors to the 2 museum, but to share the existing art stock with the public in a digital museum. According to Weng, the main features of a digital museum is to archive, exhibit and educate in the same way as the physical museum pursues this objectives [ 29 ] and most existing research creates digital musems that are a representation of a physical museum [ 28, 17, 19, 18, 29, 27, 12 ]. The ndings of Marty [ 14 ] indicate that these approaches are not su cient. This paper proposes a new approach to close the gap of previous work and the described demand of museum website visitors. 2 2.1

Proposed Approach

Digital Strolling

Modern digital museums present their exhibits only in a traditional information ltering view [ 17, 19, 18 ], by a virtual representation of the museum [ 27 ], or by providing pipelined, curated paths [ 19, 12 ]. But the ndings of Marty [ 14 ] indicate that users call for a di erent experience: \[...] online museum visitors are interested in having access to unique experiences that cannot be duplicated in museums".

Therefore we address this need by letting the user digitally stroll through the exhibits to discover unexpected results, similar to the sightseeing metaphor, established by Tezuka and Tanaka [ 26 ]. Whilst their work employs a split interface, separating a map where the user can digitally stroll from the media content that can be explored, we propose to not separate content from navigation to reinforce the strolling experience. This experience is based on physical strolling applications for museums like [ 2 ] and [ 20 ], but provides more degrees of freedom. Instead of just chaining content elements, we let the user decide which recommendations and paramedia to follow. The digital art stock of every museum consists of exhibit images and paramedia like paratexts. Paratexts are information around the exhibit that add extra meaning to it [ 1 ]. All paramedia and the exhibit's image as the central anchor build an information cluster. This cluster is strictly hierarchically structured, to increase the user's understanding, and thus the user's engagement with the system [ 3 ]. The information cluster also contains recommendations to similar exhibits with each other.

We call the experience digital strolling, when one or more information clusters are displayed as search results, related exhibits are recommended and all the clusters the user interacted with build a path, which is entirely available for the user. Similar to a browser history the information cluster string together as a path. In di erence the user shall be able to modify the path by following the recommendations within the path or drag the clusters to compare one or more exhibits. 2.2

Ranking

Interactive storytelling systems [ 20, 2 ] demonstrate the necessity of providing ranked virtual heritage artefacts in order to present a coherent information stream that can be discovered during the digital strolling. Automated processes like image retrieval are employed by e.g. Hong et al. [ 6 ] or the Google Art Project [ 5 ] but these methods still have issues in identifying contentual relationships. For example, Figure 1 shows that also unrelated images are presented as allegedly related images, because they are similar in shape and color composition, but not contentual. In order to support digital strolling, image retrieval methods are not enough. Instead, there are strong suggestions to involve a user into the curation and categorisation process of an online catalog, while an initial expert tagging is a suitable and expected starting point [ 22 ]. A classi cation harnessing the users' own words for describing an exhibit can furthermore create a contemporary understanding of the exhibitions, as advised by [ 1 ].

Thus we propose a method taking several factors into consideration: domain experts maintain semantic tags for each collection item in multiple dimensions, which also act as starting points if no other data about the user exists. In further expansion stages, these defaults are augmented with additional tags gathered from automatically Named Entity Recognitions (NER), based on text meta data elds. These tags are biased by interpretations of user's implicit and explicit interactions with collection items.

On the other side, [ 22 ] concludes with the nding that \simply providing Web 2.0 interactions, such as tagging and commenting, is not enough". A combined method of several factors as mentioned above might therefore be the right approach. 3

Exhibit platform

The exhibit platform represents a novel user interface for rich media databases as found in museums. The user can query and browse through the results and 4 also get inspired by the system's recommendations of related works for each result in a map of media tiles, as shown in Figure 2. Based on modern web technologies (HTML5, CSS3 and JavaScript) the application runs on computers with large screens, as well as on mobile devices. Its responsive design makes it independent to di erent screen sizes. For very small screens, like smartphones, a di erent interaction concept is employed that suits the smartphone's interaction possibilities. Therefore a di erent view is employed by adaptive web design. It is intended to share virtual heritage artefacts from the art stock of the Stadel Museum [ 23 ] with users, not to attract them for a physical museum visit. The user can stroll through, lter and discover images of exhibits, information about the exhibit in text, audio and video and information about the artists. 3.2

Design Principles

Role of Speed Several studies and best practises clearly indicate the importance of speed for online portals and search interfaces [ 15, 4, 21, 11 ]. At Amazon, for example, a 100 ms delay results in 1% of revenue drop [ 11 ]. A study at Bing showed that delays even under half a second have negative business impacts [ 21 ]. We therefore pay special attention to performance, which impacts the way we designed the architecture and the processes, especially in terms of semantic indexing as explained in the following chapter. Responsivity and Adaptation Half of the German population owns a smartphone (e ective date: February 2014) [ 24 ]. Hence it is necessary to support mobile devices and stationary devices at the same time to address most users. By employing Responsive Web Design (RWD) it is possible to provide a website that changes its layout to make use of di erent screen sizes [ 13 ]. Adaptive Web Design (AWD) is a di erent approach and provides two websites with a switch that reacts accordingly to the detected device class [ 25 ]. RWD is used when the interaction mechanisms stay the same on the di erent device classes and only the layout shall be adapted to the change of available screen real estate. This can be achieved with a packery system as described by [ 7 ]. AWD is used when the interaction mechanisms change on the di erent device classes. These techniques are poorly supported by the related works described. Recently the Google Art Project [ 5 ] introduced both RWD and AWD. We recommend the combination of RWD, for adapting to di erent screen sizes that are typical for high-performance devices, such as tablets and notebooks or desktops, and AWD, for switching to a website that employs di erent interaction mechanisms that are suited for the small screen.

Usability aspects Experts prefer common search portals like Google over curated, openly-ranked, reviewed, domain-speci c search portals [ 9 ]. Krug recommends to design a self-evident GUI in order to minimise confusion potential [ 10 ]. He also advises to design for scanning web sites rather than reading them to increase the user's engagement. Clear visual hierarchy is a strong factor to increase the user's engagement [ 3 ]. 3.3

Architectural Overview

The architecture has to deal with many di erent and sometimes opposing requirements. On one side, the system needs to cope with data from various sources with di erent formats and licenses. The Mediaplatform should be a place where di erent data can be used symbiotically, where the uses of single dataset can multiply. On the other side and as discussed previously, there are strong constraints regarding the application performance, feasibility and usability aspects. Re ecting those requirements, the Mediaplatform was designed as a client/server architecture shown in Figure 3.

The rst and most important step from the various data sources towards a working application is a process we call Semantic Extraction Transformation Load (Semantic ETL). Its purpose is to read the multiplicity of various input data, transform the data formats into a uni ed entity model which is suitable for di erent kinds of media, add or apply authority les, and nally streamline the data bases to the common usage patterns of end users of the Mediaplatform.

As mentioned, performance is one of the most important criteria, thus this process results in a highly optimized Apache Lucene search index, capable of answering common information needs with a single query. Once the index as well as other pre-processed structures are created in an overnight batch job, both constitute the backend of all online-operations of a server. Beside queries that go directly to the prepared Lucene index, the server contains a recommendation component capable of making semantic suggestions based on authority les or semantic taggings. 3.4

Semantic Image Recommendation

We created a hybrid combination of several approaches introduced previously: On one side, we employ an automatic index and ranking mechanism of semantic tags, on the other side we try to foster the user's curiosity by hiding these criteria behind the actual exposition items and by o ering them several possible paths through the collection. This includes automatically indexed, human-de ned tags for each collection item in the six dimensions atmosphere, association, main motive, main motive type, emotion, and subject. Currently, in the rst version of the application, those tags are de ned exclusively by experts as a preliminary step and then remain hidden behind the scenes after being indexed in the Semantic ETL process - the GUI sorely works with images. This way, users will be o ered six di erent graphical paths to continue their digital strolling through the museum in their own likeness, each showing up to four highly related images within the respective dimension as teasers, it is also possible to reveal all related images. Discovered images are no longer o ered as recommendations for the further strolling path.

Behind the scenes, we determine how strong two items are related by the number of tags they share in one of the dimensions, e.g. the main motive. So in

Figure 4, for example, for the given number of taggings in one given dimension of the images A, B and C, we try nd the largest intersection, in this case B \ A, followed by B \ C. The most related image of B would therefore be Image A over Image C.

It is easy to translate this Set Theory problem into an Information Retrieval problem: for a given image, we basically formulate a query which OR-chains every tagging of the image in the given dimension. By the example of image B, a resulting query would be \tag 2" OR \tag 3" OR \tag 5" OR \tag 6" OR \tag 7" OR \tag 8". Thereby, we let the Information Retrieval framework - Apache Lucene fed with Boolean-Should-Queries [ 16 ] - determine the proper ranking for the relatedness of collection items. This results in a very fast and powerful semantic search feature as foundation for digital strolling, bene ting from automatic indexing as well as human curated guidance. 3.5

Client-Applications

The prototype welcomes the user with featured exhibits and topics where the user can select an element or send a search query. Then the results are displayed horizontally by animations to increase the overview for the user. Gesture support ensures awless interaction on touch devices. Each result element is an information cluster, consisting of di erent media types like an image of the exhibit, information about the exhibit as text, video and audio. The elements within an information cluster are layouted according to a rectangle packer, similar to [ 7 ]. On di erent screens the elements are automatically repacked to t to the available screen size in an optimal way. Each information cluster itself can be dragged to a new position enabling the user to compare the exhibits. Hence the fact, that the amount of information clusters can increase so that the available screen real estate is not su cient to display them all at once, the user can scroll horizontally. An overview snippet provides the overview of all results like a radar, displaying the di erent packed information clusters in a minimised way.

For every exhibit the system provides suggestions of related exhibits in the six dimensions atmosphere, association, main motive, main motive type, emotion, and subject. Following a related exhibit inserts a new information cluster about this particular exhibit at the left of the predecessor.More recent results are displayed on the right, according to the European reading direction, thus the user walks along a path. Color codes help to orientate the user which results belong to the same query and where has a related exhibit been explored. Too many results would decrease the performance on tablets and decrease the overview, therefore an oblivion mechanism has been integrated. User interaction updates the information cluster's date of creation. Once a dynamically computed threshold has been exceeded, the oldest information cluster is removed from the view. The web-based platform can be accessed on desktops, tablets and smartphones. For the latter an easi ed digital strolling mechanism is presented, that displays only one information cluster at a time, see Figure 5.

(a) Search results.

(b) Similar results.

Related Work

The digital museum as an extended physical museum

Existing research focused on extending the museum's physical showroom. Wang and Shen [ 28 ] re ned 3D digitalisation techniques to present the exhibits as virtual three dimensional representation. Our approach does not focus on recreating the virtual museum as the physical museum. It introduces a possibility that cannot be reproduced in a physical museum. Closer to our approach is the Google Art Project. It presents digital collections of exhibits with the main focus on providing gigapixel images. The users are able to lter images based on properties like the exhibit material, year of production, artist name, etc. For each exhibit, the system o ers three di erent related exhibits that share a similar motive or color composition. Unfortunately there is no information provided on which basis the images are related to each other. The results create the impression, that only image retrieval methods are employed. In contrast to the Google Art Project an initial step in our approach is manual expert tagging. We want to extend this by image retrieval and user generated tags. 4.2

Strolling

Tezuka and Tanaka [ 26 ] show that the user satisfaction of a library information ltering system can be increased by employing a sightseeing metaphor. Literature is represented as places of interest on a map. In contrast to our approach we do not use a map, but the content itself to build a path. Both approaches have in common, that the user can stroll digitally to encounter unexpected information.

Similar to our digital strolling approach are on-site museum guides, that help the user navigation through collections of a physical musem. Rocchi et al. propose such a guide based on manually tagged images and texts of cultural heritage artefacts [ 20 ]. The visitor uses a location aware handheld device, which displays a life-like avatar, presenting paratexts that are related to the approached exhibits. The presented content is a seamless stream of information generated from text les, which are read aloud automatically, and images, which are presented in a cinematic slideshow. The text-image pair is joined with other text-image pairs based on their contentual context. Each text has a topic and rhetorical relations and each image is tagged concerning its content. Damiano et al.[ 2 ] follow a similar approach to support museum visitors without a prede ned path. They also present a storytelling agent as an in situ museum guide exploiting mobile devices and position aware information retrieval for exhibits in front of the device. The information units are tagged semantically enabling a seamless playback. The users can stroll in the exhibit rooms freely because the information units can be combined in any constellation possible. These on-site museum guides demonstrate the necessity of semantically tagged content in order to be able to provide a digital strolling experience. 5

Conclusion

In this paper we have presented a novel approach of a web-based digital museum that supports a unique experience that cannot be duplicated in museums: the digital strolling. We have described a semantic search feature called digital strolling, that enables the user to freely discover virtual heritage artifacts. Thereby, a hybrid ranking feature takes several factors in multiple dimensions into consideration. Also we have described a prototype that establishes the proposed method. We believe the basic idea behind digital strolling can be ported on more and di erent use cases as well as on other underlying technologies. We can conclude that it is possible to ll the gap that exists between current digital museums and the needs of the digital museums' visitors. 6

Future Work

First informal usability test with students and museum experts show, that this approach is promissing: The individuals instantly interacted in a strolling manner and reported that the recommendations raised their interest to follow them further. In order to be able to draw reliable conclusions about the proposed approach we will perform a standardised usability evaluation including usability walkthroughs and usability questionnaires according to the international norm for designing interactive dialogues [ 8 ].

According to Christensen [ 1 ] it is necessary to establish the user's participation in an exhibition platform, because they can create a contemporary understanding of the exhibitions, beyond the scienti c understanding. We will research how user input can be considered in an automated tagging mechanism, e.g. images that are often compared obviously share a property that is relevant for the users.

Currently, the process of tagging collection items is of manual kind. However, we see great potential in supporting humans in this task with automatic NER. We want to explore a carefully selected middle course between strictly prede ned curated paths on the one side and following a community-driven approach on the other.

References

1. J. R. Christensen . Four steps in the history of museum technologies and visitors' digital participation . MedieKultur , 50 :7{ 29 , 2011 .

2. R. Damiano , C. Gena , V. Lombardo , F. Nunnari , and A. Pizzo . A stroll with carletto: adaptation in drama-based tours with virtual characters. User Modeling and User-Adapted Interaction , 18 ( 5 ): 417 { 453 , 2008 .

3. S. Djamasbi , M. Siegel , and T. Tullis . Visual hierarchy and viewing behavior: an eye tracking study . In J. A. Jacko , editor, HCI'11 Proceedings of the 14th international conference on Human-computer interaction: design and development approaches , pages 331 { 340 . Springer-Verlag, Berlin and Heidelberg, 2011 .

4. Forrester Consulting . ecommerce web site performance today , 2009 . http://www.damcogroup. com/white-papers/ecommerce website perf wp . pdf.

5. Google . Google Cultural Institute, 2014 . http://www.google.com/culturalinstitute.

6. J.-S. Hong , H.-Y. Chen, and J. Hsiang . A digital museum of taiwanese butter ies . In P. J. Nurnberg, D. L. Hicks , and R. Furuta, editors, the fth ACM conference , pages 260 { 261 .

7. E. Huang and Korf, Richard, E. Optimal rectangle packing: an absolute placement approach . Journal of Arti cial Intelligence Research , 46 ( 1 ): 47 { 87 , 2013 .

8. ISO. Din en iso 9241-110 Ergonomie der Mensch-System-Interaktion - Teil 110 : Grundsatze der Dialoggestaltung (iso 9241 - 110 : 2006 ); Deutsche Fassung EN ISO 9241-110 : 2006 .

9. M. Kemman , M. Kleppe , and S. Scagliola. Just Google It - Digital Research Practices of Humanities Scholars. arXiv: 1309 .2434 [cs], Sept . 2013 .

10. S. Krug . Don't make me think! A common sense approach to Web usability . New Riders, 3rd edition edition , 2013 .

11. G. Linden. Make Data Useful , 2006 . http://www.gduchamp.com/media/Stanford DataMining. 2006 - 11 -28.pdf.

12. Louvre . Louvre visitor trails, 2014 . http://www.louvre.fr/en/routes.

13. E. Marcotte. Responsive web design, 2010 . http://alistapart.com/article/responsive-web-design.

14. P. F. Marty . Museum websites and museum visitors: digital museum resources and their use . Museum Management and Curatorship , 23 ( 1 ): 81 { 99 , 2008 .

15. M. Mayer . In search of ... a better, faster stronger web , 2009 . http://assets.en.oreilly.com/1/event/29/Keynote%20Presentation% 202 .pdf.

16. M. McCandless , E. Hatcher , and O. Gospodnetic . Lucene in Action, Second Edition: Covers Apache Lucene 3 .0. Manning Publications Co., Greenwich , CT , USA, 2010 .

17. Metropolitan Museum of Art. The metropolitan museum of art - home , 2014 . http://metmuseum.org/.

18. Museum of Modern Art. Moma | museum of modern art , 2014 . http://www.moma.org/.

19. Powerhouse . Powerhouse museum | science + design | sydney australia, 2014 . http://www.powerhousemuseum.com/.

20. C. Rocchi , O. Stock , M. Zancanaro , M. Kruppa , and A. Kru ger. The museum visit: generating seamless personalized presentations on multiple devices . In J. Vanderdonckt, N. J. Nunes , and C. Rich, editors, IUI '04 Proceedings of the 9th international conference on Intelligent user interfaces , volume 9 , pages 316 { 318 . ACM , New York and NY and USA, 2004 .

21. E. Schurman and J. Brutlag . The User and Business Impact of Server Delays, Additional Bytes, and HTTP Chunking in Web Search Presentation, 2009 . http://cdn.oreillystatic.com/en/assets/1/event/29/The%20User %20and%20Business %20Impact%20of%20Server%20Delays,%20Additional%20Bytes,%20and%20HTTP %20Chunking%20in%20Web%20Search%20Presentation.pptx.

22. R. Srinivasan , R. Boast , J. Furner , and K. M. Becvar . Digital Museums and Diverse Cultural Knowledges: Moving Past the Traditional Catalog . The Information Society , 25 ( 4 ): 265 { 278 , July 2009 .

23. Stadel Museum. Homepage , 2014 . http://www.staedelmuseum.de/sm/.

24. Statista . Anzahl der Smartphone-Nutzer in Deutschland bis 2014 | Statistik , 2014 . http://de.statista.com/statistik/daten/studie/198959/umfrage/anzahl -dersmartphonenutzer-in-deutschland-seit- 2010 /.

25. SYZYGY. Responsive vs Adaptive , 2014 . http://www.syzygy.de/studien/responsivevs-adaptive.

26. T. Tezuka and K. Tanaka . Traveling in digital archive world: Sightseeing metaphor framework for enhancing user experiences in digital libraries . In D. Hutchison, T. Kanade , J. Kittler , J. M. Kleinberg , F. Mattern , J. C. Mitchell , M. Naor , O. Nierstrasz , C. Pandu Rangan , B. Ste en , M. Sudan, D. Terzopoulos , D. Tygar , M. Y. Vardi , G. Weikum , E. A. Fox , E. J. Neuhold , P. Premsmit , and V. Wuwongse, editors, Digital Libraries: Implementing Strategies and Sharing Experiences , volume 3815 of Lecture Notes in Computer Science, pages 23 { 32 . Springer Berlin Heidelberg, Berlin and Heidelberg, 2005 .

27. The State Hermitage Museum. The state hermitage museum , 2014 . http://hermitagemuseum.org/html En/index.html.

28. N. Wang and X. Shen . The research on interactive exhibition technology of digital museum resources . In 2013 IEEE International Conference on Green Computing and Communications (GreenCom) and IEEE Internet of Things(iThings) and IEEE Cyber, Physical and Social Computing(CPSCom) , pages 2067 { 2070 .

29. T. Weng . The 19th century o cial paris salon exhibition digital museum . WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS , 12 ( 6 ): 1903 { 1912 , 2009 .