Three-dimensional acquisition and reproduction technologies are often exploited in cultural heritage field for a variety of applications such as conservation, restoration, and dissemination. Another valuable use of 3D data is to make exhibitions more accessible to visitors with impairments, allowing them to fully experience and enjoy the acquired objects. In this short paper, we explore the accessibility inherently provided by 3D representations of real-world objects, with a particular focus on the quality of the models and 3D printing, as well as the presentation aspects. To this end, we propose to apply a state-of-the-art saliency-driven process, generating a fixation map that identifies the object's salient areas that need to be reproduced with a higher definition during the 3D printing to improve the object accessibility. We present a case-study involving the full process of 3D scanning and printing the Coats of Arms in Palazzo Bo (Padova, Italy) to make them accessible to visitors with visual impairment. We employed diferent scanning techniques and applied the attention mechanism on acquired data to obtain the object salient areas and drive the printing process accordingly. Preliminary tests involving some participant feedback reveal that printing the objects with a variable detail level allows the visitors to have a better understanding of the object as a whole and to appreciate the relevant details.
In recent years, advancements in digital technology revolutionized the way we document, preserve,
and share cultural artefacts. Beyond any doubt, one of these tools is 3D reconstruction, comprising
a vast set of methods for acquiring objects, from coins [
Ø Scaling the model to size Ø Laying the model flat Ø Slicing of the model
people looking at the same subject. Additionally, we present a case-study involving the 3D scanning and printing of the Coats of Arms on display at Palazzo Bo (Padova, Italy). We scanned six objects and 3D printed them following the fixation map derived from the projection of the acquired surfaces. The main goal of the project is to create a reduced tactile “Coats of Arms wall”, so that their significance and meaning are available for visually impaired people. A preliminary study including feedbacks from blind people shows the feasibility and the potential value of the project.
Accessibility for visually impaired individuals refers to the design of services, environments, and
technologies that enable people with visual impairments to participate in society. Ensuring access to
cultural heritage to visually impaired individuals can be implemented in several ways, for instance
providing audio descriptions [
One of the challenges of accessibility is to develop a methodology to efectively create a presentation ofering the same experience to diferent people. In particular, for visitors with visual impairments we have to exclude one of the most used senses for visual arts – sight. The question that follows is: what are the visual features that make us characterise an object? And are these features also interesting for a blind person? In this regard, we propose to address this problem exploiting visual saliency. When looking at an object, our gaze unintentionally lingers on some specific areas. Indeed, by tracking the eye movements while observing some subject, we can detect which regions are visually more interesting for our sight. Analysing the eye behaviour of many subjects observing the same scene allows us to compute the so-called fixation map . The concept of fixation map was introduced in 2002 by D.S. Wooding [ 33] and consists in defining a function that outputs the amount of visual attention for a given image location. Following works aim at predicting fixation maps based on image features such as symmetry [ 34] or using data-driven approaches [35, 36, 37]. Since gaze estimation is closely related to human vision behaviour, fixation prediction models are often associated to salient object detection [ 38, 39] or used to drive other tasks, such as classification or segmentation.
In this work we propose to apply state-of-the-art fixation prediction models to the acquired 3D objects and use the resulting fixation maps to drive the 3D printing. Starting from the acquired object with texture, we rotate the 3D mesh according a reference system and create a projection on a virtual plane that is perpendicular to the original object orientation. In this way, we can use the projected texture as input for the fixation prediction and identify the areas that would result more attractive for an observer. Exploiting the 3D acquisition of the objects, we can project back the visually relevant areas and adapt the printing process and some presentation aspects according to the xfiation results. The main goal of the described process is to focus on the most salient object regions so that visitors touching the printed object can have a better understanding of the artefacts in all its parts.
Figure 1 summarises the proposed pipeline for acquisition and printing. First, the 3D scan of all the objects is performed, followed by some post-processing steps on raw data to improve the surface quality. The core part of our pipeline involves the application of fixation prediction on the acquired projected texture. This allows to efectively recognise the salient areas of the object that will guide the printing process. Finally, models are prepared and printed using two diferent technologies. In the remaining we describe a case-study where we exploited the attention mechanism for two aspects: first, we adapted the resolution of 3D printing according to the relevance; second, we focused on the most relevant regions and printed them separately with a diferent technology to ofer a better reading.
Palazzo Bo is one of the most iconic buildings in Padova: its rooms are adorned with over three thousand heraldic Coats of Arms depicted in frescoes and carved in stone (see Figure 2, left). These objects represent people who held prestigious academic positions, therefore their presence ofer unique insights into the history and culture of the place. However, traditional display methods limit accessibility for individuals with visual impairments. After an initial discussion with the museum staf, we concluded that reproducing the Coats of Arms was the most suitable choice for the project, for two main reasons: (i) Coats of Arms are omnipresent throughout the museum, adorning every wall and hall, so they are the most distinctive and prevalent feature; (ii) the museum staf usually face challenges in explaining the coats of arms to visually impaired visitors. We adopted two diferent 3D scanners for data acquisition, the EinScan Pro HD from Shining 3D (EinScan) and the Revopoint POP 3 from Revopoint 3D Technologies Inc. (POP3). The choice of using two similar tools derives from the intention of comparing a high-end instrument such as the EinScan (around 14, 000 Euros) with a low cost device (POP3 is around 700 Euros) with the idea that institutions with limited budget could possibly benefit from the same technique. Both devices are handheld and are able to capture the scene by manually moving the device around the object so that diferent points of view are acquired and automatically registered by the complementary software. The EinScan ofers diferent acquisition modes: the HD mode ofers an accuracy of 0.045 mm, Raw data cleaning
Hole filling
Back and acquires 3000 points per second, while the Rapid Scan mode ofers a maximum accuracy of 0.1 mm. The POP3 has a precision up to 0.5 mm at a working distance of 150 − 400 mm. Figure 3 gives an overview of the main post-processing steps, including noise reduction, point cloud alignment, hole iflling and surface reconstruction, performed for each acquired object in order to obtain a printable mesh. The first step involves the removal of all points that are not part of the object itself, such as the background, then the following part consists in obtaining a watertight surface starting from the point cloud, i.e. generating vertices, normals and closing the holes. Finally, since the objects are fixed to the walls of the room, their back need to be reconstructed as a plane so that after printing it can be put on a horizontal surface. This is visible in the rightmost image of Figure 3, where we can notice the additional thickness added to create a planar base. After the characterisation of salient object areas, we adapted the 3D models, isolated the identified regions of interest, and proceeded with model preparation for printing. We decided to employ diferent technologies to print diferent areas of the objects and ofer a better readibility according to the fixation maps (see Section 4.1 for details). In particular, we adopted fused deposition modelling (FDM) and stereolitography (SLA). The FDM technology was chosen to print the 3D model of the complete objects. FDM is a material extrusion technique in which a thermoplastic polymer filament is heated and a movable head proceeds to deposit the material layer by layer. We employed the Creality CR-10 Smart Pro 3D printer. This printer has a print size of 300 × 300 × 400 mm, ofers a printing precision of ± 0.1mm. In Figure 2 we show an image taken while printing a complete object with white material. The second technology we employed is SLA, used to print the surface details requiring an higher accuracy. It is a vat polymerisation method, wherein layers of a liquid contained in a vat are successively exposed to ultraviolet (UV) light. The liquid material reacts to incoming light, resulting in curing only the areas exposed to UV and causing selective solidification. We used the Formlabs Form 3 printer, characterised by a laser spot size of 85 microns, by a build volume of 145 × 145 × 185mm and a layer thickness of 25 − 300 microns. Figure 2 shows the completed print of a selected inscription detail: the object grows layer by layer from top to bottom, and thus a support structure in this case is needed while the printing proceeds. 4.1. Results We acquired 6 coats of arms: Figure 4 shows all of them with their identifiers. Table 1 summarises the ifnal results in terms of acquired points (raw data) and number of triangles for each object and device. Usually, a higher number of points suggests a higher accuracy: looking at Einscan acquisitions, we can observe that objects A, B and D have ≈ 1 points, while object F has ≈ 6 points due to the HD mode that was selected only for the last object. Regarding the POP3 acquisitions, objects C, D and E exhibit roughly half the point with respect to the other objects, denoting a lower surface resolution. Object D was acquired with the two scanners to assess the feasibility and analyse possible limitations of diferent devices. The EinScan shows a higher resolution, while the surface acquired by the POP3 is smoother and exhibits a less marked inscription. Despite the inherent challenges of manual acquisition, the POP3 managed to yield satisfactory results, largely attributed to the capabilities of its software (Revoscan 5), which played an important role in refining the acquired data. After acquisition, we used the acquired models to generate 2-dimensional texture projections on a plane and obtain an RGB image. Figure 5 shows the images used as input for visual saliency. We applied the fixation prediction method as proposed in [37] and used the original weights as provided by the authors. The resulting visual attention is shown in Figure 5, applied to two of our objects. We plot fixation maps with a colour scale representing diferent levels of attention, where 0 value means no attention and 1 indicates the maximum attention. The third and sixth images of Figure 5 show the masking applied to the objects according to the attention map, so that we have a clear interpretation regarding the most salient areas on the objects surfaces. We can notice that for all the analysed objects, we can identify two to three interesting areas exhibiting the highest attention, depending on the individual object features. For all items, one part resulting particularly interesting for our sight is the central part of the Coats of Arms, depicting the symbol representing its owner. Another interesting area for objects A and B is the small cherub on the top, while for other objects (D, E, F) the upper areas do not result particularly relevant. Finally, for some objects (e.g. item F) also the bottom part with inscriptions results attractive. We concluded that the central parts of the objects need to be printed with higher details and also to be highlighted during presentation. Also, we focused on the printing of the cherubs and the inscriptions on the lower parts of the objects to improve readability.
First, we printed the entire objects using FDM printing technique setting the layer height to 0.2mm and an infill density of 15%, Figure 6 (left) shows an object printed with PLA: the overall quality is good, except from some flat areas in which the printing layers are clearly recognizable. In particular, this is quite evident in some details (see Figure 6, center), where the resolution results altered by printing layers. Following these observations SLA printing was adopted to reproduce regions with the highest visual attention. We used Grey v4 resin, well-suited for general-purpose prototyping, particularly for models demanding intricate details, similar to ours. Figure 6 (right) shows a detail printed with SLA technology, ofering higher resolution and a better understanding of the underlying surface details.
As a preliminary result, we involved in a survey two individuals with visual impairments who volunteered to provide initial feedback. The main goal of the session was to determine the objects general usefulness, and also to assess the quality improvement ofered by the direct application of visual attention prediction. During the survey some challenges were identified, particularly regarding the initial comprehension of the objects and the influence of printing layers on readability. In particular, the perception of printing layer for FDM is significative, and brings the need of explaining that they are not part of the object. Also, diferences between FDM and SLA printing were noted, suggesting the need for refinement of printing techniques to optimize tactile perception. Regarding the efectiveness of the visual attention approach, during the survey we took note on which surface regions resulted more interesting from the tactile point of view, and we observed that the regions highlighted by the ifxation prediction were the most attractive surface areas during the tactile examination. Moreover, the participants appreciated the SLA-printed details as helpful means to improve their understanding of the whole object. Overall, the project was deemed useful in providing tactile representations of the Coats of Arms, facilitating comprehension and engagement. As a future work, we aim to extend the survey in a more structured way, collecting feedbacks from a wider range of people, and performing an extensive study about object readability driven by visual attention and fixation prediciton mechanisms.
In this paper we propose to merge 3D reconstruction and printing techniques with computer vision algorithms to enhance the experience of visually impaired visitors. We present an attention-driven method which exploits 3D scanning of artefacts and applies to the printing process of cultural heritage content. A preliminary study involving a survey highlights the efectiveness of the method, giving a strong direction for future improvements and investigations of the proposed method.
This study was funded by the European Union - NextGenerationEU, in the framework of the iNEST Interconnected Nord-Est Innovation Ecosystem (iNEST ECS_00000043 – CUP H43C22000540006). The views and opinions expressed are solely those of the authors and do not necessarily reflect those of the European Union, nor can the European Union be held responsible for them. on Haptic Audio Visual Environments and their Applications, IEEE, 2005, pp. 6–pp. [19] D. Ahmetovic, N. Kwon, U. Oh, C. Bernareggi, S. Mascetti, Touch screen exploration of visual artwork for blind people, in: Proceedings of the Web Conference 2021, 2021, pp. 2781–2791. [20] F. D’Agnano, C. Balletti, F. Guerra, P. Vernier, et al., Tooteko: A case study of augmented reality for an accessible cultural heritage. digitization, 3d printing and sensors for an audio-tactile experience, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 40 (2015) 207–213. [21] A. Reichinger, A. Fuhrmann, S. Maierhofer, W. Purgathofer, Gesture-based interactive audio guide on tactile reliefs, in: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, 2016, pp. 91–100. [22] V. Rossetti, F. Furfari, B. Leporini, S. Pelagatti, A. Quarta, Enabling access to cultural heritage for the visually impaired: an interactive 3d model of a cultural site, Procedia computer science 130 (2018) 383–391. [23] L. Cavazos Quero, J. Iranzo Bartolomé, J. Cho, Accessible visual artworks for blind and visually impaired people: comparing a multimodal approach with tactile graphics, Electronics 10 (2021). [24] J. Montusiewicz, Z. Czyż, R. Kayumov, Selected methods of making three-dimensional virtual models of museum ceramic objects, Applied Computer Science 11 (2015) 51–65. [25] D. Akca, A. Gruen, B. Breuckmann, C. Lahanier, High definition 3d-scanning of arts objects and paintings, Optical 3-D measurement technqiues VIII 2 (2007) 50–58. [26] M. Neumüller, A. Reichinger, F. Rist, C. Kern, 3d printing for cultural heritage: Preservation, accessibility, research and education, 3D research challenges in cultural heritage: a roadmap in digital heritage preservation (2014) 119–134. [27] J. Montusiewicz, M. Barszcz, S. Korga, Preparation of 3d models of cultural heritage objects to be recognised by touch by the blind—case studies, Applied Sciences 12 (2022) 11910. [28] A. Bruns, A. A. Spiesberger, A. Triantafyllopoulos, P. Müller, B. W. Schuller, " do touch!"-3d scanning and printing technologies for the haptic representation of cultural assets: A study with blind target users, in: Proceedings of the 5th Workshop on analySis, Understanding and proMotion of heritAge Contents, 2023, pp. 21–28. [29] P. F. Wilson, J. Stott, J. M. Warnett, A. Attridge, M. P. Smith, M. A. Williams, Evaluation of touchable 3d-printed replicas in museums, Curator: The Museum Journal 60 (2017) 445–465. [30] A. du Plessis, J. Els, S. le Roux, M. Tshibalanganda, T. Pretorius, Data for 3d printing enlarged museum specimens for the visually impaired, Gigabyte 2020 (2020). [31] P. F. Wilson, S. Grifiths, E. Williams, M. P. Smith, M. A. Williams, Designing 3-d prints for blind and partially sighted audiences in museums: exploring the needs of those living with sight loss, Visitor Studies 23 (2020) 120–140. [32] M. Telesinska, Multimodal 3d printed urban maps for blind people. evaluations and scientific investigations, in: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility, 2023, pp. 1–7. [33] D. S. Wooding, Fixation maps: quantifying eye-movement traces, in: Proceedings of the 2002 symposium on Eye tracking research & applications, 2002, pp. 31–36. [34] G. Koostra, L. R. Schomaker, Prediction of human eye fixations using symmetry, in: Proceedings of the Annual Meeting of the Cognitive Science Society, volume 31, 2009. [35] S. S. Kruthiventi, K. Ayush, R. V. Babu, Deepfix: A fully convolutional neural network for predicting human eye fixations, IEEE Transactions on Image Processing 26 (2017) 4446–4456. [36] W. Wang, J. Shen, Deep visual attention prediction, IEEE Transactions on Image Processing 27 (2017) 2368–2378. [37] Y. Song, Z. Liu, G. Li, D. Zeng, T. Zhang, L. Xu, J. Wang, Rinet: Relative importance-aware network for fixation prediction, IEEE Transactions on Multimedia 25 (2023) 9263–9277. [38] W. Wang, J. Shen, X. Dong, A. Borji, Salient object detection driven by fixation prediction, in:
Proceedings of the IEEE conference on computer vision and pattern recognition, 2018. [39] Y. A. D. Djilali, K. McGuinness, N. O’Connor, Learning saliency from fixations, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 383–393.