Data physicalization has emerged as a new method to represent and interact with data physically rather than digitally. Physical representations aford visual analysis in comparable ways to traditional, desktopbased visualization by introducing new capabilities, such as facilitating tactile manipulation, accessible interactions, and immersion, that are beyond traditional 2D visualizations. However, physicalization has historically been a niche aspect of visualization research due to its unique challenges. This work discusses the current challenges and highlights three areas where data physicalization can aid existing research thrusts: broadening participation, supporting analytics, and promoting creative expression.
Data physicalization—the practice of mapping data to physical form—sits at the crossroads
of various domains, including data visualization, tangible user interaction, and design [
The challenges of physicalization stem from how the majority of physicalization artifacts
are single prototypes. Without broader synthesis, these individual design explorations prevent
physicalization from maturing on the field’s broader goals, including fabrication development,
theory-building, and ethical and societal impact. To that end, previous research surveyed and
cataloged physicalizations through diferent lenses, including semiotics [
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). highlights three promising research areas where data physicalization can aid: broadening participation, supporting analytics, and promoting creative expression. Thus, this article discusses the cross-disciplinary challenges that must be addressed for physicalizations to move from simply vision to being successfully embedded in diferent domain applications.
To invest resources into addressing the challenges outlined in this article, researchers and target users must see clear benefits in the development of data physicalization. This section briefly explores the need, purposes, and benefits of physicalizing data.
Many data-driven systems use a WIMP (Windows, Icons, Menus, and a Pointer) paradigm and rely on the graphical user interface (GUI) [12]. However, the WIMP and GUI limit how and where users can represent and interact with data (i.e., context) [12, 13]. Notably, these constraints introduce challenges when users (1) work with multi-dimensional data with an inherent structure and would benefit from physical models (e.g., high-fidelity, material-realistic surgical models) compared to 2D graphical renditions or (2) work in contexts where the traditional GUI is infeasible (e.g., robotic operators supervising in an outdoor field test, doctors operating during surgeries). Data physicalizations provide the opportunity to fundamentally transform human interactions with data. By removing the constraints of the pixels and moving data representations from flat displays into the physical world, data interactions become more inclusive [14, 15], and we can broaden the ways we can experience and communicate data [16, 17, 18].
Knowing systematically how physicalization difers or fits within existing visualization tools requires a deep understanding. Due to physicalization’s cross-disciplinary nature, this understanding calls for careful considerations that are beyond conventional data visualizations [11, 19]. For example, while all visualizations must consider the expressivity of their designs (e.g., data encoding and data interactions), physicalization designers must also be mindful of the physicalization’s structural and contextual considerations [19].
Each data physicalization artifact (un)consciously reflects diferent community perspectives and values [19]. For example, physicalizations from the data visualization community mainly focus on how to efectively encode data and support analytical tasks. Physicalizations from the tangible user interface community explore how to amplify the capabilities of the human body and physical world for interaction design, while artifacts from design investigate how to identify and leverage a material’s potential. In short, by developing this deeper understanding, the opportunities data physicalization, as a field, faces might be both expanding (through values of diferent disciplines) and converging (through the advances in the field).
Through this cross-disciplinary lens, the opportunities data physicalization afords generally
align with three existing research thrusts: broadening participation, supporting analytics,
and promoting creative expression. These research thrusts are derived from the authors’
meta-content analysis of tasks presented in Bae et. al’s design space for physicalizations [19].
This work aims to outline the emerging trends of physicalization research since Jansen et
al.’s formalization of the field in 2015 [
Thrust 1: Broadening Participation. The use and need for visualizations and data are not
just confined to experts anymore. But the design of visualizations holds implicit assumptions
about the user’s sensory, cognitive, and motor abilities [15]. Scholarship highlights how
exclusively digital solutions present challenges and limitations to certain populations (e.g., people
with low vision or visual impairments [
One form of exploration is focusing on non-traditional audiences: children. The novelty of physicalizing data ofers an unprecedented method for children to engage with and better understand data. For example, Data is Yours is a toolkit [21] made out of everyday materials (e.g., paper, cardboard). The toolkit explores how constructionist practices can broaden ways of introducing children to data visualization concepts, and in turn, cultivate their data visualization literacy (DVL). Past work on children’s DVL has often relied on exclusively digital solutions, where they may come of as “black boxes” to young children and limit their embodied experiences[22]. Physical representations, in contrast, enable children to engage in embodied learning that would not be possible with a 2D screen.
Thrust 2: Supporting Analytics. Rooted in visualization, Jansen et al. [
But there continue to be underlying challenges, namely understanding which datasets would be the most meaningful to analyze and knowing how to analyze them physically. By looking at other disciplines, we see possibilities in how physicality may aid in pre-surgical planning [30] or analyzing star formation [31]. But once a dataset has been chosen, it is important to know how physicalization interfaces can support rich, dynamic interactions. Visualization shows the importance of dynamic interactions, where analysts engage in a series of interactions when exploring a dataset [32]. This reflects findings on how the number of data interactions a system supports afects the richness of the data exploration [ 19]. Thus, there is a rich opportunity to explore how physicalizations utilizing state-of-the-art fabrication techniques can support analytics and decision-making processes.
Thrust 3: Promoting Creative Expression. Engineers, scientists, and artists are building
physical artifacts guided by data [33, 34]. The cross-disciplinary space of data physicalization
reveals examples of physical artifacts that are encoded with data but do not meet Jansen et al.’s
utilitarian definition of data physicalization (Thrust 2). Collectively, these artifacts are examples
where data is used as a material source when making. Like a craftsman working with their
surrounding materials, data is now another material source and is part of the artisanal spirit
when making. For instance, Friske et al. created a data-encoded scarf where “[d]ata became one
material among others, no more or less important in design” [34]. The discussion reveals how
the authors had to negotiate between how accurately they wanted to encode the data in the
physical representation versus embracing the spirit of the material (i.e., yarn). This perspective
stands in contrast to Thrust 2, which is more cognitively oriented. In this case, data encoding is
prioritized in terms of its efectiveness in helping users carry out their analytical tasks, such
as comparing values or estimating correlations [
By understanding the benefits of physicalizing data (Section 2.2), we can highlight emerging challenges. The presented challenges are not comprehensive nor do they discuss the downsides of physicalizations (e.g., issues related to sustainability and product lifecycle). Rather they aim to open discussions on where research eforts should be targeted based on current trends.
C1: Fabrication: Constructing data physicalizations involves a large design space [19], which includes but is not limited to material choice, scale [39], interactions, and data encoding. This introduces combinatorial possibilities that a user has to consider when designing physicalizations. In addition, despite material and fabrication advancements, we lack fundamental guidance on how to leverage the analytical afordances of physical representations. This also highlights a technical challenge in understanding how to accurately map data to the given material. Users will need to consider the tradeofs between material properties (e.g., fluid-based systems [40] and smart materials over electromechanical and magnetic actuators [41]).
C2: Interpretation: Currently, there is no formal design language (analogous to Grammar of Graphics [42]) that helps users interpret data physicalizations. Many data physicalizations either use conventional 2D data visualization representations or are idiosyncratic where each encoding is unique to the creator. This starkly contrasts with the field of information visualization where there is an existing design language to help communicate data findings.
The two challenges are closely intertwined due to their overlapping focus on data encoding. For instance, interpreting a physicalization will ultimately arise from its fabrication technique. However, understanding the scope of each challenge can help tackle its associated problems. E.g., How would the interpretation of a physicalization change if only the fabrication method changed (e.g., CNC, cast and molding, 3D printing, papercraft)? Through this process, one may uncover diferent physical afordances or values [ 43] of each fabrication technique.
Data is ubiquitous where it is now woven into the fabrics of our everyday life. However, traditional methods of visualizing data digitally limit how we can represent and interact with data. Physicalization aims to expand this efort, but the field faces challenges due to its crossdisciplinary nature. This article discusses the benefits and purposes of physicalizing data to help invest eforts in addressing the outlined challenges. But these challenges will, no doubt, evolve as technology matures, society changes, and applications of physicalization concretize. To facilitate this maturation, this work serves as a discussion point to shape the field’s future.
This material is based upon work supported by the National Science Foundation under Grant No. IIS-2040489, No. IIS-2046725, and STEM+C 1933915. [10] H. Djavaherpour, F. Samavati, A. Mahdavi-Amiri, F. Yazdanbakhsh, S. Huron, R. Levy, Y. Jansen, L. Oehlberg, Data to physicalization: A survey of the physical rendering process, Computer Graphics Forum 40 (2021) 569–598. URL: https://onlinelibrary.wiley.com/doi/ abs/10.1111/cgf.14330. doi:https://doi.org/10.1111/cgf.14330. [11] P. Dragicevic, Y. Jansen, A. Vande Moere, Data Physicalization, in: J. Vanderdonckt (Ed.), Springer Handbook of Human Computer Interaction, Springer Reference, Springer, 2021.
URL: https://hal.inria.fr/hal-02113248. [12] B. Lee, P. Isenberg, N. H. Riche, S. Carpendale, Beyond mouse and keyboard: Expanding design considerations for information visualization interactions, IEEE Transactions on Visualization and Computer Graphics 18 (2012) 2689–2698. [13] A. Van Dam, Post-wimp user interfaces, Communications of the ACM 40 (1997) 63–67. [14] S. R. Klemmer, B. Hartmann, L. Takayama, How bodies matter: Five themes for interaction design, in: Proceedings of the 6th Conference on Designing Interactive Systems, DIS ’06, Association for Computing Machinery, New York, NY, USA, 2006, p. 140–149. URL: https://doi.org/10.1145/1142405.1142429. doi:10.1145/1142405.1142429. [15] K. Marriott, B. Lee, M. Butler, E. Cutrell, K. Ellis, C. Goncu, M. Hearst, K. McCoy, D. A.
Szafir, Inclusive data visualization for people with disabilities: a call to action, Interactions 28 (2021) 47–51. [16] O. Shaer, E. Hornecker, Tangible user interfaces: Past, present, and future directions, Foundations and Trends in Human–Computer Interaction 3 (2010) 4–137. URL: http: //dx.doi.org/10.1561/1100000026. doi:10.1561/1100000026. [17] J. Hollan, E. Hutchins, D. Kirsh, Distributed cognition: toward a new foundation for humancomputer interaction research, ACM Transactions on Computer-Human Interaction (TOCHI) 7 (2000) 174–196. [18] Z. Pousman, J. Stasko, M. Mateas, Casual information visualization: Depictions of data in everyday life, IEEE Transactions on Visualization and Computer Graphics 13 (2007) 1145–1152. doi:10.1109/TVCG.2007.70541. [19] S. S. Bae, C. Zheng, M. E. West, E. Y.-L. Do, S. Huron, D. A. Szafir, Making data tangible: A cross-disciplinary design space for data physicalization, in: CHI Conference on Human Factors in Computing Systems, CHI ’22, Association for Computing Machinery, New York, NY, USA, 2022. URL: https://doi.org/10.1145/3491102.3501939. doi:10.1145/3491102. 3501939. [20] D. Guinness, A. Muehlbradt, D. Szafir, S. K. Kane, Robographics: Dynamic tactile graphics powered by mobile robots, in: The 21st International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 318–328. URL: https://doi.org/10.1145/3308561.3353804. doi:10. 1145/3308561.3353804. [21] S. S. Bae, R. Vanukuru, R. Yang, P. Gyory, R. Zhou, E. Y.-L. Do, D. A. Szafir, Cultivating visualization literacy for children through curiosity and play, arXiv, 2022. URL: https://arxiv. org/abs/2208.05015. doi:10.48550/ARXIV.2208.05015, accepted to IEEE Transaction on Visualization and Graphics (also appearing in the proceedings of IEEE VIS 2022). [22] M. Resnick, R. Berg, M. Eisenberg, Beyond black boxes: Bringing transparency and aesthetics back to scientific investigation, The Journal of the Learning Sciences 9 (2000) 7–30. [23] A. Everitt, J. Alexander, Polysurface: A design approach for rapid prototyping of shapechanging displays using semi-solid surfaces, in: Proceedings of the 2017 Conference on Designing Interactive Systems, DIS ’17, Association for Computing Machinery, New York, NY, USA, 2017, p. 1283–1294. URL: https://doi.org/10.1145/3064663.3064677. doi:10.1145/ 3064663.3064677. [24] Y. Jansen, P. Dragicevic, J.-D. Fekete, Evaluating the Eficiency of Physical Visualizations, Association for Computing Machinery, New York, NY, USA, 2013, p. 2593–2602. URL: https://doi.org/10.1145/2470654.2481359. [25] R. G. Raidou, M. E. Gröller, H.-Y. Wu, Slice and dice: A physicalization workflow for anatomical edutainment, in: Computer Graphics Forum, volume 39, Wiley Online Library, 2020, pp. 623–634. [26] F. Taher, Y. Jansen, J. Woodruf, J. Hardy, K. Hornbaek, J. Alexander, Investigating the use of a dynamic physical bar chart for data exploration and presentation, IEEE Transactions on Visualization and Computer Graphics 23 (2017) 451–460. doi:10.1109/TVCG.2016. 2598498. [27] A. Veldhuis, R.-H. Liang, T. Bekker, Coda: Collaborative data interpretation through an interactive tangible scatterplot, in: Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI ’20, Association for Computing Machinery, New York, NY, USA, 2020, p. 323–336. URL: https://doi.org/10.1145/3374920. 3374934. doi:10.1145/3374920.3374934. [28] L. J. Perovich, S. A. Wylie, R. Bongiovanni, Chemicals in the creek: designing a situated data physicalization of open government data with the community, IEEE Transactions on Visualization and Computer Graphics 27 (2021) 913–923. doi:10.1109/TVCG.2020. 3030472. [29] A. Thudt, U. Hinrichs, S. Huron, S. Carpendale, Self-Reflection and Personal Physicalization Construction, Association for Computing Machinery, New York, NY, USA, 2018, p. 1–13.
URL: https://doi.org/10.1145/3173574.3173728. [30] A. Hosny, S. J. Keating, J. D. Dilley, B. Ripley, T. Kelil, S. Pieper, D. Kolb, C. Bader, A.-M.
Pobloth, M. Grifin, et al., From improved diagnostics to presurgical planning: highresolution functionally graded multimaterial 3d printing of biomedical tomographic data sets, 3D Printing and Additive Manufacturing 5 (2018) 103–113. [31] N. Imara, J. C. Forbes, J. C. Weaver, Touching the stars: Using high-resolution 3d printing to visualize stellar nurseries, The Astrophysical Journal Letters 918 (2021) L3. [32] S. Card, J. Mackinlay, B. Shneiderman, Readings in information visualization: Using vision to think, Technical Communication Quarterly 9 (1999) 347–351. [33] A. Desjardins, T. Tihanyi, Listeningcups: A case of data tactility and data stories, in: Proceedings of the 2019 on Designing Interactive Systems Conference, DIS ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 147–160. URL: https://doi.org/10. 1145/3322276.3323694. doi:10.1145/3322276.3323694. [34] M. Friske, J. Wirfs-Brock, L. Devendorf, Entangling the roles of maker and interpreter in interpersonal data narratives: Explorations in yarn and sound, in: Proceedings of the 2020 ACM Designing Interactive Systems Conference, Association for Computing Machinery, New York, NY, USA, 2020, p. 297–310. URL: https://doi.org/10.1145/3357236.3395442. [35] T. Nachtigall, O. Tomico, R. Wakkary, P. van Dongen, Encoding Materials and Data for Iterative Personalization, Association for Computing Machinery, New York, NY, USA, 2019, p. 1–12. URL: https://doi.org/10.1145/3290605.3300749. [36] L. Hale, E. Linley, D. M. Kalaskar, A digital workflow for design and fabrication of bespoke orthoses using 3d scanning and 3d printing, a patient-based case study, Scientific reports 10 (2020) 1–7. [37] R. A. Khot, J. Andres, J. Lai, J. von Kaenel, F. F. Mueller, <i>fantibles</i>: Capturing cricket fan’s story in 3d, in: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, DIS ’16, Association for Computing Machinery, New York, NY, USA, 2016, p. 883–894. URL: https://doi.org/10.1145/2901790.2901886. doi:10.1145/2901790.2901886. [38] B. Nissen, J. Bowers, Data-Things: Digital Fabrication Situated within Participatory Data Translation Activities, Association for Computing Machinery, New York, NY, USA, 2015, p. 2467–2476. URL: https://doi.org/10.1145/2702123.2702245. [39] I. López García, E. Hornecker, Scaling data physicalization–how does size influence experience?, in: Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, 2021, pp. 1–14. [40] H. Mor, T. Yu, K. Nakagaki, B. H. Miller, Y. Jia, H. Ishii, Venous Materials: Towards Interactive Fluidic Mechanisms, Association for Computing Machinery, New York, NY, USA, 2020, p. 1–14. URL: https://doi.org/10.1145/3313831.3376129. [41] M. Sturdee, J. Alexander, Analysis and classification of shape-changing interfaces for design and application-based research, ACM Computing Surveys (CSUR) 51 (2018) 1–32. [42] L. Wilkinson, The grammar of graphics, in: Handbook of computational statistics, Springer, 2012, pp. 375–414. [43] J. J. Van Wijk, The value of visualization, in: VIS 05. IEEE Visualization, 2005., IEEE, 2005, pp. 79–86.