Software visualization (SoftVis) is widely used to facilitate the process of obtaining an in-depth understanding of complex software systems. SoftVis designers can draw on a wide pool of pre-existing conceptual metaphors that model the abstract target domain to tangible source domains. As regular user-centered design methods do not provide guidance on choosing metaphorical mappings, SoftVis designers choose conceptual metaphors primarily based on their subjective similarity to the underlying data structure. We want to include image-schematic metaphors in the SoftVis design process to provide designers with guidance on choosing visualization metaphors that are also in line with the users' mental model. This could allow SoftVis designers to make more data-driven design decisions and result in SoftVis that provides better insights.
Maintaining a deep understanding of a software system, like its modular structure or
dependencies between components, is crucial for programmer productivity to understand, change,
and repair code [
Conceptual metaphors that model the abstract target domain to tangible source domains are
commonly used in SoftVis to make aspects of the target domain more understandable. By
choosing source domains from the real world, SoftVis designers want to present the software in
a familiar context [
User-centered design methods, like contextual design [
To facilitate data-driven design decisions, we can use design methods that align the SoftVis to
the software developers’ mental model of the target domain. One of these approaches is based
on image-schematic metaphors (ISM) that are extracted from the users’ subconscious mental
model [21]. Image schemas are abstract representations of basic recurring experiences in the
world that form very early in life [22, 23]. They act as pre-conceptual building blocks that we use
to conceptualize objects and events on a high level of abstraction [23]. As image schemas derive
from ”the human natural understanding of the physical world” — which is something SoftVis
metaphors commonly try to achieve [
The recurrent co-activation of an image schema with a specific target domain results in an ISM [24]. These co-activations form through subjective experiences in the real world, where the target domain can be tangible (e.g., MORE is UP, LESS is DOWN) or abstract (e.g., HAPPY is UP, SAD is DOWN). Due to their ability to describe abstract concepts and events, image schemas are increasingly being utilized for abstract data comprehension. In computer science, e.g., image schemas are used to explain blockchain technology [25] or to describe complex events for artificial intelligence [ 26]. As image schemas are linked to literal linguistic expressions [22, 27], ISM can be extracted from the user’s subconscious mental model by analyzing users’ linguistic expressions [21, 28]. Previous work indicates that user interfaces that are in line with ISM — which were previously coded from the users’ subconscious mental models — can facilitate the users’ subconscious application of prior knowledge [21, 28]. This intuitive use then results ”in an efective and satisfying interaction using a minimum of cognitive resources” [ 29]. As — to the best knowledge of the authors — no studies on ISM-based SoftVis exist, further research is needed to confirm similar positive efects for SoftVis. However, the importance of a SoftVis’ compliance with the user’s mental model for its ability to provide insights is suspected [20].
Software developers (at diferent expertise levels) are the core target audience for SoftVis tools [30]. While other target groups exist, e.g., project managers [31, 32] or students [33], we focus on software developers that work with an object-oriented programming language. In this context, we code ISM based on a list of 47 image schemas from the categories BASIC, SPACE, CONTAINMENT, MULTIPLICITY, PROCESS, FORCE, and ATTRIBUTE described by Hurtienne et al. [21]. As designers generally do not strive to match technical properties but instead match the mental model of the user [34], our primary data source are software developer interviews. However, as software developers work closely with the technical implementation, we explore programming language documentation as an additional data source.
To determine the users’ mental model of our target domain, we explore transcribed user interviews with software developers who perform software comprehension tasks. Following the coding process by Hurtienne et al. [21], contextual interviews are transcribed and ISM coded based on a list of image schemas. The identified ISM are then prioritized based on their frequency across multiple interviews and — for SoftVis designed for multiple object-oriented programming languages – their consistency across programming languages. Ambiguous ISM might further be validated by consolidated additional interviews, as described by Huber et al. [28].
To test if we can code ISM with the proposed method, we conducted contextual interviews with three english-speaking expert-level C# software developers. The interviews lasted ∼ 10 minutes each. The developers performed a software comprehension task where they searched for a bug in the source code. The software developers were instructed to think-out-loud and the interviewer did not ask any questions. We coded ISM from the three interviews and removed all ISM that were not directly related to the software system or that only appeared in one interview. Hence, Table 1 shows the ISM that were present in at least two interviews.
The second data source is programming language documentation. As software developers work closely with the actual technical implementation, the documentation might give additional insights on the image-schematic structure of the target domain. As programming languages consists of specific concepts and naming schemes, we expect resulting image schemas to generally be in line with the ISM identified in the contextual interviews.
We analyzed the first chapter (eight pages) of the oficial Microsoft C# documentation "A tour of the C# language" [35]. Table 2 shows the identified ISM.
The described analysis seems to be capable of coding ISM from software developers’ subconscious mental models. In relatively short interviews, we identified a number of ISM that were present across multiple interviews. However, we did not identify ISM for all target domains (e.g., Namespaces or Methods). This was probably due to the selected comprehension task and the short interview duration. We are confident that we will be able to collect more ISM with longer interviews and diferent comprehension tasks. As the first results suggest that most software components could be mapped to the CONTAINER image schema, we might adjust our methodology to focus more on ISM that describe relationships between the components. These ISM could play a fundamental role for designing SoftVis, as they seem to be one of the key distinctions between existing SoftVis metaphors. For example, the CENTER-PERIPHERY image schema might suggest something similar to the solar system metaphor while the CONTACT image schema might suggest something more similar to the city or island metaphor.
The proposed analysis also seemed capable of coding ISM from programming language documentation. First results indicate that ISM coded from documentation mostly match ISM coded from interviews. However, our small sample size already included one mismatching ISM. The virtual execution system .NET — which is C#-specific — has a SURFACE image schema in the documentation but a CONTAINER image schema in the interviews. While further ISM coding of the documentation is needed, this highlights potential benefits of SoftVis tools that are designed to match the mental model of the user [34]. ISM coded from documentation are probably more in line with the mental model of the language designer (i.e., design model), while the ISM coded from user interviews represent the user model, as described by Norman [36]. But ISM coded from language documentation might still facilitate design decisions, e.g., by resolving ambiguous ISM from user interviews that could not be resolved through additional interviews.
Once the list of ISM is more complete, it could help SoftVis designers to create visualization metaphors that are more in line with software developers’ mental model. ISM could provide guidance on choosing intuitive visualization metaphors and could help to enhance existing SoftVis, e.g., by highlighting mismatching ISM. As SoftVis tools are generally used for mentally demanding and time consuming comprehension tasks, even small optimizations could have a significant impact on the user experience. However, further research is needed to measure benefits of a SoftVis’ compliance with the user’s mental model.
Our future work includes further exploration of our two data sources with the proposed analysis. We will conduct more contextual interviews with software developers in diferent object-oriented programming languages and will continue to analyze programming language documentations. Additionally, we want to apply the identified ISM to SoftVis design. This includes creating novel ISM-based SoftVis metaphors and evaluating existing conceptual metaphors based on their compliance with the ISM of the target domain. Finally, we will evaluate ISM-based SoftVis tools in regards to intuitive use and their ability to provide insights compared to traditionally designed SoftVis. 2CO Communicating complexity: 2013 Conference Proceedings, Edizioni Nuova Cultura, 2013, pp. 12–18. [19] V. L. Averbukh, Approach to semiotic theory of computer visualization, Advances in
Computer Science: An International Journal 4 (2015) 44–54. [20] V. L. Averbukh, M. O. Bakhterev, D. V. Manakov, Evaluations of visualization metaphors and views in the context of execution traces and call graphs, Scientific Visualization 9 (2017) 1–18. [21] J. Hurtienne, K. Klöckner, S. Diefenbach, C. Nass, A. Maier, Designing with image schemas: resolving the tension between innovation, inclusion and intuitive use, Interacting with Computers 27 (2015) 235–255. [22] M. Johnson, The body in the mind: The bodily basis of meaning, imagination, and reason.,
University of Chicago Press, 1987. [23] A. Blackler, J. Hurtienne, Towards a unified view of intuitive interaction: definitions, models and tools across the world, MMI-interaktiv 13 (2007) 36–54. [24] J. Grady, Foundations of meaning: Primary metaphors and primary scenes (1997). [25] I. E. Khairuddin, C. Sas, C. Speed, Blockit: A physical kit for materializing and designing for blockchain infrastructure, in: Proceedings of the 2019 on Designing Interactive Systems Conference, 2019, pp. 1449–1462. [26] M. M. Hedblom, O. Kutz, R. Peñaloza, G. Guizzardi, Image schema combinations and complex events, KI-Künstliche Intelligenz 33 (2019) 279–291. [27] J. M. Mandler, C. P. Cánovas, On defining image schemas, Language and cognition 6 (2014) 510–532. [28] S. Huber, P. Schulz, E. Hauke, J. Hurtienne, Image schematic metaphors in air trafic controllers’ language (2022). [29] J. Hurtienne, Image schemas and design for intuitive use, Doctoral dissertation, Technische
Universität Berlin (2011). [30] L. Merino, M. Ghafari, C. Anslow, O. Nierstrasz, A systematic literature review of software visualization evaluation, Journal of systems and software 144 (2018) 165–180. [31] A. Schreiber, M. Brüggemann, Interactive visualization of software components with virtual reality headsets, in: 2017 IEEE Working Conference on Software Visualization (VISSOFT), IEEE, 2017, pp. 119–123. [32] M. Ogawa, K.-L. Ma, Stargate: A unified, interactive visualization of software projects, in: 2008 IEEE Pacific Visualization Symposium, IEEE, 2008, pp. 191–198. [33] A. Al-Sakkaf, M. Omar, M. Ahmad, A systematic literature review of student engagement in software visualization: a theoretical perspective, Computer Science Education 29 (2019) 283–309. [34] A. Cooper, R. Reimann, D. Cronin, C. Noessel, About face: the essentials of interaction design, John Wiley & Sons, 2014. [35] Micosoft, A tour of the c# language, 2023. URL: https://learn.microsoft.com/en-us/dotnet/ csharp/tour-of-csharp/. [36] D. A. Norman, Cognitive engineering, User centered system design 31 (1986) 2.