The rapid integration of Artificial Intelligence systems (AI) into our daily lives creates challenges with the transparency, explainability, and collaborative communication of these systems. There is a clear separation in understanding between interdisciplinary research groups, stakeholders, developers, and everyday endusers. Creating a common “language” benefits not only current conversations centering AI, but future conversations and directions. With a common “language” the spectrum of AI end-users can voice their concerns and opinions, resulting in its end-users becoming more active contributors to the conversation. This research builds a common (visual) language framework that utilizes a symbology rooted in ontology for representing components of AI systems.
As artificial intelligence (AI) surges to the forefront of major research interests, the explainability,
transparency, and communication of these systems between interdisciplinary collaborators and
the general public becomes a necessity for research, development, and adoption of AI. The rapid
evolution of the state-of-the-art in AI creates new complexities associated with these systems
[
One strategy to mitigate such mistrust and misunderstanding is to improve
communicationrelation between the various types of end-users. Building a way to communicate about these
systems could help improve comprehension, increase explainability, trust, and transparency. All
of these factors contribute to reducing misunderstandings between multidisciplinary groups and
laypeople and increasing the ethical and eficient development of these systems [
Of course, utilizing symbols to convey information is certainly not new. Indeed, we can trace
symbolic communication from pre-history [
Problem Statement The rapid evolution of AI can result in a few specific problems, such
as a correspondingly widening gap in AI democratization [
The goal of the symbolic framework is to work cross-culturally, facilitating understanding, acting as a unified language of understanding. Furthermore, the symbology and its ability to break down complex systems will help to increase explainability, transparency, and understanding of these systems. The ontology of the framework can guide users on how specific elements in an AI system work together, giving structure and guidance to both the socio-technical users and their needs. All of the components of the framework collaborate to ensure a usable and efective tool for communicating about these systems.
The foundational components for SLAi originate from the Boxology framework [
An ontology is used to describe the concepts and relationships that can exist for agents; essentially,
an ontology is a way to organize knowledge [21]. Ontologies can be utilized in many diferent
projects and in many diferent ways, from the medical field [ 22] to education [
Providing the ontological formalization to the boxology, why not use its visual representations rather than creating a symbology? If one were to only use the boxology visuals, it would be easy to imagine how large and intricate that representation could be when applied to a more complex scenario. Since the goal of SLAi is to simplify the representation of the internal workings of AI systems the symbology would need to both represent simple systems and condense complex systems to aid in overcoming issues with visually overstimulating representations. As such we must look to both human factors and cognitive sciences, taking a human-centric approach. Guidelines like those of [28] which include 9 guidelines centered around diferent elements used in visualization. While some of these guidelines directly relate to medical representation, others have a broader description and application; these include: (1) end-user’s ability to remember information, (2) end-user’s perceived efective communication, and (3) end-user’s perceived clarity and guidance to perform actions. Furthermore, aligning the usability of our framework with that of Peter Morville’s UX Honeycomb and its 7-factor design [29] can further ensure the usability of the framework. Creating a tool in parallel to a design framework, such as the UX Honeycomb, is beneficial in ensuring that the tool or framework that is being created will be usable by its intended audience. The seven factors of the UX honeycomb, in no particular order, include: (1) Useful, (2) Desirable, (3) Accessible, (4) Credible, (5) Findable, (6) Valuable, and (7) Usable, all valuable metrics to consider when developing a tool or interface. All seven factors work together to guide development towards usability and help define the priorities of our framework. Even the use of psychological concepts like the Just-Noticable Diference (JND), which describes the threshold of change that must occur to a visual for the diference to be noticeable [ 30]. JND mainly contributes to the frameworks’ want for composability, how much do we need to alter a symbol for an end-user to notice a diference, but not completely erase its original form? While this is the furture path for this research, it is necessary to keep the composability aspect in mind when developing these symbols, making it easier for composability in the future.
There is a rich history in graphical representations, symbology, and iconography that can provide qualitative methods to help the analysis of visual content and the interpretation of that content [31, 32]. Even the popular technology company Apple has a designer’s guide, describing the best practices when creating an icon, including the simplicity in design and how to develop icons that work with multiple platforms [33]. There are also tools already available that use icons and symbols to break down complex concepts like those found in the Orange Data Mining tool [34]. The orange tool has a drag-and-drop interface that uses widgets to perform various data mining techniques in a simple and non-technical way, democratizing data mining techniques.
The SLAi framework needs to have a symbology that is able to reflect the internal workings of AI, which means the symbology needs to be able to undergo transitional changes to reflect what is happening to the data in AI and have symbols that are grounded in their representation, meaning the symbolic representation reflects its real-world counterpart [ 35, 36]. Therefore, finding a balance between representing those changes without depriving it of its initial meaning and having the symbols grounded. These are all important aspects to consider since they will play a role in the usability and adaptability of the system.
SLAi must reflect both the components of AI systems, but also the end-user perception. A symbology is useless if it can not be understood by those who need to use it. To ensure that the individual symbols in our set are representative to their concepts, we ran a qualitative study to acquire some user perspectives. The study was conducted in two phases; it was distributed to 18 participants, comprising students and faculty from the Department of Computer Science & Engineering at Wright State University (WSU). For now, we have scoped the development using only perspectives of individuals more likely to be familiar with these terms, by doing so we create a more solid scientific foundation for the research ensuring that the symbols reasonably portray the technical underpinnings of AI components. 3.1. Phase I Phase I of the study used a form of end-user elicitation [37], which gathers end-user feedback by asking participants to sketch what they think represents a specific term or definition given. Specifically for the study conducted, we used the terminology from the boxology framework [38]. The terms used included: Actor, Data-input, Symbolic-input, Inference, Data-output, Symbolic-output, Statistical model, Train, Semantic model, & Transform. Each term was given a broad definition, and participants were asked to draw how they would represent this term and also provide some keywords that they associated with the term. This survey was completely open-ended, allowing the participants to freely express their perceptions. The feedback from the end-users was gathered and analysed, specifically localizing patterns that emerge between participants. The feedback gathered from this survey provided valuable end-user input for the more technical side of AI, Figure 2: Proposed Symbology demonstrating the representation that would be familiar to a group with more expertise.
After all participants finished Phase I, there was an intermission of 2 weeks before starting Phase II. Phase II required participants to fill out another survey, this time on the survey software Qualtrics. The Qualtrics survey consisted of 19 Yes or No questions that displayed one of the proposed symbolic representations created by the researcher to represent one of the specific terms. The symbols used were created by the research team and are represented in Figure 2. These symbols were constructed following the recommendations provided by the literature and an adaptation of the Boxology’s simple geometric representations. The symbols represent specific AI components and terms used in the boxology patterns. The symbols read as follows: Starting at the first row, top left and moving right: Data-Input, Actor, Data-Inference Process, Data-Output, Data-Semantic Model. Starting at the left side of the second row and moving right: Data-Training Process, Data-Transformation, Symbol-Input, Symbol-Inference Process, Symbol-Output. Lastly, starting at the left side of the third row: Symbol-Statistical Model, Symbol-Train Process, SymbolTransformation, Symbol-Semantic Model, Data-Statistical Model. Immediately following their yes or no answer, they were asked to identify in a Likert scale how confident they were in their answer, a technique used by [39]. The Likert scale was constructed from 1-5 with 1-being no confidence and 5-being confident. This technique was used to determine what changes, if any, needed to be made to the symbology to help with perception and recognition. Combining the results from both Phases of the study will aid in the construction of the symbology, ensuring that the technical aspects of an AI system are represented, but also that it is end-user-friendly and non-complex.
The nature of each phase of the experiment and its survey type required the use of a diferent evaluation method. In phase I, a bag-of-words method was used for the keywords provided by the participants. The output of this can be found in Github [40]. The results from the bag-of-words method are important when localizing common terminology surrounding a concept, getting the research closer to a successful representative symbology.
Furthermore, participants were asked to draw how they would represent the given term of interest. The drawing then underwent thematic coding, and the results from were enlightening but not in a way the researchers expected. The drawlings revealed little to no similarities between participants, with some even opting out of the drawing portion of the survey. Another issue is that many of the drawings were very detailed, highly intricate, and individualistic, resulting in the participants having no commonalities between each other (figure 3).
Phase II participants were asked to complete a 38-question survey on Qualtrics, a research survey software. The questions were iterated between a binary Yes or No question format about a displayed symbol and then immediately followed by a 5-point Likert scale rating their confidence in their previous answer. Two participants responses were dropped due to lack of completion of the survey, making the sample size 16 for phase II. The results from the 19 binary based questions showed that a majority of the participants split on their agreements and disagreements on the proposed symbol. Represented in Figure 4 are a few of the results that participant responses difered more drastically on. Participants disagreed with the researchers representation of Actor with 88.2% of participants disagreeing with its representation (figure 4). Furthermore, the participants responded that the representations for Data-input seem to align with that of the researchers, with 58.8 participants agreeing that this symbol represents Data-Input. This result can be visually depicted in Figure 4. The Likert Scale was evaluated using Fleiss’ kappa [41], which measures the reliability of agreement between a fixed number of raters, in this case the raters are the participants that took the survey.
Once the calculation for K is found, the output will lie between -1 and 1. If the output is -1 then the agreement between raters is worse than chance, meaning there is an inconsistency between raters. If the output is 1, then there is almost a perfect agreement between raters. The results of Fleiss’ kappa on the participants’ responses revealed a Fleiss’ kappa of -0.63, meaning that the raters were disagreeing more than randomly. The low Fleiss Kappa score shows that the proposed symbology developed by the researchers in phase II does not represent these components well, calling for a redesign of the symbols.
After evaluating the results from both phases I & II we can conclude that the proposed symbology created by the researchers needs redesigned. The lack of commonality between illustrations, the low Fleiss’ kappa output, and the lack of any strong agreement to the proposed symbols in the qualtrics survey proves that more representative attributes need to be incorporated into the design of the symbols. End-user feedback provided in phase I can help guide the researchers in how to incorporate such design attributes. However, it should be noted that these results may be due to such a small sample size. Sampling out of the computer science department at WSU, and with only 16 participants for phase II there was not enough participants to truly get a grasp on which symbols needed major modifications compared to those that needed minor changes. Furthermore, the lack of non-expert input can severely impact the representation of the symbology as well as, the translation of these symbols to individuals outside of the creation process and background knowledge. Having more perspectives from all end-users who are more general rather than technical can give valuable feedback on the representation of the symbology, however, starting with more technical perceptions and them simplifying seems to be a more intuitive process than vise versa. Another issue is that the symbology might simply need to be put into action, utilizing a more tutorial-based interaction rather than simply being presented in a survey. A tutorial approach would also increase the involvement of the ontology, since this research was more focused on the development of the symbology to align with the already constructed ontology of
Future work for this research is to conduct a similar study, keeping the scope to the more technically incline participants to establish a correct representation of these systems before opening up the symbology to the perceptions of more general users. The study will be opened up to all students and faculty in the college of engineering and computer science here at wright state as well as some potential recruitment from the local Air force base civilian research labs. The increase of the sample will provide more potential candidates from the study and thus increase the statistical significance of the experiment moving forward. Once there is an established symbology that reflects the perception of the technical users (ensuring the systems are correctly represented) the research will then move to gain the perspective of the general public. Moreover, Conducting a more controlled study similar to that of Fay et al. (2018) and their creation of shared symbols methodology utilizing similar tactics like the game pictionary [42] and similar papers would provide a more favorable outcome to this type of experiment. Once the base components of the symbology are created, the symbology will then be added to the ontology, and end-user testing will be opened up to the general population.
During the preparation of this work, the author used Grammarly and ChatGPT in order to: Grammar and spelling check, Paraphrase and reword. After using this tool/service, the author reviewed and edited the content as needed and take full responsibility for the publication’s content.
Alexis Ellis acknowledges support from the Advanced Air Mobility, Ohio Department of Higher Education (ODHE), OH, USA. glyphs for representing ai systems, in: HHAI 2024: Hybrid Human AI Systems for the Social Good, IOS Press, 2024, pp. 105–113. [21] T. R. Gruber, A translation approach to portable ontology specifications, Knowledge acquisition 5 (1993) 199–220. [22] M. Ivanović, Z. Budimac, An overview of ontologies and data resources in medical domains,
Expert Systems with Applications 41 (2014) 5158–5166. [23] C. Shimizu, K. Hammar, P. Hitzler, Modular ontology modeling, Semantic Web 14 (2023) 459–489. [24] M. Jarrar, R. Meersman, Ontology engineering–the dogma approach, Advances in Web
Semantics I (2009) 7–34. [25] M. Fernández-López, A. Gómez-Pérez, N. Juristo Juzgado, Methontology: from ontological art towards ontological engineering (1997). [26] P. Hitzler, M. Krötzsch, B. Parsia, P. Patel-Schneider, S. Rudolph (Eds.), OWL 2 Web Ontology Language: Primer (Second Edition), W3C Recommendation, 11 December 2012. Available at http://www.w3.org/TR/owl2-primer/. [27] A. Eberhart, C. Shimizu, S. Chowdhury, M. K. Sarker, P. Hitzler, Expressibility of owl axioms with patterns, in: The Semantic Web: 18th International Conference, ESWC 2021, Virtual Event, June 6–10, 2021, Proceedings 18, Springer, 2021, pp. 230–245. [28] D. Shaalan, S. Jusoh, Visualization in medical system interfaces: Ux guidelines, in: 2020 12th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), IEEE, 2020, pp. 1–8. [29] P. Morville, User experience design, https://semanticstudios.com/user_experience_design/, 2004. Accessed: 2025-06-02. [30] M. K. Stern, J. H. Johnson, Just noticeable diference, The corsini encyclopedia of psychology (2010) 1–2. [31] M. Rampley, Iconology of the interval: Aby warburg’s legacy, Word & Image 17 (2001) 303–324. [32] M. G. Müller, Iconography and iconology as, The SAGE handbook of visual research methods (2011) 283. [33] Apple Inc., Sf symbols - human interface guidelines, 2025. URL: https://developer.apple.com/ design/human-interface-guidelines/sf-symbols, accessed: 2025-05-29. [34] B. Zupan, J. Demšar, M. Luštrek, M. Juvan, A. Vavpetič, L. Curk, Orange: Data mining toolbox in python, Journal of Machine Learning Research 14 (2013) 2349–2353. URL: https: //orangedatamining.com. [35] M. Taddeo, L. Floridi, Solving the symbol grounding problem: a critical review of fifteen years of research, Journal of Experimental & Theoretical Artificial Intelligence 17 (2005) 419–445. [36] S. Harnad, The symbol grounding problem, Physica D: Nonlinear Phenomena 42 (1990) 335–346. [37] M. Tohidi, W. Buxton, R. Baecker, A. Sellen, User sketches: a quick, inexpensive, and efective way to elicit more reflective user feedback, in: Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles, 2006, pp. 105–114. [38] A. Meyer-Vitali, W. Mulder, M. H. T. de Boer, Modular Design Patterns for Hybrid Actors, 2021. URL: http://arxiv.org/abs/2109.09331, arXiv:2109.09331 [cs] version: 1. [39] M. Yeh, D. C. Chandra, A. John, et al., Designing and evaluating symbols for electronic displays of navigation information: Symbol stereotypes and symbol-feature rules, Technical Report, John A. Volpe National Transportation Systems Center (US), 2005. [40] kastle lab, SLAi repository, https://github.com/kastle-lab/SLAi, 2025. Commit accessed mm/dd/yyyy. [41] R. Falotico, P. Quatto, Fleiss’ kappa statistic without paradoxes, Quality & Quantity 49 (2015) 463–470. [42] N. Fay, B. Walker, N. Swoboda, S. Garrod, How to create shared symbols, Cognitive Science 42 (2018) 241–269.