=Paper=
{{Paper
|id=Vol-3136/paper8
|storemode=property
|title=Representable AI: towards a unified view of core dimensions for a visual framework
|pdfUrl=https://ceur-ws.org/Vol-3136/paper-8.pdf
|volume=Vol-3136
|authors=Angela Locoro,Paolo Buono,Marco Winckler,Silvia Corchs
|dblpUrl=https://dblp.org/rec/conf/avi/LocoroBWC22
}}
==Representable AI: towards a unified view of core dimensions for a visual framework==
https://ceur-ws.org/Vol-3136/paper-8.pdf
Representable AI: Towards a Unified View of Core
Dimensions for a Visual Framework
Angela Locoro1 , Paolo Buono2 , Marco Winckler3 and Silvia Corchs1
1
Dipartimeto di Scienze Teoriche e Applicate, Universitá degli Studi dell’Insubria, Varese, Italy
2
Dpartimento di Informatica, Universit degli Studi di Bari Aldo Moro, Bari, Italy
3
Université Nice Sophia Antipolys, Sophia Antipolis, France
Abstract
Data visualization made of multiple visualization techniques, e.g., dashboards and small multiples, is
taking the scene as data, AI algorithms and their analysis are becoming more complex and articulated.
However, still too little is said about what are the core dimensions of these interactions that may contribute
to characterize visualization techniques orchestration in scenarios where humans and AI work together
and communicate through visual languages, and what is the differential in complexity with respect to
single charts interaction. Depending on such dimensions, their level, and their combination, interaction
may require a cognitively growing effort. The present study aims at giving a unified view of complex
visual frameworks in order to identify the invariants of visualization techniques characterization, and
proposes a group of necessary and sufficient dimensions emerging when visualization techniques are the
focus of the design and may be the focus of interaction between humans and AI. The paper identifies and
discusses these dimensions, starting from the literature, and giving a characterization of each of them in
terms of constituent levels. The framework may be applied to analysis of a range of data visualization
tools and approaches, towards their concrete application to a distributed visual cognition framework
where humans-AI interactions will take place.
Keywords
visualization techniques, multi-chart interaction, conceptual framework for distributed cognition, visual
AI
1. Introduction and Background
A visualization technique refers to a graphical view of data in the form of charts (static like
bar charts and histograms, or interactive like treemaps) allowing users to analyse and visually
explore a dataset, the input and the output of an algorithm or even its intermediate steps. Whilst
the data might work with multiple chart types, it is often recommended to select the one that
ensures a message clear and accurate according to users needs [1][2][3]. And yet, multi-charts
systems (that combine two or more views) have been proved useful to show diversity and
complementary of information visualization techniques [4]. Indeed, most recent and widely
Proceedings of CoPDA2022 - Sixth International Workshop on Cultures of Participation in the Digital Age: AI for
Humans or Humans for AI? June 7, 2022, Frascati (RM), Italy
$ angela.locoro@uninsubria.it (A. Locoro); paolo.buono@uniba.it (P. Buono); marco.winckler@inria.fr
(M. Winckler); silvia.corchs@uninsubria.it (S. Corchs)
0000-0002-6740-8620 (A. Locoro); 0000-0002-1421-3686 (P. Buono); 0000-0002-0756-6934 (M. Winckler);
0000-0002-1739-8110 (S. Corchs)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)
74
Angela Locoro et al. CEUR Workshop Proceedings 74–80
Figure 1: A conceptual framework for multi-chart interaction: basement (overall) level, data level
(dashed box at the left), and orchestration (Visualization Techniques) level (at the top).
adopted tools use multi-charts as a means to design online dashboards [5, 6], animated views [7]
and graph exploration [8]. Nonetheless, envisioning how visualization techniques may act as
the primary communication means between humans and AI is not a simple task. The dangling
question here is: how to systematically describe the dependencies when combining multiple
charts to support this kind of distributed cognition and communication?
As we shall see, describing visualization techniques and the orchestration of multi-charts
is a daunting task. To the best of our knowledge, the literature have investigated diverse
aspects that contribute to the orchestration of multi-chart systems (i.e. [9]) but it still misses a
comprehensive framework showing all the dependencies. In this paper we try to combine all
previous works found in the literature and then we propose a conceptual framework aimed at
support the discussion in the community about the need of methods for describing visualization
techniques and their orchestration.
In the next sections we identify conceptual entities for describing the techniques for informa-
tion visualization. Then we identify four invariant dimensions that might help to explore the
dependencies between multi-charts systems and communication through them.
2. Towards a conceptual framework
The conceptual framework presented in this paper moves from a first conceptualization of the
interaction of a user with a visualization technique. The entities and relationships involved in
the interaction are shown in Figure 1. The basement level of the figure describes the entities
in a single chart. A user expresses a goal which might be accomplished by a task through
a set of interactions that contribute to the execution of a Viz technique (short name for
visualization technique). The user interaction might also trigger special operations for querying
and processing raw data so that the resulting processed data maps the visualization technique.
The upper layer in Figure 1 shows the orchestration level of multiple visualization techniques.
It is important to notice that all entities continuously co-evolve. Next section, we will focus on
that specific layer of visualization technique/s orchestration.
75
Angela Locoro et al. CEUR Workshop Proceedings 74–80
3. The Dimensions of the Visual Frameworks
Four invariants that play a major role for describing the orchestration of multiple visualization
techniques we identified (based on the literature and on our unifying effort). These dimensions
may affect data visualization tools design, their functionalities and limitations, and the rationale
behind many multi-charts problem-solving approaches.
3.1. Composition (Space) Levels
Most works in the literature refers to the disposition of charts along the Space dimension
([10], [11], [12]).
3.1.1. No composition
Whenever only one visualization technique is exploited, there is no composition at all. Another
example, with more than one visualization technique, is in the “overview + details” case [13],
like in the Gapminder tool, where no composition facility is provided (even if two – or more –
visualizations are present in the screen, they refer to the same visualization technique but at dif-
ferent zoom level). The composition can apply according to the other three levels: juxtaposition
of visualization techniques, superimposition and nesting. Each subsequent level may contain
the previous one.
3.1.2. Juxtaposition
It occurs when two visualization techniques are placed side by side, in whatever orientation
(high-to-low or left-to-right). In a multi-charts view, there could be juxtaposition of one or
more of these juxtaposed visualization techniques. An example of juxtaposed visualization
techniques are simple dashboards where the layout is composed of side-by-side and top-to-
bottom juxtaposed visualization techniques.
3.1.3. Superimposition
It occurs when two visualization techniques are (partially) overlapping. In geo-maps visu-
alization, different layers may be superimposed (e.g., the physical layer and the toponymy
layer).
3.1.4. Nesting
Nesting two visualization techniques means to include one visualization technique inside the
layout of a primary visualization technique, in order to extend the range of information included
in one visualization technique with a second visualization technique. Most of the time, nesting
includes superposition of elements of two or more visualization techniques. For example, graph
links may be nested inside a treemap structure, as in [14]. Nesting could also apply to the
arrangements of elements in a hierarchical layout (see for example [12]). When multiple charts
are in place, nested visualization techniques may also be juxtaposed in different guise.
76
Angela Locoro et al. CEUR Workshop Proceedings 74–80
3.2. Synchronization (Time) Levels
During interaction, visualization techniques may be positioned along the time dimension too.
Depending on the kind of visualization technique and on the layout structure, the synchroniza-
tion of views may assume levels of growing complexity.
3.2.1. No synchronization
The basic level is when no synchronization of visualization techniques is present. Typically,
this situation arises when only one visualization technique is present in the layout.
3.2.2. Sequential synchronization
Sequential synchronization arises when visualization techniques are visible one after the other
(for example, in a multi-page environment, like the one provided in Tableau Stories or dashboard-
to-dashboard drilldowns, like the ones provided in the Kibana environment [5].
3.2.3. Parallel Synchronization
In parallel synchronization, visualization techniques are visible all at a time. This is the typical
configuration of small multiples visualizations or of dashboards layout (like, for example, in
Power BI, Tableau or Kibana environments).
3.2.4. Interleaved Synchronization
When visualization techniques are all present and evolving in time, their synchronization
is “interleaved”. This kind of synchronization may be experienced in parallel, and parallel
interleaved visualization techniques may be experienced in sequence, depending on the layout
structure. An example of the first kind is the Gapminder tool [7], which provides animated time
series where points at different time are all present in the same layout.
3.3. Coordination (Tool) Levels
In case a composition exists, it must be coordinated , i.e., visualization techniques can change
congruently. For this dimension we adopt a view of coordination as defined in groupware inter-
action [15][16][17][18]. Coordination can be done according to one of the level of coordination
shown in the next paragraphs.
3.3.1. No coordination
When there is no coordination, the layout may either contain only one visualization technique or
the composition is arbitrary, i.e., no congruent changes arise. Many tools with no coordination
are available (see for example tools for single charts design or “disconnected” visualization
techniques in dashboard design, i.e., where a change in one visualization techniques does not
imply a change into another visualization technique).
77
Angela Locoro et al. CEUR Workshop Proceedings 74–80
3.3.2. Manual coordination
In manual coordination, two or more visualization techniques are related by a commonly
changing dataset, a commonly changing visual element (e.g., categorical scale) or a common
functional element (e.g., the same filter applied to two visualization techniques). An example
of manual coordination is, for example, Tableau, where filters can be set to select elements in
more than one visualization technique.
3.3.3. Coordination on demand
In coordination on demand, semi automatic functionalities may be activated on different visual-
ization techniques. For example, in the Knime a visual tool for creating data analytics pipelines
[19], components may relate and be updated congruently on demand (as soon as the pipeline is
executed) or may not, depending on the users’ needs and configurations in the system.
3.3.4. Automatic Coordination
Automatic coordination arises when the user control on the coordination of visualization
techniques is only partial or absent. An example of this is Gapminder, which provides a template
layout where visualization techniques are automatically composed and animated through a
timeline, and only the data can be arbitrarily selected by the user (not the visualization and
animation mode).
3.4. Integration (Data) Levels
Although not in the foreground with respect to the interactions among visualization techniques,
the Integration dimension (data level) included in visualization techniques is present in our
framework. Ranging from “no integration at all” to “combination/merging/ parsing” of multiple
sources, this dimension was still under-explored and under-defined. This is due to the various
complexity in data integration for which many scholars have struggled to find a solution or at
least an agreement. Many attempts made to capture, at least descriptively, this complexity are
available in the literature about data integration 1 . We are not exploring further this dimension,
leaving the levels undefined in terms of what is the complexity involved in each of them, but
only mentioning them for completeness and inclusiveness. We may briefly touch here only
the fact that integration of data may become part and parcel of the growing complexity of
visualization techniques design, interaction and evaluation.
References
[1] E. A. Bier, S. K. Card, J. W. Bodnar, Entity-based collaboration tools for intelligence analysis,
in: 2008 IEEE Symposium on Visual Analytics Science and Technology, 2008, pp. 99–106.
doi:10.1109/VAST.2008.4677362.
1
See, for example, the table schematized in http://www.cs.toronto.edu/~yuana/DI-Complexity.html for an
overview of data integration complexity.
78
Angela Locoro et al. CEUR Workshop Proceedings 74–80
[2] C. He, L. Micallef, L. He, G. Peddinti, T. Aittokallio, G. Jacucci, Characterizing the quality of
insight by interactions: A case study, IEEE Trans. Vis. Comput. Graph. 27 (2021) 3410–3424.
doi:10.1109/TVCG.2020.2977634.
[3] C. Ziemkiewicz, R. Kosara, Implied dynamics in information visualization, in: Proceed-
ings of the International Conference on Advanced Visual Interfaces, AVI ’10, Associ-
ation for Computing Machinery, New York, NY, USA, 2010, p. 215–222. doi:10.1145/
1842993.1843031.
[4] M. Q. Wang Baldonado, A. Woodruff, A. Kuchinsky, Guidelines for using multiple views
in information visualization, in: Proceedings of the Working Conference on Advanced
Visual Interfaces, AVI ’00, Association for Computing Machinery, New York, NY, USA,
2000, p. 110–119. doi:10.1145/345513.345271.
[5] Kibana, 2022. URL: https://www.elastic.co/kibana, (accessed January 2022).
[6] Tableau, 2022. URL: https://www.public.tableau.com/s, (accessed January 2022).
[7] Gapminder, 2022. URL: https://www.gapminder.org/tools, (accessed January 2022).
[8] A. Menin, R. A. Cava, C. M. D. S. Freitas, O. Corby, M. Winckler, Towards a visual approach
for representing analytical provenance in exploration processes, in: 25th International
Conference Information Visualisation, IV 2021, Sydney, Australia, July 5-9, 2021, IEEE,
2021, pp. 21–28. doi:10.1109/IV53921.2021.00014.
[9] D. A. Keim, Information visualization and visual data mining, IEEE Trans. Vis. Comput.
Graph. 8 (2002) 1–8. doi:10.1109/2945.981847.
[10] W. Javed, N. Elmqvist, Exploring the design space of composite visualization, in: 2012
ieee pacific visualization symposium, IEEE, 2012, pp. 1–8.
[11] M. Gleicher, D. Albers, R. Walker, I. Jusufi, C. D. Hansen, J. C. Roberts, Visual comparison
for information visualization, Information Visualization 10 (2011) 289–309. doi:10.1177/
1473871611416549.
[12] X. Chen, W. Zeng, Y. Lin, H. M. Al-Maneea, J. Roberts, R. Chang, Composition and
configuration patterns in multiple-view visualizations, IEEE Trans. Vis. Comput. Graph.
27 (2021) 1514–1524. doi:10.1109/TVCG.2020.3030338.
[13] B. Shneiderman, The eyes have it: A task by data type taxonomy for information visual-
izations, in: Proceedings of the 1996 IEEE Symposium on Visual Languages, VL ’96, IEEE
Computer Society, USA, 1996, p. 336.
[14] N. Henry, J.-D. Fekete, M. J. McGuffin, Nodetrix: a hybrid visualization of social net-
works, IEEE Transactions on Visualization and Computer Graphics 13 (2007) 1302–1309.
doi:10.1109/TVCG.2007.70582.
[15] Y. Laurillau, L. Nigay, Clover architecture for groupware, in: Proceedings of the 2002 ACM
Conference on Computer Supported Cooperative Work, CSCW ’02, Association for Com-
puting Machinery, New York, NY, USA, 2002, p. 236–245. doi:10.1145/587078.587112.
[16] H. Fuks, A. Raposo, M. A. Gerosa, M. Pimental, C. J. P. Lucena, The 3c collaboration
model, Encyclopedia of E-Collaboration (2008) 637–644. doi:10.4018/978-1-59904-
000-4.ch097.
[17] C. A. Ellis, S. J. Gibbs, G. Rein, Groupware: Some issues and experiences, Commun. ACM
34 (1991) 39–58. doi:10.1145/99977.99987.
[18] U. M. Borghoff, J. H. Schlichter, Computer-Supported Cooperative Work: Introduction to
Distributed Applications, Springer-Verlag, Berlin, Heidelberg, 2000.
79
Angela Locoro et al. CEUR Workshop Proceedings 74–80
[19] Knime, 2022. URL: https://www.knime.com, (accessed January 2022).
80