The rep2rep project is developing an AI tool to automatically select an appropriate representation to solve a particular problem for a particular person. A prerequisite of this tool is to understand (i.e., model) how a reader interprets a representation. But interpretations can vary wildly between novices and experts, readers of similar ability, or even the same reader in diferent tasks. We present a theory and notation (RIST and RISN) for analysing the cognitive features of a representation's interpretation, and introduce a web app to construct RISN models. These models provide information about cognitive properties of representations to guide automated representation selection to support human problem solving.
When solving problems, representation choice can be critical [
The rep2rep project is working to support people in their representation selection by building
AI tools that can analyse the problem being solved, the representational systems available
to represent the problem, and the person solving the problem. The person matters – what
is trivial for you might be dificult for me. But to accurately assess how suitable a specific
representation is for a specific person, we must analyse their interpretation of the representation.
In previous work, we identified cognitive properties of representations that influence its cost
(M. Jamnik)
upon a person [
RIST and RISN serve as the foundation of cognitive property analysis [
RIST and RISN provide means to modelling the interpretations of representations across many modalities and domains. At its core, there are four ‘schemas’, and three ‘connections’ between them. We also idenUtndoifRedoyRep‘reseintatdionRi-Scoheme mR-Dimensison’R-Symb–olPlacehcolderoDmuplicate mConectoAncnhorElquivyalence OoverlapcDiscjoint uGenerricrUnilinknDgelete Garid rMranuaalngements of schemas and links. We provide a brief sketch of the theory and notation; details are available in previous work [6, 7].
RIST has four schemas: Representation, R-Scheme, R-Dimension, and R-Symbol.
Representation
Graphic
R-Scheme Graphic
R-Dimension, Q
Graphic, Q
R-Symbol Graphic A schema associates a concept with a graphic, mapping an aspect of the interpretation to a feature of the representation. The R-Dimension schema also associates each with a quantity scale (one of Nominal, Ordinal, Interval, and Ratio [8]; the bold letter replaces Q).
Schemas may be connected in three ways: they are part of a hierarchy, e.g., an R-Symbol is one element of a1n4erors R1war-ning DShow imension; one concept is anchored beneath another, e.g., a region is anchored by bounding curves; or they are equivalent, requiring mental bookkeeping, e.g., there are two derivations of one quantity. Each connection has rules on which schemas it may join.
After building many RISN models, we observed recurring patterns; we call these idioms. To date, the idioms we have collected fit into three classes: collections of elements, how dimensions are composed and decomposed, and how coordinate systems can be read.
Having briefly described the theory and notation, we now consider how RISN models get built. An analysis consists of three steps in two phases: first, we decide on the interpretation; second, we (a) identify representation features and how they are modelled in RIST schemas, and (b) build the RISN model by connecting these schemas. We emphasise the importance of phase one – if the interpretation is not settled, we risk building an incoherent model that mixes interpretations.
To facilitate building RISN models, we have created a web app, Figure 1. This web app provides a structured environment in which schemas and links between them are easily created and updated; analysts do not need to remember every one of the schema’s slots, as the inspector panel on the right lists them. We provide extensive help and shortcuts to make building RISN models as efortless as possible, freeing the analyst to focus on the interpretation, not RISN syntax. The help and shortcuts are shown when the analyst hovers over labels and buttons in the interface; we also include these in a manual, which describes how to use the editor.
At the bottom of the window in Figure 1, we have the ‘intelligence’ panel. The RISN editor is continually analysing the model, checking for both ‘Errors’ – e.g., illegal connections – and for ‘Warnings’ – features of the model that may be correct, but are unexpected. These support analysts like static analysers support programmers: the editor can highlight features of the model so the analyst can catch mistakes, iterate more quickly, and produce high quality models. We are developing a third intelligence category, ‘Insights’ – primarily for identifying idioms.
The RISN model in Figure 1 is for a student’s interpretation of a timetable, with days presented horizontally, hours vertically, and subject names in the cells. This seemingly-trivial example has its depth made apparent by the RISN model: we consider day and time separately from subject, but bring them together into a new ‘product’: lectures. Each original dimension – day, time, subject – has concepts distinct from the those arising from the product. Without RIST and RISN, these subtleties are easy to miss; in Figure 1, these interactions are apparent.
The rep2rep project aims to be accessible to all potential analysts, so we have run two preliminary workshops on using RIST and RISN. In the first workshop, we taught the basics over two hours to five participants who are familiar with representation analysis, but not our framework. During the workshop, participants constructed simple RISN models for alternative interpretations, but we believe it was too short for them to ingest the subtleties of the theory and the features of the editor.
For the second workshop, we extended the time to four hours: we spent more time discussing the similarities and diferences between the schemas, particularly addressing the diferences between R-Dimensions, R-Schemes, and class R-Symbols. The participants were more engaged, and the models they built had fewer errors than the previous participants. Based on our experiences from these two workshops we are planning a future workshop with teachers, who are a fantastic audience for representation change, as they work with diverse representations for many readers. We believe that RIST, RISN, and our editor will be valuable to their practice, and their feedback valuable to our research.
Ongoing work with RIST and RISN involves analysing representations in the literature. We shall use our theory to determine the cognitive features of representations, from which we can estimate their eficacy; this estimate will be compared to the empirical results in the literature. Making falsifiable, verified claims based on RIST modelling will provide evidence in support of RIST as a cognitive modelling framework.
Finally, we return to the rep2rep project, and the need to adapt representation selection to the person using the representation. RIST lets us explore how people interpret representations, giving us data to design algorithms that consider both the structure of a representation, and how it will be understood by its reader, to produce appropriate recommendations for problem solving. Through RIST we gain insight into both the parameters to consider, and the potential range of values those parameters may take for diferent people; e.g., idiom use at diferent ability levels signals both a meaningful discriminant (the idiom) and its impact (the use), or the relationship between concepts and graphics (the same graphics used for multiple concepts, or one concept repeated for multiple graphics).
Supported by EPSRC grants EP/R030650/1, EP/T019603/1, EP/R030642/1, and EP/T019034/1. [6] P. C.-H. Cheng, A sketch of a theory and modelling notation for elucidating the structure of representations, in: A.-V. Pietarinen, P. Chapman, L. Bosveld-de Smet, V. Giardino, J. Corter, S. Linker (Eds.), Diagrammatic Representation and Inference, Diagrams 2020, Lecture Notes in Computer Science, Springer, 2020, pp. 93–109. doi:10.1007/978- 3- 030- 54249- 8_8. [7] P. C.-H. Cheng, A. Stockdill, G. Garcia Garcia, D. Raggi, M. Jamnik, Representational Interpretive Structure: Theory and Notation, in: Diagrammatic Representation and Inference, Diagrams 2022, Accepted for publication, 2022. [8] S. S. Stevens, On the theory of scales of measurement, Science 103 (1946) 677–680. doi:10. 1126/science.103.2684.677.