Towards Semiology of Teaching Analytics

Abstract

This paper presents a conceptual framework for designing, implementing, and evaluating notations, representations, and visualizations for teaching analytics.

Keywords

• cognitive dimensions of notations
• representational guidance
• perception of affordances
• appropriation of affordances
• teaching analytics

1 Introduction

Teaching Analytics is a new field of study that seeks to apply visual analytics methods and tools towards teachers’ dynamic diagnostic decision-making. A central concern in the emerging field of teaching analytics is the design, development and evaluation of notations, representations, and visualizations of learning and teaching processes and products and the enculturation of a “professional vision” for teachers to make the visual analytics notations, representations, and visualizations meaningful and actionable in pedagogical settings. Since teachers and learners perceive and act upon representations of their learning, the notational, emotional, informational and interactive aspects of representations must be considered in the design and development of teaching analytics systems.

This paper presents a conceptual framework to help the study of notational, emotional, and informational aspects of different kinds of notations, representations, and interactive visualizations. The primary purpose of this position paper is to present and discuss three lines of conceptual and empirical work relevant to the design and evaluation of representations.

2 Conceptual Framework

Representation as a proxy to information plays a crucial role in design in general. The nature of representations, their structures and interactions is one of the central concerns of cognitive science [ 16 ].Philosophically speaking, the function of representation is to “re-present”. Representation, in the philosophy of mind sense of the term, “is something that stands for something else”. In teaching analytics, representations “represent” the ongoing learning of the individual student and/or group of students. The technological and pedagogical aspects of representations have received significant conceptual and empirical attention in the fields of human computer interaction and learning sciences. Three following lines of conceptual and empirical work are particularly relevant for the design, development, and evaluation of representations in teaching analytics.  Cognitive Dimensions of Notations [ 1, 3 ]  

Representational Guidance [ 8, 11, 12 ]

Perception and Appropriation of Socio-Technical Affordances [ 14, 15 ]

2. 1 Cognitive Dimensions of Notations

The cognitive dimensions framework [ 1, 3 ] can help the conceptualization of the notational aspects of the teaching analytics solutions. The cognitive dimensions framework deal with the cognitive affordances of notations with respect to users engaged in activities. Gibson's ecological optics [ 2 ] and Green and Blackwell's cognitive dimensions [ 1 ] share conceptual terms such as medium and environment. Cognitive dimensions of representations, like affordances, are about the action-taking possibilities and meaning-making opportunities given actor competencies and system capabilities. The next section presents key concepts in the cognitive dimensions framework and discusses their relevance to teaching analytics. The following definitions are taken from Green and Blackwell [ 4 ]:  Information Artefacts: "the tools we use to store, manipulate, and display information" (p.5). Information artefacts are further classified as “non-interactive artefacts” and “interactive artefacts”. Representations are information artefacts and the static representations studied here are an example of “non-interactive artefacts”.  Environment: "The environment contains the operations or tools for manipulating those marks" (p.8). The environments in the NEXT-TELL context are the various “dashboards” of the Communication and Negotiation Layer, ECAAD and such for the different stakeholders.  Medium: "The notation is imposed upon a medium, which may be persistent, like paper, or evanescent, like sound"(p.8). In the case of the OLM, the medium is persistent and dynamically changed.

The cognitive dimensions framework distinguishes four types of user activity.  Incrementation: "adding further information without altering the structure in any way" (p.10)  Modification: "changing an existing structure, possibly without adding new content"(p.10)  Transcription: "copying content from one structure to another

structure;"(p.10) 

Exploratory Design: "combining incrementation and modification, with the further characteristic that the desired end state is not known in advance"(p.10)

Teaching analytics involves all four kinds of user activity. 2.1.1 Definitions of Cognitive Dimensions               

Abstraction: "An abstraction is a class of entities, or a grouping of elements to be treated as one entity, either to lower the viscosity or to make the notation more like the user’s conceptual structure" (p.24) Closeness of Mapping: "Closeness of representation to domain" (p.39) Consistency: "similar semantics are expressed in similar syntactic forms"(p.39) Diffuseness: "verbosity of language" (p.39) Error-Proneness: "notation invites mistakes" (p.40) Hard Mental Operations: "high demand on cognitive resources" (p.40) Hidden Dependencies: "A hidden dependency is a relationship between two components such that one of them is dependent on the other, but that the dependency is not fully visible" (p.17) Premature Commitment: "Constraints on the order of doing things force the user to make a decision before the proper information is available" (p.21) Progressive Evaluation: "work-to-date can be checked at any time" (p.40) Provisionality: "degree of commitment to actions or marks" (p.41) Role-Expressiveness: "the purpose of a component (or an action or a symbol) is readily inferred" (p.41) Secondary Notation: "Extra information carried by other means than the official syntax" (p.29) Viscosity: "Resistance to change: the cost of making small changes"(p.12) Visibility: "ability to view components easily."(p.34)

Juxtaposability: "ability to place any two components side by side"(p.34) The cognitive dimensions of notations defined above should be carefully considered in the design and implementation of teaching analytics solutions. For example, the dimensions of “Progressive Evaluation” and “Hidden Dependencies” can both have implications for designing notations for technology enhanced formative assessment. Representational aspects in teaching analytics are discussed next.

2.2. Representational Bias

The system of mental representations “consists not of individual concepts, but of different ways of organizing, clustering, arranging and classifying concepts and of establishing complex relations between them” [5, p. 17].

2.2.1 Definition of Internal Representations

“Internal representations are the knowledge and structure in memory, as propositions, productions, schemas, neural networks, or other forms” [17, p. 180].

From a cognitive science perspective, during learning activities information inherent in internal representations is retrieved from long-term memory and working memory. 2.2.2 Definition of External Representations

external representations are defined as the knowledge and structure in the environment, as physical symbols, objects, or dimensions (e.g., written symbols, beads of abacuses, dimensions of a graph, etc.), and as external rules, constraints, or relations embedded in physical configurations (e.g., spatial relations of written digits, visual and spatial layouts of diagrams, physical constraints in abacuses, etc.) Zhang [17, p. 180]

External representations embody environmental information, and this information can be “directly picked up” by the human perceptual systems in the Gibsonian ecological approach [ 2 ].

Teaching Analytics systems involve external representations. Representational salience and constraints of these external representations can influence the cognitive processes of information retrieval from and subsequent information storage. Suthers’ [ 8 ] conceptual and empirical work on “representational guidance” is highly relevant to the design of representations for teaching analytics systems.

Representational guidance follows from two lines of reasoning. First, the guiding ontological dimensions of representations— constraint and salience— prompt a user for what is missing as well for what is present [ 9 ]. The ontological dimensions of representations are not intrinsically social. Second, external representations play a role in guiding learning by amplifying certain kind of social interactions [ 11 ] and knowledge building interactions [ 12 ].

2.2.3 Definition of Representational Guidance

“Representational guidance" refers to how these software environments facilitate the expression and inspection of different kinds of information. [ 10 ] Representational guidance has tripartite origins in the (a) affordances of a representational notation, (b) in how that notation is realized in a representational tool such as software, and (c) in the actual configuration of representational artifacts created by users of that tool. How notations such as Smilies, Word Clouds, and Traffic Lights are represented in the software and the actual configuration of representational artifacts are issues of concern for the design and evaluation of teaching analytics applications. Finally, the socio-technical interactional aspects in teaching analytics are discussed next.

2.3 Perception and Appropriation of Socio-Technical Affordances

The notion of affordance was introduced by J. J. Gibson [ 2 ]. Gibson was primarily concerned with providing an ecologically grounded explanation to visual perception. Affordance is a deceptively simple concept that hides a radical hypothesis. Norman’s introduction of the concept of “perceived affordance” [ 6 ] brought the notion of affordance into human computer interaction. Affordances in HCI have largely been misunderstood as widgets, features and functionalities [ 13 ], despite a crucial intervention by Norman [ 7 ] himself.

2.3.1 Definition of Socio-Technical Affordance

By drawing upon ecological psychology research, Vatrapu [ 15 ] defined a sociotechnical affordance as “action-taking possibilities and meaning-making opportunities in a socio-technical system relative to actor competencies and system capabilities.”

With regard to teaching analytics, Perception of Affordances (PoA) refers to the action-taking possibilities and meaning-making opportunities that become available (that is, perceivable) to teachers in a given pedagogical situation. Appropriation of Affordances (AoA) refers to the intentional utilization of the affordances for actiontaking. AoA refers to the enactment of an interactional practice of teaching analytics (generative or creative).

3 Discussion

In summary, the preliminary conceptual framework presented here can help inform the design, development, and evaluation of notations (in terms of their cognitive dimensions), representations (in terms of their saliences and constraints), and interactive visualizations (in terms of the perception and appropriation of sociotechnical affordances) for the proposed field of teaching analytics.

Acknowledgments

This project is supported by the European Community (EC) under the Information Society Technologies (IST) priority of the 7th Framework Programme for R&D under contract number 258114 NEXT-TELL. This document does not represent the opinion of the EC, and the EC is not responsible for any use that might be made of its content.

References

[1] Blackwell , A. and Green , T. Notational systems-the cognitive dimensions of notations framework . HCI Models , Theories, and Frameworks: Toward an Interdisciplinary Science . Morgan Kaufmann2003).

[2] Gibson , J. J. The ecological approach to visual perception . Houghton Mifflin , Boston, 1979 .

[3] Green , T. Cognitive dimensions of notations . People and Computers V1989) , 443 - 460 .

[4] Green , T. and Blackwell , A. Cognitive Dimensions of Information Artefacts: a tutorial . BCS HCI Conference1998).

[5] Hall , S. The work of representation . Sage Publications , City, 1997 .

[6] Norman , D. The design of everyday things . Doubleday , New York, 1990 .

[7] Norman , D. Affordance , conventions, and design. interactions , 6 , 3 1999 ), 38 - 41 .

[8] Suthers , D. Representational Bias as Guidance for Learning Interactions: A Research Agenda . City, 1999 .

[9] Suthers , D. Towards a Systematic Study of Representational Guidance for Collaborative Learning Discourse . Journal of Universal Computer Science , 7 , 3 2001 ), http://lilt.ics.hawaii.edu/lilt/papers/2001/Suthers-JUCS- 2001 .pdf.

[10] Suthers , D. The Representational Guidance Project . Laboratory for Interactive Learning Technologies (LILT) . City, n.d.

[11] Suthers , D. and Hundhausen , C. An Experimental Study of the Effects of Representational Guidance on Collaborative Learning . Journal of the Learning Sciences , 12 , 2 2003 ), 183 - 219 .

[12] Suthers , D. , Vatrapu , R. , Medina , R. , Joseph , S. and Dwyer , N. Beyond Threaded Discussion: Representational Guidance in Asynchronous Collaborative Learning Environments . Computers and Education , 50 , 4 2008 ), 1103 - 1127 .

[13] Torenvliet , G. We can't afford it!: the devaluation of a usability term . interactions, 10 , 4 2003 ), 12 - 17 .

[14] Vatrapu , R. Toward a Theory of Socio-Technical Interactions in Technology Enhanced Learning Environments . Springer-Verlag, City, 2009 .

[15] Vatrapu , R. Explaining Culture: An Outline of a Theory of Socio-Technical Interactions . Proceedings of the 3rd International Conference on Intercultural Collaboration (ICIC 2010 ) 2010 ), 111 - 120 .

[16] Winn , W. Cognitive Perspectives in Psychology . Lawrence Erlbaum Associates, City, 2004 .

[17] Zhang , J. The Nature of External Representations in Problem Solving . Cognitive Science , 21 , 2 1997 ), 179 - 217 .