The medium of augmented and mixed reality is creating new problem spaces in the fields of learning science. The focus of this research is how to design AR experiences that support learning. These new spaces are inherently social, tangible, and real-time. These factors all point towards key design problems. When technology enters the social space how do we design for it? What is the importance of context as a design principle?
One of the many potential dangers of Augmented Reality (AR) without the careful intervention of design is the wholesale replacement of imagination with computer animation. This research will attempt to determine reusable design principles for augmenting learning spaces to ensure the maintenance of critical thinking and knowledge formation.
A central question is what theoretical frameworks can be drawn upon to inform such design principles? The Zone of Proximal Development (ZPD) is "the distance between the actual developmental level as determined by independent problem solving and the level of potential problem solving as determined through problem solving under adult guidance or in collaboration with more capable peers." (Vygotsky, 1978(Vygotsky, /1930, p. 86, p. 86). If the ZPD can be thought of as a context in which productive interactivity can take place then what design principles can be used and evaluated to generate such a context? Does keeping the user within their zone of proximal development (ZPD) ensure that constructive learning is maintained?
The meaningful learning context is defined here as the set of circumstances that is deemed relevant for the learner to build his or her knowledge. (Dettori, G, 2008) (Shute, T, 2009) Central to this research is the crucial role of context in accessing and forming knowledge.
"The distributed activity system (Pata & Väljataga, 2007), which forms when learners are realizing their goals in augmented reality, entails software, people and artifacts with different pedagogically usable functions. Mapping the pedagogical functions in case of different combinations of distributed tools and artifacts is necessary to understand the operationality and pedagogical potential of the augmented environment." (Pata, K. 2007) AR systems incorporate new ways of interacting with digital media by overlaying meaning onto the real world. Direct 3D manipulation of content is more intuitive in 3D learning and construction environments, because the direct perception of changes may foster the build-up of a mental model. The construction of psychological and physical space is one of the constituent parts in the generation of context. AR systems have been referred to as "intelligence amplifying systems to enhance human cognitive activities, such as attention, planning, analyzing problems, exploring new concepts, and decision making" (Brooks, 1995).
Main research aim: Provide design principles or reusable heuristics for developing augmented learning spaces. Research questions: What is the best way to present these principles? What evidence will count to show that they work?
In addition the use of mixed or blended reality creates a range of new challenges for the designer and the learner: i) To what extent does the design of the content effect the style of the user interface? ii) To what extent does the design of the user interface affect the range of potential interaction techniques? iii) What are the best methods for reducing the cognitive effort involved in managing interfaces (and the complex systems those interfaces regulate)?
The area of AR research is still in its infancy. Very few design guidelines and interaction patterns have been developed. As a result there are many opportunities for research. As AR applications involve virtual information registered in 3D traditional desktop evaluation techniques are not applicable. HCI problems associated with AR are also multiplied due to the vast number of different AR systems, the many types of interfaces (audio and haptic as well as visual) and the types of potential user input.
Another significant problem in the field is that the usability of an AR interface not only depends on objective measurements. The subjective user perception of interacting with the system should also play an important role during the development of an AR environment. However individual differences between users will make this difficult to evaluate.
Swan and Gabbard (2005) list a comprehensive collection of design principles found in the AR literature. Such guidelines may be used by researchers interested in particular design issues for their AR system but they raise many issues. Other work that has resulted in the development of appropriate interfaces for pervasive and ubiquitous computing technology; and explored the nature of context and the design of context sensitive technologies includes ( [Dey, 2001], [Dourish, 2004], [Lonsdale, 2004] and [Chalmers, 2004].
4 A presentation of any preliminary ideas, the proposed approach, the results achieved so far and contribution to the problem A preliminary idea for a guiding design principle lies in the difference between declarative and procedural designs. The procedural mode of guidance can be described as a linear sequence of instructions. The more precise we make a procedural representation, the more prescriptive it becomes. A declarative approach is more concerned with the many different ways that something can be achieved. "It describes the structure of a knowledge base (a map for example) rather than specifying a set of instructions for action. The creative activation of this knowledge base is at the disposal of the user because a declarative approach offers alternative options for action." (Boyle, 2002) The proposed approach will consist of the following 3 phases: Phase 1) Examine the outcomes of three existing case studies (Palace of Darius Persepolis, Materialising Sheffield and the Theatre of Memory) of virtual reconstruction research projects (where the physical location is not a part of the context). The design of the information environment (content, interface, and interaction) used in each one of these case studies will be examined in order to see the range and type of contexts that was generated from them. How adaptable were these environments? From this analysis the most significant design factors will be captured and identified in order to contribute towards the next design phase. These case studies represent procedural designs because the ability for the learner to interrogate the data set is extremely limited.
Phase 2) Examine a mobile version of the CONTSENS Cistercians in Yorkshire project (where the physical authentic location remains apart of the context). This case study represents an example of more declarative design because the learner has the ability to examine the data set in a more open way.
The archaeological learning environment (consisting of the reconstruction of 5 Cistercian abbeys in Yorkshire) was designed to allow the learner to explore the interaction between the virtual and the physical (the reconstructions and the ruins) and create a rich research context. The core idea behind the creation of the resource was that the user can learn a great deal more about a specific building or style of architecture if they are able to interact with the material in ways that is not possible, either on site, or via traditional print media. A comprehensive database of visualisations contained (but did not prescribe) the entire set of reasoning that led from the design of the 2D plans to the 3D reconstructions. The process of datamining hierarchies of evidence (intellectual transparency) in the quest for constituent parts, key narratives and evolutions of form can be explored in depth. This declarative design was essential as the total range of knowledge contained within the application could then be utilised, allowing many diverse and wide-ranging opinions to be tested and unified for further analysis. The capacity for users to have this 'active hand' in the construction of their own 'take on things' via visualisations which can be dynamically generated and transformed on the fly, in situ, and 'saved' has already been shown to contribute toward the successful augmentation of context (See Cook, 2010) An example of a successful design principle shows that the way the content is organized directly impacts on the type of interaction which is possible: Inheritance and the innate structural hierarchy of the architectural reconstructions are utilised to allow the disaggregation of a design into its original elements. Users can drill down (an example of an interaction pattern) through the final presentation of the model (the front end to the whole data set) into all the component objects and any architectural element within the model to automatically load any linked information which that element may have attached. These objects can then be extracted from their hierarchical structures, manipulated, measured and reconfigured (interaction patterns) according to the users unique research query. For example each abbey can be represented as a collection of their constituent architectural mouldings. A significant amount of the architectural style is imbedded in the structure of these mouldings along with invaluable information about their origins. The user is able to search and transform (an example of an interaction pattern) these 2D mouldings into their final 3D forms (using mobile devices in situ) in order to examine how (for instance) the Gothic style of architecture compares and potentially relates to the Romanesque style.
In this scenario it is the visualisation database which provides the digital component of the context which in combination with the real ruins allows collaborating learners to augment their rich research contexts for learning. The visual scope made available by the designer provides the 'action potential' (context as interaction) for the learner.
Phase 3) From an analysis of phase 1+2 the most significant design factors will be identified in order to create a series of AR prototypes to augment the learning space. The AR environment will map the digital information directly onto the physical world with the intention of eliminating the potential for distraction. A lack of focus on the actual object of study was reported in the feedback of phase two where users had to look away from the real world scene to look at the device.
Design-Based research (Design-Based Research Collective, 2003) will be used to identify and model technology-mediated, social learning and behaviors in order to design tools that support and promote the practices under investigation. For example, Cook (2002) has proposed a Design Research approach which revolves around evolutionary prototyping. This involves repeated cycles of: empirical work, theory/model development and tool/artifact refinement. These particular aspects are typically conceived as overlapping activities and phases (rather than as sequenced 'steps'); it is thus an evolutionary Design Research approach to analyzing the role of theory/models, empirical work and technology in learning. DBR will be applied to the phases outlined above. Two theoretical frameworks will be used to inform the methodology. The first of these is distributed cognition which highlights the importance of physical embodiment for cognition. Distributed Cognition involves the embodiment of information that is embedded in representations of interaction between individuals, artifacts and the environment. (Hollan, J. et al 2002). The second framework which will act as a lense and a measure of 'context as interaction' is the previously described Augmented Context for Development (see Cook, 2010).
The first phase of the research methodology is the literature review where the most relevant literature related to the defined problem is classified according to the different areas that are to be covered. During the evaluation phase the proposal will be tested in order to ascertain whether it accomplishes the main objectives of the research. Does the empirical evidence support our theoretically informed ideas about how you structure a context? As the study combines pedagogical and technical aspects it will involve a mixed evaluation that considers qualitative and quantitative analysis. The main aim in the evaluation phase is to understand the use of the technology developed within a context. This will be done through an analysis of various case studies. However, it is crucial for a correct evaluation to have a formal methodology. Since the project is framed in both the physical and digital domains and all technological and learning objectives must be analyzed, it is essential to consider quantitative and qualitative data. For this purpose, the Mixed Method will be implemented. This combines quantitative techniques and sources, such as closed questions or event log files generated automatically by the mobile devices with qualitative techniques, such as open questions, discussion groups or observations. Only by considering both types of information can we gain an in-depth understanding of the whole system within its context
One of the initial contributions to this research area was a proposed set of 'interaction heuristics' based on evaluation data from phase one and two. Heuristics are guidelines or 'rules of thumb' that should in future systems be able to guide the design and development of meaningful learning contexts. The initial set include guidelines such as: i)Vary the form of interaction and provide a more personalised view. ii) Improve indexing (it was deemed essential to give the user full access to all the content that was available in the system at any time) iii) Support for the evolution of practice (students claimed that it would be good to fit this practise into a larger context of their choice. iv) Provide the ability to perform deep customisation including level of detail, preferences and order of presentation (some students were at different conceptual levels and would have benefited from this ability to tailor the content to their individual level of interest) v) Users should be able to accomplish a task with a minimum of interaction steps. It should be easy for the user to learn how to use a system without prior training. vi) Reuse the real worlds (well known) level of interaction -don't make the interaction too different from how people would interact with real environments and problems. Other heuristics included providing the ability to… vii) Use simple designs where all of the interactions and elements of augmented social experiences must add value.
Another area of contribution is the use of declarative design. This is justified primarily as it gives the designer the possibility of an augmented learning context that learners may then choose to activate. In that way it is declarative. Instead of creating a context sensitive system which recommends (in a procedural way) this intervention at this point in time the feature is simply made a part of the context. The research will further extend the ZPD by incorporating the dynamic use of real time social data into the AR environment. We spread ideas around through dialogue and other forms of interaction. These ideas become 'contagious patterns of cultural information that pass from meme to meme which in turn have the ability to change the actions of a group' (Dawkins, 1976).