Holographic Learning - the use of augmented reality technology in chemistry teaching to develop students' spatial ability

Abstract

In chemistry teaching, three-dimensional molecules are traditionally visualised and drawn in 2D on paper or on a whiteboard/screen. However, the conceptual transition from 2D to 3D (i.e. spatial ability) is important for the understanding of chemistry. This is something with which many university students struggle, and they need to practice this through the visualisation of representations of atoms, molecules and reaction mechanisms. This paper reports on a study exploring how university students perceive the use of augmented reality (AR) technology in chemistry teaching. More specifically, the aim was to explore and understand what opportunities and challenges the students perceive when using AR technology for the purpose of enhancing their transition from a 2D representation of a molecule to a 3D structure visualised using AR glasses. The study was conducted during spring 2020. In this project, two groups of students of organic chemistry, in total 19 students, were given the opportunity to 'see' a holographic 3D structure of a molecule using AR glasses (e.g. Microsoft HoloLens 1). The empirical material is based on field test discussions, observation notes and 19 surveys that the students completed anonymously afterwards. The students described an immersive experience and the holographic 3D molecule was perceived as appearing like a very real object in the room. The student described the specific added value as being a support to visualising in 3D. They also stated that the amount of information on the 3D object was larger compared to a 2D representation. The challenges primarily concerned the rather narrow field of view of the AR glasses, and that training was required in order to use the device properly.

Keywords

• 1 Holographic Learning
• Chemistry
• Spatial ability
• Augmented Reality Technology
• Visualisation in 3D

1. Introduction

In chemistry teaching, three-dimensional molecules are traditionally visualised and drawn in 2D on paper or on a whiteboard/screen. However, the conceptual transition from 2D to 3D (i.e. spatial ability) is important for the understanding of chemistry. The representation and visualisation of chemical structures and reactions is a crucial, though challenging, basis for understanding chemistry [ 1 ] [ 2 ]. The challenge is that the student has to transform a flat 2D picture into a 3D form in order to realise how molecules are shaped, and thereby, how they can react with other molecules. Experts perform this transformation in their heads, without thinking about it, whereas novice students have to practice this transformation, i.e. spatial ability [ 3 ]. For many years, practicing spatial ability could only be done using analogue models, so-called ball-and-stick models [ 4 ]. However, with digitalisation, it has been possible to add new tools in order to increase students’ learning possibilities [ 5 ] [ 3 ]. In a framework from Buckley et al. [ 6 ], spatial ability, or visual processing, is elaborated and divided into two main groups, static and dynamic factors. The rotation of structures is one of the static factors that is possible to practice through the use of augmented reality (AR) technology as, through hand movements, students can rotate molecular representations [ 7 ]. Since this spatial ability is something with which many university students struggle [ 8 ] [ 4 ] [ 9 ], they need to practice this spatial ability through the visualisation of representations of atoms, molecules and reaction mechanisms. This paper reports on a study exploring how university students perceive the use of augmented reality (AR) technology in chemistry education when practicing their spatial ability. More specifically, the aim was to explore and understand what opportunities and challenges the students perceive when using AR technology to enhance their transition from a 2D representation of a molecule to a holographic 3D structure, visualised using AR glasses.

The research questions addressed in this paper are: 1. How can AR technology and AR glasses be used in chemistry teaching to enhance students’ spatial ability regarding transitions between 2D and 3D representations? 2. What is the teacher perspective regarding challenges in using AR for teaching chemistry? 3. What is the students’ experience, and what opportunities and challenges do they perceive in this process?

2. Background and literature

2.1. Teaching with technologies

Rapid technological development and changes in society have impacted digitalisation as a part of teaching and learning. Koehler, Mishra and Cain [ 10 ] state that teaching in a technology-rich environment requires specialised knowledge and an interweaving of knowledge of technology, pedagogy and content, i.e. the content to be taught (e.g. TPACK model). Further, they argue that current education is an ill-structured organisation concerning teaching with technology. Using technology to teach effectively depends on having flexible access to integrated knowledge from the different knowledge domains. In order for teachers to apply ‘new’ technologies, learn how to use them and handle the technology properly, going beyond traditional teaching methods may be perceived as both challenging and time-consuming [ 11 ] [ 12 ] [ 13 ]. Teachers may also perceive the use of digital technologies and different teaching methods as an activity that must fit into an already busy schedule in order to acquire a new knowledge base and skills on how technology, for example, can be effectively used in teaching [ 10 ]. A study by Mårell-Olsson [ 14 ] about integrating digital technologies and emerging teaching practices, such as gamification in K-12 education, found that teachers lacked knowledge about design processes, for example, from first ideas about what to achieve in teaching to the actual design of teaching modules using technologies in the classroom. In addition, teachers had difficulties finding a pedagogical balance and knowing what the added value could be for students’ learning when using what for them were new pedagogical methods and technologies with which they were unfamiliar. Consequently, teachers are facing challenges concerning how to design activities for teaching and learning in technology- rich learning environments.

In higher education, different digital and advanced technologies have been used with an aim to increase students’ engagement and interest concerning cognitive thinking and affective learning, and this idea is not new. In fact, even if using wearable technologies in higher education is not that common, the development of eye-based human computer interaction goes back as far as the early 1990s [ 15 ]. In an article, Steve Mann, often called the father of wearable technologies, mentioned that he has lived with wearable technologies and, more specifically, smart glasses, in one way or another, for almost 35 years [ 16 ]. The use of computers and other technologies as learning tools in higher education (e.g. medical and dental education) dates to the early 1970s. However, the real increase in the use of digital technologies in education came about with the introduction of personal computers in 1981 [ 17 ]. Since this time, technologies regarding how material and information are currently presented to a user have significantly evolved. Furthermore, the use of different digital technologies in education for learning purposes has been a catalyst for changing the way in which teaching is designed in order to increase students’ perceptions of meaningful learning [ 18 ] [ 19 ]. However, even if teachers argue that the use of technologies in teaching could be valuable for students’ learning, many teachers are facing challenges in the cultural shift from traditional teaching methods, i.e. constructivist and resource-based approaches [ 20 ] [ 21 ]. There is also sometimes an unwillingness to use what can be perceived as complex technologies and, through them, adapt and change to new teaching methods [ 22 ] [ 23 ] [ 21 ] [ 24 ] [ 25 ]. Regarding wearable and mobile technologies, using AR, for example, most people today are familiar with and have used the contemporary mobile game application Pokémon Go. This is an augmented reality mobile game that was developed by Niantic in collaboration with the Pokémon Company. 2.2.

What is AR?

Augmented reality (AR) technology is a technology that has the ability to overlay text, images, videos etc. onto the existing reality when merging the materialised real world with virtual objects, for example, head-mounted displays such as complex technological devices that allow the users to see computer-generated images overlaid onto the real world [ 26 ]. Craig [ 27 ] describes AR as adding digital information to the world and that a person can interact with the digital information in the same manner that they interact with the physical world. He defines AR as “a medium in which digital information is overlaid on the physical world that is in both spatial and temporal registration with the physical world and that is interactive in real time (p. 36). Further, he states that the ability to manipulate, store and retrieve information rapidly with the devices that are currently available, for example, smartphones or smart glasses, has led to the ability to do astonishing things that were not previously possible. There are multiple types of AR technologies and these are applied to a wide range of fields for different purposes and in different areas. For example, by presenting different AR applications in medical study programmes to simulate human organs and their placement within the body when learning about anatomy [ 28 ] [ 29 ], supporting patients with diabetes with their diet and physical activities [ 30 ] using AR applications as indoor navigation systems for wheelchair users to show them the available routes [ 31 ], as well as using AR technology in museums to help hearing-impaired visitors [ 32 ]. This paper focuses on the use of AR devices such as mobile handheld devices and head-mounted displays and wearables. 2.3.

Teaching with AR

The regular use of emergent technologies such as wearable technologies and AR technologies for educational purposes is still quite uncommon. One reason could be that teachers are of fundamental importance. If students are to encounter these new technologies, teachers must be confident enough to adopt a new teaching approach. According to the TPACK model [ 33 ], teachers need knowledge from three domains: content, pedagogy and technology. Thus, an interdisciplinary collaboration would be fruitful to explore these possibilities further. However, in a systematic review and meta-analysis of AR in educational settings, Garzón et al. [ 34 ] summarise the results from 61 studies conducted between 2012 and 2018, stating that AR has a medium effect on learning, in which the most explicit benefits concern learning gains and motivation. They also emphasise the importance of continuing to develop this technology further since the potential for learning is high, and that the use of AR in education has only just started. For example, a Swedish study about applying mobile devices and game-based learning using Pokémon Go as a tool for learning in elementary school, the researchers identified that some of the participating students were looking at their smartphones and the game instead of paying attention to the teacher [ 35 ]. Further, an Australian study presents three examples of the use of wearable technology in environmental sciences, cognitive and brain sciences, and teacher training [ 36 ]. The authors argue that examining how wearable technologies impact student outcomes and satisfaction is key to the teaching design. Further, they emphasise the importance of analysing student feedback and searching for factors that impact students’ learning processes.

In addition, other studies have shown that the use of AR technology in teaching has also impacted students’ motivation and engagement [ 37 ] [ 38 ] [ 39 ]. In 2002, when exploring student satisfaction, Kaufmann [ 37 ] found that the students rated learning about geometry in 3D using AR versus a PC CAD programme significantly higher (i.e. more satisfying) even though the usability of the AR programme was rated lower than the PC alternative. Hung et al. [ 39 ] identified that the students liked the use of an AR graphic book more, compared to the use of a regular picture book and physical 3D figures, and the students were most keen on using it. However, the students, in their study, expressed it hard to move between a virtual 3D representation and the 2D representation on their answer sheet. A study by Di Serio et al. [ 38 ] compared an AR teaching approach with a slide- based approach in a visual arts class. Their results showed that student motivation was significantly higher with the AR teaching approach compared to a slide-based approach. Moreover, Echeverría et al. [ 40 ] identified a difference in motivation between male and female students. The male students explored the technology beyond the designated tasks more, and therefore learned more about the technology than the subject, whereas the female students tended to complete the tasks but did not further investigate the technology. 2.4.

Teaching chemistry with AR

Some studies investigate the impact of AR technology on students’ spatial ability. Spatial ability, the mental move between 2D and 3D, can be practiced using analogue or digital tools. Spatial ability has been studied since the 1800s and forms the basis for representing and understanding chemistry [ 41 ] [ 42 ]. Spatial ability can be improved through practice and several studies have therefore scrutinised students’ representational competencies when applying digital techniques. For example, Kaufmann [ 37 ] investigated AR in geometry teaching in order to increase students’ spatial ability. He highlights the benefits of AR for increasing students’ understanding of 3D geometry. Safadel and White [ 43 ] compare AR with 2D tutorials when learning molecular modelling. Their results show that students with low spatial ability found AR more useful than students with high spatial ability. Thus, this may suggest the importance of supporting students with low spatial ability in comprehending abstract scientific concepts, for example, the transitions of 2D and 3D representations of molecules in stereochemistry. In a recent study, Habig [ 7 ] explores AR’s potential for supporting students’ learning of stereochemistry, in which gender differences were found. He showed that male students had better results than female students when using AR representations compared to conventional plastic ball- and-stick models. These gender issues have also been previously reported by, for example, Uttal et al. [ 44 ].

Furthermore, a number of studies present the challenges and disadvantages of using AR technology in teaching and learning, for example, cognitive overload in students when they are exposed to a large amount of information and the complexity of the task, as well as their unfamiliarity with the ’new’ technology [ 45 ]. Cai, Wang and Chiang [ 46 ] highlighted that low-achieving chemistry students increased their learning through AR whereas high-achieving students did not receive the same benefits. Also, the high-achieving students increased their learning using more traditional teaching methods. However, Di Serio et al. [ 38 ] argue that the positive impact of AR on motivation results in students achieving higher levels of engagement in learning activities, which involves less cognitive effort for them.

Further, the study by Cai et al. [ 46 ] states that it is important to discuss who should design and develop these AR modules. Who is responsible for ensuring that the chemistry content is correct? There could be significant errors if all the competencies described in the TPACK model [ 33 ], in this case, content knowledge in chemistry, are not used. In the study by Cai et al. [ 46 ], figures representing water molecules show pictures of a straight molecule, not bent, which would be correct, in which the hydrogen and oxygen atoms are given in the wrong colours. Thus, in our study, we wanted to combine the competencies of all three aspects of the TPACK model, i.e. technology, content and pedagogy. 2.5.

Teaching with smart glasses

As mentioned above, in traditional formal learning settings, students’ spatial ability can be practiced using, for example, analogue or digital tools. However, using interactive smart glasses, for example, Microsoft HoloLens 1, allow students to view, transform, move or rotate 3D objects using hand gestures. Mixing reality and virtual 3D objects (e.g. sometimes described as Mixed Reality) provides an interactivity in which students can explore 3D objects from different angles [ 47 ], for example, different molecules in stereochemistry. Craig [ 27 ] stated that the term “mixed reality” is often interchangeably used with augmented reality. Leonard and Fitzgerald [ 48 ] argue that one of the features of devices such as the Microsoft HoloLens is its high potential of mobility in providing visual simulations in ways that are both educationally and theoretically important. In their design-based study, they were seeking to understand the educational opportunities of using smart glasses to explore the researchable and designable aspects of the technology together. They found that students think about their experience with smart glasses (i.e. Microsoft HoloLens) in quite complex ways and the participants in their study described the use of the device as engaging, even if it was sometimes perceived as being difficult to use. Further, their students believed it had a positive impact on their learning. Further, a study by Xue, Sharma and Wild [ 49 ] investigated user satisfaction in AR applied to three cases of practical use and the influence of different factors such as age, gender, level of education, level of internet knowledge, as well as the roles of the participants in different sectors. They found that the participants’ general computer knowledge also had a positive effect on user satisfaction. In a study about ergonomic guidelines for manual material handling, Guo [ 50 ] argues that students could strengthen their understanding in hands-on training. The aim of the study was to compare the impact of students’ learning performance between an AR environment using Microsoft HoloLens and the in-class environment. In this study, Guo [ 50 ] found a significant improvement in students’ understanding of manual material handling after they had used the AR module and stated that there is a potential benefit of using AR in engineering education and training. Further, the author argues that learning and teaching procedures need to evolve in order to consider the high technological profile that, for example, most engineering students show today.

However, in a study by Leonard and Fitzgerald [ 48 ], some of their participants showed some ambivalence towards the smart glasses’ spatial affordances. For example, the ability to rotate objects was rated particularly poorly.

3. Study context and participants

The study presented in this paper was conducted during the spring of 2020 and involved two chemistry groups taking an organic chemistry course at bachelor level at Umea University. The study was designed and conducted as an additional teaching activity to the ordinary teaching, and the design was made in collaboration with a group in order to take advantage of the competencies described in the TPACK model [ 33 ], i.e. chemistry teachers (content knowledge), a chemistry education researcher (content pedagogical knowledge), and an educational researcher in technology-enhanced learning (technological pedagogical knowledge). A total of 19 students of organic chemistry, more specifically, stereochemistry, participated in the study. In this design-based research approach, the students were given the opportunity to use AR glasses (e.g. Microsoft HoloLens 1) to ‘see’ a holographic 3D structure of a molecule during a workshop. In this particular case, a nicotine 3D molecule was projected using Microsoft HoloLens 1 and the SketchUp Viewer for HoloLens (see figures 1, 2, 3 and 4). The SketchUp Viewer provided the students with the ability to rescale, move or rotate the virtual holographic molecule. Thus, the students were able to interact with the holographic object (i.e. the visualised molecule) when walking around it or standing still while moving and turning the object around using hand gestures. The students were tasked with memorising the structure of the holographically visualised molecule by viewing it and interacting with it using AR glasses and hand gestures to move or rotate it. In addition, after removing the glasses, the students had to draw a 2D representation of the molecule on paper and write the name of the molecule. The students had to solve the task individually.

In addition, a simple gamification element was added to the teaching design. The gamification element consisted of a competition. After all the students had finished their given task, a teacher checked all their answers and the students who were closest to the correct answers regarding the 2D representation of the structure and the correct name of the projected molecule won first, second or third prize. First and second prize was a pen with an integrated periodic table and third prize was a small bag of sweets.

The purpose of adding the gamification element to the teaching design was to both increase the students’ motivation to perform to the best of their ability (e.g. trying hard to solve the task), as well as for making the activity as enjoyable as possible.

4. Methodology and methods

The empirical material was collected via discussions with the students and observation notes during the workshop in the classroom. The students completed a survey individually after the activity. The survey focused on their own experience of using AR glasses, as well as their experience of the opportunities and challenges regarding the use of AR glasses for enhancing their own learning and spatial ability in stereochemistry in particular.

5. Theoretical framework

The overall theoretical framework for this study is based on Activity Theory [ 51 ] in which the key concepts of motive, goal, actions and operationalisations frame the entire study. Activity Theory allows an exploration of a context in relation to the intention of a planned activity, social relations, and the tools and materials used in an activity system. According to Leontiev [ 51 ], an activity which he regards as a system includes elements of motive, goal, actions and operationalisations. In Activity Theory, the interplay between these aspects is also explored with regards to how they affect each other in different situations (e.g. actions). Nardi [ 52 ] explains the importance of studying the role that an artefact plays in an activity system. Using the key concepts of Leontiev’s Activity Theory, the focus is not only on the individual’s actions, but also on the group’s actions in relation to the activity system. In education, for example, when teachers are teaching, they conduct operations in the classroom (i.e. procedures, different routines and practical examples of a subject). The different procedures and routines comprise combined actions and, in turn, relate to prerequisites and the school organisation. The actions and operations undertaken in the classroom are also related to the goal and motive for the teaching that the teacher is trying to pursue (see figure 5).

This research study could be seen as an exploration of an activity system in which the chosen theoretical framework is framing the study on different levels concerning, for example, motive, such as exploring if, and if so, how AR technology could enhance students’ transition of 2D representation and the 3D structure of a molecule visualised using AR glasses. The goal of the study was to investigate and understand the opportunities and challenges the students perceive when using AR technology for this purpose. The actions taken could be described as designing and examining teaching activity using AR glasses in order to reach the intended goal. The operations conducted in the classroom in this activity system could be described as the teaching design, in which the students were able to use AR glasses (e.g. Microsoft HoloLens 1) for studying a holographic 3D molecule in combination with the gamification activity. This study could be described as exploratory in nature and is inspired by and includes aspects of design-based research methods [ 53 ].

6. Analysis processes

Thematic analysis [ 54 ] was used to identify key themes and emerging patterns within the theoretical framework of Activity Theory [ 51 ], as well as construct an understanding and meaning of the collected empirical material (e.g. observation notes and student survey). Thematic analysis could be described as a process for encoding qualitative information, thereby assisting the researcher in their search for insight. This process includes multiple readings as part of an iterative process and includes the two perspectives of ‘seeing’ and ‘seeing as’ [ 55 ]. The process of ‘seeing as’ could be described as searching for repetitive patterns of significance in order to identify emerging patterns in the empirical material [ 56 ]. Ely [ 54 ] argues that an emerging theme could be described as a definition of either utterances made by all participants in a study, or as a single statement of an opinion that has a great emotional or actual meaning (e.g. significance). In this study, the phase of constructing meaning (i.e. ‘seeing as’) was conducted by searching for emerging patterns on a more abstract level, combining the participants’ observed behaviour with what they explicitly or implicitly stated during the teaching activity when using AR glasses in the classroom (i.e. observation notes). In addition, their responses to the survey concerning the described opportunities and challenges were analysed. Thus, the presented themes in the next section were formed and derived during several iterative analysis processes. The quotations in the findings section below should be regarded as illustrations of the themes that emerged in the empirical material, not merely as evidence.

7. Findings

The results are presented in five themes and illustrate the students’ perspectives on both the opportunities and challenges they experienced when using AR technology in chemistry teaching. The first theme concerns the students’ description of an immersive experience when trying the AR glasses for the first time. The second theme concerns the students’ descriptions of how they perceived the holographic 3D object projected in the classroom. The third theme concerns the students’ descriptions of how they perceived the field of view when using the device (i.e. AR glasses). Finally, the fourth theme concerns the students’ descriptions and reflections on the requirements of technical training in order to use an AR device such as Microsoft HoloLens 1 properly. The final theme concerns the students’ descriptions of the gamification element of the activity.

Descriptions of an immersive experience

When the students tried the AR glasses for the first time, the most common comments were “wow” and “so cool” when they saw the holographic molecule in front of them. Most of them also laughed and were smiling. This is in line with the responses to the survey on the question about their experience of using AR glasses. The students used expressions such as “exciting”, “exciting and advanced level of control”, “my experience was positive, and it was interesting that it’s possible to visualise 3D objects this way. It was interesting to try out a new technology”. Most of the students had some previous experience of using augmented reality technology for smartphones, for example, Pokémon Go, but not using this type of AR glasses. Their sense of excitement according to the observation notes, and in the survey, could be interpreted as them feeling an immersive experience in combination with fascination for a first-time experience when using augmented reality technology in this way. In this respect, it should be noted that the immersive experience and the perceived sense of ‘joy and fun’ could be significantly augmented not only as an immersive experience of holographic objects, but also as a firsttime experience. 7.2.

A perceived sense of a ‘real’ 3D object

In the survey, most of the students stated that the 3D object (e.g. the holographic molecule, see figure 3) was perceived as a ‘real’ object projected in the classroom, and that it was possible to interact with the object. One student explained: “It feels natural to move and turn the molecule around”. Another student stated: “I really liked the fact that it was so easy to choose and control the angle of the molecule and you could also point at the molecule/atoms”. The students also stated that it was good to be able to see the reality at the same time as using AR glasses. One student stated: “It was really good that you could still see the reality”. Another student described it as follows: “When I’m using VR (i.e. virtual reality glasses), I have a feeling that I’m trapped and it’s hard for me to focus since I usually have to use ordinary glasses. But with the AR glasses, I had none of these problems”. The students also stated that the amount of information from the holographic 3D object projected by the AR glasses was larger compared to, for example, a 2D object (see figure 4). One student described the transition between 2D and 3D representation of a molecule as follows: “It’s quite easy to move between 2D and 3D (e.g. comparing a 2D representation of a molecule drawn, for example, in the study guide and then view it as a 3D representation) than moving from 3D to 2D, since it feels hard to get all the information about the structure when writing it in 2D”. Another student stated the opposite experience: “For me, it’s harder to move from 2D to 3D than the opposite way”.

Even if the students’ own experience of the transition between 2D and 3D representation differed (e.g. which way was easier or harder to perform), all the students agreed that visualisation in holographic 3D, and being able to ‘see’ a molecule in 3D using AR technology, provided added value that increased their understanding and learning about stereochemistry, particularly regarding how to understand the information given in a 2D representation and transform it into a 3D representation (e.g. spatial ability and being able to visualise by themselves). One student described it thus: “The transition between 3D to 2D is quite hard to visualise in your head and it’s not possible to draw something that sort of comes out of the paper or towards the table. This is why a 3D object like this can facilitate the understanding of 3D structures”. 7.3.

A narrow field of view

Some of the students certainly felt challenged. A couple of students complained about a somewhat narrow field of view when using AR glasses. According to the observation notes, some students struggled when moving their heads to identify the whole image of the molecule. In addition, they were not used to walking around a virtual object in order to see it, for example, from the rear. Some students also struggled focusing on the object, particularly at the beginning, in order to get a clear view of the molecule when they put on the glasses. After some help and support and individual adjustments, all students were able to handle the device. One student explained: “It was a bit hard understanding how to use it [the device] at first, and that it [the molecule] could suddenly be out of sight if I wasn’t standing in the right spot”. Another student stated: “I couldn’t turn the molecule 360 degrees. Sometimes the device was flickering and some parts of the molecule sort of disappeared when I looked at it the ‘wrong’ way”. 7.4.

Requirement for support and training to use the device properly

First, when the students put the device on their heads, they had to adjust the glasses in order to see properly. Some students struggled more with this than others. Depending on the shape of their heads, some students had to adjust the device several times before it sat properly. For example, for students with quite oblong heads and glossy hair, the device sometimes slipped and slid down their heads, and they had to adjust it. These students often had to do this several times compared to, for example, students with a round-shaped head and hair that was not so glossy. The latter students often only had to adjust it once. In addition, the students also had to practice hand gestures, for example, tapping in the air to navigate, move or turn the molecule around. According to the observation notes, it was obvious this was something the students had not done before and were not familiar with. Quite a few of the students struggled when coordinating their hands and heads to navigate and interact with the holographic virtual object. They had to practice this several times. Some students were unable to learn how to air tap (e.g. look at the object and tap in the air using the index finger and thumb) during the activity that involved interacting with the molecule (i.e. turning it around or moving it). These students had to walk around the molecule themselves instead, and not interact with it using their hands (i.e. air tapping and navigating through menus, etc.). One student stated that moving their body around in the classroom in order to get a clear view of the molecule felt strange. “I’m not used to walking around the classroom to look at an object like that. This felt a bit odd”.

The students needed help and support to use the device since it was their first time, and they had not used these AR glasses before, even if most of them were familiar with using AR technology, for example, Pokémon Go’. Also, they were not familiar with hand gestures for navigating and interacting with holographic objects (i.e. air tapping, rotating objects, etc.), particularly not in combination with using their heads to navigate by looking at menus, and air tapping to execute a command. For example, some students were totally unable to combine looking at the molecule, air tapping and holding their index finger and thumb together, in order to rotate the molecule at the same time. However, some students were only shown once how to air tap and rotate and they were then able to manage the hand gestures properly. This was also confirmed by one student in the survey who asserted: “More advanced technology probably requires someone who can teach us how to use it, or we could have additional lessons in which we were just trained to use the technology”. 7.5.

Increased internal motivation to solve the task

A small gamification element was added to this design-based activity with the purpose of increasing the students’ efforts to try and solve the given task. Even if as researchers we did not focus on addressing questions in the survey related to this gamification element (e.g. a simple competition), it was obvious that some students were motivated by this competition. When some of the students struggled for a long time to draw the 2D representation, we told them that they could submit their answers anyway, and that this activity had primarily been designed to understand if, and how, AR technology could support their learning of stereochemistry. One of the students replied: “It’s a competition. I would like to win”. Another student said with a smile: “Ha, winning the glory is everything”.

Thus, it was clear that some of the students were motivated to make an extra effort and really try their best to solve the task, even if the added gamification element was merely designed as a simple competition. The gamification element could be interpreted to some extent as increasing the internal motivation and, moreover, increasing the efforts of some of the students when trying to solve the task.

8. Discussion and conclusion

For this study, a qualitative approach was adopted in combination with design-based research methods [ 53 ] to explore and deepen the understanding of if, and in which case how, AR technology can be used to enhance students’ spatial ability in stereochemistry in the transition between a 2D representation and the 3D structure of a molecule visualised using AR glasses. Further, the aim was to investigate the students’ own experiences of the opportunities and challenges they encountered when using what for them was a new emerging technology such as AR. It was clear that the students had an immersive experience when using AR glasses and interacting with a holographic virtual 3D object (e.g. a holograph of a molecule). In the survey, they also expressed the need to receive help and support in order to visualise 3D structures in stereochemistry. In addition, they emphasised that AR technology using AR glasses projecting holographic molecules could be a way of supporting this transition (e.g. spatial ability). The aim of this study is in line with the study by Alvarez et al. [ 36 ] in which they examined how wearable technologies could impact student outcomes and satisfaction in teaching and learning activities. In their study, they discuss the importance of analysing student feedback and identifying factors that impact the students’ learning processes. A study by Safadel and White [ 43 ] compared the use of AR with 2D tutorials in which students were studying molecular modelling. Their results showed that students with low spatial abilities found AR more useful than students with high spatial abilities. In our study, however, all the students described an increased help and support for understanding 3D objects (e.g. holographs) in stereochemistry.

In their study, Phon et al. [ 45 ] recognised that some students who were unfamiliar with AR technology experienced a cognitive overload when they were exposed to a large amount of information. The students in our study did not describe any challenges related to cognitive overload. The challenges they described primarily related to being unused to the technology involving the combination of head and hand gestures to navigate the device. The study by Echeverría et al. [ 40 ] highlighted a gender difference in motivation in which male students were more keen to explore the technology than female students. Habig [ 7 ] and Uttal et al. [ 44 ] also identified cognitive gender differences in which male students outperformed female students when using AR to visualise molecules. In our study, we found no gender differences regarding motivation and interest when exploring and using AR glasses.

Integrating emergent technologies such as AR technology in teaching, and particularly concerning the process of designing this type of teaching practice in education, is a rather complex process in which the preparedness of both teachers and students for this appears to be low (cf. [ 10 ][ 14 ]. The teaching design is very complex when combining the different domains of technological, pedagogical and content knowledge [ 10 ] in order to find a pedagogical balance [ 14 ] when designing a teaching activity in which students can use advanced technology to enhance their spatial ability, for example, in stereochemistry. The complexity also involves designing teaching activities with a pedagogical balance between actual operations in the classroom that correspond to the intended aim of what to achieve for the students’ learning processes (e.g. motive and goals, [ 51 ]). The complexity for a teacher concerns, for example, being able to possess skills and combine the three different knowledge domains [ 10 ], as well as their creativity when designing classroom activities (e.g. operations). This could be regarded as necessary skills for a teacher to possess if their aim is to enhance students’ spatial ability in chemistry using emergent technologies and also using emergent teaching practices such as gamification. In addition, it is necessary to possess these skills with regards to being aware of practical motives and goals on multiple levels [ 51 ]. It is also essential for a teacher to possess know-how regarding emergent technologies such as AR in order to achieve the motives and goals of the teaching activity, as well as the added value it offers by controlling the operationalisation of the actions [ 51 ]. However, the outcome of a teaching activity not only depends on a teacher possessing skills and creativity in the teaching design, when several individuals are involved, for example, students; it also concerns how well the students adapt to the situation when using emergent technologies that they perceive as being ‘new’, as well as their ability to become aware of their own position within an activity system (the design-based teaching activity in this study). In addition, the outcome of a teaching activity also depends on whether or not the students perceive the learning activity as being meaningful [ 18 ]. This also applies if individuals (i.e. the students in this study) become aware of themselves within such an activity system and what they can contribute regarding the use of holographic molecules and AR technology in the teaching design. In our study, we developed the AR workshop together with the chemistry teachers in charge of the course. They contributed with their chemistry content knowledge, whereas we added technological knowledge and pedagogical content knowledge. This collaboration has been fruitful. The TPACK model emphasises the importance of different competencies to address the complex, multifaceted and situated nature of teacher knowledge [ 33 ].

Furthermore, integrating emergent teaching practices, for example, gamification, in order to increase students’ internal motivation for learning, in combination with the use of AR technology to help and support their spatial ability, has been shown to be a rather complex process. Designing teaching and learning activities using both ‘new’ technologies and ‘new’ pedagogical methods requires knowledge of advanced technological, pedagogical and content knowledge [ 10 ]. However, the use of gamification as an emerging teaching practice in teaching and learning shows that teachers lack knowledge about students’ gaming culture and also appear to lack knowledge of the concept of how gamification designs (i.e. using game dynamics and game mechanics in the teaching design) might foster student motivation and commitment to their schoolwork [ 14 ].

In line with Mårell-Olsson’s study [ 14 ], the results of our study show that designing teaching and learning activities, as well as finding the pedagogical balance between the respective knowledge domains, is a complex process. A teacher needs at least some knowledge about advanced technologies such as AR (i.e. technological knowledge) and how to design teaching activities (i.e. pedagogical knowledge) in order to combine know-how on what the added values are when using AR technology and emergent teaching practices such as gamification to achieve the intended motives and goals [ 51 ] so that students can learn (i.e. content knowledge). Furthermore, in this activity system [ 51 ], we also must consider the students. Using AR glasses such as the Microsoft HoloLens 1 in the teaching design requires technological training for the students if lesson time does not merely focus on training the students to use the device properly. Thus, as one student suggested in the survey, it could be smart to conduct some activities beforehand in which the focus is on practice and using the technology (e.g. AR glasses). When the students are able to handle the device properly, it will be easier for them to focus on the content knowledge and pedagogical methods. Consequently, in higher education, this might require an increased interdisciplinary collaboration between teachers of different disciplines. For example, in this study, both chemistry teachers/researchers, chemistry education teachers/researchers and teachers/researchers with competencies in advanced technologies and technology-enhanced learning have collaborated.

This study explores and illustrates the use of AR technology and holographic objects in chemistry education regarding aspects of sustainable development and students’ experiences and perceived opportunities and challenges in this context. This study could be seen as an effort to explore how to ensure the quality (UN SDG no. 3 and no. 4) of chemistry teaching when trying to enhance students’ spatial ability regarding the alignment of the perspectives of sustainable goals such as exploring learning and training for resilient infrastructure (UN SDG no. 9) and supporting knowledge exchange (UN SDG no. 12).

9. Limitations and recommendations for future research

A methodological concern regarding this study is the selection of participants and whether including more students and adding interviews to the empirical material would have made it possible to obtain more extensive data and richer nuances. However, time constraints made further data collection impossible. For example, one restriction was that the chemistry students had other mandatory courses and assignments to complete during the semester. This meant that it would have been too complicated to include additional teaching activities to their already heavy workload. Another methodological concern is the chosen theoretical framework and the applied thematic analysis approach. Naturally, different results could have been obtained if, for example, a more theory driven-approach had been used. However, the chosen combination of Activity Theory [ 51 ] and thematic analysis [ 54 ] using design-based research [ 53 ] was regarded as beneficial to obtaining an increased understanding of using AR technology to enhance students’ spatial ability in chemistry, and to answer the research questions.

A recommendation for future research is not only to extend the number of participants and add interviews to the empirical material, but also to conduct more designed-based research with an interdisciplinary collaboration between teachers of different disciplines. This would enable the combination of specialised advanced know-how within domains such as technological, pedagogical and content knowledge. In addition, the combination of both AR and VR technologies with applications projecting 3D representations of, for example, molecules or protein structures, could be valuable. Moreover, combining gamification teaching designs in order to increase the students’ motivation to learn would also be useful. This could further broaden the understanding of students’ learning processes of spatial ability in chemistry. 10. References

References

[1] J. K. Gilbert , M. Reiner och M. Nakhleh , Visualization: Theory and Practice in Science Education , Vol. 3 red ., New York: Springer, 2008 .

[2] H.-K. Wu , J. S. Krajcik och E. Soloway, ” Promoting Understanding of Chemical Representations: Students' Use of a Visualization Tool in the Classroom , ” Journal of Research in Science Teachinig 38 ( 7 ), pp. 821 - 842 , 2001 .

[3] S. R. Hinze , D. N. Rapp , V. M. Williamson , M. J. Shultz , G. Deslongchapms och K. C. Williamson , ” Beyond ball-and-stick: Students' processing of novel STEM visualizations ., ” Learning and Instruction , 26 ( 1 ), pp. 12 - 21 , 2013 .

[4] Y. J. Dori och M. Barak , ” Virtual and Physical Molecular Modeling: Fostering Model Perception and Spatial Understanding,” Educational Technology & Society 4 ( 1 ), pp. 61 - 74 , 2001 .

[5] S. Bernholt , K. Broman , S. Siebert och I. Parchmann, ”Digitising Teaching and Learning - Additional Perspectives for Chemistry Education,” Israel Journal of Chemistry , 59 ( 6-7 ), pp. 554 - 564 , 2019 .

[6] J. Buckley , N. Seery och D. A. Canty, ”Heuristic Framework of Spatial Ability: a Review and Synthesis of Spatial Factor Literature to Support its Translation into STEM Education,” Educational Psychological Review , 30 ( 3 ), pp. 947 - 972 , 2018 .

[7] S. Habig , ” Who can benefit from augmented reality in chemistry? Sex differences in solving stereochemistry problems using augmented reality,” British Journal of Educational Technology , 51 ( 3 ), p. 629 - 644 , 2020 .

[8] A. Bongers , B. Beauvoir , N. Streja , G. Northoff och A. B . Flynn , ” Building mental models of a reaction mechanism: the influence of static and animated representations, prior knowledge, and spatial ability , ” Chemistry Education Research and Practice , 21 ( 2 ), pp. 496 - 512 , 2020 .

[9] M. Harle och M. Towns , ” A Review of Spatial Ability Literature , Its Connection to Chemistry, and Implications for Instruction, ” Journal of Chemical Education , 88 ( 3 ), p. 351 - 360 , 2011 .

[10] M. J. Koehler , P. Mishra och W. Cain, ” What is technological pedagogical content knowledge (TPACK)? , ” Journal of Education , 193 ( 3 ), p. 13 - 19 , 2013 .

[11] J. Bundsgaard och T. I. Hansen , ” Blik på undervisning: Rapport om observationsstudier af undervisning gennemført i demonstrationsskoleforsøgene. [Eng: Perspectives on teaching: Report on observational studies of teaching carried out in the demonstration school experiments],” Laeremiddel .dk, 2016 .

[12] I. Jahnke , P. Bergström , E. Mårell-Olsson , L. Häll och S. Kumar, ”Digital Didactical Designs as research framework: iPad integration in Nordic schools , ” Computers and Education , 113 , p. 1 - 15 , 2017 .

[13] E. Mårell-Olsson , P. Bergström och I. Jahnke, ”Is the Tablet a Teacher or a Student Tool? Emergent Practices in Tablet-Based Classrooms,” i In Emergent Practices and Material Conditions in Learning and Teaching with Technologies , Cham, Springer, 2019 , p. 89 - 105 .

[14] E. Mårell-Olsson , ” University Students as Co-creators in Designing Gamification Teaching Activities using Emergent Technologies in Swedish K-2 Education ,” International Journal on Interaction Design and Architecture (IxD&A) , N. 42 , pp. 47 - 69 , 2019 .

[15] A. Bulling och H. Gellersen , ” Toward Mobile Eye-Based Human-Computer Interaction,” EEE Pervasive Computing , vol. 9 , no. 4 , p. 8 - 12 , Oct.-Dec 2010 .

[16] S. Mann , ”Through the Glass, Lightly,” Technology and Society Magazine , IEEE, vol. 31 , no. 3 , pp. 10 - 14 , 2012 .

[17] P. Grigg och C. Stephens , ” Computer-assisted learning in dentistry. A view from the UK , ” Journal of Dentistry , 26 ( 5 ), p. 387 - 395 , 1998 .

[18] J. Howland , D. Jonassen och R. Marra , Meaningful Learning with Technologies, Pearson publisher , 2012 .

[19] J. I. , Digital Didactical Design - Teaching and Learning in Cross-Action Spaces , New York: Routledge, 2015 .

[20] P. A. Reynolds , R. Mason och K. Eaton, ”Remember the days in the old school yard: From lectures to online learning , ” BDJ , 204 ( 8 ), p. 447 - 51 , 2008 .

[21] K. Eaton och P. Reynolds , ” Continuing professional development and ICT: target practice,” The British Dental Journal , 205 , p. 89 - 93 , 2008 .

[22] S. Eisenstadt M , ” Exploring telepresence on the internet - the KMI stadium webcast experience,” i In Eisenstadt M, Vincent T , ( eds) The knowledge web , London, Kogan , 1998 , p. 153 - 172 .

[23] B. Gupta , D. White och A. Walmsley, ”The attitudes of undergraduate students and staff to the use of electronic learning , ” British Dental Journal , 196 , p. 487 - 492 , 2004 .

[24] I. Wagner , R. Ireland och K. Eaton, ” Digital clinical records and practice administration in primary dental care,” British Dental Journal , 204 , p. 387 - 395 , 2008 .

[25] E. Kay , ” Dental Education - shaping the future , ” British Dental Journal , pp. 447 - 448 , 2014 .

[26] D. W. M. C. V. Novak , ” Looking in, looking out: A discussion of the educational affordances of current mobile augmented reality technologies,” i Educational Stages and Interactive Learning: From Kindergarten to Workplace Training , Information Science Reference , 2012 , pp. 92 - 106 .

[27] A. Craig , Understanding augmented reality. Concepts and applications , Amsterdam: Elsevier Science & Technology, 2013 .

[28] C. Kamphuis , E. Barsom, M. Schijven och N. Christoph , ” Augmented reality in medical education?,” Perspectives on Medical Education 3 ( 4 ), p. 300 - 311 , 2014 .

[29] B. Hensen , I. Koren , R. Klamma och A. Herrler, ”An Augmented Reality Framework for Gamified Learning , ” i Advances in Web-Based Learning - ICWL 2018. Lecture Notes in Computer Science , Vol 11007 , Cham , Springer , 2018 , pp. 67 - 76 .

[30] M. Rollo , E. Aguiar, R. Williams , K. Wynne , M. Kriss , R. Callister och C. Collins, ” EHealth technologies to support nutrition and physical activity behaviors in diabetes selfmanagement , ” Diabetes Metabolic Syndrome and Obesity: Targets and Therapy , 9 , p. 381 - 390 , 2016 .

[31] L. de Oliveira , A. Andrade , E. de Oliveira, A. C. A . Soares och E. Lamounier, ” Indoor navigation with mobile augmented reality and beacon technology for wheelchair users , ” IEEE EMBS International Conference on Biomedical & Health Informatics (BHI) , p. 37 - 40 , 2017 .

[32] E. Baker och J. Z. A. .. .. (. Abu Bakar , ” Elements of museum mobile augmented reality for engaging hearing-impaired visitors , ” AIP Conference Proceedings , 1891 (1): 020033 , 2017 .

[33] M. C. Herring , M. J. Koehler och P. Mishram , Handbook of Technological Pedagogical Content Knowledge (TPACK) for Educators , New York: Routledge, 2016 .

[34] J. Garzón , J. Pavón och S. Baldiris, ” Systematic review and meta-analysis of augmented reality in educational settings ,” Virtual Reality, 23 ( 4 ), p. 447 - 459 , 2019 .

[35] J. Jaldemark , S. Eriksson Bergström och P. .. Mozelius, ” Applying mobile devices and gamebased learning in formal educational settings: Playing Pokémon Go as a tool for learning in a Swedish elementary school,” i Research Symposium: Recent trends in the digitalization of the Nordic K-12 schools Umeå University, 11 - 13 October, 2017 , Umeå, 2017 .

[36] Alvarez , Bower, d. Freitas, Gregory och d. Wit., ” The Use of Wearable Technologies in Australian Universities: Examples from Environmental Science, Cognitive and Brain Sciences and Teacher Training,” i In Dyson , Ng & Fergusson (Eds). Mobile Learning Futures - Sustaining Quality Research and Practice in Mobile Learning. 15th World Conference on Mobile and Contextual Learning , mLearn 2016 , Sydney, Australia, 24 - 26 October., Sydney , Australia, 2016 .

[37] H. Kaufmann . ”Construct3D: An augmented reality application for mathematics and geometry education , ” i In Proceedings of the Tenth ACM International Conference on Multimedia, MULTIMEDIA '02 , pp. 656 - 657 , New York, NY, USA, 2002 .

[38] A. Di Serio , M. Ibáñez och C. Kloos, ”Impact of an augmented reality system on students' motivation for a visual art course , ” Computers and Education , 68 , p. 586 - 596 , 2013 .

[39] Y. Hung , C. Chen och S. Huang, ”Applying augmented reality to enhance learning: a study of different teaching materials , ” Journal of Computer Assisted Learning , 33 ( 3 ), p. 252 - 266 , 2017 .

[40] A. Echeverría och F. N. M. Gil , ” Classroom augmented reality games: A model for the creation of immersive collaborative games in the classroom ,” 2011 . [Online]. Available: https://www.semanticscholar.org/paper/Classroom-Augmented- Reality-Games-A-model-forthe-Echeverr%C3%ADa-Gil/ 141385717d0c961f79a136a9c8b3388b516104a6?tab=abstract.

[41] M. Harle och M. Towns , ” A Review of Spatial Ability Literature , Its Connection to Chemistry, and Implications for Instruction, ” Journal of Chemical Education , 88 ( 3 ), p. 351 - 360 ., 2011 .

[42] L. L. Thurstone , ”Some Primary Abilities in Visual Thinking, ” Proceedings of the American Philosophical Society , 94 ( 6 ), pp. 517 - 521 , 1950 .

[43] P. W. D. Safadel , ” A Comparative Analysis of Augmented Reality and Two-Dimensional Using Molecular Modeling on Student Learning,” i In P. Resta & S. Smith (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 1774 - 1776 ). Austin, TX, United States: Association for the Advancement of Computing in Education (AACE), Austin , TX, United States , 2017 .

[44] D. H. Uttal , N. G. Meadow , E. Tipton , L. L. Hand , A. R. Alden och C. & Warren , ” The Malleability of Spatial Skills: A Meta-Analysis of Training Studies,” Psychological Bulletin , 139 ( 2 ), p. 352 - 402 , 2013 .

[45] E. Phon , D. Nincarean , A. Mohamad Bilal , A. Halim och N. Dayana , ” Collaborative augmented reality in education: a review , ” i Proceedings - 2014 International Conference on Teaching and Learning in Computing and Engineering , LATICE, 2014 .

[46] S. Cai , X. Wang och F. Chiang , ” A case study of augmented reality simulation system application in a chemistry course , ” Computers in Human Behavior , 37 , p. 31 - 40 , 2014 .

[47] R. Trelease och G. Nieder, ” Transforming clinical imaging and 3D data for virtual reality learning objects: HTML5 and mobile devices implementation,” Anatomical sciences education 6 ( 4 ), p. 263 - 270 , 2013 .

[48] S. N. Leonard och R. Fitzgerald , ” Holographic learning: A mixed reality trial of Microsoft HoloLens in an Australian secondary school,” Research in Learning Technology . Vol. 26 , 2018 .

[49] H. Xue , P. Sharma och F. Wild , ” User Satisfaction in Augmented Reality-Based Training Using Microsoft HoloLens ,” Computers 2019 , 8 , 9 ., 2019 .

[50] W. Guo , ” Improving Engineering Education Using Augmented Reality Environment,” i In: Zaphiris P., Ioannou A . ( eds) Learning and Collaboration Technologies. Design, Development and Technological Innovation . LCT 2018, Lecture Notes in Computer Science , vol 1092 , Springer, Cham, 2018 .

[51] A. Leontiev , Verksamhet, medvetande, personlighet: Tätigkeit, Bewusstsein, Persönlichkeit = Activity, consciousness, personality = Activité, conscience, personnalité, Moskva: Progress, 1986 .

[52] B. E. Nardi , ” Context and Consciousness: Activity Theory and Human-Computer Interaction ,” MIT Press, Cambridge, 1996 .

[53] M. Wang , V. Callaghan , J. Bernhardt , K. White och A. Peña-Rios, ”Augmented reality in education and training: pedagogical approaches and illustrative case studies ., ” Journal of Ambient Intelligence and Humanized Computing , 2017 .

[54] M. Ely , Doing Qualitative Research., London: Falmer Press, 1991 .

[55] R. Boyatzis , Thematic Analysis and Code Development: Transforming Qualitative Information , London and New Delhi: Sage publication, 1998 .

[56] J. W. Creswell , Qualitative inquiry and research design: choosing among five approaches (3 ., [updated] ed.), Thousand Oaks: SAGE Publications , 2013 .