Understanding chemical concepts requires learners to engage with information on three interconnected levels: the macroscopic (observable phenomena), the microscopic (atomic and molecular interactions), and the symbolic (chemical equations and representations). This triple nature of chemistry poses challenges for traditional teaching methods, particularly when conveying the abstract and invisible aspects of these concepts. As a result, gaps in comprehension can arise, ultimately hindering learning outcomes. This paper presents the design and implementation of an Augmented Reality (AR) training system fusing the triple nature of chemical concepts into a contextually coherent learning experience. By integrating AR technology, learners can interact with 3D models of molecules, visualise chemical reactions in real-time, and connect abstract symbols with their physical and molecular counterparts.
CEUR ceur-ws.org
enhances students’ comprehension of concepts taught in the classr1o,o2]m. F[or example, in chemistry education, laboratory activities play a vital role, allowing students to manipulate substances and handle laboratory equipment to conduct and observe chemical reactions, and understand new chemical concepts. Learning chemistry and understanding chemical concepts requires engagement on three interconnected levels, collectively known as the triple nature of chemical concepts. These levels include: • Macroscopic: this level focuses on observable phenomena such as reactions, properties, and behaviours of substances in bulk. It includes concepts like concentration, temperature, and pressure, emphasising measurable quantities and tangible observations. Pucihar)
CEUR Workshop Proceedings • Microscopic: this level examines the behaviour and interactions of individual atoms and molecules. It involves understanding structures, bonding, and the forces that govern molecular interactions, providing insight in how macroscopic properties arise from microscopic behaviour. • Symbolic: this level uses chemical symbols, formulas, and equations to represent the other two levels. It encompasses the language of chemistry, including chemical equations to describe reactions, the periodic table, and other notational systems that help convey complex ideas in a concise manner3[].
Traditional teaching methods often face challenges in efectively conveying the abstract and invisible nature of chemical concepts, which ca result in gaps in comprehension. To address this, educators frequently incorporate various training systems, particularly at the microscopic level, enabling students to develop a deeper understanding of specific chemical concepts [4, 5]. Visualisation tools for the microscopic level include static and dynamic 2D or 3D micro-representations, such as 2D static images, 3D physical static models, 2D or 3D animations, as well as virtual reality mo6d,e7l]s. [Notably, research indicates that students who engage with 3D animations tend to achieve a better grasp of the chemical concepts compared to students using static representat8i]o.ns [
Augmented reality (AR) enhances the physical environment by overlaying it with computergenerated elements. This enables students to develop important skills and literacies that are dificult or impossible to achieve in traditional technology-enhanced learning environ9m,ents [ 10]. In this paper, we present the design and implementation of an AR training system designed to fuse the triple nature of chemical concepts. By integrating AR technology, learners can interact with 3D models of molecules, observe chemical reactions in real-time, and connect abstract symbols with their physical and molecular counterparts.
As mentioned, understanding chemical concepts requires learners to engage with information across three levels. However, in traditional classroom settings or laboratory, presenting all three levels simultaneously within the same context can be challenging. Additionally, it is often dificult to ensure that information at one level is efectively connected to the others. Achieving a seamless coupling of these levels is likely to result in coherent understanding of a complex chemical concept and improve learning outcomes. The primary objective of designing our training system is to facilitate this seamless coupling of information across the three levels.
This coupling can be achieved through AR due to its ability to overlay visual content onto the physical world. By making invisible elements visible to the naked eye, AR allows us to seamlessly integrate all three levels within the real world context, such as a laboratory experiments. The design of our training system is presented in the following sections.
When designing an AR system, the goal is to create experiences that integrate seamlessly with the surrounding environment, enhancing rather than obstructing or replacing reality. To improve the understanding of chemical concepts, we use the laboratory setFitgiunrges 1() as the physical context, where experiments represent the macroscopic level of information (i.e. the observable phenomena). Since learners need to interact with laboratory equipment, it is essential for the tool being designed to enable hands-free interaction; otherwise, it would hinder their ability to conduct the experiments efectively. Therefore, we opted for a head-mounted optical see through displaFyig(ure 1).
To visualise the microscopic level of information (i.e. atomic and molecular interactions), it is essential to establish a method for transitioning from macroscopic level to the microscopic level. For this purpose, we designed tvhiertual magnifying glass, which allows the user to explore the microscopic level of information while preserving the context of the experimental setup (Figure 2left).
The final level of information to be visualised in the laboratory experiment is the symbolic layer. To incorporate this, we place labels above each interacting particle indicating the symbolic description of the molecule. Additionally, we have carefully designed molecule models to align with these symbolic descriptions. This is achieved though consistent use of colour, which indicates the type of atoms (e.g. blue for Nitrogen), the number of atoms (e.g. consistent with symbolic description) and the relative size of atoms (e.g. Nitrogen atoms are represented by larger spheres compared to hydrogen atomFsi)g(ure 2right).
One method to designing interaction withvtihretual magnifying glass could be fixing it in space and prompt learners to peak through it by moving to the predefined position where educator has placed the glass. To make the tool more engaging we decided to allow learners to handle thevirtual magnifying glass. When designing this interaction we tapped into existing afordances of handling a real magnifying glass where the interaction consists of grasping the handle and moving it in order to look at the point of interest. We implemented this interaction following two approaches: (i) through tangible object held in hand that acts as a proxy to manipulate thveirtual magnifying glass position and (ii) through a mid-air grasping gesture combined with hand movement.
We implemented two prototype systems for visualising chemical concepts, one using an AR head-mounted-display (HMD) Microsoft HoloLens 2 and the other on10a.5, ∈ Android tablet Samsung Galaxy Tab S4. The two prototypes were developed to evaluate their efects on learning outcomes when learners (future chemistry teachers) study the same content on a traditional display compared to in-context AR display. The development has been done iteratively with the experts in chemistry didactics. Their constructive feedback after each system iteration has assured the correctness of all visuals as well as improved the user-friendliness of interaction.
The AR prototype was implemented for Microsoft HoloLens1 2using the Unity3D game development engin2e. For camera pose tracking, the HoloLens inbuilt tracking system was used. To initialise the positions of augmentations, we used Vu3faonrdiaour custom-made image markers, which accurately align the virtual content with physical chemistry experiment setups. One marker is used for initialising the experiment setup of the chemical concept and is removed from the scene after initialisation. Another marker is used for tracking the physical tangible object, which acts as a prop to manipulatevtihretual magnifying glass position. The marker is continuously tracked throughout the learning scenario.
All molecular 3D models, including both interacting and non-interacting molecules, as well as relevant confounding factors, were developed as prefabs in Unity 3D. This modular design facilitates easy instantiation, manipulation, and scalability of the models within the simulation. A spawning mechanism was developed to generate non-interacting molecules dynamically. These molecules were designed to spawn at specified locations and move freely, which enhanced the realism of the simulation. To enhance user control, a button was integrated into the interface, allowing learners to toggle the visibility of non-interacting molecules. This feature enables learners to customise their view, focusing on specific interactions or phenomena without distraction.
Learners can observe chemical reactions throuvgihratual magnifying glass, a key feature of the experience. Thevirtual magnifying glass was designed as a 3D virtual object within the AR space and can be manipulated in two primary ways: 1. Using a tangible object in hand which acts as a prop: A tangible object was attached to the markerF(igure 2right), allowing learners to directly contrvoilrtuhael magnifying glass. As learners move the object, a correspondivnigrtual magnifying glass 1https://www.microsoft.com/en-us/hololens 2https://unity.com/ 3https://developer.vuforia.com/ appears above the marker with continuous tracking, allowing the observation of chemical interactions in detail. 2. Grasping the virtual object which is fixed in space: Using the HoloLens’s hand tracking and mid-air gestures, learners can also grab and repositiovnirtthueal magnifying glass with their hands.
To control the visual efects, a new shader was implemented for the lens ovfirtthueal magnifying glass. This shader creates a magnifying efect that enhances the visibility of molecules and chemical interactions, rendering them only through the lens.
The tablet version of the prototype was also implemented using the Unity 3D development environment, but deployed on a Samsung Galaxy Tab4St4ablet. Its functionality is similar to the AR prototype, ensuring consistency across platforms.
The tablet interface was designed with touch interactions. In this implementation, learners can tap and drag thveirtual magnifying glass on the touchscreen to explore chemical reactions in detail. The visual design of the tablet application mirrored that of the AR version, featuring similar colour schemes, layout, and interactive elements.
This paper presents an AR training system designed to enhance the understanding of chemical concepts by connecting the macroscopic, microscopic, and symbolic levels of chemistry. By using AR technology, learners can engage with 3D models of molecules and visualise chemical reactions in real-time, making abstract ideas more accessible.
The initial feedback from the experts in chemistry didactics has been positive. They recognise the value of combining immersive visuals with user-friendly controls to overcome the limitations of traditional chemistry education, and aim to explore its efectiveness through comprehensive user evaluation. We are currently developing additional learning scenarios to be evaluated–lab experiments supported by the developed AR training system.
This research was funded by the Slovenian Research Agency, grant number P1-0383, P5-0433, IO-0035, J5-50155 and J7-50096. This work has also been supported by the research program CogniCom (0013103) at the University of Primorska.
[1] D. A. Kolb, Experiential learning theory and the learning style inventory: A reply to freedman and stumpf, Academy of Management Review 6 (1981) 289–296.
4https://www.samsung.com/global/galaxy/galaxy-tab-s4/ [2] D. Kolb, Experiential learning: experience as the source of learning and development, Prentice Hall, 1984. URhLt:tp://www.learningfromexperience.com/images/uploads/ process-of-experiential-learning.pdf(dateofdownload:31.05..2006) [3] V. Talanquer, Macro, submicro, and symbolic: The many faces of the chemistry “triplet”,
International Journal of Science Education 33 (2011) 179–195. [4] A. T. Stull, M. Gainer, S. Padalkar, M. Hegarty, Promoting representational competence with molecular models in organic chemistry, Journal of Chemical Education 93 (2016) 994–1001. [5] M. Stief, S. Scopelitis, M. E. Lira, D. Desutter, Improving representational competence with concrete models, Science Education 100 (2016) 344–363. [6] S. M. Reeves, K. J. Crippen, E. D. McCray, The varied experience of undergraduate students learning chemistry in virtual reality laboratories, Computers & Education 175 (2021) 104320. [7] J. Á. Ramírez, A. M. V. Bueno, Learning organic chemistry with virtual reality, in: 2020
IEEE international conference on engineering veracruz (ICEV), IEEE, 2020, pp. 1–4. [8] S.-C. Chen, M.-S. Hsiao, H.-C. She, The efects of static versus dynamic 3d representations on 10th grade students’ atomic orbital mental model construction: Evidence from eye movement behaviors, Computers in Human Behavior 53 (2015) 169–180. [9] J. Carmigniani, Augmented reality: An overview, Handbook of augmented reality/Springer (2011). [10] J. Carmigniani, B. Furht, M. Anisetti, P. Ceravolo, E. Damiani, M. Ivkovic, Augmented reality technologies, systems and applications, Multimedia tools and applications 51 (2011) 341–377.