Teaching Assistant Robots in Various Fields: Natural Sciences, Medicine and Specific Non-Deterministic Conditions Roman Haskoa, Oleksandra Haskoa, Hakan Kutucub a Lviv Polytechnic National University, 12, Stepan Bandera str., Lviv, 79000, Ukraine b Karabuk University, Department of Software Engineering, Karabuk, 78050, Turkey Abstract The article describes an approach to creating a Robot Assistant as an intelligent system that uses generative artificial intelligence (GAI) and has a physical body and a voice interface for communication. To understand the actions of the assistant and increase trust in the system, Explainable AI is used, and for better efficiency - Federated Learning, that is, the robot learns on distributed data without the need for centralized collection of this data. XAI allows users to understand what algorithms and data are driving the decision- making assistant. Access to Large Language Models means that the robot assistant uses powerful language models such as GPT-3.5 to understand and generate language. This allows the assistant to understand user requests and provide information and recommendations based on a large amount of knowledge and texts. Robot interactions with the real world are becoming better and more predictable thanks to Embodied AI. It can perform tasks related to moving physical objects, understand the gestures and movements of users, and interact with objects in real-time, moving in different places and environments. The software solution is based on ROS2 with the necessary extensions for the listed technologies. All this together makes the work of the assistant effective, providing understanding and explanation of decisions, protecting data privacy and providing natural communication through the voice interface. Keywords 1 Robotics, Federated learning, Explainable AI, XAI, Embodied AI, Telepresence, ROS. 1. Introduction Assistant robots can be very useful in education in a variety of fields, including science, medicine, and areas with adverse conditions. They can provide support and assistance to students, teachers and professionals in these fields. Here are some ways that can be used: 1. Explanation of material: Assistant robots can provide explanations of complex topics using text, visual or audio materials. In the natural sciences, this can include mathematical calculations, chemical reactions, and physical laws. In medicine, they can help understand the anatomy, physiology and treatment of certain diseases. 2. Simulations and Virtual Labs: In the natural sciences, robot assistants can create virtual labs and simulations that allow students to study experiments and phenomena in a safe environment. In medicine, they can be used for medical simulation and training. 3. Personalized learning approaches: Assistant robots can adapt learning to the needs of each student. They can take into account knowledge level, interests and pace of learning to provide effective individual support. 4. Progress Tracking: Assistant robots can track student progress and provide reports to teachers or academics. This helps to identify the problems and needs of students in time. IDDM’2023: 6th International Conference on Informatics & Data-Driven Medicine, November 17 - 19, 2023, Bratislava, Slovakia EMAIL: r.hasko@gmail.com (A. 1); oleksandra.l.hasko@lpnu.ua (A. 2); hakankutucu@karabuk.edu.tr (A.3) ORCID: 0000-0001-5923-6577 (A. 1); 0000-0003-4519-610X (A. 2); 0000-0001-7144-7246 (A. 3) ©️ 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings 5. Access to resources: Assistant robots can provide access to libraries of scientific sources and publications. In medicine, this may include access to databases of medical journals and clinical guidelines. 6. Support in dangerous environments: In specific dangerous environments, such as deep space exploration, underwater operations or emergencies, assistant robots can be used to collect data, monitor and provide recommendations without risking human life. 7. Providing access to expert opinion: In teaching and research in any field, robot assistants can help students and researchers gain access to global experts and consultants through virtual conferences and communication systems. It is important to note that successful collaboration between assistant robots and humans requires careful software development, integration with real learning and work processes, and ensuring data privacy and security. Each specific industry may have its own unique requirements and challenges, and developers must take these into account. The world is witnessing the growth of more diverse learning methods, in particular, using robotics, the Internet of Things (IoT) and augmented and virtual reality (AR/VR) applications. This allows smart devices to share data over the Internet and improve the quality of learning. Virtual Reality and Augmented Reality enable the creation of immersive learning environments. For example, Microsoft HoloLens allows students to interact with virtual objects to teach science subjects. The use of generative artificial intelligence plays a special role. Successful collaboration between assistant robots and humans requires careful software development, integration with real-world learning and work processes, and data privacy and security. Special cloud web-oriented educational platforms have been developed [1, 2, 3] and a separate class of software has appeared - e-learning. Now, it's time for specialized hardware for training. Such systems are mainly built on the basis of a decentralized architecture and use the capabilities of cloud services for communication mainly through a specialized API, for example, access to ChatGPT from OpenAI. Due to the predominant focus on the Internet and cloud technologies, such systems are mostly not very effective when used in the modern hybrid learning process, which requires a quick reaction and taking into account various obstacles in the communication process. Certain features of training are present in specific fields, for example, in medical education, modern blended learning technologies using robotic educational platforms, in particular specialized telepresence robots for remote participation of students from different geographically distant places with maximum immersion in the educational process, can be particularly effective [4, 5]. Telepresence robots have been used for a relatively long time and not only in education, so we are talking about a fundamentally different, new level of their autonomy. Thanks to the use of LLM capabilities, such robots become real assistants and their remote control capability translates the efficiency and accessibility of training as well as the practical experience of future specialists to a new level with better results in their future professional activities. 2. Background The appearance of assistant robots in education was preceded by a number of stages and technological developments. Here are some key events and factors that created the basis for the emergence of this technology: 1. Development of information technology: With the advent of computers and the Internet, access to a large amount of information and interactive learning has become possible. Information technologies have become the basis for the development of educational technologies. 2. E-Literature and Learning Platforms: The launch of e-textbooks and a learning platform allows students to study online and access various learning resources, including video lessons and interactive tasks. 3. Development of artificial intelligence: The gradual development of natural language processing and machine learning technologies has made it possible to create robot assistants capable of understanding and generating human speech. This opened the door to the development of intelligent learning systems. 4. Scaling of data processing: The increase in the power of computing systems and the increased availability of large amounts of data have made it possible to create assistant robots that can quickly analyze information and provide answers. 5. Development of educational initiatives: Interest in improving the quality of education and accessibility to it has led to the funding and development of innovative educational technologies. 6. Change in learning approach: The learning paradigm is also changing. Today's students and professionals are more familiar with virtual and online learning, creating a demand for technologies that meet these needs. 7. Research in the field of education and psychology: The study of learning processes and the development of cognitive sciences and psychology contributed to the creation of more effective teaching methods and interactive systems. All of these factors have contributed to the development of teaching assistant robots that provide students and learners with access to individualized learning, additional resources, and support that can enhance the quality of learning and facilitate the effective acquisition of new knowledge. 3. Related works Current developments in robotics to assist in the educational process include a wide range of innovative technologies and robots that contribute to improving the quality and accessibility of education. Robotics in education offers an impressive range of innovative solutions that facilitate learning and increase its effectiveness. Robot-assistants for training are primarily AI-based personal assistants. By using artificial intelligence like GPT-3, they can provide personalized support to students, explain complex concepts and answer questions. Related developments should also be included here - social robots like Pepper, which can be used to interact with people with various special needs, promoting social integration and learning communication skills. At the same time, the use of multi-agent robots to create educational games allows for the development of cooperation, communication and problem-solving skills. This entire successful development process is hard to imagine without virtual and augmented reality. Thanks to these technologies, simulators and virtual laboratories have become possible, which allow students to study complex phenomena and experiments safely. And augmented reality, for example in applications for smartphones and AR-glasses, provides additional information and an interactive experience in the learning process. Special mention should be made of exoskeletons for medical education, which help students and medical staff learn anatomy, perform surgeries and research new treatments. Robots for medical education with simulators allow medical students to practice performing operations and diagnosing patients without risk to life [6]. AI support for teachers. Platforms that use artificial intelligence to analyze learning data and create personalized plans for students can make the work of teachers easier and improve the quality of learning. AI-powered learning platforms like Coursera, edX, and Udacity use machine learning algorithms to personalize learning and predict student performance. Assistant robots, such as NAO and Pepper from SoftBank, as well as robots from Boston Dynamics, can be used to teach various educational subjects. They can provide interactive lessons using speech and facial reproduction, which helps create positive interactions with students. Some works are designed to improve students' speaking skills. For example, robots from the company Leka are used to teach children to interact and develop speech skills, and companion robots for students with special needs, such as QTrobot, can be used to teach and develop children with specific needs. Special mention should be made of robots for STEM education [7], which specialize in teaching scientific subjects such as mathematics, programming and robotics. Robots such as Wonder Workshop's Dash and Dot help create interactive activities for children in these areas. These are just a few examples of current developments in the field of robotics for education. With the development of technologies and innovations, further expansion of opportunities for enriching the educational process and increasing interest in learning among students is expected. Modern developments in robotics for learning are actively developing and help improve learning processes and provide more accessible and interactive education. Such robots and technologies expand learning opportunities and contribute to the development of digital education [8]. 4. Proposed Approach The proposed approach is based on the previous experience of creating autonomous learning robots and takes into account the latest achievements in AI, in particular generative AI and LLM. The design is a fairly autonomous robotic platform consisting of the following main parts: 1. a mobile wheeled chassis with a system of motors, a control controller (for example, Arduino or STM32), a power controller with batteries and a line of sensors in the composition: - ultrasonic sensors with an emitter - infrared sensors with an emitter - radio frequency module with Bluetooth support 2. a smart unit based on the Nvidia Jetson microcomputer with a camera and computer vision support. Previous versions used a Raspberry Pi with a separate Intel Movidius to run the pre-trained neural networks, but the switch to Nvidia's platform looks more promising both in terms of efficiency and power, as well as a wide selection of specialized robotics software. 3. a user interface with an emphasis on voice input and output and additional capabilities such as a display and manipulators based on the robot arm. The feature of this voice interface is its versatility, autonomy and high degree of "intelligence" thanks to three components: - voice recognition with conversion to text - connection with OpenAI API (ChatGPT) in the format of exchanging text messages - voice synthesis based on text received from ChatGPT. In addition, the expansion of the possibilities of the user interface is provided due to the analysis of both the text and the position of the face, in particular facial expressions of the user. Thanks to this, the quality of human-machine interaction is moving to a qualitatively better level. More details about the use of ChatGPT in robotics are shown in Figure 1 by He, Hongmei [9] Figure 1: Framework of RobotGPT. Copyright by He, Hongmei. (2023). [9] The software of the described robotic platform is based on ROS2 - Robot Operating System of the second generation. At the same time, a number of features should be noted: 1. Use of generative AI (LLM) on the example of API from OpenAI, namely ChatGPT 3.5 with a two-way voice interface and a number of additional features. 2. Federated learning, thanks to which it is possible to increase the autonomy of the robot and its operation in the presence of obstacles and interruptions in connection with the cloud service. Thanks to both the capabilities of ROS and decentralized machine learning, it is possible to ensure sufficiently "intelligent" behavior of the robot in the absence of commands from the operator. 3. Explainable AI and Cooperation of Robots. This is another set of very important features of the described robot, as it allows a human to "understand" the behavior of the robot, through clarifying commands and questions with answers actually to program and improve the behavior of the robot. In turn, cooperation between individual robots, their joint interaction brings their efficiency to a qualitatively new level and allows teams of robots and people to cooperate. 4.1. Peculiarities of using generative AI (LLM) in the educational process LLMs (Large Language Models) such as GPT-3 open many possibilities for the development of educational technologies. They play key roles in the following aspects such as рersonalized learning. LLMs can create individualized learning materials based on each student's needs and skill level. They can approach each student individually and provide additional explanations or tasks to strengthen their understanding of the material. The second option is teacher and instructor support. LLMs can help teachers create curricula, materials, and tests. They can also make recommendations for improving teaching and evaluating student performance. The next option is content generation automation, question answers and explanations. LLMs can generate text and visual content for learning materials, including textbooks, articles and how-tos, and provide detailed explanations and answers to questions about various subjects and topics. This can significantly facilitate the development of learning resources and is useful for students looking for additional information or clarification on specific issues. Another option of using LLM is instant access to information. Robot assistants can quickly provide information on various subjects and topics, which helps students effectively search and find answers to their questions. Use in interactive applications. LLMs can be used in web services and learning applications where they can interact with users via text chats or voice commands. In higher education and research, LLMs can be used to quickly access a large number of scholarly articles, data, and information to support research. However, it is important to remember that the use of the LLM also involves ethical issues relating to authorship, data sources and transparency. Also, the development of these technologies requires attention to security and privacy issues in the educational context. 4.2. Federated learning and Embodied AI Federated or collaborative learning is a specific decentralized approach to training machine learning models that does not require the exchange of data between cloud services and individual robots or other client devices. Instead, the raw data on the autonomous robots is used to train the model locally. This increases the autonomy and confidentiality of data. Figure 2: Federated learning framework. Copyright by Huang, Xixi [12] Federated learning approach in medical education can be particularly useful because of the confidentiality and security of medical data. The main features of federated learning in the medical field include data decentralization, where instead of collecting all medical data in one place, federated learning allows the data to be kept on local servers or devices such as medical equipment, smartphones or desktop PCs. The model is trained locally on each device and only aggregated parameters are sent to a centralized server, which also helps reduce the risk of leaking sensitive medical data because the data itself does not leave local servers or devices. Federated learning in medical education can be used to create and improve models for medical diagnosis, image processing, patient data analysis, and many other tasks while maintaining the privacy and security of medical data. Embodied AI [6] combines computational intelligence with the physical body of a robot or agent. This approach enables robots to interact with the physical world and environment, making them ideal for use as assistants or helpers. The structure of embodied AI is aimed at solving problems of perception, planning and execution of actions in real time. The combination of embodied AI and federated learning can be useful in the development of assistant robots, as it allows improving the quality of interaction and robot learning. Several assistant robots can learn on different devices, and then combine their knowledge in one centralized model using federated learning. The use of federated learning helps to store user data on their devices, which allows for a higher level of privacy. Assistant robots equipped with embodied AI can work with these models without disclosing users' personal information. 4.3. Explainable AI and Cooperation of Robots The proposed approach based on the ROS2 [13] operating system of robots together with the use of generative AI, described software architecture, support for federated learning, explainable AI and communication capabilities between robots for control and their interaction allows the simultaneous use of several personalized "smart" assistant robots. In addition, if necessary, other robotic components are involved in the educational process, for example, manipulators based on robot hands. Explainable AI (XAI) is an approach to creating intelligent systems that are able to explain their decisions and actions in a human-understandable form. This approach is particularly important in robotics, as robots can interact with humans in a variety of domains, and explaining their actions can be critical for safety, trust, and acceptance. Explainable AI in robotics involves clearly explaining decisions. Robots, thanks to XAI, try to reveal how they came to a certain decision or action. This may include displaying the logic, calculations and data used to make decisions. XAI can provide visual representations of its decisions and processes occurring inside the robot. This can be graphical charts, diagrams or other visual tools. Figure 3: Example of cooperation between a team of robots and people. Copyright by R.Hasko [3] In this example, the interaction of several robots with people in a team. Individual assistant robots interact with each other using available communication channels such as radio, infrared or ultrasonic and with people mainly through a voice interface. 5. Results The proposed approach makes it possible to create qualitatively better assistant robots for various fields of use and to integrate them into existing or new educational systems. The modular approach allows you to adapt the system to specific needs. The emergence of LLM has opened up new opportunities for AI, and its use in the described assistant robot opens up completely new opportunities compared to previous versions of robots. At the same time, the emphasis on the use of a "smart" voice interface significantly improves human-machine interaction. The use of federated learning and specialized AI-oriented microcomputers ensures a high level of autonomy and independence for such a robot. Thanks to the use of XAI, the robot can be applied in various fields, in particular in medicine to explain diagnoses and treatments and as an autonomous moving platform with explanations of its behavior and in many other fields. This approach is essential to ensure safety and trust in robots in various human interaction scenarios. Taking into account the possibilities of interaction between individual robots for their teamwork makes it possible to effectively use the described robotic platform both in the educational process and in many other fields. 6. Conclusion and Future Directions In this article, we proposed a modern approach to the creation of highly mobile intelligent assistant robots as a synergy of robotics with modern artificial intelligence technologies and a convenient user interface with an emphasis on voice input-output as the most convenient for humans. 7. References [1] Hasko, R. et al. MEdOps - Medical Education with Emphasis on Robotics. IDDM-2022: 5th International Conference on Informatics & Data-Driven Medicine, November 18–20, 2022, Lyon, France. CEUR Workshop Proceedings. – 2022. – Vol. 3302 : P. 266–271. [2] Josie Hughes at al. Embodied Artificial Intelligence: Enabling the Next Intelligence Revolution. 2022 IOP Conf. Ser.: Mater. Sci. Eng. 1261 012001 [3] Hasko, R., Shakhovska, N. Tripled Learning: Conception and First Steps. CEUR Workshop Proceedings, 2018, 2105: 481–484. [4] A. Latip, Y. Andriani, S. Purnamasari and D. Abdurrahman. Integration of educational robotic in STEM learning to promote students' collaborative skill. Journal of Physics: Conference Series, Volume 1663, 5th Seminar Nasional Matematika dan Pendidikan Matematika (SENATIK) 2020 12-13 August 2020, Semarang, Indonesia [5] Hasko, R. et al. A mixed Fog/Edge/AIoT/robotics education approach based on tripled learning[J]. CEUR Workshop Proceedings, 2020, 2616: 227–236. [6] Jiafei Duan et al. A Survey of Embodied AI: From Simulators to Research Tasks. doi:10.48550/arXiv.2103.04918 [7] Shakhovska, N., et. al. The Student Training System Based on the Approaches of Gamification[J]. Advances in Intelligent Systems and Computing, 2020, 938: 579–589. [8] Shakhovska, N., et. al. The method of big data processing for distance educational system. Advances in Intelligent Systems and Computing, 2018, 689, pp. 461–473 [9] Darmawansah, D., Hwang, GJ., Chen, MR.A. et al. Trends and research foci of robotics-based STEM education: a systematic review from diverse angles based on the technology-based learning model. IJ STEM Ed 10, 12 (2023). https://doi.org/10.1186/s40594-023-00400-3 [10] Oña Edwin Daniel; et. al.. "Robotics in health care: Perspectives of robot-aided interventions in clinical practice for rehabilitation of upper limbs". Applied Sciences. 2019, 9 (13): 2586. doi:10.3390/app9132586. ISSN 2076-3417. [11] He, Hongmei. (2023). RobotGPT: From ChatGPT to Robot Intelligence. 10.36227/techrxiv.22569247. [12] Huang, Xixi & Ding, Ye & Jiang, Zoe & Qi, Shuhan & Wang, Xuan & Liao, Qing. (2020). DP-FL: a novel differentially private federated learning framework for the unbalanced data. World Wide Web. 23. 10.1007/s11280-020-00780-4. [13] Steve Macenski at al. Robot Operating System 2: Design, Architecture, and Uses In The Wild. https://doi.org/10.48550/arXiv.2211.07752