The aim of the study is to take advantages of modern information technology to solve educational problems and improve the quality of education. This review uses the capabilities and tools of virtual reality and machine learning to create a learning platform for children with disabilities. The document also describes the partial practical implementation of the system. The platform is designed to solve students' potential problems with learning and socialization. The study concludes by highlighting aspects that require further research and development.
Autism is a life-long illness that has a very big impact on the children's personality as
well as their families or caregivers. Diagnosis can give an understanding of why a child
or teenager is different from their peers, as well as open the way to support and services
in education, health and social care, access to volunteer organizations, and contact with
other children and families with similar experience. All this can improve the lives of
children and their family members. The main manifestations of autistic behavior are
usually present in early childhood, but are not always noticeable in the circumstances
in which the child's condition is manifested, such as when he/she is in kindergarten or
in elementary or secondary school. Autism is largely related to a number of
comorbidities [
Therefore, VR can offer the benefit of presenting abstract concepts through visual means (such as bubbles with virtual character lines) and seamlessly allow you to change the environment (for example, change the color of the ball or make the table disappear), which may be difficult or even impossible in a real environment. VR can also portray different scenarios that may be unworkable in a therapeutic "real world" setting given the natural social constraints and resource issues. Thus, VR is well suited for creating interactive skills training paradigms in the main areas of defection for individuals with ASD. Machine learning in education is a form of personalized learning that can used to give each student a personalized educational experience. In these conditions, students focus more on their own education, can follow the most comfortable pace for them, and make independent decisions about what to study [16-15].
Machine learning provides an opportunity for better organization and management
of the curriculum. This helps to accordingly differentiate each job and better understand
each student's potential. As a result, it improves the quality of the analysis of which
work is best for the teacher and which teaching methods are most effective for the
student. This method facilitates the work of teachers and students and makes their
learning process more comfortable and fun. It also increases the level of involvement
and interest in learning, which increases the effectiveness of education. Machine
learning has the potential to make teachers more effective by performing tasks such as
classroom management, scheduling, and more. Thus, teachers can focus on tasks that
cannot be accomplished by AI and which require human intervention [
Artificial intelligence is a term that describes an algorithm based on its own knowledge base to deliver results that have a significant impact on users. With this concept in mind, it is easy to understand the importance of machine learning in this field. The connection to virtual reality becomes clearer when one considers the increasing impact that machine learning has on it. A simple headset that displays digital images does not require artificial intelligence. However, virtual reality does depend on highly accurate AI algorithms to simulate reality.
Old-school teachers find it difficult to get used to machines that have the ability to think and learn. Such people do generally not perceive machine learning suggestions as a tool that can change the field of education due to lack of attention. However, the introduction of such technology in learning will not avoided. Machine learning in the form of predictive analytics is used to draw conclusions about things that may happen in the future. For example, using a high school student’s record set, predictive analytics might tell which one is more likely is expelled because of an academic failure or even a predictable score on a standardized exam, such as ACT or SAT. Machine learning in the form of adaptive learning can used to exclude problematic situations or encourage the thirst for knowledge. Adaptive learning is a technological or online education system that analyzes student’s work in real time and modifies teaching methods and curriculum based on this data. 2
Our main goal is to help improve the learning and social skills of students with special learning needs using virtual reality and machine learning. Children, especially when they are very young, do not understand their differences from others and because that they primarily need to be taught to realize their identity and social behavior properly. Later they can learn various subjects, such as mathematics, biology, history, art, music, physics, etc. The teacher or assistant teacher chooses a lesson according to the level of knowledge and social skills of the student, and the child is going through a lesson, as he wants. Because the lesson is designed in the form of a game, the child will not even notice that he is automatically receiving new knowledge and skills.
The virtual reality system also requires the use of non-player characters who need to
respond to the environment. That is, for a full immersion, they must not only follow the
written code for interaction, but also "learn" to do something a person would. For
example, in the case of a road situation, a “smart” non-player character may respond to
the road situation based on information from his or her visual and auditory senses. This
is precisely what the virtual reality environment needs to develop. Children who see
NPC behavior will take this as a basis for their interactions with the system [
There are studies that have already implemented similar systems, but all of them are informative due to the lack of standardization of virtual reality systems (hardware and software). An example is the ClassVR technology, which offers kids exciting, beautifully focused training programs, virtual reality education resources. You can search by topic, topic, or even keyword and find ready-made collections or individual resources to create your own lesson [1-3]. Computer expert Jack Clover studies the relationship between machine learning, artificial intelligence, and virtual reality. Another example is the research and implementation of Second Life ™ VR island virtual reality, where the locations are: school classroom, school canteen, playground, camping, racing track, fast food restaurant, hardware store, apartment, coffee shop, sporting goods store and Central Park.
Examples of social lessons that a child can learn [
For a comprehensive picture of the essence of the system under study, we depict a tree of goals. In order to represent the functionality and interaction of processes in a system, there are many methodologies. The IDEF0 methodology is chosen to represent this information system because it is the most common one for use where it is necessary to illustrate methods that require simple and clear definitions. In addition to the inputs and outputs, the methodology has controls and mechanisms. The former serves as the kind of guide and instruction on which the interaction of functional blocks should take place. The second is, in fact, people with certain skills, equipment, hardware, etc. who serve as tools when executing the system. The following four types of arrows are used for diagram A0. Input: Student's self-study, choosing a lesson by a teacher. Output: “Test results for the student”, “Test results for the teacher”. Controls: “AR interface”, “Information technology tools (glasses, controllers, tablets, etc.)”, “Recommendations for the education of students with special needs”. Mechanisms: “Student with special needs”, “Teacher”, “Hardware”. After determining the A0 level, which represents the main purpose of the system, at least one decomposition is made (usually two). Although, if required by the system, up to eight decompositions can be made for detail and clarity in illustrating the main functionality. After breaking down the main functional block “Virtual reality platform using ML for teaching children with special needs”, five new features are created: “Start the lesson”, “Play a theoretical scene”, “Complete a practice scene”, “Analyze student activity through ML”, “Complete the lesson”.
All the arrows that are used in the previous diagram remain at this level. In addition,
several new inputs and outputs have added: “Running the theoretical part of the lesson”,
“Running the practical part of the lesson”, “Data on student activity”, “Running the
standby window”, “Data analyzed” [
For better demonstrate how the “Complete a practice scene” function block decided to
make a second decomposition. As a result, this functional block has been broken down
into three others that clearly demonstrate its activities: “Expect to trigger”, “Work on
the student's actions”, “Respond to student action”. Also included here are the inputs
and outputs: “Student action”, “Selected reaction for interaction”. Since machine
learning is a very important component of our system, decomposition of the functional
block “Analyze student activity through ML” is recommended and necessary for
detailing the information system. After breaking this block, four new features are
created that clearly and simply represent this process: "Initialize a new method object",
"Generate input and output data for training", "Model training", "Use the constructed
model to analyze the student's activity”. To combine these four function blocks, you
need to add auxiliary inputs and outputs to the diagram. As a result, three new arrows
are created: “Initialization of indicators”, “Input and output data”, “The model is built
as a result of training”.
To present a simplified version of the basic functions that will implemented in the
information system, a hierarchy of system tasks is a good option. This includes all five
major function blocks that were included in the A0 diagram shown earlier and the main
function of the A-0 diagram, which is the first general system diagram.
Because of the systematic analysis of the information system, four diagrams were
constructed using the IDEF0 methodology. The first diagram represents the main
function of the developed system. After the first decomposition of the main functional
block into five functions, a general system view is created. Next, to better illustrate the
individual processes, we made two more partitions of the selected functional blocks.
After performing this system analysis, you can be convinced of the simplicity, accuracy
and clarity of the IDEF0 methodology [
The following system is created in a way that meets all the requirements of web
accessibility, software and universal design [
In the future, to implement this application-using machine learning, it is necessary to choose its methodology and algorithm. Since the primary purpose is to classify the performance of the lesson, it is very convenient to use the reference vector method (SVM). It is the most common to solve the classification and regression problem.
The main feature of the method is that it is more accurate than other estimation methods. As the virtual reality system is implemented, it will run from a computer. This means that the computing power of the device is sufficient to work out a method algorithm. Classified objects can divided only linearly in individual cases. For the most part, they are not linear. Therefore, to solve the linear distribution problem, kernel functions that project data from a low-dimensional space into a multidimensional one are used. With the right kernel function selected, objects can linearly separate by a hyperplane in multidimensional space. Thus, kernel functions serve as a directing space.
The main task of the algorithm is to find the most correct line, or hyperplane, which
divides the data into two classes. SVM is an algorithm that receives data at the input
and returns a line separating that data [
Before the lesson begins, the child will review the theoretical scene to know what to do during the practical part of the lesson. For example, during a lesson on proper behavior on the street and specifically how to cross the road, the system will show the student how to do it - wait for the green traffic light, see if there is an underground crossing, if so, then he/she can go there to make the road safer. In the practical part, the child can do what he/she wants, but if he/she wants to cross the road to a red light, then the system simply will not allow it. One of the lessons that teach social behavior - on the street has practically implemented. There is also a non-playable character in the scene to help the child to understand when to cross the road and when not. Such avatars in the next versions of the application will learn and choose the path of their behavior. For example, crossing them through a pedestrian crossing with a traffic light or going into an underground crossing. The findings analyzing, it can concluded that children will become more socialized after the lessons are over. It can noted also that their level of knowledge will increase and the number of conflicts will decrease. However, learning through virtual environments will only be effective if parents also work with the children and explain why they should do so, not otherwise. The system is designed with the help of virtual reality capabilities and machine learning tools, because virtual reality is a powerful tool that helps students with learning disabilities improve. This helps the system anticipate the student's steps, so it can offer an alternative that the student can make.