For individuals with autism spectrum disorder, art is a significant tool of self-expression. As children and adults with autism spectrum disorder are often visual thinkers, art is a natural way to express their feelings and how they see the world. Art therapy is used in a variety of settings to support people of all ages with mental health and physical disorders. Regular art therapy sessions help students to communicate with peers and family, develop self-confidence and control their emotions. Being familiar with art materials will also help a child with autism develop stronger fine and gross motor skills and give them more flexibility in unfamiliar situations. Holding a brush or pencil, making a necklace by stringing beads, or sculpting with clay creates control of sensation and achievement of mastery. However, traditional art therapy tools are not always suitable for use by children with autism in case of increased sensory sensitivity, which makes it impossible to work with paint, brushes, or listen to certain musical compositions, etc. One of the effective tools that therapists use to help people with autism better communicate and connect with others and the world around them is virtual reality. Autism therapists and researchers began using virtual reality in the mid1990s, using the technology to create environments to help individuals with autism simulate a variety of situations. Virtual art therapy was no exception, and research in this field is extremely relevant. Art therapy sessions were conducted using a Meta Quest 2 standalone virtual reality helmet and controllers. The software used is Open Brush and Drawing Desk, which have no alternatives among free VR drawing software.
Art therapy is the process of using a certain form of art to help people deal with health issues,
depression, various mental traumas, and more. Art therapy is understood as an intervention by visual
arts with the aim of diagnosis and positive influence on the psychological and emotional state of a
person. Unlike going to a therapist, uncovering life issues that have been experienced, a person can use
art to help communicate their emotions more comfortably. The founder of art therapy is Adrian Keith
Graham Hill, who in 1945 in the book "Art Against Disease" described how he recovered while treating
tuberculosis in a sanatorium with the help of "a simple act of drawing" [
Autism (autism spectrum disorder, ASD) is becoming an increasingly common diagnosis, and recent
research from the Centers for Disease Control and Prevention of Diseases, USA indicates that every 44
children are diagnosed with an autistic spectrum disorder [
Since most children with autism have difficulties with verbal self-expression, art becomes a tool of
self-expression for them. It has been proven that children with autism often have an advantage in art,
many of them are quite talented in mathematics, drawing, music, and even drama [
One of the characteristic features of autism spectrum disorders is difficulty with verbal and social communication. In certain cases, individuals with autism do not speak and cannot use language to communicate. In other cases, it is difficult for them to process the language, to maintain a smooth and easy pronunciation. It is difficult for individuals with ASD to recognize facial expressions and body language. As a result, it might be challenging to distinguish an anecdote from a real story, or sarcasm from sincerity. Meanwhile, many individuals with autism have an extraordinary ability to think visually. Many can use this ability to process memories, record images and visual information, and express ideas through drawings or other artistic means. However, children with ASD may not have the opportunity to use artistic media, or it can be difficult, for example in large classes. Art therapy offers therapists the opportunity to work one-on-one with such children to develop a wide range of different skills.
While children with high-functioning autism may have access to art education alongside their peers, it is nearly impossible for children with more severe forms of autism. An example would be fine motor skills, as many children and adults on the autism spectrum have difficulties with it. That impairs the ability to draw, usage of scissors, or play instruments. Although most children with autism can acquire these skills over time, they will be slower than their peers. The problem is also the desire of a child with ASD to participate in group studies. Involving a child with autism in a theatre or dance performance can be very difficult, especially if skill is expected. Children with autism often have gross motor disorders. This makes it difficult to catch or hit a ball, and it makes it difficult to do activities such as dancing or certain types of instrumental music.
One of the proposals to overcome these challenges is multimedia art for children, teenagers, and
young people of different spectrums. For example, Dr. Naiara Belart García emphasizes the use of
digital art interventions to improve emotion recognition and communication skills in patients with an
autism spectrum disorder. Emphasis is placed on the use of digital painting via virtual reality [
The Rewire.education resource (https://rewire.education/) offers VR drawing as a motivation for children with ASD in graphomotor exercises using virtual reality pen and paper. Although the researchers claim that children with ASD are good with gadgets, tablets, computer games, etc., in virtual reality they can behave quite differently than they do in real life.
Researchers from Drexel University's College of Nursing and Health Professions and the School of Biomedical Engineering, Science and Health Systems studied VR as an art therapy tool, acknowledging the difference in activation of the prefrontal cortex between two different drawing tasks in virtual reality [11]. The influence of fragrance in the process of virtual painting is described in [12], it is also described that self-expression based on virtual reality is embodied visual expression, and generates new artistic and imaginative reactions. VR has been shown to have the potential to improve psychological health and well-being through creativity, enhanced imagination, and interactivity [13].
In a review of the use of art in virtual reality [14], it was concluded that latter seems to be a useful tool in neurorehabilitation. The art therapists and psychomotor therapists in [15] agreed that the virtual reality tools can be sufficient in treatment process. Such conclusion was also made by authors in [16], [17]. Positive influence (i.e. a reduction in the erroneous movements) was observed by researchers [18], who used VR to experiment with art therapy for neurorehabilitation of patients with stroke. Such improvement in the performance when interacting with an artistic stimulus was called Michelangelo effect, and it was confirmed by [19]. One of the drawbacks of the VR in therapy is the lack of eye contact between therapist and client, also it is difficult to analyze facial expression [20].
People with autism can communicate through virtual reality to share their experiences as in Beholder project (https://bom.org.uk/), whose founder, Matt Clark, describes the life of his autistic son. The goal of the project is to use virtual reality in cooperation with highly functional autistic artists.
The integration of virtual reality in art therapy is described by Liat Shamri Zeevi in "Making art therapy virtual: integration of virtual reality in art therapy with adolescents" [21]. It is noted that, in general, the VR technique can be especially useful for the treatment of children who find it difficult to create traditional art therapy in the clinic. Virtual reality can also be a therapeutic alternative for clients who fear making mistakes in their work, as it allows them to explore experiences without any physical or real-world consequences. For virtual reality art therapy clients who do not see themselves as creative, artwork in this tool can help them develop specific ideas and find ways of self-expression that are not available through other guided imagery techniques. Difficulties in such therapy are also described, for example, when the art therapist observes the client's work on the monitor, instead of looking at the client himself, disconnection from the client may occur because of attention split. In addition, users (mostly in the first sessions) may experience mild dizziness, a feeling that in extreme cases may last for a short time after the therapy session is over. This makes it critical for future research to explore the use of VR in art therapy for clients with different clinical, physical, or mental conditions. Some limitations are also the availability of powerful hardware and virtual reality tools.
Art therapy for a child with autism is recommended even at the age of two or three. In a world that can otherwise be confusing and often overwhelming, art can offer a solution to a problem that better suits the child's thinking style. Although it is important to work closely with a child's art therapist, as these professionals have experience in art and counseling, it is possible to actively work with a child at home, helping expand on what was learned in therapy. However, the following strategies can help jumpstart their creative journey.
Isotherapy, as an art therapy method, is creative, spontaneous, or thematic drawing. The drawing is an effective tool as it allows the child with ASD freely express his thoughts, feelings, and experiences through the language of images, symbols, and colors, as well as free himself from negative experiences and "paint" his attitude towards the environment. This direction is the foundation of art therapy, as it involves drawing on all possible surfaces, starting from paper and canvas, and ending with wooden and glass surfaces, fabric, etc. A wide variety of tools can be used for drawing, such as paints, pencils, oil and artistic pastels, felt-tip pens, and chalk [22].
All the above points to the need for further research to evaluate the effects of virtual art therapy for different age groups (children, adolescents, with different mental health conditions, etc.) in the short and long term.
For more than 70 years, creative art therapy methods such as visual art therapy (isotherapy), music therapy, dance, drama, and other therapies have been used in the psychotherapy and counseling of people with disabilities of all ages, especially children with ASD. A certified professional (art therapist) who has completed an approved program in a specific specialty in creative art therapy can help develop life skills and promote healthy self-expression in children with autism. As mentioned, over the past few years, autism specialists and researchers in this field have published studies on how art therapy has a positive effect on children with ASD. Research has shown that, in general, art therapy can reduce behavioral problems in children with autism in certain problem areas, including social communication behavior, flexibility, and self-esteem. Indeed, art therapists are finding that regular art therapy sessions can help children with ASD both at school and at home with things like managing their emotions, interacting with peers and family members, and building self-confidence.
Very often, children with autism lack the basic skills needed in the areas of communication, cognition, learning, generalization, and motor coordination, etc. These skills are developed as they grow up, but art therapists must be aware of the difficulties that children with sensory oversensitivity experience. Therefore, it may be helpful for an art therapist to regularly consult with a pediatric physical therapist or an ASD specialist to assess the child's specific sensory issues. VR has several benefits for students with ASD, such as helping overcome excessive shyness, influencing children who are very hyperactive or have heightened emotions, and more. In this case, virtual isotherapy can fulfill the role of real VR meditation.
A Use Case Diagram can be considered a good starting point for discussing key project participants and processes without going into too many implementation details. These UML diagrams are also the most popular type of UML behavior diagram category and are used to analyze the functionality (use cases) and interaction with different types of agents (actors) of the system. So, as you can see in Fig. 1 in the developed system there are 4 actors, i.e. the main entities: • A child with autism who undergoes the process of traditional tactile art therapy followed by integration with virtual art therapy; • An art therapist who forms a list of tools for art therapy, while taking into account the personal capabilities of each child with ASD; • An ASD specialist and makes decisions regarding the use of art therapy tools personally for a child; • VR art applications developer.
An actor A child with ASD has such options of use as: "undergo traditional art therapy", "undergo digital art therapy using virtual reality technologies." In its turn, the "Art therapist" use case forms a list of therapeutic art measures, as it is connected by the <<include>> relation with the "Add to the list with digital art therapy" use case. Art therapy also includes the formation of a set of applications for digital art therapy.
The use case "Decide on the most relevant software for digital art therapy" is performed by the Psychologist, supplemented by an assessment of these applications according to the requirements of Universal Design (UD) and WCAG requirements (WCAG stands for Web Content Accessibility Guidelines, more can be found at https://www.w3.org/WAI/standards-guidelines/wcag/).
The use cases "Propose Digital Art Therapy Software" and "Establish Economic Indicators" belong to the "Software Developer" and include "Create Digital Art Therapy Software Rating", which will be used to analyze the results obtained for a specific child with ASD, as well as to shape the future experience using virtual art tools. It should be considered that "Art therapist" and "Psychologist" are guided by the personal characteristics of the child who participates in art therapy.
To log in to the system, use the option "Undergo traditional art therapy", a set of which is formed by the "Art therapist". Also, this entity takes into account the personal characteristics of a child with ASD and is guided by the recommendations of the psychologist. The <<include>> link combines "Evaluating existing software" and "Forming a set of software applications for digital art therapy." The set of offered software for virtual reality belongs to the entity "Software developer, freelancer".
The entity "Child with ASD" also performs digital art therapy "undergo digital art therapy using virtual (augmented) reality."
All entities actors ("Art therapist", "Psychologist" and "Software developer, freelancer") participate in the formation of virtual art therapy tools.
Fig. 2 shows the sequence diagram. It consists of four system objects: Child with ASD, Art Therapy, Therapist, VR developer. To activate the system, the child with ASD logs into the system, after which the Therapist checks the role and selects art instruments. A child with ASD gets access to these tools, tasks are completed. The Therapist checks and stores the received information. A VR developer creates a set of VR art tools, and adds personalized software applications. After that, the system reports whether new tools have been added. Art Therapy includes a set of art tools that form a VR Therapist and VR developer.
The visual representation of the graph in the form of an activity diagram is presented in Fig. 3, where the initial state is set by the Art Therapist, and transitions to the state of art interaction with a Сhild with ASD. Then the initial parameters of therapeutic art tools are selected, if everything is correctly designed, then the transition to interaction with a freelance developer of virtual reality takes place.
The visual representation of the graph in the form of an activity diagram is presented in Fig. 4, where the initial state is set by the Art Therapist, and transitions to the state of art interaction with a child with ASD. Then the initial parameters of therapeutic art tools are selected, if everything is correctly designed, then the transition to interaction with a freelance developer of virtual reality takes place.
On the recommendations of art therapists and autism specialists, the following stages of art therapy (namely, isotherapy) for a child with ASD were suggested (Fig. 4).
STAGE 1. Usually, at this stage certain preparations take place before the isotherapy itself, namely: actions and procedures related to determining the emotional state of a child with autism, determining his sensory sensitivity, state of anxiety, etc. are carried out. This will give an understanding of how flexible the art therapist can be in choosing artistic art materials and what experiments he can conduct with them. It should be remembered that one of the main goals of art therapy for autistic children is to increase tolerance to unpleasant sensory stimuli, to encourage autistic children to "tolerate" materials with textures or paints with smells that they avoid in everyday life. The goal is to choose such art materials for children that they can use at the initial stage of isotherapy, and over time, at subsequent art sessions, raise the threshold of sensory tolerance.
An interesting point in Stage 1 is the choice of colors for the isotherapy session. In theory, there are many approaches and methods of selecting color combinations. In this work, it is proposed to modify the Dembo-Rubinstein method for evaluating the basic set of colors by a child with autism. The DemboRubinstein method is distinguished among other methods by its universality, which consists in the fact that, in addition to testing children with ASD, it is possible to involve absolutely everyone who participates in the art therapy session or is related to it, for example, the parents of an autistic child.
STAGE 2. This stage is mostly related to the creative process, namely the direct creation of a picture using colored pencils or paints. The art therapist is responsible for the maximum involvement of a child with ASD in the creative process of isotherapy, which requires maximum concentration of attention, concentration, and the creation of an appropriate creative environment.
At the same time, various obstacles and certain deviations may arise. This is the excessive excitement that arises because of a strong fascination with the drawing process and forces the art therapist to show a certain ingenuity to correct these processes. Parallel interaction of an art therapist with an autism specialist, as well as interaction with the parents of an autistic child, is not excluded. As already mentioned, certain problems related to excessive sensory sensitivity of the child may appear. This is caused by the fear of tactile interaction with paints, excessive reaction to the smell of art materials, etc. In this case, the art therapist will try to exclude tactile interaction with paints, choose brushes with a longer handle or replace them with pencils.
STAGE 3. At this stage, the art therapist must perform actions related to a certain distancing of the child with ASD and the drawing created by him. If at the previous stage it was necessary to stimulate a child with autism to be creative, to focus his attention on drawing, then already here the process of viewing the resulting drawing comes to the fore. It is necessary to partially remove the person who participated in isotherapy from the created art to provide an opportunity to perceive his work in a different way, to experience new emotions while viewing this drawing. Such distancing and detachment will allow a child with ASD to experience new emotions and experiences, which will provide an opportunity to get a projection of the current life situation and to recognize the problem of such a child more deeply. Stage 3 should also contain interesting interactions, for example, evaluating the virtual drawing and comparing the obtained results with the real tactile drawing. If a child with ASD avoided pencils and paint due to excessive sensory activity or sensitivity, did virtual art therapy help partially or completely overcome these barriers?
STAGE 4. The fourth stage is to verbalize or materialize in some way what the child with autism is feeling and thinking. The complexity of this stage includes the possibility that an autistic child may be partially or completely non-verbal because according to statistics, approximately 25% of all autistics are non-verbal. It is likely that the child's parents will be involved in the discussion of the picture, as well as an autism specialist, for example, a psychologist. The result of the discussion of the obtained art product is, in fact, a reflection of the inner world of a child with autism, therefore all small details, color accents, and combinations have a special meaning and should be interpreted accordingly by an art therapist and a worker of psychological-medical-pedagogical consultation. At the same time, the discussion should take place in a relaxed manner, without the art therapist imposing his vision or opinions.
At the final stage, the art therapist and autism specialist must consider certain aspects of the assessment: • Color. What does this or that color means for a child with ASD, and what associations appear? By synthesizing basic colors and their combination, you can better understand the creativity of such a child and his experiences; • Number of depicted objects; • The size of the depicted elements of the picture; • The semantic load of these objects (concrete, abstract, etc.); • Psychological interpretation of the symbols of the picture, which helps the artist to understand the deep meaning of the inner experiences of the child; • Integrity and integration of the created picture (united by a common theme, presence-absence of logical connections between the elements of the picture); • Key, main, and secondary objects in the picture (exclusively determined by the child during the discussion, pointing to them); • The presence of unfilled areas of the picture.
In Ukraine, the development of special educational software applications using augmented and virtual reality technologies for people with autism is only in its nascent stage, because many issues related to domestic theoretical aspects of teaching people with autism, as well as material support in inclusive schools, need to be resolved classes In work [23], it is proposed to use VR/AR technologies to study the discipline of "Social and everyday orientation" and also investigated interaction with 3D avatars, as well as immersion in virtual scenes for acquiring social skills [23].
Art therapy sessions were conducted using a standalone Meta Quest 2 virtual reality helmet and Touch Controllers (https://www.meta.com/quest/products/quest-2/). As of today, there are several applications for VR drawing for Quest 2, with the help of which you can learn virtual creativity, because the manipulators provide high accuracy, and the set of art tools is impressive. The table lists the VR applications for Meta Quest 2 and provides a brief comparison of them.
2D drawing
3D drawing + + + + + + + + +
15 20 15 10
The software used in the research is Open Brush and Drawing Desk, which have no alternatives among free software for VR drawing. It should also be noted that this software was not developed for children with ASD and does not take into account all the requirements for the implementation of comfortable virtual drawing and does not provide a personalized art approach. In this regard, the authors of this work are developing virtual reality, which will be oriented towards the inclusive category, in particular for people with autism. A virtual reality scene prototype was developed, a virtual drawing scenario was created, 3D models were selected, and the interaction of virtual content was described. The software is developed using Unreal Engine 5.1 using the Blueprint visual programming language.
We will present the features of the procedure using virtual reality, in which the art therapy technique "Drawing on a free topic" was used. 3 autistic children, one 10-year-old girl and two 12-13-year-old boys, took part in our pilot study. The children were familiarized with the procedure, but it is worth noting that they experienced some alertness and tension when putting on the helmet.
All participants were given the same instruction, “Draw anything you want using available tools. When the picture is ready, give it a name." We will give a brief description and psychological analysis of the obtained art products and a reflection on the peculiarities of the procedure.
Figure 5, which was created by an autistic girl, and depicts a person, was created very quickly. It should be noted that the child learned the instructions well, immersed himself in the process of drawing, and by his non-verbal reactions, it is possible to trace his enthusiasm for the creative process and his concentration on the task. Observing the course of the child's performance of the task, we note that at the initial stage, more interest was aroused in the study of materials and color gamut than in the creation of the picture itself. The drawing itself was created quite quickly. Analyzing its psychological and art therapy component, we can notice the predominance of blue and gray colors, which hypothetically can indicate dominant dysthymic states, melancholy, sadness, anxiety, and at the same time prudence and analyticalness and fixation or even being stuck in these states. Accordingly, the dominance of the cognitive component over the emotional component can be traced both through the color gamut and through the nature of the resulting image (the presence of a clearly defined head as the main element of a person). The absence of eyes also indicates isolation and a desire to isolate oneself from reality, which reflects the personal and social position of an autistic child. However, placing this human figure in the center of the canvas indicates comfort and acceptance of this position.
Note that the child refused to give the name of this picture. In the future, art therapy work with this child should be directed to the description of the drawing by answering the following questions: "Who did you draw?", "What is her name?", "How old is she?" "What is her mood?", "What does she do?", "What does she want?", "How is she like you?"
Figure 6 (12-year-old boy) broadcasts through the color scheme and symbols of the predominance of focus on the inner world. However, it is worth emphasizing the combination of gray and orange colors, which signal, on the one hand, a lack of emotions, depression, and apathy (gray color) and pleasant feelings and experiences, warmth, and the desire to stand out among others (orange). Since the orange color was added during the completion of the drawing, it can be perceived as an internal signal of being filled with positive emotions, which the child was able to chaotically convey precisely through the color.
Approximate questions that should be asked to an art therapist when working with such a child in order to get extensive information about his inner picture of experiences: "What is placed in the center of the picture", "What do you like most about this picture?" "Would you like to change something about it", "Which color do you like best in this picture?" "Would you like to give this drawing to someone?".
Of all the proposed questions, the child answered only the first question, indicating that the picture shows both an eye and a planet, which is how he interprets the central object of the created picture.
Considering the answer to the question about the depicted object in the picture, they (the eye and the planet) are evidence of a deep contemplation of one's own "I" and the scale of the child's inner experiences, which are constantly in a dual struggle of opposites (positive-negative, exalted-depressed).
It is worth noting that the title of this drawing is also missing, as the subject refused to give it.
Analysis of the drawing (Figure 7) of a 13-year-old boy allows us to draw conclusions about a diverse palette of experiences since many colors and objects are observed. Note that the color range varies from black to yellow and orange, which indicates dynamism and inner expressiveness of emotions. Considering the answers to the questions about the main objects, they defined a yellow circle and a black square. This indicates a certain dichotomy, both in emotional and cognitive states, and the presence of contradictions of these components, as parts of the individual's mental reality, trying to dominate each other.
The presence of unrelated elements in the obtained picture is quite expressive, which indicates the lack of integration of structures and parts of the mental reality of an autistic child. After receiving the drawing, the art therapist (as an option with the help of a chatbot) should build a dialogue based on the subject of the drawing, based on the following questions: "What is drawn in blue?", "What is drawn in red?", "What is drawn in black?", "What do you like most about this picture?", What do you dislike most about this picture?", "Which element of this picture would you like to become? Why?"
In virtual drawing, to the detriment of real life, you can cancel the action and start drawing again. The child will feel more comfortable, realizing that if he makes a mistake, he can easily undo it. It is also an ideal option for people who have sensory sensitivity and are afraid of contact with paint or other art materials. The paint will not get on clothes and not pollute the room! No need to clean your brushes: In VR, you can instantly remove all the paint from your brush with the push of a button on your controller. You can also quickly and easily set up your workspace and start drawing. The absolute advantage is the cost of virtual art materials. After all, you can paint as much as you want without buying new brushes, paints, and canvases.
One of the disadvantages of virtual drawing is the lack of tactile feedback, such as applying a brush to the canvas, although developers are trying to solve this problem (vibration and sound effects when drawing). It is difficult to control the pressure of the brush on the canvas. The authors also faced another problem - the level of awareness of virtual reality sessions. Full VR immersion requires prior preparation and it may take a lot of time to prepare a child with ASD for a virtual art therapy session. The helmet can cause discomfort to the controls-tracking takes some getting used to. All these aspects should be taken into account when developing specialized software and conducting virtual art therapy sessions.
During a virtual art therapy session, it is also important to monitor the emotional state of a child with special needs. In recent decades, recognition of emotions based on facial expressions with the help of, for example, webcams, cameras in smartphones, etc., allows to obtain information about the quality of education and understand which corrective techniques are effective in the process of interaction between a child and a teacher [24]. However, such recognition can be complicated in the case of isotherapy in virtual reality mode because the use of a VR helmet prevents recognition of facial expressions and eye tracking. Moreover, there are almost no software tools that would do this in realtime. In this regard, the authors are working on the creation of an artificial intelligence system that will help art therapists, psychologists, and parents monitor the activity phases of a child with special needs during an art therapy session in a virtual reality helmet. This system is designed to solve the task of recognizing the child's emotional state, and ensuring interaction between all participants in the virtual isotherapy process will be implemented with the help of machine learning and a specially developed chatbot, including data analysis methods [25], [26].
Based on the results of isotherapy in children with ASD, we state that the first stage for this category of children turned out to be the longest and most exciting (according to the subjective assessment of the art therapist according to specific criteria). The child experiences interest and joy, which he demonstrates through the selection of tools and colors and their approbation on the virtual canvas.
During Stage 2, children are usually immersed in a creative process that is not accompanied by reactions, but is associated with complete immersion, which is evidence of the "inner workings of mental processes."
At Stage 3, which is related to distancing, children begin to distance themselves from the created picture and perceive it as not their own. At this stage, the first difficulties arise, where specialists must be actively involved (art therapist, PMPC employee, etc.). It is the projection of the inner world onto the picture and interaction with it that helps to establish contact with oneself and the outside world.
The fourth stage was the most problematic, which fully reflected the difficulties of children with ASD, which are related to verbalizing their own emotions, states, and experiences. All the children refused, very briefly, and reluctantly answered the questions proposed by the art therapist.
Therefore, at this stage, it is advisable to intensify work with interaction with a chatbot, which will help the child to describe "repressed" experiences in a comfortable way.