=Paper=
{{Paper
|id=Vol-2590/short10
|storemode=property
|title=Creation of a Hardware-Software Complex for the Transmission of Odor in Virtual Reality Environments
|pdfUrl=https://ceur-ws.org/Vol-2590/short10.pdf
|volume=Vol-2590
|authors=Artem Smolin,Konstantin Malyshev,Anna Spiridonova,Andrey Mironov,Yuliya Konstantinova
|dblpUrl=https://dblp.org/rec/conf/micsecs/SmolinMSMK19
}}
==Creation of a Hardware-Software Complex for the Transmission of Odor in Virtual Reality Environments==
https://ceur-ws.org/Vol-2590/short10.pdf
Creation of a Hardware-Software Complex for the
Transmission of Odor in Virtual Reality Environments*
Artem Smolin1[0000-0001-7242-6471], Konstantin Malyshev2[0000-0002-8455-5876], Anna Spiri-
donova3[0000-0003-0724-0405], Andrey Mironov4[0000-0001-5336-8800], Julia Konstantinova5[0000-
0001-8957-3566]
1,2,3,4,5 ITMO University, Saint-Petersburg, Russia
smolin@itmo.ru, malyshevko@itmo.ru, spiranna@list.ru,
mr_blum@mail.ru, konstantinova_yuliya@inbox.ru
Abstract. One of the main tasks to enhance perception and deeper immersion in
virtual reality environments is the use of different sensory channels. This article
discusses the creation of a prototype device for the reproduction of smells, which
is used in conjunction with virtual reality headsets. This device will allow the
user to immerse in the virtual space more fully.
Keywords: Virtual reality, Sensor channels, hardware and Software complex.
1 Introduction
During the evolution mammals obtained the ability of the nervous system for multi-
modal perception to combine different inputs of the sensory system, such as light,
sound, smell and taste. This ability led to the enhanced detection or identification of a
particular stimulus.
A reflex caused by the stimulus is a stereotypical reaction of the mammalian organ-
ism, to something that prompted this reflex. Based on this theory, it can be assumed
that we can increase the degree of user’s involvement in the virtual reality environ-
ments, by increasing the number of sensory inputs to enhance the stimulus.
The sphere related to the degree of immersion of users in the virtual reality is now
actively developing, a variety of technological devices are being created, ranging from
various controllers to complex systems with climate change. All this allows to increase
the degree of involvement of users into the virtual reality.
Based on the above, within the framework of this work, it was decided to create a
prototype of a smell reproduction device that can be connected to various virtual reality
headsets and integrated into new or existing applications.
* Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons Li-
cense Attribution 4.0 International (CC BY 4.0).
2
2 The Use of different sensory channels in virtual reality
environments
At the moment, various technological prototypes are being created. They use differ-ent
sensor channels in both unique and combined formats [1]. For example, experiments
are being conducted on the effects on the taste buds of the person.
The Nourished project is a VR simulator that can help people with allergies to per-
ceive the full sensory experience of eating (recreating the full sensory experience of
eating, without actual food). The main components of the development are: Oculus vir-
tual reality headset, food detection devices, motion sensor and aroma diffusers (see Fig.
1).
The system creates visual effects using virtual reality, as well as simulates sounds
during meal. In this case, the user eats healthy food in the form of cubes printed on a
3D printer (algae, flour from shiitake mushrooms, etc.) [2].
Fig. 1. Project Nourished.
Experiments on the symbiosis of smell and other ways of influencing perception be-
gan in the 1960s, when watching a film was followed by special cards that a person
could scrub and feel a certain smell.
One of the developed commercial prototypes of the device for transmitting smells is
multisensory VR mask Feelreal (see Fig. 2) [3]. The mask is attached to the viewer’s
face and to one of the virtual reality headsets, and then a scent generator with seven
removable cartridges releases odors into the nose area.
3
Feelreal SDK is attached to this method, which will allow developers to add various
individual scents to their projects. In addition, using the special Feel Real Player, it is
possible to use different flavors while watching movies [1].
Fig. 2. Multisensory VR mask Feelreal.
3 Creating a prototype device for transmitting smells in virtual
reality environments
For the first version of the prototype, experiments were conducted, on the basis of
which we received data on changes in interest and involvement in the interaction pro-
cess after the inclusion of another sensory channel.
Before creating a prototype of this device, research was carried out, which resulted
in a method that meets the following criteria:
The number of odors available for use in the system;
Control the strength and saturation of the smell;
Possibility of introducing additional solutions that affect the ability to perceive
odors;
Possibility of combining odors to obtain a complex aroma;
Ability to easily integrate the game engine and prototype.
The research work was divided into three stages. At the first stage, methods for re-
producing odors were investigated to enable the device user to fully evaluate it, and
the following methods for the technical implementation of the prototype were also
investigated and thought out:
Use of high-pressure cylinders;
Spraying scents and odors in a natural way;
Storing the smell in its natural form or using an analog (flavoring agent) and spraying
with an odor injection system.
4
As a result, the third method was chosen, in which the spray was carried out using an
odor injection system.
The second stage of the study was the search for a technical solution to control the
previously selected method of implementing odor reproduction.
First of all, methods for controlling the injection of odors and regulating the amount
of odors were investigated. As a result of this work, a large number of control methods
were developed and the option of using a water pump and valves with a tap controlled
by motors without gearboxes and tubes used in home aquariums was selected. After
working on the next version of the prototype, a different method was chosen, due to the
low reliability of the first one.
The following technical solutions for the control of valve and pump were also inves-
tigated:
Development of unique management Board;
Using a ready-made Arduino Board and writing unique firmware;
Using a ready-made solution, such as injecting water into the engine.
As a result, the second method was chosen: using a ready-made Arduino-based Board
and a raspberry Pi mini-computer as a web server to receive commands and execute
them on the device.
Fig.3 shows the final layout of the prototype, without interaction with the mini-com-
puter.
For this device the microcontroller Arduino Nano was chosen because of its simplic-
ity and capabilities, as well as due to compatibility with other more powerful counter-
parts. In addition, thanks to this microcontroller, it is possible to expand the function-
ality without high costs. The control device was built on the ARM architecture, with
the possibility of wireless connection methods.
As a result of the undertaken work, we can highlight the following components,
which should have interaction with the system and with each other:
Control device;
Microcontroller;
Engines;
Pump.
On the Fig. 4 there is demonstrated the third iteration of the scheme, in which the chan-
nels of exchange commands (black lines), power (red dotted lines) and the following
components of the system are indicated:
The motor driver;
Pump driver;
Power supply;
Voltage converter;
Microcontroller;
Control device;
Pump;
5
Engine;
Valve.
Fig. 3. First prototype diagram.
Fig. 4. Device diagram.
6
The final version of the device diagram, which is shown in Fig. 5 represents a complete
diagram of the interaction of the device components and external sources.
Fig. 5. Component interaction scheme device and external sources.
4 Preparing the created prototype for operation
To prepare the created prototype for operation, the following tasks were required:
PC setup;
Configuration of the control device;
Calibration of devices.
To control and configure the controller and the control device from a personal com-
puter, it was necessary to have the PuTTy program, which allowed connecting to the
device via the Serial port and SSH. Also, to connect to the Serial port, a special con-
verter was required, which was implemented as a USB device.
When using Unix-like systems with a kernel version higher than 4.4, the driver for
our controller was embedded in the kernel. To flash the Arduino controller, it was nec-
essary to install the Arduino IDE software package for the required version of the op-
erating system. 30 4.3.1.2
Since the control device is available without a pre-installed OS, we needed to use
the Orange Pi portal [4] and find a suitable version of the operating system for us.
In this case, it was Ubuntu Server, after using which we recorded the image on a
MicroSD memory card using the Win32 Diskimager utility, which is distributed under
the OpenSource license. After recording to the microSD card, it was required to connect
it to the device via the SERIAL port, for this purpose, using connectors we joint the
Converter in USB and outputs for the UART.
7
In the process of connecting at a speed of 115200 b / s, process of the first start
appeared. Since the terms of reference already specified which language and libraries
should be used to function the program code on the control device, the required pack-
ages were installed:
NodeJS;
Git;
Python 3;
GCC;
Make.
The last three points are required for successful Assembly of packages used in the Node
Js project. The Serial package was built using the command: npm install -g serial.
As a result of the first launch of the device, it was clear that the standard package
manager that comes with NodeJS was not enough. It was decided to use the third-party
package manager Yarn, which is installed using the command: npm install -g yarn.
After successful installation, you it is possible to launch it in the folder with the
already created project using the command: yarn install.
For the voltage Converter to work, its initial calibration was required. For this pur-
pose, a multimeter was used, which was switched to the voltage measurement mode.
The input contacts were supplied with a voltage of 12V, the output was connected with
the contacts of multimeter, and then the device was calibrated until the necessary 5V
was obtained on the multimeter.
5 Final testing
To test the developed prototype, there was chosen a VR location created by Borrowed
Light Studios. It represents a journey into Vincent van Gogh's the Night Cafe. [5] To
enhance the effect of immersion, the smells of coffee and brandy were used, those
drinks were typical in the cafes of van Gogh's epoch (see Fig. 6).
Further prospects for the modification of the project are to change the way the air
supply is controlled; instead of engines and valves, it is supposed to use an electromag-
netic nozzle, which will allow for a more accurate supply of odor and increase the speed
of response to a request, since the reaction time to the task will be minimal.
The new version of the prototype is supposed to be connected directly to the Oculus
Quest helmet, using Bluetooth technology, thus abandoning the Pro-jutting device in
the form of a web server.
8
Fig. 6. Prototype of the device with odor reproduction.
6 Conclusions
Virtual reality technologies are actively used in society in all aspects of its existence:
education, commerce, entertainment, etc.
Nevertheless, there is a number of unsolved problems, one of which is the creation
of a software and hardware complex for immersion in virtual reality environments with
the full use of touch channels.
As a result of the undertaken work, a prototype of a smell reproduction device for
virtual reality environments was developed. The development of this device included
the following stages:
Selection of technology of reproduction of smells;
Technical development of the device;
Selection of test applications;
Selection and preparation of smells.
References
1. Papagiannis H. Augmented reality. Everything you wanted to know about future
technology. – Moscow: Eksmo, 2019. – P. 123
9
2. Project Nourished, http://www.projectnourished.com/
3. Multisensory VR mask Feelreal, https://feelreal.com/
4. Orange Pi Portal, http://www.orangepi.org/downloadresources/
5. The Night Cafe - An Immersive VR Tribute to Vincent van Gogh,
6. https://www.borrowedlightvr.com/the-night-cafe/