Currently, modern technologies are rapidly developing and are in demand among society. It is possible to notice a variety of sensors that allow performing actions automatically, software for performing a number of tasks [1,2].
It has been found that a certain part of the society cannot use most of the applications due to the respective defects. Software or application developers pay little attention to maintaining or developing projects that could solve common problems.
Therefore, it was decided to develop a model that would allow solving most problems in the use of information technologies at the level of the operating system with the help of a single software tool.
Using voice assistants is becoming a normal everyday thing. Various companies are implementing voice assistants in their applications. The reason for this is to simplify the use of the product due to additional technologies. In [3], the principle of using ready-made solutions in voice recognition is considered. In addition to voice assistants, gesture management is rapidly developing. Currently, companies are implementing such solutions in their cars in order to provide their customers with a certain convenience in using their product.
The work [4] shows an example of the application of gesture control, which is adapted to another system.
At the moment, there are various sensors and software that allow solving a number of people's problems. But the search for software that would allow solving specific problems in the use of different applications from different manufacturers was not found. Therefore, it was decided to develop a model that can solve different problems of different users using voice and gestures. Thanks to ready-made solutions in a certain programming language, as well as their combination, you can create a universal and unique solution that can be in demand -this is a feature of this model.
Different products are adapted for different tasks [5,6]. But no universal one was noticed. Thanks to the use of innovative technologies, it is possible to create a software application that could solve a number of problems without additional costs from the user.
Different programming languages have similar data libraries. Thanks to this, you can solve similar problems in different programming languages.
Computer vision is in demand among the famous. Its application has various directions: automation of actions, tracking of objects, processing of input data, etc. [7,8].
As an example, the Python programming language and the OpenCV data library are used. This library allows you to process both streaming videos and images [9].
Another off-the-shelf solution that is needed is voice recognition. Thanks to voice recognition, it is possible to adapt certain actions to the application [10]. There are several libraries that allow you to implement voice recognition. Among the well-known are Pocket Sphinx and Speech Recognition [11,12]. Pocket Sphinx allows you to create a model that will recognize only those words that are listed in the dictionary [13]. Speech Recognition is a model that is trained using a set of specified words. The set of words that includes Speech Recognition can be enough for most tasks.
The project contains various directories and executable files, as well as additional configuration files for saving user settings.
The structure of the project is divided in such a way that it would be easy to navigate through it. There are SOLID design principles, where the first principle is applied to the project structure (Single Responsibility Principle), which indicates the division of responsibility between components [14].
Figure 1 shows a diagram of the application components.
Libraries. Two libraries are used -OpenCV and Speech Recognition. The diagram shows the connections between the libraries and other system components.
Voice recognition. This directory contains a basic set of functions and classes that will allow recognition and processing of input data, where the specified user action can be performed as a result. Since the set of necessary words can be large, the commands themselves are placed in a separate file, where the system can be flexibly scaled in the future.
Gesture recognition. A directory that contains the necessary file that will help recognize gestures. Component interfaces. This directory contains the interfaces of all project classes.
The precondition for creating interfaces is the second principle of SOLIDclasses are open to extension, but closed to modifications (Open-Closed Principle).
Main and configuration files. The main file (main.py) contains a set of functions that work compositionally with speech and voice recognition modules. To save the user configuration, a separate file (.conf) is created.
At the beginning of working with the program, the user has the opportunity to make some settings or directly start managing actions automatically (Fig. 2). The setup includes several stages: determining the position of the eyes, nose, and main voice commands. When the setup process is complete, the data will be saved to a separate file. The setting is required in order to reduce the possibility of incorrect processing of user commands. The program can run until the "stop word" is triggered. The voice recognition module will wait for the corresponding word when it needs to pause the program execution process. Otherwise, all commands will be processed.
Since the application runs in a single thread, only one of the action management methods can be executed at a time. In this case, you need to use 2 separate streams for gesture control and voice control. Thanks to this, the user can perform actions faster and more conveniently.
Voice assistants and gesture control are implemented in various products or systems of companies. Having analyzed the state of the problem of the use of existing applications in society, it was found necessary to create some automation to solve most of them.
This work reflects the principle of project construction, which allows you to manage actions with the help of various existing means. Thanks to the combination of several components, a new unique and universal goal can be achieved.
For further scaling of the project, SOLID design principles were used, thanks to which the application is built in such a way that the structure of any module can be changed without problems.