For a modern person, the loss of any limb or organ is a big problem. Not everyone can measure it and try to remove these restrictions. Thanks to modern means and technologies, it becomes possible not to be a person with disabilities, but to purchase augmented ones. Already, bionic prostheses of the upper extremities use myoelectric potentials for control, have wireless data transfer technologies and their functionality will soon be inferior to modern smartphones. As a result, hand prostheses begin to constitute a powerful tool for improving the functional capabilities of human limbs, thereby improving the quality of the user. The paper presents the goals and objectives set during the development of a hardware-software system for an automated reprogrammed hand prosthesis for people with disabilities, as well as the rationale for its implementation. Parts of the developed system, technical characteristics and software and algorithmic solutions used in the development are described. In conjunction with direct performing motor mechanisms and a full-scale model of a bionic prosthesis, the development is intended for use in the eld of medical rehabilitation of the upper limbs. The system under development will allow you to customize gestures implemented by the prosthesis through an application on a personal computer. Direct control over the prosthesis can be carried out both on the basis of bioelectric potentials and on the basis of additional human-machine interfaces, in particular, voice and graphic control interfaces in a mobile application. Through the use of additional interfaces, it will be easier for a person to go through the initial rehabilitation period in which he needs active muscle training to use myoelectric sensors. Also, the system provides the opportunity to use a bionic prosthesis for those people whose nerve endings were damaged and are no longer able to conduct neural impulses. In the developed system, it was possible to con gure up to 1000 gestures, which the user can independently create and modify. The system was tested on a patient with an amputated left upper limb and was noted by him and rehabilitologists as promising.
Unfortunately, in modern society the percentage of people with disabilities is
constantly growing [
Over the past decades, advances in the development of 3D printing, light and
powerful engines, capacious batteries, as well as the growth of the capabilities of
electronic computing devices have created the conditions for the appearance of
prostheses that are as close as possible in their functions to human limbs to such
an extent that they can be replaced by cosmetic and traction types of prostheses
[
All today's companies developing bionic prostheses are focused on two areas
of development [
The most interesting area of study is the management system, since work in
its eld has just begun and it has a long way to go to become more convenient
to use. The main problems of the control system of modern bionic prostheses
are [
The prosthesis being developed will reduce the period of post-traumatic rehabilitation of a person and will make it possible for those people who are not available to use myoelectric sensors. 2
The aim of this work is the development, research and testing of the hardwaresoftware system of an automated reprogrammable bionic prosthesis of the hand, with modular control. The prosthesis can be controlled both on the basis of bioelectric potentials and on the basis of additional interfaces. Through the use of additional interfaces, it will be easier for a person to go through the initial rehabilitation period in which he needs active muscle training to use myoelectric sensors. Also, the system will provide the opportunity to use a bionic prosthesis for those people whose nerve endings were damaged and are no longer able to conduct neural impulses, which are required for using myoelectric sensors.
To control the bionic prosthesis, it will be possible to use controls based on the graphical user interface, voice control, as well as based on bioelectric control using myoelectric sensors. The presented interfaces will convert the recognized prosthesis commands into control signals for the movers to form gestures for the prosthetic hand. In addition to providing the basic functions of the prosthesis, such as squeezing / unclenching individual ngers and the arm as a whole, the modular control approach will allow the user to independently con gure and add new functionality for personal use by creating or editing gestures implemented by the prosthesis. 3
Scienti c novelty of the solutions proposed in the draft 1. The proposed use of natural language recognition for the control system of the prosthetic arm has no analogues in the areas of rehabilitation and prosthetics of the upper limbs of a person according to the analysis of open sources. 2. The developed system for storing, transmitting and performing prosthetic gestures allows us to abstract from the implemented system of motor mechanisms and control devices and to create new ones or change old gestures. 3. A modular control system based on the interaction of myoelectric sensors, voice and graphic control, will signi cantly expand the functionality of the system. 4
In process rehabilitation, during the management of the prosthesis, the patient must think what kind of simple movement he would like to make with his phantom arm, for example, bend his arm at the elbow, squeeze his hand into a st, etc. Myoelectric sensors read signals from spinal motor neurons and convert them into prosthetic commands. The problem with prostheses is that their sensors are attached to the remains of the muscles at the patient's amputated limb. Twitching of these muscles set the prosthesis in motion. But it often happened that the muscles were damaged too much, due to which the movements of the prosthesis were very limited. About 50 percent of patients simply refused the prosthesis due to its incorrect work. That is why many experts have long been trying to nd an alternative approach to solving this problem.
The system of automated reprogrammable control of the prosthetic arm is a system based on voice and combined (voice-myoelectric) control. No more trying to recreate or trying to simulate the movements of a phantom hand. Instead, you can pronounce a number of simple phrases and not necessarily in a strictly speci ed form or case, as well as realize even more complex movements as a result of a combined command. 5
The structure of the hardware-software system of the prosthetic hand can be divided into three main parts (see Fig. 1): 1. The full-scale model is a complex consisting of a model of the forearm from the point of view of ergonomics, as close as possible to the appearance of a real forearms and sensors and actuators integrated into it devices; 2. The microcontroller part of the control system provides control of the mechanical e ect of the drives on the prototype part of the prosthesis, taking into account the given command, as well as reading / transmitting data between the sensors and the controller; 3. The software part of the control system is the interface of the system's interaction with the user and provides the ability to control the prosthetic hand, create new actions performed by the prosthesis, and change the current ones. The hand is a complex system and is of particular importance to humans, and the anatomy of the hand is characterized by the presence of small bones and articulating joints of various types. As a result of the analysis of the anatomical features of the arm, the minimum necessary group of muscles, joints and tendons was selected that were su cient to perform basic movements: 1. Muscles - extensor and super cial exor of the ngers, dorsal interosseous, vermiform; 2. Joints - distally and proximal interphalangeal, metacarpophalangeal; 3. Tendons - super cial and deep.
Based on the assumptions made, a graphic 3D model of the hand prosthesis
was developed in Google SketchUp 2018 and Autodesk 3ds Max 3D modeling
environments. When considering the horizontal section of the prosthesis (see
Fig. 3), you can see that the model provides channels that simulate the work
of the deep exor tendons of the nger exors (A). For the supporting part of
the model are the elements of the articulation of the phalanges of the ngers
between themselves, which mimic the distal (D), proximal interphalangeal (C) and
metacarpophalangeal (B) joints. A group of servos connected to the phalanges
of the ngers by small cables that mimic the functioning of the above muscles
and tendons is responsible for the model [
The STM32F407VET6 microcontroller [
Gestures are stored on the SD card located in the prosthesis. Work with the sd card le system is carried out through the FatFs library; 4. Management of actuators of the full-scale model. In particular, the display of information for the user, engine driver control via CAN bus;
User interaction with prostheses is carried out through the use of two applications. The rst is the HandControl desktop application, which provides the user with the ability to reprogram the actions performed by the prosthetic arm. The second is an android application that acts as a control device for the prosthetic arm. It contains the choice of the prosthesis operating mode (combined, direct or myoelectric control mode), sending a request for the execution of gestures, as well as the implementation of voice control.
The desktop application is developed in C# using following means: 1. Technology WPF (Windows Presentation Foundation), which is a replacement for WinForms technology for developing graphical applications in C#. The technology is based on the use of DirectX when rendering graphics in an application and using XAML to describe application graphics.In addition, this technology allows you to most e ectively implement the MVVM programming pattern, which was used in this application; 2. Framework Material Design for styling and extension functionality of the basic elements of WPF; 3. The Json.NET Framework to enable conversion of .Net objects to their JSON equivalent and vice versa by matching the names of the properties of the .NET object with the names of the JSON properties and copying them; 4. Framework Fody, which implements PropertyChanged events in classes that implement the INotifyPropertyChanged interface. 6
The system under development expands the capabilities of existing prosthetic analogues by integrating voice control and makes it possible to abstract the development of the prosthesis model from the control system. In addition, users of prostheses will be able to pass the adaptation period to the bionic prosthesis much faster at the initial stages due to the combined control system, when they can refuse myoelectric control.