The process of inclusion of a person with special needs into society can be improved with different information technologies. For people with hearing impairments, it should be based on speech-to-text converter. The main functions of such an application should allow the user to see the text on the screen of the device. To create such an application, its main functions were chosen based on the analyzed analogs, so the designed application met all the necessary demands. The UML diagrams were chosen to model the designed application, called CLON. The software was tested with different single words, phrases, and sentences. Speech-to-text, hearing impairment, risk low, speech recognition, quick answer, convert The effective development of society can be determined by how government measures are aimed at human well-being. This development can be considered at a high level if the state directs its forces to help the least protected and most vulnerable members of society. Issues of social assistance and protection of the rights of persons with disabilities are considered relevant throughout the civilized world. Having hearing impairments causes troubles for not only persons with such disability, but also for their families, friends, all the other members of society, and creating a mean to improve such communication, will be of greater help to the society of hearing people. Taking into account the level of the electronic devices spread, such helpful mean should be designed and developed as an information technology. The object of research is the process of communication between people with hearing impairments. The subject of the research is a development of an application as a mean of interaction between people. The practical value of the results is to develop and test new tools and methods for converting speech into text, saving human resources and money, and building convenient, useful, and important for people with hearing impairments software and systems. The data that will be extracted and directed to the development of such a science as speech recognition, other systems that use the human voice to convert to text will be very useful and will improve these systems.
speech, voice message
Ukraine
2021 Copyright for this paper by its authors. with the speech-to-text keyword, the Scopus database as a result gives 953 documents, 470 of those papers were published in 2017-2020. The word cloud from the abstracts of these documents is in fig. 1 (numbers and common words were removed).
The results of the implementation of speech-to-text methodology can be used in various fields, such
as advertisment, automatic keywords extraction, education, linguistics, military, data analysis, etc.
[212]. The peculiarities of translation of spoken language into sign language were investigated in [
The speech-to-text method was used to assess the risk of Alzheimer’s disease when patients` texts
were analyzed to find special markers [
In [
The effects of assistive technology for students with severe disabilities (reading and writing) were
investigated in [
The peculiarities of stemming as a part of non-formal conversation converting into the text were
investigated by authors who concluded, that the non-formal sentences should be normalized to formal
ones to be used for text classification [
Let us analyze the capabilities of the developed software product in comparison with modern analogs. Programs similar to those present on the market are products of foreign production: RogerVoice, Live Transcribe, App MyEar, AVA, Voxsci. 2.1
RogerVoice (https://rogervoice.com/) is the world's first mobile application for creating subtitles for telephone conversations, ie converts mobile voice calls into a much more accessible text format. Developed by deaf engineer Olivier Jeannel. The application uses speech recognition technology to convert voice to text, so deaf and deaf people can read what the other person is telling them.
The application was created thanks to a successful campaign on Kickstarter, a beta version of the application is currently available, which can be downloaded from the RogerVoice website (Fig. 2a).
The application reads human language and translates it into text. To do this, the application uses a smartphone microphone and Google's language API (Fig. 2b), which supports more than 70 languages. To ensure that the transcription is as error-free as possible, machine learning technologies are responsible for recognition. But Live Transcribe can not only decrypt the voice, but also notify the user that someone wants to talk to him, and also allows you to participate directly in the dialogue, giving answers using the built-in keyboard. The spoken text is perceived by the phone's microphone and delivered to the screen of the Android phone via Wi-Fi or other network connection. This can be useful for people who do not hear and attend conferences or lectures, for example. Spoken words will appear on the phone of the person who has the applications.
The MyEar app was developed by Gerald Isobe, a deaf golfer and his son Brandon.
Gerald grew up reading his lips, but was disappointed to understand only 30% of what he said, and tired of asking people around him, "What are you talking about?"
This program was created based on these frustrations, and it is a program that he and others use to communicate with their hearing colleagues, friends, and family.
Direct pricing. It costs a one-time $ 9.99 with free updates when we launch a new software update.
For emergencies. If a police officer stopped you, would you be able to effectively understand what he was saying, for example, at night? Most likely not. Here the MyEar application can help. It will record what the officer is saying so that you do not guess or try to understand what the officer is saying at night and at any other time. Last minutes of a meeting or presentation, and no time to call an interpreter. Sometimes last-minute meetings can happen and then just open the MyEar program and you can use it for long presentations and you will understand what it is about.
Influence on new words and topics. TV subtitles and ASL translators do not translate 100% word for word what people say on TV or on the phone. However, you may not realize this until you try the MyEar app. The MyEar app will translate every word a person says to you.
AVA
Lip reading can be more difficult in a group of people, and this is one of the main reasons for creating AVA. If a person who is deaf or suffers from hearing problems is with a group of friends, he or she can force those friends to join the program - then the person (s) with hearing impairments will see the live broadcast of the group conversation. The language is perceived by the phone's microphone, and the name of the speaker is displayed on the screen before the person speaks.
AVA works with employers, teachers, event organizers and other accessibility professionals who seek to fully engage their deaf and hard of hearing members.
How can Ava fit into your daily life? See how users use Ava → Ava gives you a whole new level of autonomy for many situations in your daily life. 2.5
Voxsci (https://www.voxsci.com/) is a language-to-text program that converts voicemail messages into texts and emails that you can store, search and share. Expenses start at £ 5 a month for 30 voicemails or emails.
Listening to voice messages can be very inconvenient. VoxSciences provides a paradigm shift by transcribing voice messages into text messages. This gives voice messages a quantum leap for joining email, SMS, and instant messaging on the same basis with all the inherent benefits such as texture search. Our VERBS engine (virtual engine for basic language recognition) converts voice messages into text messages and delivers them as e-mail, SMS, or via the API. Voicemail to Text (SMS) is ideal for personal or corporate voicemail systems.
Voice messages, transcribed and delivered by e-mail, are mainly used by call centers, comment or contest lines, and corporate voicemail systems.
The Client's Voice is a market research technique that provides a detailed set of needs and customer needs. It includes analysis of feedback from various sources, such as e-mail surveys, the Internet, and IVR. VoxSciences provides a key component to facilitate the analysis of audio feedback for analysis, providing transcription of transcripts almost in real-time in the business class through our API.
Therefore, we can conclude that the availability of different operating systems does not depend on the price of the application. Therefore, applications that are free offer the use of the application on different operating systems. From the ratings, we can conclude that the applications are not difficult to use, and they are popular among users, and often used. Applications were evaluated according to reviews on official websites, Play Market, App Store (Fig.6 presents the feedback about MyEar, with an overall rating 4.1 and positive text reviews; Fig.7 presents a review of Live Transcribe. Reviews are mostly positive, often mentioned the usefulness of the application). In general, there are no negative reviews, there are simply lower ratings, where people are dissatisfied with the quality of translation from language to text, dissatisfied with the pricing policy, or have additional wishes, such as quick answers. Therefore, it can be concluded that the complexity of development depends on the technologies used to develop the application and whether the application uses a network connection.
Thus, we can conclude that all analogs are competitive, have many advantages, but also have disadvantages. The disadvantages that emerged during the comparison of analogs are: Difficulty of use - complex interface.
VoxSci All OSs 5 2 3 3 High fee for the application - people like the free application more. A small number of settings - users would like to be able to give short answers, adjust call filters. Some operating systems are not supported. Complexity of development - many companies create their own technologies for language recognition, which makes the application more difficult to use and more expensive.
Compared to analogs, the developed application will have many advantages and will include and correct those disadvantages that have analogs. The main advantage of the application will be that it will allow you to configure call filters, the first time will not charge for the use of services, and will allow you to provide quick answers. 3
Basic Success Scenario: This section of the specification describes a "success scenario". That is, actions that lead to the successful completion of events in the main process, for example: The user installs the CLON application. The user logs in / registers (or skips this step) in the application. The user gets acquainted with the settings and interface of the application. The user selects a live broadcast recording point. The application shows the user the text that has just been spoken. The application analyzes the text and offers quick answers. The user selects one of the suggested answers or selects a typing item independently.
Baseline extensions or alternate streams: This section indicates all other possible scenarios that lead to the successful completion of the main scenario or alternative scenarios that lead to incorrect completion of the precedent. In this case, after processing all possible extensions of the precedent, the aircraft must ensure the return of the user to the main scenario, if the aircraft does not provide an alternative course of events. For example, baseline extension: The user has logged in to the application. The user makes settings for the application - language and color settings.
The user makes conversation filters.
operations: public: grant access to application functions, provide access to filters, provide access to application settings.
Add class objects from the class chart with the following names: System User, Application, Convert Speech to Text, User Interface, Use Microphone.
Add links that connect class objects to names: System User, Application, Convert Speech to Text, User Interface, Use Microphone. Add Message 1 - 14: Log in / register in the system. Configure the application. Exit the application. Convert recorded speech into text, provide access to a microphone. Record speech.
The development of the activity diagram shows how the program will work at the stage of speech perception from the speaker. The activity diagram assumes that after the user selects a text item, the system will record the speech in the device memory. After this step, the program uses API, which was specially developed for text recognition. API performs speech recognition. Then the program will create text based on the speech and pass it to the program interface. Then the user will be able to see the text.
The sequence diagram is constructed from the classes of the class diagram in chronological order.
The deployment diagram, that visualizes the hardware and software of the designed system, is in fig. 14. The nodes are CLONE application, API and text analysis Servers, and a connector between server and host.
Type of application (mockap, prototype). The application provides login or registration so that the user's settings are saved, his statistics are processed and improvements are provided.
If the user is not yet registered in the system, one can register.
You can also sign in with a social network, such as Twitter or Facebook.
The following prototype shows how the user will interact with the person. When a person says something, it will be automatically written as text in the user's application.
The following mockap shows how a person will interact with the application. To get the text from the person speaking, the user needs to click on the button to start writing the text. The user can also go back, select options from the radio button, such as add text yourself, use quick answers.
Next, the program must analyze the text so that the user can provide a quick response from the proposed options, which will be voiced. If the user does not find the right answer, he can type the text and it will be sounded through the phone speaker. The first version will only receive speech from the user and translate it into text.
The first example will look like this. The user will be asked to say something.
After that, there will be a pause of 3 seconds after which the speech will be interrupted. After that, the user of the application will see the text that was said by another person.
Text to be spoken by the user: Hello it's my first program speech recognition app.
You can see that the recognition was almost accurate. One word did not match - program - CLON interpreted it as a problem. Other results are satisfactory.
The following versions are needed in order to improve the results and implement all these features for the mobile application.
Testing will follow these steps.
1. Single words recognition. At this stage, a certain vocabulary of some words will be checked to see if there are any difficulties with a certain category of words, or whether the recognition error does not depend on the category of the word.
2. Phrases recognition. At this stage, certain phrases consisting of two or more words will be checked. Here you can see how recognition occurs when more words flow.
3. Sentence recognition. Check whole sentences. Check more than two sentences. 6.1
The first check is a check of certain words, pronounce for example. Fig. 20 shows the results for the personal pronoun I recognition.
The result (bye) is not satisfactory. The next pronoun is you.
The result is just as disappointing. The recognized word is yo. The next pronoun is he.
As the result, the application did not recognize the speech. The next pronoun is she.
The result is also not successful. Instead of the pronoun, skip is recognized. Now you can try to recognize some other words, much longer. For example, you can start with greetings. The first greeting is hello.
Then other words were checked, both complex and easy to pronounce and to perceive.
The first word is kernel.
Then the word is linguistics.
Thus, we can conclude that the more words, the better the recognition. The text is self-correcting and it is possible that the artificial intelligence itself can correct or add the right words.
The next sentence is How tall are you.
The third sentence is Computational linguistics is the scientific and engineering discipline concerned with understanding written and spoken language from a computational perspective, and building artifacts that usefully process and produce language, either in bulk or in a dialogue setting.
Many language recognition problems are constantly being studied, improved, and understood. One such problem is modeling the process of translating speech into text. Translating speech into text is one of the problems that the science of speech recognition is trying to solve.
Speech recognition is the process of converting a speech signal into a text stream. This technology is used to solve such tasks as computer control, voice information, dictation, phonogram transcription. This technology is also used for speech recognition in Google translators, as well as in applications that help people with hearing impairments to communicate.
In the last few years, the demand for communication applications between people has been growing, because it is a fast way to communicate. Modern messengers are used for messaging, but their idea prompted people around the world to create a similar application for people with hearing impairments. This is one of the first and defining steps to solving the global problem because people with disabilities live among us and also want to communicate. Not everyone understands sign language and not everyone can immediately understand that a person has hearing problems, so this application is very necessary.
The system developed a system that recognizes the speaker's text and translates speech into text yes, people with hearing impairments will be able to "hear" their interlocutor and understand him if he does not know sign language. Therefore, the developed application can and should be improved by expanding its functionality for further use in commercial companies seeking to help people with hearing impairments, as well as those entrepreneurs who intend to monetize such applications to increase their profits. You can provide the following options: adding filters for speech perception - that is, the conversion of speech into text will be more accurate if you specify the topic of conversation; integration with modern platforms; adding advertising banners of interactive format, namely mini-games based on the canvas model of the user's Internet browser. Sciences and Information Technologies, CSIT, 2016, pp. 190-192. DOI: 10.1109/STCCSIT.2016.7589903 44. V. Vysotska, Linguistic Analysis of Textual Commercial Content for Information Resources Processing, in: Modern Problems of Radio Engineering, Telecommunications and Computer Science, TCSET, 2016, pp. 709-713. DOI: 10.1109/TCSET.2016.7452160 45. V. Vysotska, V. Lytvyn, Y. Burov, P. Berezin, M. Emmerich, V. B. Fernandes, Development of Information System for Textual Content Categorizing Based on Ontology, volume Vol-2362 of CEUR Workshop Proceedings, 2019, pp. 53-70. 46. R. Bekesh, L. Chyrun, P. Kravets, A. Demchuk, Y. Matseliukh, T. Batiuk, I. Peleshchak, R. Bigun, I. Maiba, Structural modeling of technical text analysis and synthesis processes, CEUR Workshop Proceedings, 2020, 2604, pp. 562-589 47. A. Yarovyi, D. Kudriavtsev, Method of Multi-Purpose Text Analysis Based on a Combination of
Knowledge Bases for Intelligent Chatbot, CEUR Workshop Proceedings, 2021, 2870, 1238-1248. 48. Y. Bodnia, M. Kozulia, Web Application System of Handwritten Text Recognition, CEUR
Workshop Proceedings, 2021, Vol-2870, pp. 1323-1337. 49. A. B. Rianto, Mutiara, E.P. Wibowo, P.I. Santosa, Improving the accuracy of text classification using stemming method, a case of non-formal Indonesian conversation, Journal of Big Data, 2021, 8 (1), art. no. 26. 50. O. Kuropiatnyk, V. Shynkarenko, Text Borrowings Detection System for Natural Language
Structured Digital Documents, CEUR workshop proceedings, 2020, Vol-2604, 294-305. 51. O. Cherednichenko, N. Babkova, O. Kanishcheva, Complex Term Identification for Ukrainian
Medical Texts, CEUR Workshop Proceedings, 2018, Vol-2255, 146-154. 52. A. Berko, V. Andrunyk, L. Chyrun, M. Sorokovskyy, O. Oborska, O. Oryshchyn, M. Luchkevych, O. Brodovska, The Content Analysis Method for the Information Resources Formation in Electronic Content Commerce Systems, CEUR Workshop Proceedings, 2021, 2870, pp 1632-1651 53. V. Kuchkovskiy, V. Andrunyk, M. Krylyshyn, L. Chyrun, A. Vysotskyi, S. Chyrun, N. Sokulska, I. Brodovska, Application of Online Marketing Methods and SEO Technologies for Web Resources Analysis within the Region, CEUR Workshop Proceedings, 2021, 2870, pp 1652-1693. 54. N. Antonyuk, L. Chyrun, V. Andrunyk, A. Vasevych, S. Chyrun, A. Gozhyj, I. Kalinina, Y.
Borzov, Medical news aggregation and ranking of taking into account the user needs, CEUR Workshop Proceedings, 2019, 2488, pp. 369-382. 55. V. Andrunyk, A. Vasevych, L. Chyrun, N. Chernovol, N. Antonyuk, A. Gozhyj, V. Gozhyj, I.
Kalinina, M. Korobchynskyi, Development of information system for aggregation and ranking of news taking into account the user needs, CEUR Workshop Proceedings, 2020, 2604, 1127-1171. 56. V. Andrunyk, L. Chyrun, V. Vysotska, Electronic content commerce system development, in: Proceedings of 13th International Conference: The Experience of Designing and Application of CAD Systems in Microelectronics, CADSM, 2015. 57. A. Demchuk, B. Rusyn, L. Pohreliuk, A. Gozhyj, I. Kalinina, L. Chyrun, N. Antonyuk, Commercial content distribution system based on neural network and machine learning, CEUR Workshop Proceedings, 2019, 2516, pp. 40-57. 58. L. Chyrun, V. Andrunyk, L. Chyrun, A. Gozhyj, A. Vysotskyi, O. Tereshchuk, N. Shykh, V.
Schuchmann, The Electronic Digests Formation and Categorization for Textual Commercial Content, CEUR Workshop Proceedings, 2021, Vol-2870, pp. 1816-1831. 59. A. Berko, I. Pelekh, L. Chyrun, M. Bublyk, I. Bobyk, Y. Matseliukh, L. Chyrun, Application of ontologies and meta-models for dynamic integration of weakly structured data, in: Proceedings of the IEEE 3rd International Conference on Data Stream Mining and Processing, DSMP, 2020, pp. 432-437. DOI: 10.1109/DSMP47368.2020.9204321 60. A. Berko, I. Pelekh, L. Chyrun, I. Dyyak, Information resources analysis system of dynamic integration semi-structured data in a web environment, in: Proceedings of the IEEE 3rd International Conference on Data Stream Mining and Processing, DSMP, 2020, pp. 414-419. DOI: 10.1109/DSMP47368.2020.9204101 61. I. Pelekh, A. Berko, V. Andrunyk, L. Chyrun, I. Dyyak, Design of a system for dynamic integration of weakly structured data based on mash-up technology, in: Proceedings of the IEEE 3rd International Conference on Data Stream Mining and Processing, DSMP, 2020, pp. 420-425. DOI: 10.1109/DSMP47368.2020.9204160