Laryngectomy leads to voice loss. Current voice restoration techniques are insu cient. New method is proposed based on silent speech interface and digital copy of user's voice. Analysis of arti cial voice perception, particularly personality attribution, is conducted as a part of this project. voice restoration machine learning silent speech synthesis arti cial voices per-
Laryngectomy is an invasive, radical surgical treatment of laryngeal cancer
How is our personality perceived by others? What factors should be taken into
account while analysing this issue? Most of the studies on personality perception
have focused on visual modality. Recently it has been noticed that, along with
visual, other e.g. aural (voice) and haptic (touch) modalities could play a
significant role in that process
Other researchers also reach the conclusion that, among other factors, the sound of our voice plays crucial role in personality perception. Nass & Lee (2001) conducted an experiment with participants listening to an introvert or extrovert arti cial voice in natural context. The subjects (introverts and extroverts) showed similarity attraction to one of the voices (also introverts and extroverts). It shows that voice is one of factors that contribute to personality perception.
Some authors even postulate a concept of arti cial personality
The subject of arti cial voices is especially important in the context of emerging array of voice assistants (e.g. Alexa1, Siri2, Google Home3, Cortana4). The perception of such systems would be an interesting research eld in cognitive science. However, the same approach can be applied in order to get important information on how to help a speci c group of people with the use of synthesised voice. 3
Let us consider people that have lost their voice. Laryngectomy is an invasive,
radical chirurgical treatment for laryngeal cancer
Criterion 1) { (VR 2) esophageal speech (ful lls Criterion 1 and Criterion 2, does not meet
Criterion 3) 1 Alexa: https://developer.amazon.com/alexa, retrieved 30.04.2017, 20:00 2 Siri: https://www.apple.com/ios/siri/, retrieved 30.04.2017, 20:00 3 Google Home: https://store.google.com/ca/product/google home/, 30.04.2017, 20:00 4 Cortana: https://www.microsoft.com/en-us/cortana, retrieved 30.04.2017, 20:00 retrieved { (VR 3) electrolarynx (ful lls Criterion 1 and Criterion 3, does not meet
Criterion 2) As can be seen none of those methods ful ll Criterion 1, Criterion 2 and Criterion 3 at the same time. Due to the drawbacks of the current methods of voice restoration after laryngectomy, our research group has proposed an alternative. { (VR 4) Intelligent interface based on neuromuscular input and digitized biological voice 4
Application of novel technologies from the eld of arti cial intelligence could
help the group of patients following laryngectomy. The main project goal is to
create a complete system of communication for laryngectomees, combining two
approaches:
{ (A1) AlterEgo system
System that we propose draws from both (A1) and (A2), combining and modifying them.
The training session: 1. neuromuscular signals from the face and neck of the patient are detected using EMG sensors. At the same time his/her voice is recorded; 2. recordings are manually transcribed to text; 3. once the voice is recorded it is converted into an arti cial voice model, but with "biological trace" of an user (M1); 5 Lyrebird AI: https://lyrebird.ai/, retrieved 30.04.2017, 20:00 6 CereProc: https://cereproc.com/, retrieved 30.04.2017, 20:00 4. based on information from 1. the machine learning model is built with EMG signal-derived features and text transcriptions as a predicted values (M2). Speech production: 1. patient tries to speak normally (that's the di erence between "silent speech" interfaces and the system proposed by us). Here, mouth movements are desired increasing naturalness of speech; 2. the physiological movement signals are collected from his/her mouth and neck muscles with EMG sensors; 3. the system predicts the text that should be produced by similar movements basing on the previously built model (M2); 4. this text is converted according to previously built digital copy of user's voice (M1); 5. afterwards the sound (speech) is played from a speaker attached to the patient's body. It is aimed to be a place that the patient will not see and take little space. On the current stage of the project a JBL Go speaker is attached to the user's forehead with an elastic band.
There have been rst attempts of using silent speech system to help patients
following laryngectomy
The project raised numerous research questions. Among others: { (RQ 1) Would the system be natural for patients? { (RQ 2) How users would perceive their own body with a speaker attached to it and how others will perceive them? { (RQ 3) How to integrate solutions into a system for a speci c task (voice restoration after laryngectomy)? { (RQ 4) How to manage complexity of the language? { (RQ 5) How to build suitably fast system that allow use in everyday situations? { (RQ 6) How to provide a system that will allow to prosody control?
Investigation of naturalness of the voice will be the rst step within (RQ 1). The very basic hypothesis that we posited is that users will perceive a voice, even with lower quality, with their "biological trace" as more natural than the best (on the matter of voice quality) available text to speech voice (H1).
In order to test (H1) we have established two main tasks: { (T1) Analize current literature on the subject of arti cial voices perception and psychological aspects of laryngectomy; { (T2) Conduct empirical study that will answer this question.
As people attribute personality traits to arti cial voice, thus output of our system could a ect the way how patients will be perceived by others e.g. could be perceived as extravert, deceptive etc. Based on that assumption we argue that it is crucial to choose a better way: building entirely arti cial voice (maybe with assigned arti cial personality) or arti cial voice with patient's \biological trace".
Issues to be considered are the psychological aspects of laryngectomy.
Operation is a daunting experience for patients and their relatives. The study of
Bussian et al. (2010) suggest that psychiatric disorders a ects approximately
one fth of laryngectomy patients. The mail survey study with a large group
of respondents after laryngectomy
Still, a study by Vilaseca & Chen (2006) could be mentioned. They showed that although patients identi ed speech among their most important problems, no correlation was found between speech and long-term quality of life (QOL). There is a need to establish which factors connected with voice, contribute to the self-perception. Such study would be greatly aided by the system we are going to create, where various factors can be manipulated and their e ects can be measured. 6
After analysis conducted in previous section, we have drawn rst preliminary
conclusion:
(Conclusion 1) Patients may prefer a system with "biological trace" of their
voice. This is based on the premise and assumption:
{ (P1) Other people seem to perceive a part of people personal traits basing
on their voice
Combining (Conclusion 1) with our criterion of naturalness (Criterion 2) leads to: (Conclusion 2) The e ort to create digital model of patient's voice is justi ed and it should be further investigated experimentally within (T2).
It seems that users could perceive a voice, even with lower quality, with their "biological trace" more natural than the best (in the respect of voice quality) available text to speech voice. As we can see is it worth building arti cial voice with "biological trace" of a patient. We argue that the di erence between arti cial voice with or without "biological trace" of patient's is worth further investigation, due to the fact that literature suggests that there is a di erence in perception of those, but do not specify what is the character of this di erence.
During the conference we have encouraged a discussion on the following questions: { (Q1) People have most probably developed a well working system of how to attribute emotions and personality traits to others based on their voice. What are the possible cognitive biases in the context of emergence of arti cial voices? { (Q2) Is it really valuable to use patient's natural voice in our novel voice restoration technique? Do we need to try to falsify (H1) more ercely and why?