=Paper=
{{Paper
|id=Vol-415/paper-21
|storemode=property
|title=A system for control of hearing instrument selection and adjustment based on evaluation of correct transmission of speech elements and features
|pdfUrl=https://ceur-ws.org/Vol-415/paper20.pdf
|volume=Vol-415
|dblpUrl=https://dblp.org/rec/conf/cvhi/PlingeB07
}}
==A system for control of hearing instrument selection and adjustment based on evaluation of correct transmission of speech elements and features==
https://ceur-ws.org/Vol-415/paper20.pdf
Conference & Workshop on Assistive Technologies for People with Vision & Hearing Impairments
Assistive Technology for All Ages
CVHI 2007, M.A. Hersh (ed.)
A SYSTEM FOR CONTROL OF HEARING INSTRUMENT SELECTION AND
ADJUSTMENT BASED ON EVALUATION OF CORRECT TRANSMISSION OF SPEECH
ELEMENTS AND FEATURES
A. Plinge, D. Bauer
Institut für Arbeitsphysiologie an der Universität Dortmund
plinge@ze3.de bauer@ifado.de
Abstract: Modern digital hearing aids provide unprecedented means of compensating for hearing
impairments. However, this comes at the price of adjusting complex processing parameters. To
achieve an individual optimum, all processing parameters have to be carefully pre- and re-adjusted.
The required time and effort can often not be made available. To provide an independent means of
optimizing speech intelligibility, we developed a dedicated solution using mostly off-the-shelf hardware
and a dedicated software. With this, a pc-literate hearing impaired person can independently check his
aided speech reception. The result can be a valid basis for improvement of the hearing instrument
parameter adjustment.
Keywords: hearing impaired, individual adjustment, hearing instruments, speech reception
1. Motivation
When acquiring a new hearing aid, a hearing impaired person usually receives an initial fitting by an
audiologist. After a habituation period of two to three weeks, a corrective fine-tuning is advised. There
is usually little time for that task. In cases of severe sensory hearing deficits more time than is
available is often required.
The audiologist uses a pure-tone audiogram as basis for the hearing aid adjustment. Sometimes, a
supra-threshold scaling procedure is added; narrow-band noise is used to avoid the sinusoid that is
not well representing natural speech. Most often Word lists are used to achieve a fine tuning of the
parameter settings. Standardised word tests are administered to assess a measure for speech
intelligibility and the degree of hearing loss. In Germany, typically the Freiburger, Kündig or Sotcheck
word lists, and sentence test (e.g. Marburger Satztest) are utilized for historical reasons. In clinical
research, further speech audiometric techniques have been investigated (cp. Kollmeier 1995).
We may identify a number of shortcomings in the standard process:
First, given the correct choice of the instrument, there is little time for the audiologist to achieve an
individual optimum by utilizing all of the fine-tuning features of modern digital hearing aids.
And secondly, often no attempt is made to introduce an analysis that is able to identify weak points in
the transmission of speech on the basis of single speech elements or speech features in order to
resolve the deficiencies by readjustment or change of the configuration of the hearing instrumentation.
In cases of severe sensory damage with typically narrow dynamic ranges, a precise adjustment of
AGC and spectral compression characteristic is the most salient condition for achieving an individual
optimum of speech intelligibility. (We assume that resonances caused by the mechanical coupling
have been sufficiently flattend.) It is of utmost importance that any valuable feature within the usable
spectral range is transformed onto a well audible level that yields best possible individual distinctness.
A. Plinge & D. Bauer
A third issue is that the user has usually no means of control over the quality of speech transmission
as it is given to him by these more or less “prescribed” procedures. So he is often unable to judge
whether a different instrument or a different adjustment might improve his or her speech reception.
Beyond the adjustment of the basic input-output characteristics, modern hearing aids offer a lot more:
Especially noise suppression algorithms and external “add-on” microphones. Here also efficient test
procedures are usually lacking or simply not offered by the standard audiological procedures. The
user might want to find out if and when in situations with external noise surroundings, one and which
one of the internal noise-reduction procedures may give him a sufficient signal-to-noise ratio....or if the
use of external microphones and which of the various configurations is required. The user is here too
in urgent needs of more and useable guidance.
In order to address the aforementioned various issues of quality control, a prototypical device
consisting of a standard personal computer, dedicated software and inexpensive extra hardware was
developed. The device can be used as calibrated sound source, emitting building blocks of speech to
be listened to and evaluated by the user. In this regard, the approach can be compared with the
Otometry introduced by J.A. Victoreen (1973). But instead of damped sinusoids, actual complex
speech feature signals are used; signals representative for all salient speech element classes, not
only vowels.
With this device, the user himself can check - in the privacy of his own home or a room in a nearby
facility of an interest group etc. - the functionality of his hearing aid, identify shortcomings and find out
what programmes to use and when to employ additional devices as FM microphones or to move the
microphone nearer to the speaker.
2. Level measurement and hearing loss
Figure 1: normal and impaired hearing thresholds
To characterize a person's hearing ability, basic frequency-dependent isophonic levels are measured.
The threshold, as measured with sinusoids, is called minimum audible pressure (MAP). The upper end
of useable sound pressures is marked by the levels above which pain and damage are inflicted on the
ear. Below that, the level of uncomfortable loudness (UCL) is usually measured. In between lies the
hearing area, here the most comfortable level (MCL) is measured. Salient spectral speech energies
are marked by the phonetic symbol of the corresponding sound. It can be seen that these different
energies span a considerable dynamic and frequency range. All speech sounds in the speech area
(based on natural levels of a speaker in 1m distance) fit comfortably in the hearing area of a normal
hearing person (figure 1, left).
When drawing the detection threshold of a person with severely impaired hearing, the residual hearing
area is considerably smaller, and the loss to be compensated can be identified directly. Almost all
people with cochlear hearing loss suffer from loudness recruitment, i.e. the fact that a sound level
increased above the threshold grows in loudness much faster than for a person with normal hearing.
2
A. Plinge & D. Bauer
Modern digital hearing instruments can be adjusted to achieve a very good compensation for the
frequency dependent recruitment if a significant number of frequency bands is adjusted separately.
As clear and instructive as pure tone measurement charts may appear - the hearing ability is far more
complex than a two dimensional area plot can convene.
First of all, the perception of mixed sounds and varying levels underlies various psychoacoustical
effects not portrayed here. When hearing a sound of a given frequency, weaker sounds may not be
perceived when close by in frequency or time, they are “hidden” or masked by the original sound,
which is called spatial and temporal masking, respectively. For persons with sensorineural hearing
deficits, the number of independent critical bands is diminished drastically from 24 to e.g. 3. Therefore,
these persons suffer from very strong masking (cp. Moore 1998), which results in the inability to
separate speech from surrounding noise.
This problem is even more aggravated, if within the speech stream - despite best fitting - perceptive
gaps are produced, since some speech elements cannot be made audible by pure amplification. This
occurs often in cases of severe high tone losses; speech elements or features that reside in these
regions of the basilar membrane are “lost” for cortical processing; even worse: they produce wrong
segmentation in word entities. Such a grave disturbance of the natural segmentation can at a certain
point no longer be compensated by context-analysis; speech intelligibility drops markedly and these
hearing impaired individuals are forced to rely mainly on lip-reading.
In order to optimize speech intelligibility a hearing instrument or add-on device can apply a large
variety of nonlinear measures (cp. Plinge & Bauer 2005). Such complex non-linear processing has to
be fine tuned to each specific hearing loss individually, making the fitting procedure a complex process
with a multitude of parameters to be adjusted. To maximize the benefit for the specific user, testing the
users’ ability to perceive speech using hearing instruments with add-on devices, and their respective
parameter setting, is evaluated.
The pure tone audiogram gives a rough assessment of the hearing loss only at threshold. Word and
sentence tests are done above threshold; they can be used to verify in what proportion the goal of
restoring speech understanding was achieved. However, they do not show which speech elements
are poorly or not at all perceived. A tool allowing sufficient analysis of transmission of speech
segments is urgently needed. To fill in that quality-assessment gap and provide persons with impaired
hearing with an independent and inexpensive tool to test the combined (over-all) speech
understanding ability, the system presented here was designed.
3. Hardware and Calibration
The System can be assembled using a PC or laptop, a USB sound card, high quality speakers, some
acoustic insulation, a linear microphone, and (initially) a calibrated sound pressure source (cp. figure
2).
Figure 2, system setup
3
A. Plinge & D. Bauer
Figure 3, Software, judging a single sound
The use of a USB sound card avoids electrical distortions in the audio signals, which would not only
falsify measurements but could also annoy or injure the hearing-aid user. The speaker has to be
selected carefully. It should have a near linear frequency characteristic and yield enough sound
pressure to produce speech enhanced by 30dB plus 12dB reserve for linearization. A few affordable
consumer speakers were found to have sufficient quality. The acoustic insulation is used to reduce
reflections (in the otherwise untreated living- or workroom) to a sufficient low level. The (high quality)
microphone is used for sound level calibration and frequency linearization, so it should be stable in
level and have a linear frequency response.) For this, a cheap alternative to expensive microphones
(industry standard for calibrated sound measurements) was devised, consisting of a small battery
powered circuit board with a quality electret microphone capsule.
The software contains tools for the general setup and calibration, such as frequency characteristic
compensation, level measurement and distortion control. By recording white noise, the speakers’
frequency characteristic can be measured (approximately). From this measurement, an FIR filter with
linear phase and the inverse frequency characteristic is calculated and subsequently used to linearize
the sound output between 500 and 5000 Hz. In order to produce defined sound pressure levels, the
system is calibrated using a calibrated sound pressure source. A 1kHz sound of 96dB(A) is produced
and recorded by the microphone. So the recorded sound level of the speaker can be deduced, and the
speaker volume adjusted to an adequate level.
4. Software Tests
In order to test and evaluate the perception of individual speech elements, prerecorded sounds are
played, individually or in groups, with a calibrated normal-speech level set to the equivalent of the
speaker’s level at a distance of 1m from the hearing aid. To achieve the effect of distances smaller or
lager than 1m, the level can be adjusted in terms of “virtual distance”, i.e. an increase of 6dB is
portrayed as 0.5m speaker distance in the software (according to the square root law of sound
pressure variation from a localized source). This is done to make the effects of varying speaker
distances more transparent to the user and let him relate to a real world scenario. In this way he/she
can experience the maximum speaker distance that guarantees a loudness level that yields good
comfort and best distinctness of the sound(s) under control.
Additionally, the effect of adding various surround noises can be evaluated and the (hopefully)
beneficial effect of the hearing instrument’s noise reduction algorithms.
The virtual distance setting can be classified by the user as either “weak, but audible”, “comfortable”,
“loud”, or “too loud”. These levels represent the levels shown defining the hearing area in figure 1. In
order to convey speech naturally, the first three should be achievable with the hearing aid. If
everything is optimally adjusted, “comfortable” should be located in the 1m distance (+/-0dB), “weak”
4
A. Plinge & D. Bauer
be several meters away (-18dB) and loud near the ear (+12 to 18dB), just as a normal hearing person
perceives without any supplemental instrumentation.
4.1 Vowels
Using band filtered vowels with calibrated natural levels, it is possible to test transmission in different
frequency bands. The vowel components are played back consecutively in a group and the user is
asked to adjust the virtual speaker distance in a way that percept of each is pleasant and distinct.
Hereby, the basic compression characteristic of the hearing aid is tested. The hearing instrument
should be able to amplify and compress all components of vowel-like sounds so that they are shifted
onto a level of most comfortable hearing and most distinct classification. If, e. g. the second formant of
/i/ is too weak or of /E/ is too loud, the local compression in that band has to be re-adjusted.
The optimum levels of the test vowels can be individually set, thereby providing a quantitative
measurement in virtual distance (and equivalent dB's), so that the required re-adjustments are made
transparent. The result can be commented upon and printed together with all levels, which should give
any audiologist or hearing aid dispenser a clear picture of what is to be done.
If the user is willing and able, he can also try to establish “weak” and “loud” setting for all vowel-
components, thereby testing the ability of the hearing aid to reflect modulation of each frequency
band. If a “too loud” setting is achieved too easily, the safety mechanisms of the hearing aid should be
checked and the amplification may have to be reduced.
4.2 Fricatives
With the same interfaces, sets of fricatives can be used. While a good adjustment may be achievable
for /f/ and /S/, weakly articulated sounds, as e.g. the German /C/, may be only audible when drastically
amplified, and the /s/ may even then remain inaudible due to severe high tone losses (here special
compensatory replacement measures can be applied, cp. Plinge & Bauer 2005). The hearing impaired
user can discover his personal limits with any available hearing aid. If, for example, he can perceive
the /C/ only at virtual distances of less than 25cm, it becomes obvious that this may not be practical in
everyday situations.
Voiced fricatives can be tested in a similar fashion. It is to be expected that they are audible, but they
may not be distinguishable from each other or voiced sounds if the hearing instrument fails to transmit
the fricative component.
4.3 Plosives
Since the plosives are vital to segmentation in German, as in all Indo-European languages, it is
important that they are well beyond threshold and, favourably, also discriminable. Using plosives in
minimal context (/ph/ etc.) the basic perception can be evaluated. Playing all unvoiced plosives in a
loop, both their audibility and discriminability can be easily established.
Plosives in vowel context are used to test the temporal characteristics, specifically the required
plosive-enhancement property of the hearing instrument. Since severely hearing impaired persons
may require excessive amplification for plosive features, an optimal setting may not be technical
feasible with state-of-the-art hearing instrumentation. As vowel-plosive combinations are provided for
all voiced and voiceless plosives, critical cases can be identified.
5. Hearing in Noise
One very important issue for hearing impaired persons is hearing in noisy surroundings and groups of
speakers. Masking and reduced dynamic range are efficiently disabling the person to make out weak
speech elements in noise and to separate simultaneous sound streams, as a normal hearing person
easily does (cp. Bregman 1990). The inability to locate the sound source or to separate several
5
A. Plinge & D. Bauer
sources sadly forces him or her to avoid situations which bring her or him in contact with more than
one person at a time, which has serious psychological and social implications. Digital hearing aids
offer a variety of noise reduction mechanisms (cp. Edwards 2000). These are limited due to the small
microphone distances enforced by behind-the-ear technology; efficient reduction of low frequency
surround energy without disturbing the wanted speakers’ energy remains a problem to be solved by
different configurations and future work (cp. Bauer & Plinge 2004).
5.1 Single channel noise reduction
By mixing pre-recorded noise to the choice of phonemes given, the hearing aid's (single channel)
noise suppression ability can be evaluated. Since the virtual distances can be widely adjusted, the
hearing impaired user can directly compare even unrealistic scenarios, like having the speaker talk in
25cm distance to the hearing aid in a crowded office. By comparing the listening experience with
different noise suppression algorithms and different settings, he can learn when to use and what to
expect of each, preparing for the real situation.
5.2 Directional microphones
Using a second speaker (or another concentrated noise source) outside the normal frontal range of
direction–of-arrival, the function of directional microphones can also be tested. After setting up a noise
source in form of the speaker at a given angle of e.g. 80 to 110°, the hearing impaired user can
experience up to what level directional side noise is sufficiently cancelled and up to what level he can
still have an undisturbed perception of the frontal speaker.
5.3 FM microphones
Ideally, the use of an FM Microphone handed to the speaker should yield an extremely good signal to
noise ratio and maximum levels, especially if the microphone is placed about less than 6cm beside the
mouth in head-set fashion. It should enable the person with impaired hearing to hear everything that
he or she could hear in down to 12cm virtual distance in the previous tests. This can be more or less
the case, depending on the quality and construction of the microphone, as well as the quality of the
radio transmission and input processing in the hearing aid.
In order to test any given equipment, the user can repeat all tests with the FM microphone mounted to
the speaker (6cm beside the membrane). For testing the microphones ability to suppress noise, the
second speaker is placed directly opposite the one with the microphone. When running the noise
tests, the noise signal is played back atop of the speech signal, and the user can evaluate the quality
by varying the noises virtual distance. He can relate these results to real situations like a second
interfering speaker nearby, sitting in a car, train or crowded office.
6. Conclusions and Outlook
Using such a system, an individual optimum fitting of all parameters can be evaluated in important
hearing scenarios. This should enable hearing impaired users to request first, an individually optimal
fitting of all parameters of all programs and second, if required, an alternative choice of their hearing
instrument. So they can finally draw best possible benefit from existing technology in hearing
instrumentation and furthermore indicate future developments, such as replacement of sounds or
features, that are inaudible today, by audible substitutes. As a next step, we will introduce the system
into practice in private and clinical applications.
References
Bauer, D. and A. Plinge (2004) Assistive Technology for people with impaired hearing, CVHI.
Bregman, A.S. (1990), Auditory Scene Analysis, MIT Press
6
A. Plinge & D. Bauer
Edwards, B. (2000), Beyond amplification: signal processing techniques for improving speech
intelligibility in noise with hearing aids. Seminars in Hearing, vol. 21(2): pp. 137-156.
Edwards, B. (2002), Signal processing, hearing aid design and the psychoacoustic Turing test, IEEE
Proceedings of the International Conference on Acoustics, Speech and Signal Processing, May 13-
17, Orlando, FL, vol. 4, pp. 3996-3999.
Kollmeier, B. (1995), Computer controlled speech audiometric techniques for the assessment of
hearing loss and the evaluation of hearing aids, in Psychoacoustics, Speech and Hearing Aids,
World Scientific.
Moore, B.C.J. (1998), Cochlear Hearing Loss, Whurr, London
Plinge, A. and D. Bauer (2005), Genesis of wearable DSP structures for selective speech
enhancement and replacement to compensate severe hearing deficits, Assistive Technology -
From Virtuality to Reality, Proceedings of AAATE 2005, IOS Press
Victoreen, J.A. (1973), Basic Principles of Otometry, Charles C Thomas Pub Ltd.
7