This article concerns the classification of household sounds for the development of an application to help people with hearing disabilities to resolve everyday situations which may present them with serious problems: telephone calls, the sound of the door bell, an alarm clock, the completion of a domestic appliance programme, etc. It presents the study carried out in order to obtain the best acoustical parameters by adopting training and evaluation techniques using GMM models with a varying number of Gaussian components. Parameters such as MFCCs, ZCR, Roll-Off Point and Spectral Centroid are tested using the classifier.
A fire alarm or a telephone call at an ungodly hour warning of danger can be sounds which are perceived to be urgent when they are detected. Hearing problems or simply a high level of noise are factors which sometimes make it difficult for the human ear to identify these sounds. Deaf people experience the issues that stem from not being able to detect or identify sounds on a daily basis. Studying the techniques and algorithms which enable this task to be performed automatically, not simply based on overcoming an intensity threshold like the majority of products available on the market, is being viewed as significant technological progress which will offer huge benefits to people with hearing disabilities.
It is about improving their autonomy and independence when performing everyday tasks which due to their limitations are often either extremely difficult or impossible. Developing an application which can detect and classify the various sounds which may emerge in a home is considered to be a fundamental requirement towards helping to improve the quality of life of people with hearing impairments.
This article presents the preliminary results of the study carried out in order to obtain a set of parameters relevant to the classification of impulsive sounds such as door bells, alarm clocks, a baby crying, with a high degree of accuracy and reliability.
The stages making up this kind of system can be analysed in three distinct modules
In order to test and validate the recognition system, it is essential to have a sound database with
representative size. In order to achieve this, part of the commercial database “Sound Scene Database
in Real Acoustical Environment”
The tests were performed using three different types of sound (100 signals per type): telephones, alarm clocks and door locks, all obtained using a sampling frequency of 44100Hz. The selection was made on the grounds that these sounds are commonly found at home.
In order to extract the characteristics, the sounds in the DB were segmented into 20-millisecond
frames prior to analysis
For each frame, 13 cepstral coefficients, the zero crossing rate, the Spectral Centroid and the Roll-Off Point were calculated, providing a 16-parameter vector. These parameters are briefly described below. 3.2.1. Mel Frequency Cepstral Coefficients (MFCCs): This is a perception parameter based on the FFT. After calculating the logarithm for the FFT magnitude of the signal, the bins (minimum units in the spectral domain) group together and soften in line with the Mel Frequency scale which is defined mathematically in formula 1: Mel ( f ) = 2595 log 10 ⎜⎝⎛ 1 +
f ⎞⎟
700 ⎠
(1)
3.3. Classification method
3.2.3. Spectral Centroid: This parameter measures how strong a sound is. The centre of gravity is
assessed based on the information obtained from the Fourier Transform. It is defined as:
where X t [n] represents the nth sample of the Fourier Transform for the t frame.
3.2.4. Roll Off Point (RF): This parameter represents the frequencies below which 85% of the energy
in the audio spectrum resides. It is commonly used to identify sounds from musical instruments. In this
context, the percussive sounds and note attacks usually have more energy, which is why this is a
measure of the existence of abrupt signal changes. Equation 5 defines this characteristic, M [ f ]
being the energy of the signal in frequency bands higher than f . The maximum value of f is delimited
by the size and the sample ratio of the acoustical band.
Lastly, a DCT (Discrete Cosign Transform) is performed in order to decorrelate the vector of the
resulting parameters. The function defining this process is as follows:
(2)
(3)
(4)
(5)
(6)
k =20
c ( p ) = ∑ log m n ( p ) cos
q
n =1
3.2.2. Zero Crossing Rate (ZCR): This is the number of zero crossings which occur in the analysis
frame. In the signal spectrum, high (or low) frequencies imply high (or low) zero crossing rates. As
such, the number of zero crossings is an indicator of the signal’s high frequency content. The
mathematical definition is shown in equation 3:
There are several probabilistic classification techniques, but they are not all appropriate for
recognising sounds that are not related to speech
K gm( x) = ∑ w ⋅ g (μ k , ∑ k )(x)
k k =1 k ∑ wi = 1 i=1 i ∈ {1,......, K} : wi ≥ 0 An initial experiment was carried out with the aim of observing the complexity that the classification involves by only using the first two cepstral coefficients (MFCC1 and MFCC2) and the probability awarded by the Gaussian Mixture Model. Figure 2 shows how the separation performed by the classifier is reasonably high. The same test using the third and fourth coefficients, MFCC3 and MFCC4, produces different distributions with more overlapping areas, as shown by the graph in figure 3.
A study has been carried out in order to obtain the optimum parameter combination, using different data sizes in training. Exhaustive tests have been performed, using mixture models of 3 to 8 Gaussians for 4 different training database sizes.
The results obtained are presented using two measures for the success rate: by frame, calculating the average success of all frames, and by sound, by assigning each sound the most likely type according to the average success of the frames of which they are composed and by calculating the average success rate for all sounds of the same type.
The results obtained are divided into the three points below. 50 33 Training DB size
15
In a training session with 33 sounds of each type, varying the GMM parameter, the best results are obtained by using 5 or more Gaussians, as can be seen in figure 5. Also in this case, the percentage of correctly classified sounds is 100%.
Training the Gaussian Mixture Model with sizes of 66, 50, 33 and 15 sounds of each type, the number of correctly classified frames goes down slightly as the number of sounds used to train the model decreases. In terms of sound classification 100% of sounds are correctly classified in all cases. Figure 4 shows the best results. 0,902
0,9 0,898 0,896 0,894 0,892 0,89 0,888 0,886 0,884 0,882 0,894 0,893 0,892 0,891 0,89 0,889 0,888 0,887 0,886 0,885 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 3 4 7
8 5 6
Nº Gaussians
MFCCM0FCCM1FCCM2FCCM3FCCM4FCCM5FCCM6FCCM7FCCM8FCMC9FCCM1F0CCM11FCC1C2Renotlrlo-OidffPoint ZCR Fig. 6. Best parameters in evaluation by frame MFCCM0FCCM1FCCM2FCCM3FCCM4FCCM5FCCM6FCCM7FCCM8FCMCF9CCM10FCCM11FCC1C2enotlrlo-OidffPointZCR
R Fig. 7. Best parameters in evaluation by sound The best result has an overall frame classification precision of 90.1% using a database containing 66 sounds, 8 Gaussians and a combination of 8 parameters: MFCC0, MFCC1, MFCC2, MFCC4, MFCC9, MFCC11, Centroid and Roll Off Point.
At this work a first study of household sounds classification has been made. The used database has been used to evaluate the different acoustic parameters from the audio and the classification model. The used signals are clean and they don’t contain any type of background noise. In the future we want to create an own database which not only will add a wider range of sounds, but it will also take into account the existence of different possible noises, such as electrical appliances, outdoor traffic noises,…