=Paper=
{{Paper
|id=Vol-2544/paper6
|storemode=property
|title=On the Development of a Non-Invasive Pathologies Identification System by Qualitative and Quantitative Characterization of Infant Crying and the Application of Intelligent Classification Models to be Used in Rural Environments
|pdfUrl=https://ceur-ws.org/Vol-2544/paper6.pdf
|volume=Vol-2544
|authors=Carlos A. Reyes-Garcia,IAlejandro. A. Torres-Garcia,M. Antonia Ruiz-Diaz
|dblpUrl=https://dblp.org/rec/conf/irehi/GarciaTD18
}}
==On the Development of a Non-Invasive Pathologies Identification System by Qualitative and Quantitative Characterization of Infant Crying and the Application of Intelligent Classification Models to be Used in Rural Environments==
https://ceur-ws.org/Vol-2544/paper6.pdf
On the Development of a Non Invasive Pathologies
Identification System by Qualitative and
Quantitative Characterization of Infant Crying
and the Application of Intelligent Classification
Models to be Used in Rural Environments)
Carlos A. Reyes-Garcia Alejandro. A. Torres-Garcia M. Antonia Ruiz-Diaz
Ciencias y Tecnologías Coordinación de Ciencias Universidad Politecnica de
Biomédicas Computacionales Tlaxcala
Instituto Nacional de Instituto Nacional de Zacatelco, Tlaxcala, Mexico
Astrofísica, Óptica y Electrónica Astrofísica, Óptica y Electrónica mariaantonia.ruiz@uptlax.edu.mx
Tonantzintla, Pue., México Tonantzintla, Pue., México
kargaxxi@inaoep.mx kcobaimskywalker@gmail.com
Abstract— The detection of pathologies in the early stages decades. Several studies refer to both the subjective auditory
of a baby's life has been one of the major challenges to analysis of voice and speech and to automatic acoustic
overcome for the medical sciences. Lack of means of analysis in adults. However, with regard to newborn crying,
interpretation of this normal physical manifestation of the there are few automatic methods, some based on classical
child has made this task extremely complicated. The discovery approaches such as the Fourier transform and the
that the crying wave, as the sole initial means of autocorrelation analysis [1] [2] [3] [4] [5] and others in
communication of babies, contains information about their parametric techniques [6] [7]. These methods allow us to
neurophysiological state, has opened the possibility of estimate the main acoustic quantitative characteristics, such
interpreting that state and to diagnose diseases from a few days
as the frequency of vibration of the vocal cords, the
of birth. In this paper we present the efforts to develop a
practical integral system to automatically identify pathologies
resonance frequencies of the vocal tract, linear prediction
in newborn babies by selecting quantitative features and which coding (LPC), Mel frequency cepstral coefficients (MFCC),
also highlights different types of qualitative characteristics on etc. In recent years, several authors propose classification
the newborn infant crying through appropriate acoustic methods for a wide range of pathologies. Reyes et al. [8], [9],
processes. And, in each case, after the features are selected or [10] have investigated normal, deaf and asphyxiating
identified uses them to recognize the inherent pathology. Once newborns through neural networks, evolutionary model
the system is complete and fully tested we pretend to offer selection and fuzzy logic, Poel et al [11] present results on
rural nurses, general doctors, researchers and scholars a tool the classification of crying in newborns normal disorder and
as a mean to make noninvasive diagnostics and as an related to hypoxia using radial-based function neural
information support to allow them to have a solid perspective networks with a general classification performance of 85%.
on relevant crying events, and to facilitate the development and
unification of standards for the assessment or comprehensive The cry of the newborn reflects the development and
description of the crying wave. possibly the integrity of the central nervous system, so that
its analysis is an attractive non-invasive means to assess the
Keywords— analysis of infant crying, automatic physical state of babies from very early stages of life. In the
identification of qualitative characteristics, classification of analysis of infantile crying, it is also important to identify the
crying, non-invasive diagnosis. qualitative characteristics, since they provide relevant extra
information that allows to identify variations or similarities
I. INTRODUCTION between normal and pathological crying, as well as to
. The crying of newborns is a functional expression of differentiate between different pathologies. Generally the
basic biological needs, emotional or psychological conditions analysis of the qualitative characteristics is done manually,
such as hunger, cold, pain, cramps and even joy [1]. It by means of visual perception (inspecting spectrograms) and
requires a coordinated effort of several brain regions, mainly auditory (listening to the crying recordings) of specialist
brainstem and limbic system, and is related to respiration and doctors, who according to what they see and hear can make a
pulmonary mechanisms. Its characteristics reflect the diagnosis . This paper presents the approach to develop a
development and possibly the integrity of the central nervous system which will integrate a model section method to use
system. Therefore, the analysis of infant crying is a suitable quantitative features and a method that allows the automatic
non-invasive complementary tool to assess the physical state detection of crying units and ehich uses a model called
of infants, particularly important in the case of premature "dodecagrama" that allows to automatically identify the
infants. Specifically, the distinction between a regular crying melody type of the crying units, and finally with the values
and one with abnormalities is of clinical interest. Being of the fundamental frequency. they automatically identify
economic and without contact, the study of the crying of the distinctive qualitative characteristics such as shifts, glides
newborn baby has had an outstanding growth in the last and noise concentrations of crying units.
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0)
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX
IREHI 2018 : 2nd IEEE International RuralIEEE
and Elderly Health Informatics Conference
In the recordings there can be cries with high pitch, with low
pitch, sharp cries, etc. and the variation of the intensity,
since the infants can reduce or increase the intensity of their
crying at will in the same recording. Crying detection has
been carried out manually in various works such as [1], [13].
In this work, the crying unit detection method is part of an
interface implemented in MATLAB. Based on experimental
tests, crying units smaller than 200ms were eliminated
because they are very short-duration sounds that do not
provide useful information for further analysis. We also
Figure 1. Automatic Infant Cry Recognition (AICR) Process
defined an energy threshold (U (e)) applied to the signals,
II. The Automatic Infant Cry Recognition (AICR) and which, based on [14] and our experiments, is obtained
Process as follows:
This process, in general, is performed through two phases;
first, the processing of the signal to obtain the acoustic
characteristic vectors and the second phase to identify the
type of cry by means of a classifier, Figure 1 shows this
process. where En is the energy of the short time signal.
2.1 Signal Processing Phase
There are two different processes to extract acoustic features
from the infant cry wave. One is to get quantitative features
and the other is to obtain the qualitative features.
In the case of the quantitative features, during the signal
processing phase, each signal was divided in segments of
1sec. Each segment was subdivided in 50 ms windows, then
generating 19 windows out of every one second sample.
Later, from each window 16 coefficients MFCC were
extracted, with which a total of 304 coefficients by vector
were obtained. After adding the label of the class, each
vector has 305 attributes. For the experimentes shown in this Figure 2. General scheme of the process of extraction and
paper we used 507 samples of normal cry and 879 of deaf processing of qualitative characteristics
cry. At the end the size of the matrices generated for each
Figure 2 shows step by step the operation of the proposed
type of cry were as follows: normal cry 507x305, deaf cry
method starting with the detection of crying units. After the
879x305. The quantitative features extracted for our
identification of the melody type is made, a modification of
purposes were Mel Frequency Cepstrum Coefficients
the method presented in [15] was developed, which we have
(MFCC) which have a frequency similar to the one of the
called the dodecagram method, which is part of the feature
human ear, which is more sensitive to certain frequencies
extraction interface. In Figure 2 the dodecagram is shown in
that to others. In this way, an approximation to the form in
which each crying unit is positioned at the center of lines f
which the ear perceives the sounds is obtained.
and g. The value of the lines is determined by the value of
the fundamental frequency of the first window. The next step
For the qualitative features the extraction process begins
is to code the unit of crying by means of the following rules:
with the detection of crying units. This is carried out with
1 if the value of the fundamental frequency passes to a higher
the purpose of using them for a later analysis, such is the
row, 0 if the The value of the fundamental frequency remains
case of [12], in which the average duration of the crying
in the same row and -1 if the value of the fundamental
signals, the mean of the fundamental frequency of the crying
frequency passes to a lower line. Where the number 1
as well as its melodic form are analyzed.
corresponds to an increase in the fundamental frequency, the
0 without changes, and -1 to a decrease in the fundamental
In the recordings are sounds that are not useful for the
frequency. Therefore, we can see that the unit of crying
analysis of infant crying, such as the sounds produced by the
shown in Fig. 1 has a melodic form of type: descending-
environment and the inspiratory sounds produced by the
ascending.
infants before emitting a unit of crying, to which the doctors
call inspirations. Another important point that should be To identify the shifts, the differences of the fundamental
considered is the variety of environments and devices in frequencies along the signal are measured, if the difference
which the recordings are acquired as well as the intensity exceeds 100Hz it is considered shift (there can be more than
and type of crying of the infant. one in a crying unit).
In the same way to identify the glides the differences of the 100. The stop criterion was to reach 8,000 fitness function
fundamental frequencies along the signal are measured, if the evaluations.
difference exceeds 600Hz in a very short time it is
considered glide (there can be more than one in a crying Table 1. Results of generating solutions with the Hybrid Fuzzy-
unit). A vibrato is defined as a series of waves with at least Genetic Algorithm for the classes of Normal vs Hipoacusic (deaf)
four movements of ascent and descent in the fundamental infant cry, where (E.A) is Evaluation Accuracy, (T.A) is the Test
frequency Accuracy and Storage is the percentage of data kept after the
reduction/selection stage.
III. CLASSIFICATION METHODS:
With the purpose of provide the interface of the system with
the best classifiers available several hybrid classification
methods were tested, all of them implemented with a Table 2. Results of FRNN and PCA for the classes of Normal vs
combination of fuzzy systems with evolutionary algorithms Hipoacusic (deaf) infant cry, where (E.A) is Evaluation Accuracy
or neural networks. Some of them were tested for the and (T.A) is the Test Accuracy. PCA refers to the percentage of
quantitative features and some for the qualitative ones. They information we want to preserve
are described next.
3.1. The Quantitative Experiments
For classifying the quantitative features, in this case, we
experimented with a method that first reduces the size of
features and instances from the data base by means of a
selection process based a hybrid fuzzy-genetic algorithm as Additionally, the results of an average of 10 experiments
presented in [16]. The hybrid fuzzy-genetic algorithm for with other method that combines vector reduction with PCA
feature and instance subset selection combines a Hybrid and a Fuzzy Relational Neural Network (FRNN) are
Meta-Heuristic (HMH) algorithm and a Fuzzy Self- presented in Table 2. The FRNN receives as input a training
Adaptive Genetic Algorithm whose crossover operator is a set and returns as output a fuzzy relational matrix. Before
Rotary Circular Crossover (RCC) based on Half Uniform
presenting the training data to the FRNN, PCA is applied to
Crossover (HUX). The best individual in the initial the dataset and the experiments are performed by using 2, 7,
population is used as the initial solution of the HMH with 26 and 65 features respectively from the transformed matrix.
the purpose of improving its fitness. This method –also The training, evaluation and test sets used in the experiments
proposed in this work- is a combination of simulated were the same than those used to obtain the results presented
annealing, taboo search and hill-climbers algorithms which in Table 1. Experiments with 3 and 5 linguistic properties are
allows us to speed up the convergence of the initial reported, the input membership values were obtained with
population. The genetic algorithm adjusts its own control the Trapezoidal membership function, with fd = 6 and fe =
parameters while running the algorithm by means of two 1.5. Three epochs were completed to train the Neural
fuzzy inference systems. For a complete description of the Network. The classification was performed with the max-
functioning of the selection process the reader is refered to min composition.
the work presented by Leon-Barranco et al in [16].
This process was carried out once each input feature of the
3.2. The Pattern Classification Stage. training samples was transformed in membership values to
each of the assigned linguistic properties, i.e., once a vector
We first note that the amount of data is very large, so it will containing m features was transformed in a 3m, or 5m-
be useful to select variables or features just as to select dimensional vector, and the fuzzy relational matrix was
instances with the intention of reducing computational returned by the FRNN. If we consider that the 826 instances
resources and, to improve the accuracy of the classification with the 304 features are the 100% of data, then 0.66%,
process. We have 1386 instances or vectors in the infant cry 8.55%, 16.45% and 49.34% was the storage size used for 2,
data base, and each infant cry vector is represented by a 7, 26 and 65 principal components, respectively.
vector of 304 features. All these 1386 instances belong to 2
classes (normal and hypo-acousic infant cry). What we want 3.3 The Qualitative Experiments
to do is to find a smaller feature and instance subset that can
better represent the infant cry dataset avoiding the need of For the classification experiments when using the qualitative
using all data, without degrading the classification accuracy features four adaptive neuro-fuzzy classifiers were
of the system. implemented and adapted [17-20], which are; Neuro-fuzzy
The results of evaluating the Hybrid Fuzzy-Genetic adaptive with linguistic modifiers classifier (ANFCLH),
Algorithm are presented in Table 1. The baby’s crying neuro- fuzzy classifier with linguistic modifiers and selected
dataset has 1386 instance with 304 features each. Ten characteristics (LHNFCSF), neuro-fuzzy conjugate gradient
training, evaluation and test sets were formed, i.e., for each classifier (SCGNFC) and accelerated conjugated gradient
experiment 826, 280 and 280 instances for training, neuro- fuzzy classifier (SSCGNFC). The SCGNFC and
evaluation and test respectively were randomly selected SSCGNFC systems are optimized by scaled conjugate
from the baby’s crying dataset. The presented results are gradient algorithms. In these two systems, the k-means
from experiments with a maximum population size Mp = algorithm is used to initialize fuzzy rules. Also, the Gaussian
membe rship function is only used for descriptions of fuzzy IV. DISCUSION
sets. The other two systems are based on linguistic modifiers As can be observed, the classification results can be more
(LH) tuned by scaled conjugate gradient. accurate when using the quantitative features. Besides the
3.4 Experimental Results high efficiency shown by the hybrid fuzzy-genetic algorithm
since, after the process of instance/features
The system was tested with two sets of samples, one selection/reduction, with only a small fraction of the data
obtained from the Chillanto database of the National Institute base it was able to obtain a test accuracy of 97.35%. This
of Optical Astrophysics and Electronics of Mexican infants, result was higher than the highest obtained with the FRNN
in which it was tested with a base of Normal cries against and PCA method when keeping the 70% of information whose Test
that of Hipoacusic (deaf) cells collected at the National Accuracy was of 87.3 % for the same set of samples.
Rehabilitation Institute in Mexico. In addition, for the
purposes of the international project mentioned in the Regarding the accuracy in the recognition when using
Aknowledgements, we tested the classifers in another qualitative features, it can be seen that in the case of Normal
environment to differentiate between premature and term vs Hipoacusic (deaf) infant cry the NFC_LH_FS recognizer was
baby crying registered in the University Hospital of Liège, in the one that obtained better results with 77.25% and in the
Belgium to assess the versatility of the system. case of Premature vs. Term two classifiers, NFC and
NFC_accelerated, obtained the highest precision that is 84.52%..
Although the results seem not to be very high in any way,
Table 3. Classification results with each of the classifiers for the they are very encouraging, since they were obtained using
classes of Normal vs Hipoacusic (deaf) infant cry. The results only the qualitative characteristics.
of different iterations will be displayed together with the
average accuracy and the standard deviation. All four classifiers are trained through a series of iterations
following a 10-fold Cross Validation scheme. Tables 3 and 4
show the partial results that each classifier obtains at the end
NFC f NFC_LH NFC_accelerated NFC_LH_FS of each iteration (fold).
61.54 61.54 69.23 76.92
Although better results have been obtained using quantitative
100 100 100 100
characteristics, the presented study is relevant since the
78.57 78.57 78.57 85.71
extraction of qualitative characteristics is important because
69.23 69.23 61.54 46.15 besides allowing a description very close to that used by the
85.71 92.86 85.71 92.86 medical specialists when doing a perceptive study of the
71.43 78.57 71.43 78.57 waves of baby crying acceptable classification results can be
61.54 61.54 61.54 76.92 obtained through which an initial diagnosis can be made and
76.92 76.92 76.92 92.31 the right advise of first care be provided.
76.92 76.92 76.92 76.92
V. CONCLUSION
69.23 61.54 69.23 46.15
Average By the results obtained we are supported to
Accuracy
Perceptually, there is a great similarity between the crying
75.1099 75.7692 75.1099 77.2527
waves of healthy and deaf babies, so, obtaining a recognition
Standard
accuracy of 97.35% when using the quantitative features
Deviation
encourages us to continue with our work to finish the non
8.7061 8.6885 8.8066 11.334
invasive diagnostic tool.
The accurate results obtained with our developed models for
the processing of quantitative features reinforced with the
Table 4. Classification results with each of the classifiers for the descriptive properties of the qualitative characteristics
Premature vs. Term classes. The results of different iterations reflects a high potential for its later application in the
will be displayed together with the average accuracy and the development of non-invasive diagnostic systems.
standard deviation
In the same way, in the case of the classification of
NFC f NFC_LH NFC_accelerated NFC_LH_FS premature and term cries in which 84.52% was obtained, it
100 100 66.67 100 allows to see a greater possibility of application of the
100 100 100 83.33
qualitative characteristics in domains other than the
diagnosis. In this case the greater precision can be explained
100 100 100 85.71
83.33 83.33 83.33 83.33
by the difference between the two types of crying caused by
83.33 83.33 83.33 50
the improved maturity of the phonatory apparatus of babies
born at term.
85.71 42.86 71.43 71.43
100 83.33 100 66.67 The system and the models still require more studies and
85.71 71.43 100 85.71 experiments with various modifications, such as the use of a
Average combination of both qualitative and quantitative
Accuracy characteristics tanking the advantages shown by each one,
84.52 81.90 84.52 72.14 including other methods that we have tested for the same
Standard purpose.
Deviation With the results obtained it is possible to ensure that the use
17.96 17.43 15.64 20.65 of both quantitative along qualitative characteristics for their
classification for various purposes opens a large window of [8]. Reyes-Galaviz, O. F., Tirado, E. A., & Reyes-Garcia, C. A.
application opportunities limited only by the imagination of (2004).Classification of infant crying to identify pathologies in
recently born babies with ANFIS (pp. 408-415). in Lecture Notes in
the developers. The first proposed task in progress is to finish Computer Science 3118, Springer Berlin Heidelberg. Computers
a robust, easy to use and friendly interface that can be Helping People with Special Needs, edited by Klaus Miesenberger et.
handled and the results interpreted by any non-specialized al., Springer, Berlin, 2004, pp 408-415.
health personal. An application is thought to be installed in [9]. Cano, O.S.D., Escobedo, D.I., Suaste, I., Ekkel, T., Reyes Garcia,
smart phones with the possibility to be integrated to C.A.: A Combined Classifier of Cry Units with New Acoustic
Attributes. In: Martínez-Trinidad, J.F.,Carrasco Ochoa, J.A., Kittler,
telemedicine systems and which could be particularly useful J. (eds.) CIARP 2006. LNCS, vol. 4225, pp. 416–425. Springer,
in rural environments where the where the specialists do not Heidelberg (2006).
arrive and the diagnostic devises are extremely limited. [10]. Amaro-Camargo, E., & Reyes-García, C. A. (2007). Applying
statistical vectors of acoustic characteristics for the automatic
classification of infant cry. InAdvanced Intelligent Computing
Theories and Applications. With Aspects of Theoretical and
ACKNOWLEDGMENT Methodological Issues (pp. 1078-1085). Springer Berlin Heidelberg.
[11]. Poel M., Ekkel T. Analyzing infant cries using a committee of neural
The authors are grateful to the project MX14MO06 networks in order to detect hypoxia related disorder 2006
International Journal on Artificial Intelligence Tools 15 3 397-410.
"Techniques of analysis and classification of voice and facial [12]. K. Michelsson, K Eklund, P. Leepänen, H. Lyytinen, “Cry
expressions: application to neurological diseases in newborns characteristics of 172 Healthy 1- to 7-Day-Old Infants”, International
and adults" of the Mexico-Italy scientific and technological Journal of Phoniatrics Speech Therapy and Communication
cooperation executive program financed by AMEXID of the Pathology, vol. 54, 2002
[13]. K. Wermke, W. Mende, C. Manfredi, and P. Bruscaglioni,
SRE and the Ministry of Health. Foreign Affairs of Italy. “Developmental aspects of infant's cry melody and formants”, Med
Eng Phys, vol. 24(7-8), pp. 501-514, 2002.
REFERENCES [14]. I Saratxaga, I Luengo, E Navas, I Hernández, J Sánchez, I Sainz,
“Detección de PITCH en condiciones adversas”, IV Jornada en
Tecnología del Habla, Universidad del País Vasco – Euskal Herriko
[1]. Michelsson K, Michelsson O. Phonation in the newborn, infant cry. Unibertsitatea pp. 13-18,2006.
Int J Pediatr Otorhinolaryngol 1999;49 (Suppl 1):S297-S30. [15]. Benyó Z. Várallyay G Jr., Illényi A. Melody analysis in the newborn
[2]. Fuller BF. Acoustic discrimination of three types of infant cries. Nurs infant cries. pages 11-14, Diciembre 2009.
Res 1991; 40(3):156-160. [16]. Agustin Leon-Barranco, Carlos A. Reyes-Garcia, and Ramon
[3]. Laufer MZ, Horii Y. Fundamental frequency characteristics of infant Zatarain-Cabada, “A Hybrid Fuzzy-Genetic Algorithm”, in Lecture
non-distress vocalization during the first twenty-four weeks. J Child Notes in Computer Sciences (LNCS) 4113, edited by D.-S. Huang,
Lang 1977; 4(02):171-184. K. Li, and G.W. Irwin, Springer, Berlin, 2006, pp. 500-510, ISBN: 3-
[4]. Reggiannini B, Sheinkopf SJ, Silverman HF, Li X, Lester BM. A 540-37271-7, ISSN: 0302-9743.
Flexible Analysis Tool for the Quantitative Acoustic Assessment of [17]. Cetişli B, Barkana A. Speeding up the scaled conjugate gradient
Infant Cry. J Speech Lang Hear Res 2013; 56(5):1416-1428. algorithm and its application in neuro-fuzzy classifier training. Soft
[5]. Sirviö P, Michelsson K. Sound-spectrographic cry analysis of normal Computing 14(4):365–378, 2010.
and abnormal newborn infants. Folia Phoniatrica et Logopaedica [18]. Cetişli B. Development of an adaptive neuro-fuzzy classifier using
1976, 28(3):161-173.Hevia-Montiel N, Molino-Minero-Re E, linguistic hedges: Part 1. Expert Systems with Applications, 37(8),
Carrillo-Bermejo AJ. Tortuosidad discreta como medida pp. 6093-6101, 2010.
morfométrica en tumores cerebrales. Rev Mex Ing Biom. 38(1):188– [19]. Cetişli B. The effect of linguistic hedges on feature selection: Part 2.
98. 2017 DOI: 10.17488/RMIB.38.1.13 Expert Systems with Applications, 37(8), pp 6102-6108, 2010.
[6]. A. Fort, C. Manfredi, “Acoustic analysis of newborn infant cry [20]. Schönweiler R, Kaese S, Möller S, Rinscheid A, Ptok M. Neuronal
signals”, Med. Eng. Phys, vol. 20, pp. 432-442, 1998. networks and self-organizing maps: new computer techniques in the
[7]. Kheddache Y, Tadj C. Resonance Frequencies Behavior in Pathologic acoustic evaluation of the infant cry, Int. J. Pediatr. Otorhinolaryngol.
Cries of Newborns. Journal of Voice 2014. 38 (1) (1996) 1–11.
http://dx.doi.org/10.1016/j.jvoice.2014.04.007