=Paper=
{{Paper
|id=Vol-1507/dx15paper35
|storemode=property
|title=Processing Measure Uncertainty into Fuzzy Classifier
|pdfUrl=https://ceur-ws.org/Vol-1507/dx15paper35.pdf
|volume=Vol-1507
|dblpUrl=https://dblp.org/rec/conf/safeprocess/MonrousseauTL15
}}
==Processing Measure Uncertainty into Fuzzy Classifier==
https://ceur-ws.org/Vol-1507/dx15paper35.pdf
Proceedings of the 26th International Workshop on Principles of Diagnosis
Processing measure uncertainty into fuzzy classifier
Thomas Monrousseau1 , Louise Travé-Massuyès1 and Marie-Véronique Le Lann1,2
1
CNRS, LAAS, 7 avenue du colonel Roche, F-31400 Toulouse, France
2
Univ de Toulouse, INSA, LAAS, F-31400 Toulouse, France
e-mails: thomas.monrousseau@laas.fr, louise@laas.fr ,mvlelann@laas.fr
Abstract data [5] [6] [7], the use of fuzzy logic type-1 or type-2 [3]
or statistical models.
Machine learning such as data based classification Fuzzy logic is a multi-valued logic framework intro-
is a diagnosis solution useful to monitor complex duced by Zadeh [8] that is known to be more efficient for
systems when designing a model is a long and ex- representating uncertainty and impreciseness than binary
pensive process. When used for process monitor- logic. In previous work, a fuzzy classifier named Learning
ing the processed data are available thanks to sen- Algorithm for Multivariate Data Analysis (LAMDA)
sors. But in many situations it is hard to get an ex- has been proposed by Aguilar [9]. This classifier can
act measure from these sensors. Indeed measure originally process simultaneously two different types
is done with a lot of noise that can be caused by of data: quantitative data and qualitative data. A real
the environment, a bad use of the sensor or even number contains an infinite amount of precision whereas
the conversion from analogic to numerical mea- human knowledge is finite and discrete, thus LAMDA is
sure. In this paper we propose a framework based interesting because there is no solution proposed in the
on a fuzzy logic classifier to model the uncertainty literature to process in a uniform way heterogeneous data
on the data by the use of crisp (non fuzzy) or fuzzy and to handle in a same problem quantitative data and
intervals. Our objective is to increase the num- qualitative data is often a complex subject. A new type
ber of good classification results in the presence of data, the interval, has been introduced by Hedjazi [10]
of noisy data. The classifier is named LAMDA to model uncertainties by means of crisp intervals. In this
(Learning Algorithm for Multivariate Data Anal- paper we propose an extention to fuzzy intervals in order to
ysis) and can perform machine learning and clus- improve its application to process noisy data measurements
tering on different kind of data like numerical val- but with the capacity to handle others features types like
ues, symbols or interval values. “clean” data or qualitative features. Moreover the algorithm
should stay low cost in term of memory and computation
1 Introduction time to enable the method to be embedded on small systems.
Data classification is the process of dividing pattern space
using hard, fuzzy or probabilistic partitions into a number In the first part of the paper the LAMDA algorithm is
of regions [1]. Classification algorithms are more and more shortly presented then in a second time a method to use the
used nowadays in a world where it is not always simple to algorithm to classify noisy data is introduced. This method
get a model of complex process. On the opposite it is easier is in two parts: the first presents a general solution to model
to get data on systems by monitoring and store it. Differ- uncertainty on data with crisp intervals based on confidence
ent types of classifiers can be used depending on the sit- intervals and the second shows an improvement to model
uation. The principal ones described in the literature are Gaussian noise with fuzzy intervals. In both cases examples
artificial neural networks, k-nearest neighbors, support vec- of application are introduced to show the improvement of
tor machine, decision trees, fuzzy classifiers and statistical the method compared to the use of the data without trans-
methods. formation.
Most of the time, data are issued from sensor measure-
ments and are corrupted by noise. This noise can have dif- 2 LAMDA algorithm (Learning Algorithm
ferent origins, for example environment disturbances, bad for Multivariate Data Analysis)
use of the sensor, hysteresis effect or numerical conversion
and representation of the data. Many domains of applica- This section presents the principle of the LAMDA algo-
tion have to deal with noise problems like medical diagno- rithm.
sis [2], biologic identifications [3] or image recognition [4].
Uncertainty can be understood in two ways: the first is the 2.1 General principle
uncertainty directly present in the data like noise and the LAMDA is a classification algorithm based on fuzzy logic
second can be assimilated as the reliability of a feature in- created on an original idea of Aguilar [9] and can achieve
side a class. In this paper we consider only the first case. To machine learning and clustering on large data sets.
avoid noise problems in classification some solutions have The algorithm takes as input a sample x made up of N
been provided previously, for example the transformation of features. The first step is to compute for each feature of x, an
269
Proceedings of the 26th International Workshop on Principles of Diagnosis
Where xn is the n-th feature of the sample x, ρj,n is
the mean of the n-th feature for the class j and σj,n is
the standard deviation of the n-th feature for the class j.
• Qualitative data:
Qualitative can take values in a set of modalities. The
membership function of qualitative data returns the fre-
quency of modality taken by the feature into the class
during the learning phase. We introduce a qualitative
variable with K modality {Q1 , ..., QK } and the fre-
quency Φkj of the modality Qk for the class j. The
membership is described by:
f (xn ) = (Φ1j,n )q1 ∗ ... ∗ (ΦK
j,n )
qK
(4)
Figure 1: Summarized scheme of the LAMDA algorithm k
q = 0 if xn 6= Qk
with
adequacy degree to each class Cj , j = 1..J where J is the q k = 1 if xn = Qk
total number of class. This is obtained by the use of a fuzzy
adequacy function. So J vectors of N adequacy degrees • Intervals:
are computed, these vectors are called Marginal Adequacy
Degree vectors (MAD). At this point, all the features are in The membership function for interval data is a function
a common space. Then the second step is to take all the which tests the similarity between two fuzzy intervals.
MADs and aggregate them into one global adequacy degree In this case similarity is defined by two components:
(GAD) by means of a fuzzy aggregation function. Thus the the distance between the intervals and the surface that
J MAD vectors (composed of N MADs) become J scalar these intervals have in common. Indeed the class pro-
GADs, the higher the GAD, the better the adequacy to the totype for crisp interval data is a mean interval. The
class. The simplest way to assign the sample x to a class is similarity function is:
to keep as result the class with the biggest GAD.
All the process is summarized in Fig. 1. R
1 V µA∩B (ξ)dξ ∂[A, B]
S(A, B) = ( R +1− ) (5)
2.2 Fuzzy membership computation 2 V µA∪B (ξ)dξ $[V ]
During the learning step, the algorithm creates prototype
data for each class and for each feature. These data are
called classe descriptors or prototypes; they can be for ex- where µX (x) is the value of x in the fuzzy set X,
ample means or variances. We define as Cj,n the class pro- ∂[A, B] is the distance between intervals A = [a− , a+ ]
totype of the n-th feature for the class j. and B = [b− , b+ ]and $[X] is the size of a fuzzy set
As previously mentioned the first step of the algorithm into a V universe. This is described by:
is a comparison between the sample vector x and all the Z
Cj,n . This operation is performed with membership func- $[X] = µX (ξ)dξ (6)
tions and gives as result a membership adequacy degree. V
Thus M ADj,n is the MAD for the j-th class and the n- In the case of crisp intervals and in a universe between
th feature. As the framework is based on fuzzy logic, all 0 and 1:
memberships are numbers in the [0,1] interval. The general
membership function is: 1 $[A ∩ B]
S(A, B) = ( + 1 − ∂[A, B]) (7)
2 $[A ∪ B]
M ADj,n = f (Cj,n , xn ) (1)
The class prototype Cj,n depends on two things: the type where $[X] in this case can be replaced by the length
of data and the function used. Some functions may require of the interval:
only one data into Cj,n whereas others need a list of param-
eters. $[X] = upperbound(X)-lowerbound(X) (8)
In the following section, some examples of membership
functions are presented. and distance ∂[A, B] is defined as:
• Quantitative data: ∂[A, B] = max[0, max(a− , b− )−min(a+ , b+ )] (9)
Many functions are available for this kind of data. For In the case where an interval feature is used the pro-
example the Gaussian: totype for a class j is given by [ρn− n+ n−
j , ρj ] where ρj ,
n+
(xn − ρj,n )2 respectively ρj represents the mean value of lower
− 2
2σj,n bounds (respectively upper bounds) of all the elements
f (xn ) = e (2) belonging to class j for this feature.
Once the MAD are computed whatever the feature
or the binomial function: type, it is possible to perform any type of processing
f (xn ) = ρxj,n
n
.(1 − ρj,n )1−xn (3) as described on Fig. 2
270
Proceedings of the 26th International Workshop on Principles of Diagnosis
of confidence (for example a confidence interval of 95% is
an interval in which the exact value of the measure can be
found with a probability of 95%). Introducing x̂ the mea-
sured value and l the length of a centered on zero confi-
dence interval based on the measurement error, the interval
used by the algorithm is calculated: X = [x̂ − 2l ; x̂ + 2l ].
The main aim of the transformation is to improve the clas-
sification on the transition zones where data is really sensi-
tive to noise and a small change can modify the output of the
classifier. The use of intervals to model uncertainty is effec-
tive only if the “clean” data is relevant for the classification
problem. If it is not the case a better solution is to remove
the irrelevant feature. It will in most cases provide better
output results. This expresses the fact that if the “clean”
data is difficult to classify it is not improved by using confi-
dence intervals.
3.2 Experiments
A set of data has been created for an application test which
Figure 2: Projection principle for heterogeneous feature can be interpreted as sensors time evolution of a continuous
types process. This set of data is composed by three quantitative
(numerical) features of 101 samples that are shown on the
Fig. 3. Three classes are specified and used as targets for
2.3 Marginal adequacy degree merging the classifier. These classes are chosen arbitrarily to repre-
Once all the features are grouped into the membership space sent different behaviors of a system that could be healthy
the next step of the algorithm is to transform the MAD vec- or failure modes. Nevertheless the classes are built to make
tors into a set of single value which depicts the global mem- all the data relevant for the system monitoring which means
bership of the sample to a class. These values were intro- the three features do not have a global negative impact on
duced in section 2.1 and are called GAD. To perform this the classification results.
transformation a fuzzy aggregation function Ψ is used. The three features x, y and z are defined by the following
The aggregation function is the following: time functions:
Ψ(M AD) = α.γ(M AD) + (1 − α).β(M AD) (10) −t
• x=e 2
where γ is a fuzzy T-norm and β is a fuzzy T-conorm. t
α parameter is called exigency indicator. It enables to give • y = 12 · e 4 − 1
more or less significance to the union operation and the in- • z = tanh(t − 5)
tersection operation. Two fuzzy T-norm and T-conorm are
currently implemented in the algorithm, the min-max and
the probabilistic. For example if min-max is used, (10) be-
comes:
Ψ(M AD) = α.min(M AD)+(1−α).max(M AD) (11)
When all GAD are computed they give the membership
of the data x to each class. The final result depends on the
application but the simplest way to give a result is to class
the sample in the class which has the highest GAD. A limit
membership can also be fixed: if no GAD is higher than the
limit, the sample is defined as unclassifiable.
3 Uncertainty modeled with crisp intervals
3.1 Method presentation
Every data measurement is performed with noise. In some
cases noise has enough bad effect to increase the error of
classification. Thus the point is to model the imprecision of
the data to decrease the number of bad classifications.
A technique used in several fields of application is the
use of intervals to symbolize data uncertainty [11] [12]. So
we are suggesting a framework where numerical data are Figure 3: Data used to test the intervals method
transformed into intervals to model imprecision.
In a situation where the probability law followed by the This example is used to measure the improvement in the
noise on a variable is unknown, it may be possible to ob- classification results in the case of all data are noisy. Artifi-
tain a confidence interval. It is an interval in which the cial noise is added by the following: x is the ideal variable
real value of the measure is present with a certain amount without noise and x̂ the noisy variable, x̂ = x + Y with
271
Proceedings of the 26th International Workshop on Principles of Diagnosis
Figure 5: Example of approximation of a Gaussian fuzzy
interval by a triangular fuzzy interval
Figure 4: An example of data corrupted with a noise in the
interval [-0.5 ; 0.5]
4 Modeling Gaussian noise with fuzzy
intervals
Y a random variable following a uniform distribution on an 4.1 Fuzzy interval method presentation
interval I. Most of the time, noise on physical measure follows a Gaus-
sian distribution centered on the real value. Thus it is inter-
The experiment has been performed with these condi- esting to model this specific kind of uncertainty. Neverthe-
tions: α parameter of (10) is set at 0.8 with the [min,max] less, it is difficult to handle fuzzy intervals with an exact
functions to compute the fuzzy aggregation and the mem- Gaussian shape. That is why we suggest approximating the
bership function used for quantitative data is the bino- Gaussian with a triangular fuzzy interval. This interval is
mial.[min, max] aggregation is chosen because experiments described with a lower boundary x− and an upper boundary
on the algorithm showed that this kind of aggregation pro- x+ : X = [x− ; x+ ] which leads to a similar description as
vides better results on noisy data that the probabilistic one. crisp intervals. So:
A first classification without any noise gives a result of 91% µX (x− ) = 0 and µX (x+ ) = 0 and µX ( x +x
+ −
)=1
of good classification. Then the experiment is repeated a 2
great many times to avoid statistical mistakes. In this case, with µX (x) the fuzzy value of x into the fuzzy set X. As
the experiment has been run fifty thousand times, x̂ is re- a Gaussian of ρ mean is centered on the true measure value
computed at each new run. Results are given on table 1. + −
the maximum fuzzy value of the triangle x +x 2 is equal to
ρ. To compute x− and x+ we propose to use the full width
at half maximum (FWHM) that can be calculated this way:
Interval for ran- [-0.3 ; 0.3] [-0.5 ; 0.5] [-2 ; 2]
p
dom data F W HM = 2 2ln(2) · σ (12)
Mean success
with σ that is the standard deviation of the measure.
percentage 89.9% 84.7% 79.6%
Thus for a Gaussian function that has a mean value ρ and a
with binomial
standard deviation
p σ the approximated
p interval X is defined
function
Mean success by X = [ρ − 2 2ln(2) · σ; ρ + 2 2ln(2) · σ]. An example
percentage with 91.9% 89.8% 70.3% of this approximation is given on Fig. 5.
interval function
Until now all the implementations of the LAMDA algo-
Table 1: Table of results for the crisp intervals method rithm were using only crisp intervals despite the fact that
the general method was introduced. The class prototype is
now a triangle interval computed with the means of upper
and lower boundaries of the data used to train the algorithm.
Thus the membership function is still a similarity measure
As it can be seen, this method provides an improvement
between two fuzzy intervals like in (5) but it is necessary to
on the results in the two first cases where noise deteriorates
redefine the distance function between the intervals. A solu-
the classification with the quantitative method but when the
tion has been proposed to measure a distance with the center
data is still globally consistent. In these cases, the intervals
of gravity of triangular fuzzy intervals [13]. In the present
method gives better results than binomial method 82% of
situation:
the time. But when noise amplitude is much higher than the
data like in the [−2; +2] error interval, the interval method a+ + a− b+ + b−
does worse in general than the binomial function. ∂[A, B] = | − | (13)
2 2
272
Proceedings of the 26th International Workshop on Principles of Diagnosis
with A = [a− ; a+ ] and B = [b− ; b+ ], A and B being
triangular fuzzy intervals like described in this section.
The intersection A ∩ B needed in (5) is calculated with an
analytical solution based on geometry and trigonometry. It
avoids numerical integration that could be less precise and
longer to compute.
4.2 Experiments
As we did previously with the crisp method, a test is per-
formed with a Gaussian noise on the same data set (Fig. 3).
The test is done in the same conditions as in the previous
section. The difference is on the construction of the noisy
data x̂ = x + Y . Y is now a random variable that follows a
normal distribution of standard deviation σ and centered on
0. Results of the simulation are given on the table 2.
Figure 6: Representation of iris data by class
σ 0.2 0.5 0.7 1
Mean success
percentage 83.2% 79.8% 79.8% 79.6% The classifications are performed using the cross-
with binomial validation method. The percentages of well classified data
function for the two methods are:
Mean success
percentage with 86.8% 82.5% 77.2% 71.3% • using binomial function (scalar): 81.3%
crisp interval • using fuzzy triangular intervals: 94.0%
function
Mean success Once again the classification rate is increased by the use
percentage with 93.1% 84.5% 79.3% 74.8% of the fuzzy interval method instead of the binomial one.
fuzzy interval
function 5 Conclusion
We presented in this article two methods to model uncer-
Table 2: Table of results for the fuzzy intervals method
tainty for classification applications. An example showed
that these methods can improve classification results even
Similarly to the previous test, the interval method in- when the signal to noise ratio is high. The second method
creases the rate of good classifications until the standard de- based on fuzzy intervals demonstrated that try to model
viation σ becomes too high and the binomial function pro- more precisely the probability law of the noise can pro-
vides better results. This point is reached here for σ = 0.7 vide better results than use confidence intervals modelled
which corresponds to a signal to noise ratio (SNR) of 6 dB by crisp intervals. However this process to model uncer-
for the signal with the smallest amplitude. Also it is im- tainty reveals limits when the SNR reaches a low level. A
portant to notify that in all cases the fuzzy interval provides future important work is to limit the classification error of
better results than the crisp interval method. the interval method at the level of the numerical method.
These methods will now be tested on data out coming
4.3 Experiments on iris dataset from a real industrial process.
As a second example we use the classical iris dataset[14]. Another way to manage uncertainty on classifiers like
This dataset contains four features: sepal length in cm, LAMDA could be to use type-2 fuzzy functions [15]. This
sepal width in cm, petals length in cm and petal width is an expansion of classical fuzzy logic where the member-
in cm. All these features are measured for three types of ship functions give in output a fuzzy interval which can be
flower: iris Setosa, iris Versicolour and iris Virginica which used to model variance of the data.
constitute three classes. It is easy to classify without any To provide a better solution to manage uncertainty in the
error the iris dataset by using only the petals information LAMDA classifier it can be useful to extend the problem to
that are in general most relevant that the sepals ones. Thus the qualitative features. It is often difficult to determine if a
only the sepal sizes are kept in this test to simulate the qualitative element is close to another, for example the color
noise. The figure 6 shows the repartition of the data in the "orange" is closer to "red" than "blue". But on small training
2D space of the sepal features. dataset consider this kind of information can improve final
classification results. This could be done by using similarity
We assume that the data follow a normal distribution matrix which are already used in some artificial intelligence
centered on a mean µj,n and with a standard-deviation σj,n . problems.
This hypothesis can be verified by using a statistical test. LAMDA algorithm can work with a feature selection al-
The Kolmogorov-Smirnov test has been used for each class gorithm named MEMBAS (Membership Margin Based Fea-
with a 5% significance level, it shows that the hypothesis is ture Selection) [16]. This algorithm uses LAMDA classes
true for the iris Setosa and the iris Versicolour but not for definitions and its membership functions to provide an ana-
the iris Virginica. Nevertheless all the data are processed as lytical solution for the feature selection. A future work will
if they follow a normal distribution. be to measure the impact of the interval use on MEMBAS
algorithm to perform selection on noisy data.
273
Proceedings of the 26th International Workshop on Principles of Diagnosis
References [14] Fisher R.A. {UCI} machine learning repository, 1936.
[1] J. C. Bezdek. A review of probabilistic, fuzzy, and http://archive.ics.uci.edu/ml.
neural models for pattern recognition. Journal of Intel- [15] J.M. Mendel, R.I. John, and F. Liu. Interval type-
ligent and Fuzzy Systems, Vol. 1, No. 1:pp 1–25, 1993. 2 fuzzy logic systems made simple. Fuzzy Systems,
[2] E. Alba, J. Garcia-Nieto, L. Jourdan, and E. Talbi. IEEE Trans. on, Vol. 14, No. 6:pp 808–821, Dec. 2006.
Gene selection in cancer classification using pso/svm [16] L.Hedjazi, J.Aguilar-Martin, and M.V. Le Lann.
and ga/svm hybrid algorithms. In Evolutionary Com- Similarity-margin based feature selection for symbolic
putation, CEC 2007. IEEE Congress on, pages 284– interval data. Pattern Recognition Letters, Vol.32,
290, Sept. 2007. No4:pp. 578–585, March 2012.
[3] Scott Ferson, H. Resit Akqakaya, and Amy Dunham.
Using fuzzy intervals to represent measurement error
and scientific uncertainty in endangered species clas-
sification. In Fuzzy Information Processing Society,
1999. NAFIPS. 18th International Conference of the
North American on, pages pp 690–694, Jul 1999.
[4] Zhang Weiyu, S.X. Yu, and Shang-Hua Teng. Power
svm: Generalization with exemplar classification un-
certainty. In Computer Vision and Pattern Recognition
(CVPR), 2012 IEEE Conference on, pages pp 2144–
2151, June 2012.
[5] Arafat Samer, Dohrmann Mary, and Skubic Mar-
jorie. Classification of coronary artery disease stress
ecgs using uncertainty modeling. In Computational
Intelligence Methods and Applications, 2005 ICSC
Congress, 2005.
[6] Kynan E. Graves and Romesh Nagarajah. Uncertainty
estimation using fuzzy measures for multiclass classi-
fication. Neural Networks, IEEE Transactions on, Vol.
18:pp. 128–140, 2007.
[7] Prabha Verma and R.D.S. Yadava. Fuzzy c-means
clustering based uncertainty measure for sample
weighting boosts pattern classification efficiency. In
Computational Intelligence and Signal Processing
(CISP), 2012 2nd National Conference on, pages 31–
35, 2012.
[8] L.A. Zadeh. Fuzzy sets. Information and Control, vol.
8:pp. 338–353, June 1965.
[9] Carrete N.P. and Aguilar-Martin J. Controlling selec-
tivity in nonstandard pattern recognition algorithms. In
IEEE Transactions on Systems, Man and Cybernetics,
volume 21, pages 71–82. IEEE, Jan/Feb 1991.
[10] Hedjazi L., Aguilar-Martin J., Le Lann M.V., and
Kempowsky T. Towards a unfined principle for rea-
soning about heterogeneous data: a fuzzy logic frame-
work. International Journal of Uncertainty, Fuzzy-
ness and Knowledge-Based Systems, Vol. 20, No. 2:pp.
281–302, 2012.
[11] B. Kuipers. Qualitative Reasoning: Modeling
and Simulation with Incomplete Knowledge. The
MIT Press,Cambridge, Massachusetts, london edition,
1994.
[12] Lynne Billard. Some analyses of interval data. Journal
of Computing and Information Technology, CIT 16:pp
225–233, 2008.
[13] Hsieh C. H. and Chen S. H. Similarity of general-
ized fuzzy numbers with graded mean integration rep-
resentation. In Proceedings of the Eighth International
Fuzzy Systems Association World Congress, volume
vol. 2, pages pp. 551–555, Taipei, Taiwan, Republic
of China, 1999.
274