=Paper=
{{Paper
|id=Vol-2540/paper54
|storemode=property
|title=None
|pdfUrl=https://ceur-ws.org/Vol-2540/FAIR2019_paper_65.pdf
|volume=Vol-2540
}}
==None==
https://ceur-ws.org/Vol-2540/FAIR2019_paper_65.pdf
Operational classifier development using
quasi-open set semi-supervised training and
GANs
Emile Engelbrecht1 and Johan du Preez2
1
University of Stellenbosch, Stellenbosch, Western Cape, South Africa
18174310@sun.ac.za
2
University of Stellenbosch, Stellenbosch, Western Cape, South Africa
dupreez@sun.ac.za
1 Extended Abstract
Deep learning methods for classifier development have been found wanting when
translated to real-world applications. The most notable drawbacks of deep learn-
ing is the cost associated with gathering annotated samples and the assumption
that all is known about the domain. Within this work we study and alter deep
learning techniques to develop operational-grade classifier models whilst address-
ing these two limitations.
For classification, it is assumed that each data sample encountered by the
model will belong to some category/class in the domain which can be determined
through studying of the input samples. Some classification examples are:
– The introductory machine learning MNIST digit data-set contains images of
hand-written digits from 0 - 9. A classifier model would be required to study
an input image and determine its corresponding digit (0, 1, 2 ..., 8 or 9).
– Symptoms of patients are used to determine their corresponding disease.
A classifier model could be built to study symptoms and predict diseases
automatically.
In the developing world where the majority of the population cannot afford
doctor visits due to lack of doctors and/or money, having a classification system
capable of quick free diagnoses is of immense benefit. This example is one of
many which shows the potential of the 4th industrial revolution.
Deep learning has in recent years seen growing success in training classifier
models. Typically deep models are developed/trained using two learning regimes,
supervised or semi-supervised learning.
Supervised learning requires that each input sample used to train the model
has to be annotated/labelled to indicate which class it belongs to. Models then
study (many) labelled samples and learn the differences between the various
classes in the domain. Given that the model trained well, new never-before seen
input data can be accurately classified during the operation/testing phase. Su-
pervised learning has shown high accuracy scores with deep neural networks
given that a large number of annotated samples are provided [1] [2]. Manually
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0)
2 Emile Engelbrecht and Johan du Preez
annotating samples within applications, however, is expensive which lead to the
development of the second learning regime, namely semi-supervised learning.
Semi-supervised learning research has achieved similar accuracy results to
supervised learning while requiring much less labelled data [3]. This is due to
models learning from both labelled and unlabelled data simultaneously. Labelled
data samples make known the various classes to the system, whilst unlabelled
data aids in learning class properties. Semi-supervised learning consequently
addresses the cost issue relative to manually annotating data samples. For both
supervised and semi-supervised learning, however, it is still required that all
classes in the domain be known by the system.
Within real-world application domains there might be (many) classes which
are not known to designers. For the symptoms/disease example above, unknown
classes might be the symptom-disease correlations yet to be discovered by doc-
tors. In such cases there is no certainty as to whether an unlabelled sample
corresponds to one of the classes known to the system or not. This violates the
assumption of semi-supervised training, causing it to break down. It still, how-
ever, remains fundamental for operative classifiers to classify over all classes in
the domain, known or unknown, as samples from all classes will be encountered.
To the best of our knowledge, no work has addressed a semi-supervised learn-
ing scenario where unlabelled samples might also belong to categories outside
of those known to the classification system. We therefore first formally define
this setting, which we coin a quasi-open set, after which we propose a learning
regime to handle quasi-open sets. This learning regime is called quasi-open set
semi-supervised training and requires known classes be correctly classified whilst
simultaneously requiring unknown classes (only seen in the unlabelled data) be
classified as ’other’.
Training under quasi-closed semi-supervised learning develops a model ca-
pable of classifying over all classes in the domain even though each class might
not be explicitly expressed to the system. Models are therefore able to train
using vast unfiltered and unrestricted unlabelled sets (as would be available in
application) which was previously un-attainable using deep neural networks.
Our proposed method for quasi-closed semi-supervised learning uses genera-
tive adversarial networks (GANs) in a similar fashion to general semi-supervised
learning using GANs [3]. An additional framework is, however, added to handle
unknown classes. Experiments are done using MNIST by providing labelled sam-
ples for some classes and unlabelled samples for the same and different classes.
Example accuracy scores reach upward of 96.23% when 500 labelled samples
were provided for 7 classes whilst unlabelled data was provided for all 10 classes.
Our method is ensured to remain non-domain specific to allow ease of translation
to any application.
With this new learning regime and our proposed method, we are able to
build operational classifiers (capable of classifying over all classes in the domain)
without requiring each individual class be made known to the system. This
approach greatly extends the range of automated classification tasks that can
be addressed in a cost effective manner.
Quasi-closed semi-supervised learning 3
References
1. Makantasis, Konstantinos and Karantzalos, Konstantinos and Doulamis, Anastasios
and Doulamis, Nikolaos: ”Deep supervised learning for hyperspectral data classifi-
cation through convolutional neural networks”.2015 IEEE International Geoscience
and Remote Sensing Symposium (IGARSS), pg. 4959–4962 (2015)
2. Cireşan, Dan; Meier, Ueli; Masci, Jonathan; Schmidhuber, Jürgen: ”Multi-column
deep neural network for traffic sign classification”. Neural Networks. Selected Papers
from IJCNN 2011. pg. 333–338 (2012)
3. Dai, Zihang and Yang, Zhilin and Yang, Fan and Cohen, William W and Salakhut-
dinov, Ruslan R: Good semi-supervised learning that requires a bad gan. Advances
in neural information processing systems, pg. 6510-6520 (2017)