In automated health services based on text and voice interfaces, there is need to be able to understand what the user is talking about, and what is the attitude of the user towards a subject. Typical machine learning methods for text analysis require a lot of annotated data for the training. This is often a problem in addressing specific and possibly very personal health care needs. In this paper we propose an active learning algorithm for the training of a text classifier for a conversational therapy application in the area of health behavior change. A new active learning algorithm, Query by Embedded Committee (QBEC), is proposed in the paper. The methods is particularly suitable for the text classification task in a dynamic environment and gives a good performance with realistic test data.
The application context of the current paper is the development of automated
therapeutic conversational interventions for behavior change [
Recurrent neural networks are popular for text understanding but they require a large corpus of labeled training data, which is difficult to collect. In addition, natural language communication is an example of a non-stationary learning environment where the evolution in the conversational culture over time and populations require a local customization and maintenance of the classifier, possibly even at the level of an individual customer.
One approach for the maintenance and continuous improvement of a classifier in the
production environment is to use active learning (AL) methods [
In this paper we demonstrate an application of active learning in the classification
of short text messages, tweets, from a social media platform using a text classifier based
on recurrent Neural Networks, RNNs [
In pool-based active learning [
The AL process starts with an initial set P0 of labeled tuples of K feature vectors xk and corresponding labels lk, i.e.,
P0 = fxk; Lkg; k = 0; ; K 1 (1) The Initial classification model M0 is developed using P0. Next, a new set S1 is selected from the pool. The samples are manually labeled by a human oracle, for example, a health counselor. The new training data P1 is produced by adding the samples S1 to P0. The model is updated and deployed. The same update cycle can then be continuously repeated.
The selection of the next batch Sj+1 of B samples can be based on many different criteria. The minimum requirements for a jth iteration are 1. novelty: Pj \ Sj+1 = ; 2. richness: xn 6= xm; 8n; m 2 Sj+1 i.e., the B new samples in Sj+1 should be novel and they should be different from each other. 2.1
In the popular Query-by-Committee (QBC) method [
dj = D [C0(xk); C1(xk); ; CR(xk)] (2) where D[] is some measure to compute the disagreement.
In a typical case, the disagreement is based on vectors of class likelihoods given by the classifiers pr(xk) = Cr(xk). In a committee of two classifiers, the disagreement can be defined as a norm of the difference dk = jp0(xk) p1(xk)j.
The committee often disagrees on very similar samples and therefore the basic
algorithm does not provide the required richness for the new sample collection. A pareto
optimal solution is needed to meet both the novelty and richness conditions. The
richness is related to the nearest neighbor problem (NN). The k-nearest-neighbor searching
problem (kNN) is to find the k -nearest points in a dataset X Rd containing n points
to a query point q 2 Rd under some norm. There are several effective methods for
this problem when the dimension d is small (e.g. 1,2,3), such as Voronoi diagrams [
The selection of the new samples based on a disagreement of a committee assumes
a certain variability among the committee members [
In this paper the proposed method is to use the committee in another feature space derived from the outputs of the model. A multi-class classifier is often developed using the one-hot encoding principle where the classifier produces a vector of class likelihoods pr(xk) = M (xk) for a feature vector xk. The likelihoods represent the class predictions for the testing data. In geometric sense the likelihood vectors pr(xk) span an orthonormal space, a class space of the current classifier, where each axis represents a class.
The proposed method is a variation of the QBC method where selection task is performed in the class space of the current classifier. The class space is a metric lowdimensional space and there the committee can be based on conventional classification tools, e.g., based on a random forest or other relatively light algorithm. The training of the committee of classifiers and testing of them on a new data can be performed in an end-user device. Therefore, this enables a local active learning of the classification model.
The processing steps of the proposed method are described in Algorithm 2 below.
1: repeat 2: Use classifier Mj to get class likelihood vectors for data pr(xk)8xk 2 Pj 3: Use QBC method defined in the class space to select the new samples for labeling. 4: Add new samples to the training data Pj+1; j = j + 1 5: until Stopping criteria are met.
In this paper we call the modified method Query by Embedded Committee (QBEC), to separate it from the conventional QBC. In the current paper the committee is embedded in the class space. Naturally the same can be performed also in another output space, for example, corresponding to intermediate layers of the network. 3
Let us start with a synthetic example to illustrate the differences between QBC and the proposed QBEC method, and their benefits over random sampling from the pool. 3.1
The original synthetic data is shown in Fig.4(a) with two classes illustrated by red crosses and blue circles. A random forest (RF) classifier was designed for the set P0 with 100 labeled samples. In the QBC method a committee of two RF classifiers was designed using a different initialization. The selection of new samples was based on selection of the samples with the largest difference in the class likelihood values between the classifiers. Fig. 4(b) shows an example of a QBC sampling. A random sampling used in the reference condition is illustrated in Fig. 4(c). The QBC method clearly takes more samples from the class borders that the random sampling method. The QBEC method also focuses on the class borders but puts more emphasis on the borders between classes rather than outer borders.
The accuracy in the training in the three methods is shown in Figs. 2(a) and 2(b). The QBC and QBEC has a similar performance in the first batches but the accuracy of QBEC method keeps improving at the point where the performance of QBC saturates. This may be understood in this case by comparing the selections in the two methods in Fig. 4(b). In the QBCSC the sampling focuses on borders between the classes while in the conventional QBC solution a large number of samples are selected from the outskirts of the feature space which is less relevant for the class confusions measured by the accuracy.
In this paper the content is from Twitter, which is a popular short-text messaging platform. The content was selected by keywords that relate to smoking and tobacco use. In the typical flow of content the test system gave approximately 1000 tweets per day when excluding repetitions (re-tweets) of the same message.
In the current paper the content is manually classified into three classes: sustain
talk, change talk, and neutral communication. The two first classes are considered
important elements in many therapeutic techniques for substance abuse, such as CBT [
In this paper we use a typical architecture for a text classifier based on a state-of-the-art deep learning RNN tools. The text classifier model has six components, presented in Table 1.
The embedding layer is meant to map each word of the input text into a low dimensional embedding vector, while the bidirectional layers get higher level features from the input, dropout being used for regularization. The hard attention layer is used for global re-weighting of hidden layers and the desired class label is chosen using regular dense layer with softmax activation. 3 The ethical and legal approval of the data collection was granted, and handled according to, by the Internal Committee for Biomedical Experiments (ICBE) of Philips. The initial classifier C(0) was trained using a manually classified set of 2398 tweets. Examples of typical tweet types and their counts in the initial training set are shown in Table 2. In addition, an independent test data set with manually labeled tweets was used for testing. The performance of C(0) in an independent training set is poor; the accuracy is barely above 0.5. In the following experiment the active learning process was executed sequentially so that the current dump of tweets about the target topic was downloaded once a day, classified using classifier C(n). Approximately one percent, typically around 30 tweets, were selected to the manual labeling using one of the selection methods. The samples were manually labeled and included in the training set, and subsequently used in the training of the next model C(n+1).
The numbers of new labeled samples resulting from daily 23 iterations in the three methods are illustrated in Figs. 4a-c. First, it seems that random selection rarely picks samples from change talk category while those are much more common in the two other methods. The accuracies in the three methods are shown in Fig.4d. In the random selection the accuracy does not clearly improve over the iterations but in the two other methods there is a clear improving trend. Unlike the results with the checker board data, there is not really a difference in accuracy between QBC and QBEC, although, the computational requirements and processing time in QBEC is obviously significantly lower than in QBC. In this paper we propose a new algorithm for active learning in the application of text classification in the application of health counseling. The training of a a classifier for a specific complex talk type requires a large labeled data base which is typically difficult, expensive, and time consuming. There may also continuous concept drift in the target area, for example, due various cultural influences.
A popular approach for active learning is to use a disagreement in a committee of classifiers to select samples for manual labeling and inclusion into the training. These methods are commonly called Query-by-Committee (QBC) methods. The QBC methods require that multiple classifiers are trained for the task. In applications where classifier is complex, a.e.g, a deep neural network model, and requires a long training time this may be problematic. In the method introduced in the current paper the committee selection is performed in a low dimensional space spanned by the likelihoods of the current classifier model. In this case the actual classifiers of the committee can be fairly simple. The method is called Query-by-Embedded-Committee (QBEC).
We demonstrate the performance of QBEC first using synthetic data. The performance of QBEC turns out to be superior to the random selection of training samples and it, surprisingly, exceeds the performance of QBC. One may speculate that this is because the embedding based on the prediction likelihoods inherently zooms the committee to zoom into areas where the disagreement is largest.
In a second experiment we trained a complex classifier for classification of tweets related to smoking behavior into three classes. The classes represent change talk, sustain talk, and neutral communication of the writer in relation to tobacco use. This is a very challenging classification problem requiring a large labeled data base. In the active learning experiment 1% of tweet content downloaded on each day was manually labeled and included in the new model. It was shown that QBC and QBEC outperform random selection of samples. However, the results of the two methods are similar. However, it should be noted that the computational of QBEC is significantly lower than in QBC. Therefore, the sample selection in QBEC could be performed even in a customer device such as a smart phone.