Artificial Intelligence has been a major breakthrough in many domains. Now, it has started automating health care domain through Natural Language Processing and Computer Vision applications. As a part of it, researchers are now focusing more on mining health related information from the text shared through social media and clinical trials. This paper explains about our system for health care text classification tasks conducted by Health Language Processing (HLP) Lab. We experimented with representing the target classes available in task 1 and task 2 as vectors. The classification has been performed using Support Vector Machine. To compute the representation for target classes, we used traditional methods available in Vector Space Models and Vector Space Models of Semantics. In this shared task, the task 1 is about distinguishing the tweets mentioning ”adverse drug reaction” from the ones which do not. The task 2 is about distinguishing the tweets that includes personal medication intake, possible medication intake and non-intake. The preliminary results are satisfying in-order to continue the research in developing a representation method for target classes.
The objective here is to represent the given tweets into its equivalent numerical representation in-order to carry out classification.
Document - Term Matrix (DTM) and Term Frequency - Inverse Document Frequency (TF-IDF) representation methods are used in which the given tweets T = t1; t2; t3; :::; tn are presented as a matrix D with the dimension m n. Here m represents the number of tweets and n represents the number of unique words present in the tweet collection T .
D = dtm(T )
D = tf idf (T )
U V T = svd(D) In the above equation, U represents the distributional representation of tweets with the dimension of m m, V T represents the distributional representation of the words with the dimension of n n and represents the significance of the basis vectors present in U and V T . In detail, column vectors in U are the Eigen vector of DDT which represents the column space, column vector in V T are the Eigen vector of DT D which in turns represents the row space and the diagonal element of are the squared Eigen values of DDT and DT D. The computation of DDT finds the cross co-occurrence of the words in the Matrix D. Finally, the resultant column vector in U is taken as D to for further steps.
We have experimented to represent the target classes as an entropy vector by summing up the tweets vectors available in the matrix D with respect to the target class. This can be mathematically represented as, In DTM the frequency count of the words alone are considered to form the representation for tweets3. In TF-IDF, along with the frequency count of the words, frequency count of the words appearing across the tweets (inverse document frequency) are also taken into the consideration4. This re-weighting scheme in TF-IDF gives higher weights to the rarely occurring word and lower weights to the frequently occurring word.
The matrix computed from the previous section undergoes matrix factorization to get the distributional representation of tweets. These vectors can be seen as a semantic representation of tweets, as the vector produced out of matrix factorization becomes the basis vector representation of matrix D. Here Singular Value Decomposition (SVD) is used to perform the matrix factorization5,6.
In above equation, Ce represents the class embedding (entropy vector of the class) , tc represents the target classes per
tweet and C represents the available unique target classes. The dimensions of the class embedding in Vector Space
Model representation is 1 n and 1 m in Vector Space Models of Semantics representation.
(
i=1
F = f eatures(D; Ce)
The distance, similarity and correlation between the class embedding and tweet vectors are measured to get the feature matrix in-order to perform the final prediction.
Here F is the feature matrix with the dimension m (5 numberof uniquetargetclasses). The measured features are Dot Product, Euclidean Distance, Chebyshve Distance, Bray Curtis Dissimilarity and Correlation7.
This section details about how the proposed approach is applied on Task 1 and Task 2 data sets. Task 1 is a binary classification problem8 and task 2 is a multi-class classification problem9. The dataset for both the tasks are provided by shared task organizers and its statistics are given in Table 1 and Table 2. Each task’s data set includes training data, development data and test data.
Train Dev Test The tweets in the given datasets are represented as a matrix using methods described in VSM and VSMs sections. The available target classes per class is mentioned in Table 1 and Table 2. The submitted runs varies only with representation but further classification remains same for all the runs. In task 1, the given data is represented as DTM in run1, TF-IDF in run2, DTM followed by a SVD in run3 and TF-IDF followed by a SVD in run4. While performing SVD we have taken column vectors from the U as a basis vector representation for tweets. The dimension of the vectors is equal to the number of instances. Similar to task 1, task 2 is also computed with the four types of representations.
The class embedding for target classes are computed by summing up the tweet vectors that belonged to the respective classes. In this way, for task 1 we have computed two class embeddings (ADR mentioned and ADR not mentioned). For task 2, we have computed three class embeddings (personal medication intake, possible medication intake and non-intake).
On successive computation of class embeddings, the features are computed between the tweet vectors and class embeddings as mentioned in Feature Learning Section. These measures are taken as the attributes and given to the classifier to make the final prediction. In task 1, one class SVM is used to handle the label biasing problem. In task 2, SVM with RBF kernel is used to make the final prediction.
In task 1, it has been observed that except TF-IDF, the other representation methods shows higher error in training the one class SVM with the ADR mention. Based on this, in submitted runs the training model is based on the tweets in which the ADR is not mentioned. The observed training error rate for task 1 is given in Table 3. In task 2, applying SVD tends to appear as the over fitted model by giving constant accuracy for 10 - cross 10 - fold validation. Hence, we avoided to submit the multiple runs for task 2. We have submitted the model based on DTM and class embedding. The final submitted runs were evaluated by the shared task organizers and the obtained results are given in Table 4 and Table 5.
The preliminary approach to class representation method attains considerable accuracy in both the tasks. It has been observed that the imbalance in the target classes is the core reason for low score. Especially in the proposed class 1 2 3 4
ADR not mentioned 930 ADR mentioned 930 ADR not mentioned 195 ADR not mentioned 856 Table 4: Task 1 Results
0.057 0.056 0.087 0.186
0.093 0.109 0.204 0.481 Table 5: Task 2 Results
0.071 0.074 0.121 0.268
for classes 1 and 2 0.569
for classes 1 and 2 0.462 representation the entropy of the target class vector is directly dependent on the number of instances that belonged to the respective class. Hence the future work will be to focus on handling the label biasing problem, which is a common scenario with many practical applications.