CIQ or Classi cation of Insincere Question task in FIRE 2019 focuses on di erentiating proper information seeking questions from di erent kinds of insincere questions. As a part of this task, we (team A3-108) submitted di erent machine learning and neural network based models. Our best performing model which was an ensemble model of gradient boosting, random forest and 3-nearest neighbor classi ers with majority voting. This model could correctly classify 62.37% of the questions and we secured third position in the task.
In recent years, community question answering forums have seen an upswing.
The number of users of such forums has recorded exponential growth. Di erent
toxic, malicious, hate related posts throw the biggest challenges to most of them.
In this task, an attempt has been made to lter out malicious content from the
forum of Quora 1 that will keep their platform more secured for users.
The task aimed at distinguishing true information seeking questions (ISQ) from
non-information seeking questions (NISQ). Six ne grained classes were designed
for this classi cation and distribution of them in the given training corpus is
shown in table 1. This task is motivated by an earlier task 2 which focused
on the binary classi cation of sincere questions from the insincere ones. The
current task is a ner counterpart of question classi cation posted at Quora. As
the statistics of the below table suggests, the dataset is a highly imbalanced one
where 2 classes constitute majority of the samples.
Preprocessing plays a vital role in tasks where the input data is in textual format.
We did not use any external tokenizer for tokenizing the input. The punctuations
were discarded and the white space acted as a delimiter between the words.
We used TF-IDF vectors at character and word levels for this task. We
experimented with classi ers individually as well as their ensembles. Di erent voting
procedures were also tried out. In hard voting, the class labels are predicted
based on majority voting among the participating classi ers. In the case of soft
voting, the voting classi er picks out the maximum of the sums of the predicted
probabilities computed for the constituent classi ers. The following were
implemented using scikit-learn [
{ Linear SVM { Multinomial Naive Bayes (mNB) { Adaboost (Adaptive Boosting) { Gradient Boost (GB) { Random Forest (RF) { k-Nearest Neighbor (k-NN) { Voting Classi er We tried various combinations of word and character level n-grams for the classi cation. By performing grid-search, we observed that combining both word unigrams and bigrams outperformed character level n-gram TF-IDF vectors as well as the combination of character and word level n-grams. The nal submission was a hard voting classi er consisting of gradient boosting, random forest and 3-nearest neighbors classi ers.
Neural Network Models
We have also experimented with neural network based sequential classi ers,
where we utilized word level features as inputs to the LSTM [
Di erent classi ers were trained to predict the class of each question. We include
the top performing system outputs in table 2.
We could observe that boosting methods Gradient boosting and Adaboost [
Based on above results, we try to automatically analyze training data to
understand the di cultly present in the Community Question Answering task.
For that, with basic tokenization and cleaning we applied LDA [
We also used LDA model to analyze training text by plotting them using
T-SNE [
We presented our supervised approaches for the FIRE task of classi cation of insincere questions (CIQ) in Quora for English. From our experiments, we can argue that for low resource and imbalance task such as CIQ, traditional machine learning algorithms with feature engineering outperform recent neural network based approaches. Adaboost classi er with word unigram and bigram TF-IDF features performed the best among all the classi ers. Huge amounts of unlabeled questions from Quora can be explored to improve the clustering techniques.