Mental health is important at every stage of life in the modern world. The eRisk gives two tasks on detection of depression and anorexia respectively. In this paper, we do not take any temporal information or other corpus to support the tasks. We employ a TF IDF with SVM as classi er, a CNN+LSTM based deep neural network and a simple keywords based method to very whether those methods can learn the mental information from sparse space with unbalanced small data sets. The task results show the simple keywords model gets the best results in task 1 of f1 0.47 and the best recall of 0.71. In task 2, the simple keyword model gets the best recall score of 0.76, CNN+LSTM model gets the best f1 score of 0.36.
Mental health is important at every stage of life in the modern world. According
to WHO, globally, more than 300 million people su er from depression, the
leading cause of disability. More than 260 million are living with anxiety disorders.
Many of these people live with both[
Since we are in the Internet era, many people submit their comments, texts,
photos or videos to the social networks. For the people with mental health
problems, it is easier to analyze their mental situation through the memories written
in the social networks. CLEF 2018 gives two sub tasks in eRISK(early risk
prediction on the Internet) task[
As we took part in the task from April, we don't have enough time to train complex models. To the time sequences prediction problems, temporal feature is one of the important aspects to be considered, external corpus or more complex semantic models are also e cient. In this paper, we deal with this task in simple ways to explore whether simple models or some key features will work for this time correlation task without special temporal features considered. To achieve these goals, we construct a traditional CNN+LSTM based deep neural network, employ TF IDF represented features on SVM model and speci c keywords selected method respectively. We only submit the tenth chunk, and the nal chunk scores show some exciting results.
The remainder of this paper is organized as follows: Section 2 gives related works. The three veri ed method will proposed in Section 3. Section 4 presents the results. Section 5 makes the conclusion and future work. 2
Emotion recognition related tasks have been held for years in di erent
conferences or workshops. For example, sentiment analysis in Twitter[
The eRisk 2018 contains two sub-tasks, one is Task 1: Early Detection of Signs of Depression, and the other one is Task 2: Early Detection of Signs of Anorexia. The tasks are both running on the contents crawled from social networks with temporal tags. Our team takes part in the two sub-tasks, we propose three methods to detect the early signs of mental disorder.
Keywords model Marked as TUA1B in Task 2 and TUA1C in Task 1 is a
simple model in which using some emotional words as targets to measure the
authors' situation. Research in [
TF IDF with SVM Marked as TUA1D in Task 1 and TUA1A in Task 2.
This is a traditional method to assess the situation of mental health. We train
the SVM model using a linear kernel API of sklearn[
CNN+LSTM model Marked as TUA1A and TUA1B in Task 1, TUA1C in
Task 2. We construct a CNN+LSTM based deep neural network as the
detection models which can be seen in Table 1 using keras[
In this model, the contents of every chunk are preprocessed into one-hot features with an index value of the corresponding word in vocabulary instead of the word itself. Before feeding into the network, adding a padding process to format the length, the "maxlen" is selected as 2000. Other hyperparameters chosen for this model are as follows: "input length" of Embedding is set to the length of vocabulary, Task 1 is 429700 and Task 2 is 156593, "embedding size" is 128; The dropout factor is 0.25; For convolution layer, the " lters" is 64, setting "kernel size" to 5, "padding" with "valid", "strides" with 1 and "activation" is "relu"; Giving MaxPooling the parameter of 4 as "pool size"; For LSTM layer, the output dimension is 70; The following Dense layer is a one cell classi cation layer with "Activation" being "sigmoid". For the compiling, choosing "binary crossentropy" as "loss" and "adam" for "optimizer". The "metrics" is the default setting as "accuracy". Epoches is added as 20 to maximize the accuracy and minimize the loss of training data. In our experiments, the nal loss in Task 1 is 0.000108512764422, in Task 2 is 0.00640664884888 respectively and accuracy are both 1.0. 4 4.1
Data prepossessing The crawled contents for two sub-tasks are formatted in XML les. Data sets are divided into ten chunks under posting timeline. For Task 1, the data sets contain training and testing sets of 2017 and this year's data for testing. In Task 2, the les only contain training data and testing data, cause this sub-task is a new task in eRisk this year. Contents for every IDs are organized with ve tags: one root tag named < W RIT IN G > means one formatted posts, four child tags named < T IT LE >, < DAT A >, < IN F O > and < T EXT > orderly.
< T IT LE > tag gives the title of this post, < DAT A > tag marks the post data, < IN F O > tag reminds which platforms the posts are crawled and the nal < T EXT > tag contains the contents people write-in. As < T EXT > tag and < T IT LE > tag may both have no contents in one < W RIT IN G >, to gain more textual information, during processing, we combine both T IT LE and < T EXT > contents together, and extend the texts to one sentence. Although most of the IDs have a lot of < W RIT IN G > tags with e cient contents in one chunk, there are still IDs with none content in both < T EXT > tag and < T IT LE > tag of one chunk and cannot extract useful information except temporal information (not used in this paper). At this moment, the chunk with none content can only contain the texts from former chunks. All of the contents of IDs are extracted into ten chunks, in which chunk1 only contains the current texts of this chunk, the next chunk can contain the contents of former one and contents contained in current chunk, following this way, we get split contents for chunk1, chunk2, ,chunk10. Specially, for Task1, the training and testing data sets of 2017 are combined into one data set as new training corpus.
For the risk detection using keyword model, the < T EXT > tag and < T IT LE > tag combined sentences of IDs will be very chunk by chunk without summing them together. 4.2
Evaluation results Employing the models mentioned above with the extracted data, we submit the chunk 10th results of Task1 and Task 2 respectively. Table 2 and Table 3 show the results of Task 1 and Task 2 respectively.
Our nal chunk results of three models can be checked in Table 2 and Table 3. In Task1, the keywords model gets the best F1, precision and recall scores of 0.47, 0.35 and 0.71, two CNN+LSTM models get almost the same results. The two CNN+LSTM models are just two times running feeds with the model. The recall keywords model gets is one of the top recalls in the 11 teams, in which the best is 0.95. In Task 2, we employ three models, the keywords model only gets the best recall of 0.76, CNN+LSTM model get the best F1 and precision scores of 0.36 and 0.42.
TF IDF with SVM gets both 0 value in Task 1 and Task 2, this makes us confused. This model with l2 normalization of TF IDF features predict all the IDs as label "2", no risk ID detected in the tenth chunk. Those results make the model unbelievable, thus gets the 0 evaluation results nally. Evaluations chunk1 chunk2 chunk3 chunk4 chunk5 chunk6 chunk7 chunk8 chunk9 chunk10 ERDE5 ERDE50 F1 P R 9.87% 9.95 % 9.96 % 10.03% 10.17% 10.23% 10.29 % 10.35% 10.52% 10.55% 9.63% 9.64 % 9.19 % 8.87% 8.90% 8.70% 8.58 % 8.40% 8.47% 8.02% 0.31 0.47 0.23 0.37 0.44 0.32 0.41 0.44 0.39 0.43 0.42 0.44 0.42 0.38 0.48 0.44 0.38 0.53 0.46 0.38 0.58 0.46 0.37 0.61 0.45 0.34 0.65 0.47 0.35 0.71
As we only submit the tenth chunk results, for a fully analysis of the models, we calculate the ten chunks results of ERDE5, ERDE50, F1, P and R for keywords model in Task 1 and task2, which shows in Table 4 and Table 5 separately.
In Task 1, checking Table 2 and Table 4, we can nd keywords model get the best F1 scores from the rst chunk, while in Task 2, comparison the Table 3 and Table 5, The keywords model perform worse than in Task 1, though its recall Evaluations chunk1 chunk2 chunk3 chunk4 chunk5 chunk6 chunk7 chunk8 chunk9 chunk10 still perform good than CNN+LSTM method. Considering CNN+LSTM model only, with less negative samples CNN+LSTM model gets higher results in Task 2 compared with the two years data of Task 1. This maybe the small scale of the data sets cannot fully train the deep neural network, with more negative data, the model may learn more unbalanced information.
For keywords model, the keywords used are "depression" and "body" respectively for Task 1 and Task 2. They both get the best recall scores, While "depression" gets better results than "body" in F1 and Precision. These may indicate that in depression risk signs, people are often using the obvious words like "depression" to express themselves, while in anorexia situation, the disorder of eating may not almost focus on "body", there are maybe other hidden words existing. In other words, depressed people tend to express in the social networks directly and anorexia people are more di cult to detect. The team of FHDO gets the best detection among all the teams of F1 score of 0.85, we are looking forward to see their perfect models to be published. 5
In this paper, we propose three models to detect the risk of depression and anorexia respectively. Our results show the CNN+LSTM and keywords model can e ciently cover these task, while SVM model got unbalanced training. Our experiments show with more negative data sets, the CNN+LSTM model become sensitive about the data and need more tuning to train a suitable model for depression task. Our keywords model get the most e cient recall scores among the three models, even gets a good position among all teams, that proofs the key information is important to deal with these tasks.
In the future, we will use the test data to tune the CNN+LSTM model for a better detection ability combined the keywords information. Further more, a basic model by SVM without l2 normalization will also be checked. This research has been partially supported by JSPS KAKENHI Grant Number 15H01712.