In this paper, we present the method we developed when participating to the e-Risk pilot task. We use machine learning in order to solve the problem of early detection of depressive users in social media relying on various features that we detail in this paper. We submitted 4 models which di erences are also detailed in this paper. Best results were obtained when using a combination of lexical and statistical features.
The WHO (World Health Organization) reports that \the number of people su ering from depression and/or anxiety increased by almost 50% from 416 million to 615 million" from 1990 to 20131. Depression and Bipolar Support Alliance also estimates that \major depressive disorder a ects approximately 14.8 million American adults" and \annual toll on U.S. businesses amounts to about $70 billion in medical expenditures, lost productivity and other costs" (http://www.dbsalliance.org).
Depression detection is crucial and many studies are devoted to this challenge
[
Recent related work focus on people communication and social media post
analysis to detect depression. Rude's study shows that depressed people tend to
use the personal pronoun (\I") more intensively than others [
These previous studies show that some cues and features extracted from social media posts can be related to depression. In this paper, we report our investigations on using various features in order to answer the e-risk challenge 1 http://www.who.int/mediacentre/news/releases/2016/
depression-anxiety-treatment/fr/
as described in [
In our participation runs, the features we used to characterize posts are of two types: lexicon-based (extracted using NLTK toolkit3) and numerical features. These features are used in a machine learning method using Weka.
The remaining of this paper is organized as follows: Section 2 provides an overview of the model we used. Section 3 details the di erent features we implemented to train di erent models. In Section 4 we detail the 4 runs we have submitted and the underlying models and present the results. In Section 5 we discuss the results and depict future work. 2
2 https://www.reddit.com/ 3 NLTK is a platform for building Python programs for natural language processing that interfaces easily with text processing and machine learning libraries (www.nltk.org)
The model is composed of three modules. In the rst one, we pre-process the XML les that contain the users' posts. The second module aims at extracting the features. Notice that while some features capture information from any textual parts, others focus either on the Title part (corresponding to the initial post) or on the Text part which corresponds to comments on the initial post. The feature extraction module is extensible: while we developed some features, new features can easily be added. Then, in the formatting module, we select a subset of the features to be used in the model. 3
We developed di erent types of features. Some have linguistic foundation while others are more statistically-based. We distinguish lexicon-based features from other numerical features.
For lexicon-based features, we rely either on previous observations on
depressive subjects' behaviour [
Features are calculated for each user as follows: we rst calculate the feature value for each of his or her post or comment, then we average the value over his or her posts in the chunk ; when several chunks are used, we average the feature values obtained for each chunk for the considered user.
We also used some other numerical features that are described in Table 2.
The details of the features are described in [
We submitted 4 runs corresponding to 4 models. The features that were used for each model are listed in Table 3. While we used model GPLA to start with, the other models were introduced later on. The second column of Table 3 indicates the chunk number when each model was introduced. The 4 runs corresponding to our 4 models were performed with the Random Forest learning algorithm under the Weka platform using the default parameters.
In order to decide whether to issue a decision for a subject or wait for more chunks, we used the prediction con dence rate that Weka generates for each prediction. We set a threshold (estimated using samples of depressive subjects) and we only issued decisions that had a prediction con dence that exceeds the selected threshold. The evolution of the threshold for each model through the runs and according to the chunks can be tracked using Table 4, a threshold of 0.5 basically means that all predictions are considered.
& related drugs and Wikipedia list4.
"everywhere", "everything" a lot.
to each word: Negative, Positive or Ambiguous we then calculated the frequency of each category
by Gualtiero B. et al. [
4
The evaluation takes into account not only the correctness of the output of the
system (i.e. whether or not the user is depressed) but also the delay taken to
emit its decision. To this aim, the ERDE (Early Risk Detection Error ) metric
proposed in [
Our best results when considering ERDE measures are obtained using model GPLC which does not use POS results nor the most frequent n-grams. Including them in the model slightly improves F1 measure mainly because of higher recall
14 Variation of the number of posts
15 16 17 18 19 20
the number of posts is generally small.
of comments
(0.60 against 0.50) (see Table 5). Our run GPLB had the 2nd best Recall (0.83) across participants and GPLA the 5th. 5
In the runs we submitted we consider 19 features. However, some additional features are worth studying. In future work, we aim at considering temporal features such as the date of the posts, part of the day, etc. Moreover, we would like to modify the way features are calculated : in the case of lexicon-based features, each lexicon item would be a distinct feature. By this way, we would obtained a richer representation of each user and potentially a better detection. 12. C. Strapparava and A. Valitutti. Wordnet-a ect: an a ective extension of wordnet. Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, May 2004, pp. 1083-1086, 2004.