In this paper, we present our approach to the detection of self-harm at eRisk 2020. The main objective of this shared task was to identify as soon as possible if a user presents signs of committing selfharm by using their posts on Reddit. To tackle this problem, we used a representation called Bag of Sub-Emotions (BoSE), an approach that represents the posts of the users in a set of sub-emotions, in combination with a Bag of Words. With this strategy, we were able to capture the sub-emotions and topics that users with signs of self-harm tend to use. For the early classification, we choose five different strategies based on the temporal stability shown by the users through their posts. Our approach showed competitive performance in comparison with other participants. Additionally, the interpretability and simplicity of our representation present an opportunity for the analysis detection of different mental disorders in social media.
Self-harm is defined as the direct and intentional injuring of body tissue with
the intent to commit suicide [
In this work, we describe the joint participation of INAOE-CIMAT, two
research centers from Mexico, at eRisk@CLEF shared task 1. For this
participation, we used a representation called Bag of Sub-Emotions (BoSE), an approach
based in the use of fine-grained emotions to capture specific emotional topics on
posts [
The remainder of this paper is as follows: Section 2 presents some related work for the self-harm detection task and early predictions. Section 3 describes our text representation. Section 4 and Section 5 presents the experimental settings as well as the obtained results. Lastly, Section 6 depicts our conclusions. 2
As previously described, self-harm is a mental disorder associated with the intent
of committing suicide or directly damaging the body. Most works related to the
detection of the signs of self-harm in social media content focus on the analysis
of the post content, mostly considering different kinds of word based features
[
In psychology, it has been established the correlation between emotions and
mental disorders, and the study of the manifestation of emotions in language is an
active research area [
Generate Sub-Emotions. The creation of sub-emotions used the lexical
resource from [
As described above, the obtained sub-groups of words allow separating each coarse emotion in different topics. These topics help to capture more specific emotions expressed by the users in their posts. Figure 2 shows some examples of the kind of sub-emotions automatically generated by the proposed approach. Analyzing this figure in detail, it can be appreciated that words with similar context tend to group together. It can also be noticed that even for the same emotion each group of words shows a different topic. For example, for the Anger emotion has one group related to the topic of "fighting and battles" and another group about "loud noises or growls". In another example, the Surprise emotion has one group to "art and museums", whereas other groups contain words related to "accidents and disasters", or "magic and illusion".
Text to Sub-Emotions. Once generated the sub-emotions, we masked the users’ posts by replacing each word with the label of its closest sub-emotion. To do this process, first, we calculated the word embedding vector of each word in the vocabulary of the users using FastText. Then, we obtained the cosine similarity between each word vector and the sub-emotions. Finally, the closest sub-emotion was selected to replace the word.
After documents were masked, we built the BoSE representation by using
histograms of sub-emotions. Basically, each document was represented as a vector
of weights associated to sub-emotions, where weights are computed in the tf-idf
fashion. Figure 3 describes the whole process to create the representation. In [
This year’s shared task was a continuation of the eRisk 2019 T1 task [
Used classification strategies 1. Run 0: it considered only the training set from the previous edition, and employed the BoSE representation. A user was classified as committing selfharm if his/her probability of belonging to the positive class was higher than 0.60 in two consecutive predictions. 2. Run 1: it is similar to Run 0, but the model was trained using the depression data set from the eRisk 2018 task. 3. Run 2: It combined a BoW representation with BoSE, and trained the classification model using the self-harm and depression datasets together. 4. Run 3: It is similar to Run 2, except that the training was done using the self-harm dataset. 5. Run 4: It employed the BoW and BoSE representations trained with the depression and self-harm datasets; a positive prediction was generated when its probability was higher than 0.55.
Here, it is important to note that the used approach presents two main differences with respect to our previous year strategy. On the one hand, the addition of the users’ vocabulary to the BoSE representation, which allow to capture some specific words related to self-harm, and, on the other hand, the use of the 2018 depression dataset in the training phase, which aims to build a more robust classification model by taking advantage of the existing relationship between self-harm and depression. In Table 1 we show the stragy used for each run. We first trained and evaluated our model using the 2019 eRisk dataset. This experiments helped us to select the best parameters before sending the predictions to the server. The 2019 dataset contains two categories of users: self-harm and control. For this configuration experiment, we used the users’ whole post histories, performed a cross-validation strategy, and considered the F1 over the positive class as evaluation measure. Table 2 presents the obtained results. It compares the results using the BoSE and BoW representations, trained exclusively with the self-harm data set as well as adding the depression data set. These results show that adding information from the depression collection helped to improve the classification performance. This could be due the lack of data using only the self-harm dataset. In Table 3, we present five of the most relevant words of each dataset (depression and self-harm). We can appreciate that the model captures some differences between both problems. For example, for self-harm, most relevant words are related to physical damage with some mental problems, and for depression words are more related to emotional problems.
For the submission of results, we trained the model using all the information from all the users of the training dataset. Then, using the five classification strategies previously mentioned, we detected the the users who presents signs of committing self-harm. Table 4 shows the results obtained by the five strategies over the 2020 test data set. The strategy named as Run 3 was the one which obtained the best results; it consists of the usage of BoSE and BoW trained only over the self-harm dataset. In this strategy, a user was identified as committing self-harm if the probability of the positive class was higher than 0.60 in two consecutive predictions, indicating a temporal emotional stability of the user. We can appreciate that this approach also obtains the best ERDE prediction, which imply a good prediction with relatively less information. The usage of both representations, indicate that not only the emotional information but also the presence of certain words (associated with certain topics) are important for the detection of people who commit self-harm. In table 5 we show some of the most relevant sub-emotions related to self-harm and the topics they capture. Some topics are related to negative aspects, like hate, criticise or refuse. An interesting sub-emotion captured automatically by our model is related to young people, where people that commit self-harm usually is a teenager or closer to that age.
Self-harm anger11 unsociable, crowd, mischievous disgust17 condemn, criticise, refuse, repudiate fear5 dreadful, hate, bad, nasty negative18 adolescence, teen, juvenile trust22 impatient, desire, anxious 5
In this paper, we presented our approach for the eRisk 2020 shared task 1, which consists in deciding as soon as possible if a user presents signs of self-harming by using his/her post history in chronological order. For this, we proposed the use of a representation that combines a bag of words with a bag of sub-emotions, which was created using a lexical resource of emotions and FastText sub-word embeddings. The main idea of our approach is to capture specific fine-grained emotions and topics that a user committing self-harm tend to express through his/her posts. Our approach differs from other methods in its simplicity and interpretability, particularly against approaches that use several different features and complex classification models. In the test set, it obtains competitive results, showing an opportunity for a deeper exploration on the usefulness of modeling the emotional information from users that have the risk of committing self-harm or suffering from another mental disorder.
This research was supported by CONACyT-Mexico (Scholarship 654803).