Depression is one of the most debilitating mental health diseases. Detecting the presence of depressive symptoms in the early stages of the disease is essential to reduce further consequences. As the study of language and behaviour is a pivotal component in mental research, social network content positions itself as a helpful tool. This paper introduces a general framework to analyze variations in the individual's use of language over time on social media. We present a novel approach using temporal word representations to quantify the magnitude of words movements. This framework allows us to evaluate if words evolution can reveal the presence of depressive tendencies. We adapted diferent temporal word embedding representations to our framework and assessed them in Reddit benchmark datasets. Our results achieve high competitiveness compared with state-of-the-art methods, showing the potential that time-aware word representation models can bring to early detection scenarios.
Depressive conditions afect around 300 million people [
A growing body of evidence shows that proper interventions can significantly reduce the
adverse efects of the disorder [
In this context, initiatives such as CLEF Early Risk (eRisk) emerged as an online competition focusing on early risk prediction on the internet. Five editions have been held since 2017 [18, 19, 20, 21, 22]. In 2017, the organizers introduced the pilot task on Early Detection of Depression (EDD) [18]. The task consisted of sequentially processing pieces of users’ posts to detect early traces of depression. Organizers proposed the task in two editions (2017 and 2018, and the third edition, 2022, is currently ongoing). The challenge provides train and test data, composed of the history of writings of positive and negative Reddit users. The user’s writings are sorted chronologically and split into ten subsets or chunks for each user. Each chunk contains 10% of the posts. This division into chunks is especially relevant, as participants will process each chunk sequentially to detect pieces of evidence as soon as possible. To properly consider the earliness of the decision of participant systems, organizers defined time-aware classification metrics.
This paper proposes a novel approach to predict and detect early depressive symptoms. Our framework tries to capture changes in individuals’ language use over time. The idea is based on the premise that, when people develop a depressive disorder, they usually change their use of language. We intend to seek patterns in how individuals’ language evolution can manifest changes in their mental health states. For example, in an individual with a healthy mental condition, the word life appears in the context of words that are related to positive or neutral terms (great, going on, ok, good). Conversely, if that same individual begins to develop depressive symptoms, life would start appearing in alarming contexts (end, ruined, miserable) in the progressive temporal periods.
In particular, we introduce the use of temporal word embeddings for early detection of signs of depression. Temporal models have been traditionally used to capture the language and cultural evolution, focusing on meaning shifts in long periods (decades to centuries). For example, the word gay originally meant cheerful or pleasant, and only after the decade of 1990 acquired the primary meaning of homosexuality [23]. Unlike all previous work on temporal word representations, we propose using these models as a resource to detect mental health shifts in the short term. We present a general framework for the EDD task. It calculates the magnitude of word movements over diferent time slices. These movements over time are the features considered during the training and inference classification phases. The model computes the movement of a word as the distance between the temporal word embedding for the user in a particular chunk and a reference static word embedding that captures general language.
We tested two temporal embeddings models. (1) Temporal Word Embeddings with a Compass (TWEC) [24], built on the word2vec architecture. This model generates one unique embedding per word, ignoring the local context where the word appears. (2) Dynamic Contextualized Word Embeddings (DCWE) [25], which is based on pre-trained language models (PLMs), such as BERT [26]. PLMs provides contextualized embeddings, i.e, capturing context-specific vectors. We evaluated our methods in the EDD eRisk benchmark collections of 2017 and 2018. Our proposal provides an interesting exploratory window of how individuals’ usage of words evolves under certain mental states. Despite not using any linguistic resource or hand-crafted features specific to depression, we obtained competitive results among the participants in both editions. Our ifndings suggest that temporal representation methods can be a reliable solution for capturing the evolution of depressive symptoms. In this regard, the adaptability of our framework also makes it easy to transfer these methods to other related disorders without additional efort.
The remainder of the paper is structured as follows. Section 2 provides a review of related work about depression detection on social media and the use of temporal models. Section 3 presents our proposed framework. First, we introduce the general idea and then the actual approaches implemented. We describe the objective task and the collections used in our research, along with the experimental configuration and results in Section 4. Finally, Section 5 discusses the main conclusions derived from this work and proposes future lines of work.
Researchers from the psychology, medicine or linguistics fields have performed extensive work on topics related to mental health conditions and language. These works found interesting connections between language use and mental health [27, 28, 29]. In recent years, social media has become an essential source to provide data related to mental health disorders [30, 31, 32, 33, 34]. This amount of valuable information allows researchers to analyze user language and behaviour. Many studies have investigated how social network content can predict and identify mental conditions. As a result, computational methods have been applied to detect the presence of disorders like schizophrenia [35], suicidal ideation [36, 37] or eating disorders [38].
Data coming from social platforms like Twitter [39], Facebook [40] or Reddit [31] has been
used to model the characteristics of mental disorders. De Choudhury et al. [
Detecting depression in the early stages has great importance in reducing the disorder’s impact. Therefore, when building automatic alert systems, the sooner the alert emission on the temporal sequence, the better. For this reason, eRisk organized the first shared task on early risk detection
2https://clpsych.org/
of depression in 2017. To tackle the time aware nature of the task, the organizers proposed
Early Risk Detection Error (ERDE)3, a new metric which considers both accuracy and the delay
of the predictions. In the first edition of the task, participant systems used traditional machine
learning models such as bag-of-words, topic models, standard classifiers, or neural networks
approaches [18]. They also experimented with feature extraction of diferent depression lexicons
[
In the second edition, participants proposed new approaches leveraging word embeddings
and Deep Neural Networks to capture the semantics of the individuals and combined them with
the best performing features of 2017 [19]. Most of the participants improved their methods from
2017. Furthermore, some of them also developed classifiers supporting incremental classification.
For instance, in Funez et al. [
Lately, researchers have adopted neural language models as the most common way to capture
semantic features from texts. These models encode the word meanings as vectors, known as
word embeddings. Words that appear in similar contexts will be closer in a multi-dimensional
space. Traditional word embedding architectures, including Word2vec [
() The introduction of context-dependent approaches addressed the first limitation. Most
contextualized embeddings come from modern transformers architectures, such as ELMo [
() A growing body of research focused on the second limitation, creating approaches known as Temporal Word Embedding Models (TWE) or Diachronic models. However, learning embeddings in temporal contexts is a complex task. Traditional temporal methods relied on the use of static neural models. Due to the stochastic nature of neural networks, the models trained in diferent temporal moments will have vector spaces with diferent coordinate systems. This means that the temporal models from the time intervals must be aligned before using them to
compare representations, which comes at computational and performance costs. Static-based TWEs considered diferent alignment strategies. Most of them assume that words at nearby periods are highly similar, applying strategies to minimize the distance between contiguous vector space models (VSM).
A common way of approaching the learning of TWE is to divide data into time slices and
learn static embeddings of each one independently, and then proceed to the alignment of the
models [
Recently, research eforts are centred on detecting language change in contextualized word embeddings [60, 61, 62, 25]. Pre-trained transformers made it possible to contextualize the training of temporal embeddings for shorter time slices, e.g., single years. However, it is worth noting that contextualized embeddings perform worse than static embeddings for semantic change detection tasks [63] so far. In this regard, Dynamic Contextualized Word Embeddings (DCWE) [25] is a recent approach that combines the strengths of contextualized word embeddings with the flexibility of dynamic word embeddings. DCWE jointly models temporal and social information using pre-trained language models across time intervals. The authors demonstrated that considering time information can even improve downstream tasks such as sentiment analysis.
We based our proposal on the idea that individuals developing depressive symptoms will eventually change their language usage. They express new feelings, thoughts, and emotions, which will induce changes in their writings. Consequently, depressed individuals will vary the use of words, and many words will appear in previously unseen contexts. We designed a framework to analyze the variations in the individual’s use of language over time. Our method locates word shifts from one time step to another and quantifies these meaning changes. Next, we will introduce the overall view of the framework for later on going on the diferent implementations that we tested.
Our framework calculates two types of word embeddings: reference and dynamic. Reference embeddings are static and capture standard and neutral use of words. Contrarily, the dynamic embeddings represent the meaning of words for a user in each specific time interval, i.e., we compute a diferent dynamic vector space for each chunk and user. As a result, we can quantify the magnitude of word movements over diferent time slices. We obtain these movements by computing the distance between the dynamic word embeddings and the reference vectors. Then, the movements of words from chunk to chunk are the features considered during the training and inference classification phases. The general framework is divided into three main phases: () construction of reference embeddings, () construction of dynamic embeddings and () magnitude calculation of word movements. The first two phases depend on the selected temporal model. The last one (computation of word movements) is model-independent and remains the same regardless of the variant.
(i) First, we construct the global reference space, . This global space is atemporal, considering the full training corpus. We train a model computing a reference vector for each word present in the corpus vocabulary with the whole training corpus.
(ii) After computing the reference space, we consider the temporal chunks for a user. We define as the temporal spaces of each chunk, with 0 ≤ ≤ . Each point in () corresponds to the usage of the word at time . The training process of the reference and temporal spaces varies depending on the embedding model considered. This work presents two variants by using static and contextualized embedding architectures.
(iii) Finally, we quantify the magnitude of the word shifts that occurred over time. Our framework obtains the word shifts on the user’s writings of each chunk by computing the distance of each word from the chunk to that same word in the reference vector space . We define these word shifts as the movement of the word at time . The word shifts ( ) are from now on called deltas. Bigger values of deltas will represent larger deviations from the reference. If a word is not present in a specific chunk , we consider that its meaning has not changed, and its delta has zero value. To analyze variations in the individual’s use of language, we construct a matrix of deltas based on their writings. We defined this user matrix as Δ . Thus, for an user , we construct the matrix as equation 1:
1 · · · 1⎛ 11
⎜ 1 Δ = ⎜⎜ ⎜ ⎝ 1 · · · 1 1 ⎞ ⎟ ⎟ ⎟ ⎟ ⎠ (1) where rows represent the temporal chunks of that user, and columns the movements for each word present in the vocabulary, ∈ . Each row of the user matrix represents the word shifts for a diferent chunk. For instance, the row (Δ []) captures all the word shifts that happened for the individual during the chunk .
We addressed the problem as a binary classification (non-depressed vs depressed), and trained the classifiers at chunk-level. Therefore, each row of the users’ matrix is a diferent data sample. In the training phase, each row is labelled as 0 or 1 depending on whether the training user is negative or positive. In the inference phase, we feed the trained classifiers one user chunk at a time in chronological order. If the classifier’s decision over that chunk is negative (nondepressed), we continue processing the subsequent user’s chunks. If the decision is positive, we emit an alert for the user. However, if our classifiers reach the last temporal chunk and the decision is still negative, we classify the test user as non-depressed. We give more details of the classification process in the subsection 4.2.
This subsection describes the diferent choices for building the word representations. We implemented two variants based on the adaptation of existing temporal models. We explain how we constructed the reference and temporal vector spaces. After computing those spaces, the subsequent calculation of words shifts remains the same in both variants. 3.1.1. Temporal Word Embeddings with a Compass (TWEC) First, we adapted a temporal model based on word2vec, known as the TWEC method [24]. TWEC proposes the use of atemporal vectors as references, i.e. as a compass, when training the specific representations to a given time interval. This model fits very well with our framework since we can take advantage of the compass construction and use it as our reference vector space. TWEC approach is based on the premise that words that have changed over time appear in contexts with words whose meaning changed slightly. For that, authors exploit the word2vec architecture, which employs a dual representation of words (input and output embeddings). TWEC keeps output embeddings frozen across time and trains the temporal slices updating only their input embeddings. The frozen layer acts as an atemporal compass, allowing all the temporal models to share the same coordinate system. In the original study, the authors trained the compass with the whole experimental datasets. Then, they divided the collections into temporal chunks and used the compass to generate the temporal spaces. However, due to the eRisk task’s nature, at inference time, we do not have access to the entire history of the writings of a test user. That is, proceeding as in the original TWEC application would be cheating on the eRisk task. In this case, our model needs to process individual chunks sequentially and make early alerts using only the information related to that chunk. Therefore, we needed to modify TWEC to leverage it for our task slightly.
Our adaptation of TWEC is graphically depicted in Figure 1. First, we construct the input and output weight matrices, what we define as and , by training the word2vec CBOW architecture on the entire training collection. We ignore the resulting matrix and consider the output matrix as our reference space for temporal chunks. Therefore, the reference space model captures a general use of language by the Reddit users (including depressed and control users). After is calculated, we can start computing all the temporal chunk models associated to each user and to a specific time . We train every temporal model using the user’s writings for that chunk as input. As output, we keep the embeddings frozen to ensure that all the temporal models share the same coordinate system. This process remains the same when calculating the temporal spaces for training and test users for the subsequent classification task.
Full collection of training users
Atemporal reference compass For each user and its temporal chunks ...
...
...
Initialization 3.1.2. Dynamic Contextualized Word Embeddings (DCWE) We experimented with another TWE variant to calculate the reference and temporal models using contextualized word embeddings. In this case, we adapted the Dynamic Contextualized Word Embeddings (DCWE) [25] proposal to compute the vector spaces. The idea is to exploit the transformers’ architecture for generating context-based embeddings. In this model, we can distinguish two main components: () Dynamic module and () Contextualizer, illustrated on Figure 2. Words are first mapped to dynamic representations and then are contextualized. For both components, we used the standard BERTBASE language model, with 12 attention layers and a hidden layer with 768 dimensions. The resulting word embeddings are called for word and time step . The dynamic component is inspired by temporal models proposed in early works in the literature. Authors model a vector ofset that depends on word and time. This ofset ( ) is generated by a time-specific feed-forward network (FFN). To model the temporal drift, authors impose a random walk prior over . The random walk enforces smooth changes of the dynamic embeddings over subsequent times. Consequently, the distance between words that are far apart in time will be greater. The ofset is summed to the static embeddings ˜, which are initialized using BERT pre-trained embeddings.
The concatenation of ˜ and embeddings is the dynamic component of the final embedding. Then, these dynamic embeddings are contextualized using BERT (component in the Figure). The contextualizer applies masked language modelling (MLM) as the training objective. The outputs of both components are the final dynamic contextualized embeddings. We note that, due to the maximum length limit of BERT, we need to split the chunks into individual posts. This results in diferent dynamic word embeddings for each word appearing in the writings.
With DWEC, we produce dynamic contextualized embeddings. However, we still miss the reference space vector to compute the temporal word shifts. To create , we have taken advantage of the previously static embeddings. We believe that the BERT embeddings, trained FFN
BERT
Input embeddings + +
+ Words' Post
Dynamic Contextualized
Embeddings
Chunk Post
Users in large collections of documents, would work as a good reference space in our solution. Finally, we can compare the temporal word shifts from the reference embeddings. For clarification purposes, this method is illustrated in Figure 3. As previously mentioned, we perform the whole process at the post level. Therefore, we calculate the deltas also at the post level. Thus, to quantify the word shifts for each chunk, we averaged all the distances of the words’ used in that time interval 4.
This section covers the experimental settings and evaluation of our proposed approaches. First, we briefly describe the collections used to train and test our classifiers. We also introduce the evaluation metrics that consider both the efectiveness and the delay in making the decisions. Then, subsection 4.2 presents the implementation details of our experiments. Finally, we discuss the results and compare our performance against state-of-the-art baselines.
The collections used in this work correspond with the eRisk task early risk detection of depression (EDD) from 2017 and 2018. This task is focused on sequentially processing the publications posted by users on Reddit5. There are two classes of users, depressed and non-depressed. Table 4If a word has not occurred in any of the posts’ chunks, the shift value for that word is zero (0) 5https://www.reddit.com/
DELTAS CALCULATION
Reference Static Embeddings
Dynamic Embeddings
DCWE
Post
Chunk Deltas from Post Users Average Deltas from
Chunk 1 summarizes the details of each collection. Note that both datasets are highly unbalanced, as there are many more control users. User’s writings are chronologically sorted into ten chunks. Thus, each chunk contains 10% of the subject’s history. Alerts of the user developing depression must be emitted as early as possible, deciding each user is depressed or not based on the sequential chunks of information. For this purpose, classifiers are given one chunk at a time, and then they have to label the user as depressed, non-depressed, or they need more chunks before making a decision.
Regarding the evaluation metrics, organizers proposed standard classification metrics such as F1-measure, precision () and recall (). Additionally, as the main objective of the task is to emit alerts as early as possible, a novel time-aware measure was defined by Losada and Crestani [31]. This new metric, called Early Risk Detection Error (ERDE), considers both the correctness of the predictions and the delay it took to emit the alert. ERDE is defined by: (, ) = ⎧ ⎪ ⎪ ⎪ ⎪⎨ if is False Positive (FP) if is False Negative (FN) ⎪0() · if is True Positive (TP) ⎪ ⎪⎩⎪0 if is True Negative (TN) is the binary decision taken by the systems with delay for the user. The cost function 0() is a monotonically increasing function of and , the penalization threshold: 1 0() = 1 − 1 + − (2)
The delay is measured by counting the number of writings () processed before deciding. Particularly, organizers proposed the use of ERDE5 and ERDE50, where 5 and 50 represent the number of posts from which the metric penalizes more quickly. Finally, and costs are set to 1, and cost is the proportion of depressed subjects from the total subjects in the test set. Being error measures, lower values of ERDE mean better performance.
4.2.1. Filtering process When calculating the word deviations, we have experimented with diferent lexicons filtering out terms by their total term frequency in the corpus. The total unique words from the 2017 and 2018 corpora are 42588 and 62455, respectively. With this in mind, we have considered three diferent settings to see how the words’ frequency might afect the performance of our solutions. () Filter out the 100 most frequent words, () filter out the 100 most frequent words and only use the next 10000 most frequent terms [64] and () consider all the terms in the corpus. The selection of terms was made only over the training data, unseen terms on test data were discarded. For selecting the best filtering strategy, we cross-validated it on the training split while optimizing for the 50 metric. The results reported in this paper use the first configuration, as it is the setting that obtained the best values in the validation stage. This seems to point out that temporal shifts from words with a a high term frequency may add noise to the process, while keeping low frequent terms can improve the classification. We leave for future work a thorough comparison between the impact of the term frequency and term selection methods on the performance of our temporal solutions. 4.2.2. Classification process We perform a binary classification task (depressed vs non-depressed) by considering as features only the word shifts at each chunk. In this process, we used a standard classifier, Support Vector Machine (SVM) of Python sklearn. We decided to experiment with SVM’s motivated by its remarkable results in the eRisk tasks compared to other classification techniques. We experimented with diferent kernels of SVM during the validation stage. Parameters were optimized to minimize the 50 measure. We applied a grid search strategy (ranging from 2017 2018
Team FHDOB
Team UNSLA
Team FHDOA
SS3 [
The objective of this paper was to evaluate the adaptation of temporal embedding models to early detection scenarios. For that, we compared our results against the best participants and state-of-the-art methods for the eRisk early task in 2017 and 2018. Table 2 shows the baselines along with our proposed methods (ΔTWEC_SVM and ΔDCWE_SVM).
In 2017, there were 30 submissions from 8 international participants. All the participants’
results’ are detailed in [18]. We report the best performers from the participants in 5,
50 and 1. We also include the SS3 method [
In 2018, there were 45 systems from 11 diferent institutions. Again, the participants’
performance can be consulted in the CLEF oficial working notes of 2018 [ 19]. Coincidentally,
the best performers for each metric were from the same research groups as in 2017. FHDOB’s
incremental solution disregarded almost all the metadata features [67], and obtained the best
results in 50 and 1. The UNSLA method [
We can conclude from the overall results that capturing changes in individuas’ use of language over time show remarkable performance in early detection scenarios. Our proposed approaches obtained competitive results compared to the state-of-the-art, especially considering the timeaware error measures. Even in some cases, we outperformed all the reported values for the task. Additionally, we have to stress that, in contrast to most previous solutions, () our methods do not apply an elaborated set of hand-crafted and specifically designed combination of features, such as the exploit of depression or emotion lexicons seen at prior works. () The decisions do not rely on complex models, ensembles of diferent classifiers, etc. In contrast to most proposed approaches, our proposal is easily adaptable to other domains and could even be combined with other specific features without any additional efort.
This paper explored the potential of language evolution in early depression detection using publications from social media. A basic idea inspired our work: when people develop a depressive condition, they regularly change their use of language, i.e., the way they communicate with others. We proposed and evaluated a novel early detection framework to assess this assumption. This framework adapts temporal word representations to calculate the magnitude of word movements over time. Particularly, we implemented two temporal techniques based on () the use of static embedding models and () contextualized embedding models. We evaluated both approaches on two datasets provided at eRisk 2017 and 2018 in CLEF international forum. Our results show how useful it can be to consider the evolution of words as features to detect early traces of depression. By considering only word shifts as features, we ranked among the top state-of-the-art models. Another interesting aspect of our framework is that we do not rely on depression-specific features or elaborated decision mechanisms to emit the alerts.
The flexibility of our framework opens many research lines for future work. First, we plan to extend the training of the reference space model over a larger number of users to increase its stability. In this regard, we are interested in using other datasets to confirm our results, considering other social platforms and languages. Additionally, this adaptability also allows us to investigate the commonalities in terms of words’ evolution with other similar diseases such as anorexia, self-harm or suicidal ideation. Finally, we will explore ways to improve the performance of our solutions. We will analyze other classification algorithms and feature selection methods better suited for handling large scale data and noisy features. [13] N. Colineau, C. Paris, Talking about your health to strangers: understanding the use of online social networks by patients, New review of hypermedia and multimedia 16 (2010) 141–160. [14] J. Suler, The online disinhibition efect, Cyberpsychology & behavior 7 (2004) 321–326. [15] C. M. McHugh, A. Corderoy, C. J. Ryan, I. B. Hickie, M. M. Large, Association between suicidal ideation and suicide: meta-analyses of odds ratios, sensitivity, specificity and positive predictive value, BJPsych open 5 (2019). [16] R. N. Golden, C. Weiland, F. Peterson, The truth about illness and disease, Infobase Publishing, 2009. [17] L. Manikonda, M. De Choudhury, Modeling and understanding visual attributes of mental health disclosures in social media, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2017, pp. 170–181. [18] D. E. Losada, F. Crestani, J. Parapar, eRisk 2017: CLEF lab on early risk prediction on the internet: experimental foundations, in: International Conference of the Cross-Language Evaluation Forum for European Languages, Springer, 2017, pp. 346–360. [19] D. E. Losada, F. Crestani, J. Parapar, Overview of eRisk: early risk prediction on the internet, in: International conference of the cross-language evaluation forum for european languages, Springer, 2018, pp. 343–361. [20] D. E. Losada, F. Crestani, J. Parapar, Overview of eRisk 2019 early risk prediction on the internet, in: International Conference of the Cross-Language Evaluation Forum for European Languages, Springer, 2019, pp. 340–357. [21] D. E. Losada, F. Crestani, J. Parapar, eRisk 2020: Self-harm and depression challenges, in:
European Conference on Information Retrieval, Springer, 2020, pp. 557–563. [22] J. Parapar, P. Martín-Rodilla, D. E. Losada, F. Crestani, Overview of eRisk 2021: Early risk prediction on the internet, in: Experimental IR Meets Multilinguality, Multimodality, and Interaction - 12th International Conference of the CLEF Association, CLEF 2021, Virtual Event, September 21-24, 2021, Proceedings, Springer International Publishing, 2021, pp. 324–344. [23] V. Kulkarni, R. Al-Rfou, B. Perozzi, S. Skiena, Statistically significant detection of linguistic change, in: Proceedings of the 24th International Conference on World Wide Web, 2015, pp. 625–635. [24] V. Di Carlo, F. Bianchi, M. Palmonari, Training temporal word embeddings with a compass, in: Proceedings of the AAAI conference on artificial intelligence, volume 33, 2019, pp. 6326–6334. [25] V. Hofmann, J. Pierrehumbert, H. Schütze, Dynamic contextualized word embeddings, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, 2021. [26] J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), Association for Computational Linguistics, 2019, pp. 4171–4186. URL: https://doi.org/10.18653/v1/n19-1423. doi:10.18653/v1/n19-1423. [27] D. J. Harper, Discourse analysis and’mental health’, Journal of Mental Health 4 (1995) 347–358. [28] M. E. Jensen, E. A. Pease, K. Lambert, D. R. Hickman, O. Robinson, K. T. McCoy, J. K.
Barut, K. M. Musker, D. Olive, C. Noll, et al., Championing person-first language: a call to psychiatric mental health nurses, Journal of the American Psychiatric Nurses Association 19 (2013) 146–151. [29] T. Althof, K. Clark, J. Leskovec, Large-scale analysis of counseling conversations: An application of natural language processing to mental health, Transactions of the Association for Computational Linguistics 4 (2016) 463–476. [30] G. Coppersmith, M. Dredze, C. Harman, K. Hollingshead, From adhd to sad: Analyzing the language of mental health on twitter through self-reported diagnoses, in: Proceedings of the 2nd workshop on computational linguistics and clinical psychology: from linguistic signal to clinical reality, 2015, pp. 1–10. [31] D. E. Losada, F. Crestani, A test collection for research on depression and language use (2016) 28–39. [32] S. MacAvaney, B. Desmet, A. Cohan, L. Soldaini, A. Yates, A. Zirikly, N. Goharian, Rsddtime: Temporal annotation of self-reported mental health diagnoses, arXiv preprint arXiv:1806.07916 (2018). [33] A. Cohan, B. Desmet, A. Yates, L. Soldaini, S. MacAvaney, N. Goharian, Smhd: a large-scale resource for exploring online language usage for multiple mental health conditions, arXiv preprint arXiv:1806.05258 (2018). [34] D. Mowery, H. Smith, T. Cheney, G. Stoddard, G. Coppersmith, C. Bryan, M. Conway, Understanding depressive symptoms and psychosocial stressors on twitter: a corpus-based study, Journal of medical Internet research 19 (2017) e6895. [35] M. Mitchell, K. Hollingshead, G. Coppersmith, Quantifying the language of schizophrenia in social media, in: Proceedings of the 2nd workshop on Computational linguistics and clinical psychology: From linguistic signal to clinical reality, 2015, pp. 11–20. [36] B. Desmet, V. Hoste, Online suicide prevention through optimised text classification,
Information Sciences 439 (2018) 61–78. [37] M. De Choudhury, E. Kiciman, The language of social support in social media and its efect on suicidal ideation risk, in: Proceedings of the International AAAI Conference on Web and Social Media, volume 11, 2017. [38] T. Wang, M. Brede, A. Ianni, E. Mentzakis, Detecting and characterizing eating-disorder communities on social media, in: Proceedings of the Tenth ACM International conference on web search and data mining, 2017, pp. 91–100. [39] X. Chen, M. D. Sykora, T. W. Jackson, S. Elayan, What about mood swings: Identifying depression on twitter with temporal measures of emotions, in: Companion Proceedings of the The Web Conference 2018, 2018, pp. 1653–1660. [40] M. A. Moreno, L. A. Jelenchick, K. G. Egan, E. Cox, H. Young, K. E. Gannon, T. Becker, Feeling bad on facebook: Depression disclosures by college students on a social networking site, Depression and anxiety 28 (2011) 447–455. [41] M. De Choudhury, M. Gamon, S. Counts, E. Horvitz, Predicting depression via social media, in: Seventh international AAAI conference on weblogs and social media, 2013. [42] M. De Choudhury, S. Counts, E. Horvitz, Social media as a measurement tool of depression https://doi.org/10.1145/3178876.3185999. doi:10.1145/3178876.3185999. [58] Z. Yao, Y. Sun, W. Ding, N. Rao, H. Xiong, Dynamic word embeddings for evolving semantic discovery, 2018, pp. 673–681. doi:10.1145/3159652.3159703. [59] R. Bamler, S. Mandt, Dynamic word embeddings, in: International conference on Machine learning, PMLR, 2017, pp. 380–389. [60] M. Giulianelli, M. Del Tredici, R. Fernández, Analysing lexical semantic change with contextualised word representations, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 3960–3973. URL: https://aclanthology.org/2020.acl-main.365. doi:10. 18653/v1/2020.acl-main.365. [61] R. Hu, S. Li, S. Liang, Diachronic sense modeling with deep contextualized word embeddings: An ecological view, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 3899–3908. URL: https://aclanthology.org/P19-1379. doi:10.18653/v1/P19-1379. [62] M. Martinc, P. K. Novak, S. Pollak, Leveraging contextual embeddings for detecting diachronic semantic shift, in: N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, S. Piperidis (Eds.), Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020, European Language Resources Association, 2020, pp. 4811–4819. URL: https://www.aclweb.org/anthology/2020.lrec-1.592/. [63] D. Schlechtweg, B. McGillivray, S. Hengchen, H. Dubossarsky, N. Tahmasebi, Semeval-2020 task 1: Unsupervised lexical semantic change detection, arXiv preprint arXiv:2007.11464 (2020). [64] H. P. Luhn, The automatic creation of literature abstracts, IBM Journal of research and development 2 (1958) 159–165. [65] Q. Le, T. Mikolov, Distributed representations of sentences and documents, in: International conference on machine learning, PMLR, 2014, pp. 1188–1196. [66] M. L. Errecalde, M. P. Villegas, D. G. Funez, M. J. G. Ucelay, L. C. Cagnina, Temporal variation of terms as concept space for early risk prediction., in: CLEF (Working Notes), 2017. [67] M. Trotzek, S. Koitka, C. M. Friedrich, Word embeddings and linguistic metadata at the CLEF 2018 tasks for early detection of depression and anorexia., in: CLEF (Working Notes), 2018.