We present an online system that recommends web-based reading passages to users based on free long-form text-based elicitations of how they're feeling right now. The system combines natural language processing techniques used to extract users' intent with an information retrieval system to yield relevant and useful narratives for users. An eight week long user study found that most people who used the system reported better mood at the end of their interaction with the system. Interestingly, our study also discovered greater user engagement with randomly recommended narratives than with narratives selected for users based on their written descriptions of their own mental states. These observations could constrain the future design of self-help recommender systems.
• Human-centered computing; • Information systems → Content ranking; Personalization; Query intent; Recommender systems; • Applied computing → Psychology; bibliotherapy; personalization; human factors; recommender systems
Bibliotherapy is the use of books as therapy for mental distress
and depression [
The key psychological premise of bibliotherapy is that the reader
begins to identify with a particular character in the book, and thus
is able to observe a situation related to their own predicament
HealthRecSys ’20, September 26, 2020, Online, Worldwide.
© 2020 Copyright for the individual papers remains with the authors. Use permitted
under Creative Commons License Attribution 4.0 International (CC BY 4.0). This
volume is published and copyrighted by its editors.
from suficient psychological distance. This allows them to think of
possibly solutions, which they eventually realize might also apply
in their own situation [
Thus, efective bibliotherapy requires insight into the patient’s condition, and the ability to recommend readings that the patient is likely to find relatable in their current condition. Since this is a complex and sensitive task, the assistance of a trained psychotherapist is generally advised for formal bibliotherapy.
It is unlikely that the role of a human therapist could be replaced
via algorithmic recommendations in formal bibliotherapy for formal
mental health treatments. However, bibliotherapy is not always
formal. The friendly neighborhood librarian’s suggestions for a book
to read after hearing a teenager’s anxiety about a recent distressing
event is an equally valid, albeit, informal type of bibliotherapy [
Given the plenitude of digital reading resources available today,
it is surprising to find that the prospect of conducting informal
bibliotherapy using algorithmic recommendations has not been
well-studied. We note an online bibliography system described in
the literature using collaborative filtering based approach based on
system-defined interest category tags [
In this paper, we propose a recommender system for informal
bibliotherapy that identifies relevant recommendations for users
based on long-form written elicitations from them. We also report
preliminary results from a user study that assesses whether such
systems are useful for participants, and whether delivering
personalized recommendations for bibliotherapy is a good idea. Given the
ongoing debate about the nature of ‘filter bubbles’ generated by
recommender systems [
Our system is essentially a personalized search engine for a set of online narratives (primarily blogs) related to personal resilience and growth. The system’s key novelty is the use of text responses elicited from users to suggest readings to them. We use classic information retrieval methods to index this corpus of narratives, natural language processing methods to transform long-form user text inputs into search queries, and query expansion and ranking techniques to extract relevant articles given users’ searches. We describe the system in greater detail below. To correlate all details described here, our system itself is accessible at this URL for reference. 2.1
Users interact with our system using anonymous IDs, preserving privacy at source. We collect no identifying information by design. However, for users participating in user studies for compensation (discussed below), there is an option to enter an email address for future identification.
New registrants to the system are encouraged to watch a video, prominently displayed on the landing page, describing the possible modes of engagement with the system and asked to select a unique ID from a set of randomly generated ID strings. Returning users sign into the system using the IDs they selected at the time of registering.
We elicit user mood responses on a 5 point scale, with adjectival descriptions of a rating of 1 being terrible and a rating of 5 being amazing. A corresponding smiley emoticon reacts to the user’s input. Following this numeric elicitation, the system asks the user whether they want to type in more about their mood (see Figure 1A). If they click ’yes’, they see a ‘type-it-out’ text box in which they enter long-form text describing how they’re feeling and why they’re feeling that way (see Figure 1B). If they click ’no’, they directly see reading recommendations (see Figure 1D).
Users see recommended readings sequentially in a carousel view. They may choose to rate these narratives by personal relevance on a scale of 1 to 5, where 1 represents highly irrelevant and 5 represents highly relevant. We explain in the introduction to the system that relevance, in the context of our system, means how close a particular narrative is to what users are experiencing in their life and whether they can learn something from it, and realise what must be done to get things back on track in their own lives.
The web service itself follows a Model-View-Controller (MVC) architecture. We use Jinja2 to create the application’s views. All the data-related logic is handled by an SQLite database, which forms the Model component of the architecture. Python (with Flask) acts as the Controller and acts as an interface between Model and View. 2.2
Narratives are selected based on whether they are prominently related to first person accounts of mental resilience and personal growth through periods of sadness and distress. While the system is designed to handle a large number of narratives, we conducted the user study we describe below using a restricted set of 30 total narratives, which are treated as individual documents by our information retrieval system.
We apply a standard natural language pipeline for converting each document into a bag-of-words representation. Each one is tokenized, stemmed, and lemmatized and stop words are removed. The bag of words associated with each document is stored in an inverted index. 2.3
Our system treats the text entered in the ‘type-it-out’ box as an
implicit query, and uses it to identify relevant blogs for the user. We
ifrst extract keywords from the text using the RAKE algorithm [
Our basic expectation in designing this system is that it will help people cope with stress and unhappiness by connecting them with ifrst-person narratives of surviving dificult situations. Such a proposition can be operationalized in many diferent ways, making it hard to test. However, we conducted a user study with a fixed operationalization to see if we could characterize the value of the system in some quantitative form. In our study, we focused on answering two questions: • Does greater engagement with the system predict improvements in mood? • Does personalization of recommendations based on users’ descriptions of how they’re feeling actually produce greater user engagement?
Mood ratings are already collected in the system as described
above on each user visit. We operationalize engagement using
relevance ratings assigned by users to the readings our system ofers
them, a common practice in recommender systems research [
We recruited participants soliciting interest from people who wanted
to develop a skill set for dealing with stressful situations in life or
those who wanted to ofer their contribution to improving mental
health care. Our primary source of participants was word of mouth
mixed with chain referral sampling. In total, 190 people signed up
for the study, but only 36 people completed all planned sessions,
yielding an attrition rate of 81%. While high, this is not unexpected,
since self-help studies, focusing as they do on people experiencing
or prone to experience mental distress frequently exhibit similar or
even higher attrition rates [
In the interest of maintaining ecological validity, we asked participants to simply use the system at least once in a span of 3-7 days over a period of eight weeks, and exactly seven times in total. A reminder email was sent to each participant four days from the last time they had participated. The IRB Board at IIT Kanpur reviewed and approved this protocol.
We made a slight modification in the design of the system used for the user study by inserting a short questionnaire in the system at the point where participants are asked whether they wanted to type in something about their mood (see Figure 1C). The questionnaire consisted of 3 short answer questions selected from a bank of 21 questions, rotated such that participants didn’t have to respond to the same question twice across 7 sessions.
This addition was necessary because we did not want to force participants to write about their mood, but at the same time wanted text samples to guide the recommender system. By adding 3 short questions, we obtained suficient text content to guide recommendations even for participants who did not want to respond in longform to the mood description probe. Thus, for participants who responded in long-form, we used their long-form responses to generate recommendations. For participants who chose not to respond in long-form, we generated recommendations using answers to the questionnaire questions. The questionnaire is available on the version of the system linked to from this paper, but is not expected to be a part of the actual system in deployment.
Also, we fixed the number of readings each person would read and rate in a particular session at 10, and changed our recommendation algorithm to ensure that 5 of these 10 readings would be generated based on the users’ text inputs, treated as queries by our system, and the rest would be randomly selected from the remaining narratives. 3.3.1 Engagement with the system is weakly correlated with improvement in mood. Our first research question in this project was to identify whether engagement with our system resulted in improvement in participants’ mood ratings. To characterize this relationship, we estimated the mood trend for each participant across the seven sessions for which they gave us mood ratings. For each user, we fit an ARIMA(1,0,0) model to tease apart the underlying trend line from transient fluctuations. We correlated this estimated trend in mood with our measure of user engagement - the average relevance scores assigned to the readings each participant rated across all seven sessions. Greater engagement was expected to yield higher average relevance scores, viz. the participant felt that they could identify with the themes or protagonists strongly in several narratives. As is visually evident in Figure 2, we found a moderate positive correlation between these two quantities ( = 0.25), although this relationship did not reach statistical significance because of the small number of participants who completed the study ( = 0.14). 3.3.2 Engagement with narratives is selective and heterogeneous. While the average relevance score across narratives gives a reasonable summary of a user’s engagement with the system, it also conceals important information about the specificity users’ interests in self-help readings. To uncover this information, we show the average relevance ratings for all user-narrative pairs in the user study in Figure 3a.
The interesting observation here is the lack of any global structure in the heatmap, which would have indicated clusters of user or narrative similarities, which in turn would have suggested value in collaborative filtering-based approaches for making recommendations in this domain. The absence of such structure recommends caution in applying such approaches: it looks like users are highly selective about which narratives they find personally relevant, and there is very little similarity in these preferences across users. Tolstoy said that every unhappy family is unhappy in its own way. It looks like individuals find resonate with self-help narratives also each in their own way. 3.3.3 Personalized recommendations create significantly lower engagement. The last important insight our data reveals answers our second research question: do participants find value in receiving recommendations matching their own personal narratives of their present sense of being. Recall that each user rates 5 personalized and 5 randomly selected narratives in each session, efectively generating a two condition within subject manipulation of personalization.
Figure 3b plots the average relevance rating for all participants in each of these two conditions - while rating narratives selected by the RS, and while rating randomly selected narratives from the same corpus. Interestingly, the randomly generated narratives are rated significantly higher than personalized narratives (two sample t-test = 4.4, < 0.001). Thus, it looks like personalized recommendations are actually reducing user engagement in our user study. 4
In this paper, we have presented a recommender system that attempts to mimic the role of a counsellor suggesting informal bibliotherapy. The key novelty of the system is the use of natural (a) Average relevance rating for each blog-person tuple across seven elicitations (b) Average relevance rating across participants while rating personalized and random suggestions from the recommender system language processing to transform long-form narratives generated by the user as a description of their current mental condition into relevance cues used to retrieve semantically related readings.
We also report preliminary results from a user study, wherein
we found that engagement with the system is weakly correlated
with elevated mood among participants who completed the study.
This correlation, however, should not be mistaken for causation. It
is certainly possible that experimenter-demand efects induced by
responding on the same system over and over again might account
for much of the measured improvement [
The user study also uncovers intriguing evidence supporting the
case that conventional recommendation strategies may not work
very well in self-help settings. The pattern of ratings displayed
by our participants suggests that this domain does not lend itself
very easily to collaborative filtering based recommendations. Our
controlled within subject manipulation of personalization also
reveals that content-based recommendations appear to lead to lower
user engagement than random recommendations from the same
thematic corpus of readings. These observations substantiate recent
theoretical proposals about the inappropriateness of conventional
recommendation strategies in self-help settings [
The possibility that suggesting narratives resembling the user’s
personal narrative of distress may backfire is not counter-intuitive.
It is quite conceivable that suggesting too related a narrative might
cause the user to not acquire suficient psychological distance from
their own predicament for therapeutic mental transformations to
occur. This observation joins the ever-growing list of unexpected
dificulties that the psychological feedback loops perpetuated by
recommender systems generate [
The research we have reported in this paper is very much a work in progress, and we are in the process of extending it in two dimensions. One, collecting data from more participants will enable us to draw conclusions with greater statistical confidence, as well as perform more interesting sub-cohort analyses using individual diferences between users. Two, as stated above, interview-based debriefing of study participants may give us more direct evidence to support the inferences we are currently drawing indirectly from data analysis.
We are grateful to our study participants for their support in testing the system.