=Paper=
{{Paper
|id=Vol-2848/user2agent_paper_9
|storemode=property
|title=A Snooze-less User-Aware Notification System for Proactive Conversational Agents
|pdfUrl=https://ceur-ws.org/Vol-2848/user2agent-paper-6.pdf
|volume=Vol-2848
|authors=Yara Rizk,Vatche Isahagian,Merve Unuvar,Yasaman Khazaeni
|dblpUrl=https://dblp.org/rec/conf/iui/RizkIUK20
}}
==A Snooze-less User-Aware Notification System for Proactive Conversational Agents==
https://ceur-ws.org/Vol-2848/user2agent-paper-6.pdf
A Snooze-less User-Aware Notification System for
Proactive Conversational Agents
Yara Rizk Vatche Isahagian
yara.rizk@ibm.com vatchei@ibm.com
IBM Research AI IBM Research AI
Cambridge, Massachusetts Cambridge, Massachusetts
Merve Unuvar Yasaman Khazaeni
munuvar@us.ibm.com yasaman.khazaeni@us.ibm.com
IBM Research AI IBM Research AI
Cambridge, Massachusetts Cambridge, Massachusetts
ABSTRACT from a wide set of users while customizing these models to
The ubiquity of smart phones and electronic devices has individual users’ preferences.
placed a wealth of information at the fingertips of consumers
as well as creators of digital content. This has led to millions CCS CONCEPTS
of notifications being issued each second from alerts about • Human-centered computing → User models; Natural
posted YouTube videos to tweets, emails and personal mes- language interfaces; • Computing methodologies → Ma-
sages. Adding work related notifications and we can see how chine learning.
quickly the number of notifications increases. Not only does
this cause reduced productivity and concentration but has KEYWORDS
also been shown to cause alert fatigue. This condition makes Alerts, Notification, User-centric, Proactive Chatbots
users desensitized to notifications, causing them to ignore
ACM Reference Format:
or miss important alerts. Depending on what domain users Yara Rizk, Vatche Isahagian, Merve Unuvar, and Yasaman Khazaeni.
work in, the cost of missing a notification can vary from 2020. A Snooze-less User-Aware Notification System for Proactive
a mere inconvenience to life and death. Therefore, in this Conversational Agents. In Proceedings of ACM Conference (Con-
work, we propose an alert and notification framework that ference’17). ACM, New York, NY, USA, 4 pages. https://doi.org/10.
intelligently issues, suppresses and aggregates notifications, 1145/1122445.1122456
based on event severity, user preferences, or schedules, to
minimize the need for users to ignore, or snooze their notifi- 1 INTRODUCTION
cations and potentially forget about addressing important Due to increased adoption of digital transformation, we are
ones. Our framework can be deployed as a backend service, bombarded with hundreds of notifications every day on our
but is better suited to be integrated into proactive conver- smart phones (and other electronic devices) from calendar
sational agents, a field receiving a lot of attention with the reminders, Twitter notifications, and Facebook posts. Then,
digital transformation era, email services, news services and there are work related event notifications including work
others. However, the main challenge lies in developing the emails, calendar invites, deadline reminders, etc.
right machine learning algorithms that can learn models Handling notifications as soon as they arrive can cause dis-
ruption of work, loss of focus on specific tasks, and decrease
of overall productivity in the workplace, especially when
Copyright © 2020 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
notifications arrive at the rate of minutes instead of hours.
Checking them in bulk increases the risk of important noti-
fications falling through the cracks. Furthermore, depending
on when users prefer to check their messages (mornings ver-
sus lunch breaks or evenings), missed deadlines and delayed
response to urgent alerts may increase.
Even worse is experiencing alert fatigue, a problem faced
in many domains from business to healthcare [5], where too
many notifications make individuals desensitized to them.
This, in turn, causes them to miss some critical notifications
IUI'20 Workshops, Cagliari, Italy Rizk et al.
that may have detrimental consequences. For example, in interrupt users with notifications by monitoring users’ event
healthcare, critical test results may be missed, leading to the streams in an application. In intrusion detection, an alert
death or sever impairment of patients. In business, significant management system that post-processes alerts to determine
amounts of capital may be lost or workers may be put in whether to issue notifications or not has been proposed and
harms way as a result of missed alerts. aims to minimize the number of missed critical alerts [7].
In this work, we propose an alerting framework that can
be embedded within proactive conversational agents or de- 3 BACKGROUND AND DEFINITIONS
ployed as a backend service, to issue, suppress and aggregate To better understand the need for our framework and the
notifications. This system attempts to reduce alert fatigue associated design choices, in this section, we highlight the
through selective issuance and suppression of notifications. different complexities and dimensions that our framework
A learning algorithm models the behavior of the observed needs to take into consideration. We begin by providing
system (e.g. estimates the number of expected alerts) and a basic understanding of what an event is, something that
user preference to determine which alerts to issue, suppress happens in a dynamic system that is characterized by specific
or aggregate and the means of alert notification. User pref- features/conditions. These events can occur at a high or low
erence can be learned by observing user behavior and fine frequency rate, can be either discrete or continuous with
tuned with explicit feedback from the user through natural monotonic or oscillatory trajectories. Next, we define alerts
language conversation with the proactive chatbot. In addi- and notifications in this framework.
tion to reducing alert fatigue and minimizing the risk of
missing important notification, t his f ramework enhances • Alert: A message that is generated by a watcher of a
worker productivity and concentration by minimizing dis- dynamical system when an event occurs.
tractions. The main set of research questions that must be • Notification: A message that is sent to a user when
addressed to enable such an alerting framework has to do the conditions of an alert are satisfied.
with how do we learn from user data the effective user-specific Alerts are associated with dimensions such as severity (e.g.
rules to issue, suppress and aggregate notifications. error, warning, info, not available), criticality (e.g. critical,
Next, Section 2 briefly s urveys related work. S ection 3 non-critical), urgency, and duration (e.g Repeated vs. one-
introduces the background and notation. Section 4 details shot). Notifications are also associated with dimensions such
the proposed framework. Section 5 discusses the challenges as the notification cycle (periodic vs. aperiodic), and the
facing an intelligent user interface to enable the framework. notifications channels (email, slack, etc.). These dimensions
Section 6 concludes with final remarks. add to the complexity of the problem. To further highlight
the complexity, we provide two exemplar scenarios below.
2 RELATED WORK Consider a non-professional stock market trader who is
Some work about alert and notification systems exists to interested in monitoring the prices of specific mutual funds.
control notification issuance. Rule-based systems were the She is at work during specific times of the day and cannot be
most commonly adopted approaches. Consel et al. [2] devel- available to check the prices at certain hours and is offline
oped an alerting system for assisted living that reminded an most of the weekdays after 6 pm or is working from home.
elderly person to take their medication or lock the doors, for These conditions infrequently change based on some other
example. The system relied on predefined rules to determine personal commitments. It would be ideal if a system can learn
when to issue notifications based on their priority. Bazinette the user’s availability and run queries to issue notifications
et al. [1] allowed users to customize the notifications from about prices or events during these times to avoid periodic
a wide range of sources through a single system. AlertMe notifications that will be ignored or having any important
provided users with the capability of customizing alerts and news buried in a sea of periodic updates. Furthermore, noti-
used semantic analysis to issue custom alerts [4]. fications can also be thought of as potential signals to take
More recently, machine learning based approaches have actions. For example, in case of money invested in shares, the
been investigated in restricted domains. Tomavsev et al. [8] user can get a notification to either sell her shares because
developed an approach to issue alerts by predicting whether the market has potentially reached a peak point and might
patients’ condition will deteriorate based on test results. Un- fall soon (in this case the alert will depend on the price at
like [8], our work also suppresses or aggregates instanta- which user bought shares and the market’s volatility). This
neous notification of non-urgent alerts. Motivated to prevent action signaling alert can also be sent internally to other
missing any notifications, Oh et al. [6] proposed an intelli- subsystems, hence setting off a chained process.
gent notification system that used user context to provide Consider a travel preapproval process at research insti-
timely notifications. Finally, Iqbal et al. [3] proposed a notifi- tutes that handles employees’ travel expenses to conferences.
cation management system that determined the best time to A director could configure the alerting system to notify her
Snooze-less Notifications IUI'20 Workshops, Cagliari, Italy
of any new or pending travel requests. It would be great if the are intelligently generated based on the information from the
system intelligently issues or suppress notifications by query- various data sources and components before a notification
ing a data store to estimate how many employees will submit scheduler determines when to emit the notification, while
a travel request to an upcoming conference. Then, instead of taking into account the user’s schedule.
issuing notifications for each submission, it will aggregate
them and send a single notification. If the system knows Opportunities for Learning
that there are pending requests for an upcoming conference There are multiple opportunities for learning (depicted by
but employees are also submitting requests to another con- “brain" clip-arts in Fig. 1) from users in this framework both
ference at a later date, it will issue a notification about the in a direct and indirect fashion. At the overall system level,
upcoming conference but suppress all other notifications to users’ direct feedback through the UI can be used as re-
prevent the former requests from being overlooked. wards/punishment in a reinforcement learning framework.
While its possible to handle such scenarios today using The system would then learn a model to issue, suppress and
rule-based systems, that approach suffers both from an in- aggregate notifications. Indirect feedback can also be used by
crease in the number of rules as well as the complexity of keeping track of which notifications, issued by the conver-
rules which make them harder to create and maintain. Fur- sational agent, were snoozed, ignored, opened immediately,
thermore, these rules are typically enterprise focused and deleted without being opened, etc. Focusing on individual
not user focused. Given the recent advancements in artificial components of the system, machine learning algorithms such
intelligence, we envision a user focused learning framework as deep neural networks can be adopted to classify alerts,
within a proactive conversational agent. learn user behavior, determine the notification modality, se-
lect the notification channel, and schedule notifications.
4 METHODOLOGY
The framework consists of two main components (Fig. 1): pre- 5 INSIGHTS AND CHALLENGES
dictive alert (in blue) and notification (in green) management. The proposed framework has multiple advantages. It can
A conversational agent with a user interface (UI) allows users intelligently reduce the number of issued notifications thus
to customize their alerts and notifications, through natural reducing alert fatigue. Less notifications imply less distrac-
language, as well as provide feedback to train the model. tions; therefore, the users’ productivity and concentration
would increase. Allowing users to provide feedback would
Alert Management System enable better customization of the learned model. A byprod-
The alert management system consists of multiple sub- uct of the system deciding whether to issue or suppress an
components. First, an alert customization module takes the alert and how to aggregate alerts is implicitly sorting the
user’s input and provides recommendations to the user based tasks which improves productivity. Our domain agnostic
on the system’s behavior. It can be modeled as a classification implementation allows us to easily extend the framework to
and clustering problem where similar alerts are clustered and other domains, especially considering its utility in diverse
then classified based on relevance to the user. Depending on fields from business to healthcare. Unlike other approaches
the amount of available data, more sophisticated learning in the literature, our invention does not adopt a rule-based
algorithms can be adopted to achieve more accurate results. approach to determine which alerts to issue/suppress, and
Then, an event watcher is configured based on this customiza- targets a broader scope than some of the other approaches.
tion to monitor the system, making use of the system model Furthermore, it does not require knowledge of the under-
and predicted behavior to optimization observations. When lying model of the dynamical system it is observing and
a change happens in the system to trigger an alert, the alert corresponding alert and notification modules. Learning from
generation engine creates and forwards it to a classifier that unstructured or semi-structured natural language messages
utilizes a taxonomy to characterize the alert. An alert aggre- generated by the modules reduces the integration overhead.
gator determines whether to issue or to suppress an alert The proposed framework faces multiple challenges that
or to aggregate alerts and issue them at a later date. This must be addressed before real-world deployment. Customiz-
information is sent to the notification management system. ability and generalizability are two conflicting goals. Finding
the right tradeoff between both will be crucial to the success-
Notification Management System ful deployment of the framework. Furthermore, classification
The notification management system consists of multiple models make mistakes; and mistakes in one domain can have
user-centric components. User behavior is modeled to in- more severe consequences than in other domains. For ex-
fluence the modality, communucation medium (channel), ample, how can we guarantee that the classification models
format and time of notifications. Users also have the ability will not make mistakes in life and death situations? Can we
to directly set their preferences using the UI. Notifications place safeguards around them to ensure that such mistakes
IUI'20 Workshops, Cagliari, Italy Rizk et al.
Figure 1: Detailed Workflow of the Alerts Framework
are avoided or caught in time? Furthermore, it is difficult to be integrated within proactive chatbots, email services, news
determine why certain machine learning models make the services or other services that generate notifications. The
decisions they do. This reduces users’ trust in the system and system would classify the severity of various alerts, and learn
acceptance of it. Therefore, explainability is a crucial com- from user data while maintaining confidentiality and privacy
ponent of this system: how can we allow machine learning of the data. The main challenge, and focus of future work,
algorithms to explain their decisions to non-expert users to lies in finding the right data and machine learning models
gain their trust. Another challenge is integrating the system that can learn customizable and generalizable models.
with applications that are generating the events that our
system uses to evaluate the alerts and send notifications. REFERENCES
From a user interaction perspective, it is important to de- [1] Vincent Bazinette et al. 2001. An intelligent notification system. IBM
sign the conversational interface in a user friendly manner to Research Division, Thomas J. Watson Research Center, PO Box 704 (2001).
[2] Charles Consel, Lucile Dupuy, and Hélène Sauzéon. 2015. A unifying no-
maximize the usefulness of the collected data while minimiz- tification system to scale up assistive services. In The 17th international
ing the learning overhead or burden on the user when using SIGACCESS conference on computers & accessibility. ACM, 77–87.
the system. The feedback interface should also be simple [3] Shamsi T Iqbal and Brian P Bailey. 2008. Effects of intelligent notification
enough to encourage users to provide meaningful feedback. management on users and their tasks. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems. ACM, 93–102.
Clarifying user intents, and enabling disambiguation partic-
[4] Asterios Leonidis et al. 2009. Alertme: A semantics-based context-
ularly when the user requests the creation, modification, and aware notification system. In 33rd International Computer Software and
deletion of alerts through a conversational interface would be Applications Conference, Vol. 2. IEEE.
crucial. Another challenge is to empower users to customize [5] Daniel R Murphy, Brian Reis, Dean F Sittig, and Hardeep Singh. 2012.
notifications, provide feedback, and effectively incorporate Notifications received by primary care practitioners in electronic health
records: a taxonomy and time analysis. The American journal of medicine
that feedback into the reinforcement learning model.
125, 2 (2012), 209–e1.
[6] Hyungik Oh, Laleh Jalali, and Ramesh Jain. 2015. An intelligent notifica-
6 CONCLUSION
tion system using context from real-time personal activity monitoring.
This work proposes an alerting and notification system that In International Conference on Multimedia and Expo. IEEE, 1–6.
is application-agnostic and personalizable based on predic- [7] Tadeusz Pietraszek and Axel Tanner. 2005. Data mining and machine
tive and adaptive machine learning algorithms. A method of learningâĂŤtowards reducing false positives in intrusion detection.
Information security technical report 10, 3 (2005), 169–183.
notifying, escalating, and suppressing alerts should tap into [8] Nenad Tomašev et al. 2019. A clinically applicable approach to continu-
different types of available datasets to learn user-centric mod- ous prediction of future acute kidney injury. Nature 572, 7767 (2019),
els including: 1) dynamical system behavior, 2) user behavior, 116–119.
and 3) event modeling and classification. This framework can