Many intelligent systems can be personalised by end-users to suit their specific needs. However, the interface for personalisation often trades of the degree of personalisation achievable with time, efort, and level of expertise required by the user. We explore two approaches to end-user personalisation: one asks the user to manually specify the system's desired behaviour using an end-user programming language, while the other only asks the user to provide feedback on the system's behaviour to train the system using reinforcement learning. To understand the advantages and disadvantages of each approach, we conducted a comparative user study. We report participant attitudes towards each and discuss the implications of choosing one over the other.
• Human-centered computing → User interface programming.
IUI Workshops’19, March 20, 2019, Los Angeles, USA
Copyright © 2019 for the individual papers by the papers’ authors. Copying
permitted for private and academic purposes. This volume is published and
copyrighted by its editors.
shown that this information is crucial to meeting
participation [
To facilitate eyes-free participation in a meeting, we are
studying the use of a computer vision system to extract the
visual information that is important for equitable meeting
participation. Information that may be extracted include
attendee identity [
The ‘most useful subset’ varies not only from context to context, but also from user to user. A goal of an intelligent system is to automatically adapt to the context or to users’ preferences to achieve a personalised experience and improve performance. It is not possible for system designers to correctly anticipate user preferences in many contexts, so a naïve heuristic-based solution to the subset selection problem would not be satisfactory. Thus, it is important to have a personalization interface. However, it is challenging to enable users to specify preferences for automated adaptation without requiring excessive time and efort.
This paper explores two alternatives for end-user personalisation of such a system. One approach is to learn automatically from how the user interacts with the system and from user feedback, which is well suited to reinforcement learning models. Reinforcement learning typically requires a large number of feedback instances from the user to reach the desired level of personalisation. The reward signal for reinforcement can be given explicitly (e.g., by the user marking whether behaviour was appreciated or not) or implicitly (e.g., if the user turns the volume down for some notifications, or actively queries the system for information). End-users may not be able to give a fully-informed response because they cannot visually perceive all information in the environment, and may not realise when insuficient or incorrect information has been given. This further leads to users having less control and therefore less ownership and trust of the system.
An alternative approach is to enable the user to directly specify their preferences, using a simple end-user programming paradigm to define the system’s behaviour, such as rule-based programming (e.g., if this, then that). Explicit rule-based programming has the potential for avoiding user frustration and increasing ownership of the system. However, users might struggle to specify their needs in such ways; or to anticipate what information they may want in diferent situations. To design such systems efectively, we were motivated to understand strengths and weaknesses of each approach. Background work in this area is in Appendix B. 2
To explore personalisation for an eyes-free meeting, we created a meeting simulator. It creates a readily-available source of well-controlled meeting scenarios. Each generated meeting contains a variable number of attendees. For each attendee, the following 8 attributes are generated: relation, name, pose, activity, eye gaze mode, location, clothing and gender. These are further elaborated in Table 1 in Appendix A. These were chosen based on current inference capabilities of computer vision systems, as well to provide enough diversity for interesting personalisation opportunities.
Each simulated meeting contained 3–8 attendees. Participants provided names of colleagues prior to the study, allowing us to constrain each simulated meeting to contain 1–3 attendees known to the participant. The value for each attribute was randomly chosen, with constraints. For example, if an attendees’ activity is typing, then the eye gaze mode will be looking at the phone or laptop.
Two interfaces were built to personalise meeting overviews, corresponding to rule-based programming and reinforcement learning. Both used the same meeting attributes as the basis for personalised notifications.
This interface supports the manual creation of diferent sets of rules for diferent global contexts (e.g. diferent meeting sizes). The user would define the global context first (e.g. total number of attendees > 5). Then they would define the associated notifications (e.g. provide the name for all attendees). They could have a diferent set of rules for a diferent context (e.g. if the total number of attendees less than or equal to five, provide the name and location for all attendees). Alternately, a user may want diferent notifications based who is speaking. Then one possible set of rules could be: when there is an attendees whose relation is known and activity is speaking, tell me the eye gaze mode for all attendees; when there is an attendee whose relation is unknown and activity is speaking, tell me the pose for all attendees. The second consideration is to set notifications for specific attendees. For example, the user might want to know diferent information for attendees based on their relation. Possible rules could be: tell me the name and activity for attendees whose relation is known, tell me the location and pose for attendees whose relation is unknown.
Based on this, we divide our rule-based programming interface into two parts to support two level nesting if-clauses: the if-clause to set the ‘global context’ condition, and the if-clause to set notifications for specific attendees when the ifrst-cause is satisfied (Figure 1). The first part enables users to set the ‘global context’ condition for the whole meeting, which could be: when the total number of all attendees {=, <, > x }, x is a number between 0 and 8; when the total number of attendees whose one attribute is or is not one certain value {=, <, > x }; when there is an attendee whose one attribute is or is not one certain value and another attribute is or is not one certain value. The second part is setting notifications for specific attendees when the meeting satisfies the ‘context’ condition, which could be: if attendees’ one attribute is or is not one certain value, then tell me these attributes.
Users can enter an arbitrary number of rules, and an arbitrary number of if-clauses and notifications per rule. Users can also review, edit and delete existing rules (Figure 1).
In reinforcement learning [
Reinforcement learning problems can be solved using
Qlearning and an ϵ-greedy policy [
In the context of our meeting simulator, the input to the Q-network includes the features of each attendee, which are the values of each attribute. Each instance of the attendee’s attributes used in the rule-based programming interface is a feature for the reinforcement learning model. There are 17 features in total, which includes 4 features for name (known A, known B, known C, other), 2 features for relation (known, Edit Rule Enter Rule
Part 1: Set the "global context" condition for the whole meeting
Relation is Known
Any Attendees Name PRAoecltsaievtiitoyn KUnnokwnonwn Eye Gaze Mode Number > 2 Number Name Relation Pose Activity Eye Gaze Mode > 0 < 1 = 2 3 4 5 6 7 8
Select this condition Selected conditions: When the total number of attendees > 2 for attendees whose relation is known
Part 2: Set notification for specific attendees
Pose is Sitting Al Attendees Name Relation PAoctsievity SStitatnindging Eye Gaze Mode Other
Name Number Name Relation Location Pose
Activity Eye Gaze Mode
Clothing
Gender
Select these notifications Selected notifications:
Notify me the name for attendees X whose pose is sitting Submit Delete
X
unknown), 3 features for pose (sitting, standing, other), 5
features for activity (reading, typing, speaking, listening,
other), and 3 features for eye gaze mode (screen projector,
other member in meeting, other). The Q-network consists
of three fully connected (FC) layers with rectified linear
unit (ReLu) activation functions [
We focus here specifically on the moments at the very beginning of a meeting, which we call the meeting overview, rather than an ongoing assistive experience throughout the meeting. While we recognise the importance of ongoing assistance, we have chosen to focus only on the meeting overview to make the scope of our study tractable.
Fifteen participants were recruited using convenience sampling, 8 female and 7 male. Participants spanned four organizations and worked in a range of professions including real estate planning and surveying, business operations, interaction design research, machine learning research, and biomedical research. Six participants self defined as programmers, and among them, five participants had experience in machine learning from taking courses or research experience. We will refer to our participants as P1, P2, ..., P15.
In this experiment, we considered a scenario where participants need to join a meeting with a few colleagues that they know and some people that they do not know. We told the participants that they had dialed into the meeting late without any video, but that the system could give them an overview of what was happening when they entered the meeting. They were then given a detailed explanation of the various attributes. The task was to train the system to only deliver information of interest to the participant.
The experiment used a within-subject design. Our two conditions were rule-based programming and reinforcement learning. All participants performed the task twice, once in each condition, in counterbalanced order.
Participants were first introduced to the meeting simulator. Then there were two sessions. In each session, participants used a diferent adaption technique. There were 3 sections in one session: default setting, training, and testing. Figure 3 describes this procedure and timeline. In the ‘default setting’ section, participants entered a simulated meeting, where they were notified about all information about all attendees.
In each ‘training’ section, participants were asked to use
the two strategies we provide to personalise the
notification system for 20 minutes to train the system to only give
them the information they want to know to understand the
situation. The participants’ cognitive load for this training
process was measured using the standardised NASA Task
Load Index questionnaire ([
Participants also filled out a questionnaire about their satisfaction level with using this approach. This questionnaire consists of four questions on a 7-point scale. Q1: after test meeting 1, how relevant was the notification to your interest? Q2: after test meeting 2, how relevant was the notification to your interest? Q3: How successful were you at creating the rules/how successful do you think the system was at learning your preferences? Q4: In general, how satisfied are you with the notifications after training the system?
At the end of the experiment, we conducted a short semistructured interview to ask about participants’ experiences and opinions. We asked what participants struggled with when using the two techniques, their opinions about the advantages and disadvantages of the two techniques they experienced, as well as what would be the ideal approach for them to personalise such a system. Participants were encouraged to think-aloud throughout the experiment. The entire study was audio recorded and transcribed.
For the rule-based programming approach, the researcher also helped participants navigate the interface. The researcher did not help participants with developing rules or formulating rules. Considering the time constraint of the experiment, this enabled participants to focus on the rule-based approach in comparison with the reinforcement learning approach, rather than the specific interface that may require training to fully understand. The experiment took about 60 minutes and participants were compensated £20 for their time. 4
Our quantitative comparisons showed that participants reported a significantly lower cognitive load, and significantly higher satisfaction scores, when using rule-based programming rather than reinforcement learning. Our complete statistical analysis is presented in Appendix C.
The rest of this section presents our qualitative analysis of participants’ experience during and after the experiment.
Even though all participants were able to personalise the system by using rule-based programming, they also asked questions or requested help from the researcher. The following sections present common issues participants experienced. Dificulty understanding the filtered information. Some participants had dificulty understanding the meetings with the notifications constrained by their rules. This means that participants had false expectations about the notifications they would receive based on their rules, and therefore had dificulties uncovering what was happening in the meeting based on the filtered information. This issue always happened at the beginning of the training, and was always resolved with the help of the researcher. For example, P5 described her frustrations:“I want to know when people are speaking, whether people are listening and what they are doing now, this includes much more information. For example, when I imagine this, I thought I would know the people who are speaking and who are listening, and what exactly they are doing. When I told you to use the rules, it lost too much information.” In this case, P5 entered rules to set the notifications to tell her who is speaking and who is listening, but since there was no one speaking or listening in the meeting overview (the beginning of the meeting), there was no notification. However, P5 had dificulty understanding this logic. Dificulty maintaining multiple rules. While the system supports participants in creating multiple rules, it is also more dificult to keep track of multiple rules and understand how they work together. For example, P2 asked the researcher when creating multiple rules: “So all of these rules are stacking up?” P4 questioned how multiple rules work together after trying multiple rules in one meeting: “Okay, the last one was very short, did some of these [rules] override each other?”
Participants also became confused with notifications when dealing with multiple rules, even when knowing how they work together. For example, P6 created diferent rules for meetings with diferent sizes. After one meeting, P6 mentioned he was not satisfied with the information. The researcher inquired further, and mentioned that the meeting is a small size and the notifications were given based on his rules for small meetings. Then, P6 said: “This is a small meeting? Oh, then yes, I am satisfied. Sometimes I get confused.” Confusion about what the interface supports. Some participants expressed confusion about the capability of the interface. For example, P2 said, “I don’t know [if] what I need, or what I am asking for, is more granularity in control or things which can’t be given.” Some participants wanted functionality the interface did not provide. For example, P3 commented, “The way I was hoping is to be able to train the system to take out the stuf I don’t like to hear about, like I don’t want to know the clothing for everyone.” However, the interface only supports users to specify attributes they want to hear. Some participants were unaware of functions the interface supported. P11 commented, “Still there some things, instances or features, like other activities, are not that informative, but I guess it is the best we can do, I can not just grows out the other activity.” Although, in fact, the interface can support users to specify if the activity is other, then...
Dificulty giving binary feedback. Arriving at a binary judgment about each notification was described as dificult by many participants. Frequently, they expressed a desire to be able to give richer feedback. In keeping with best practice for fast convergence of reinforcement learning, participants were instructed to only give positive feedback to the system when they were completely satisfied with the notification, i.e., when there is neither any unnecessary nor missing information. However, participants struggled to understand this concept and usually needed further explanation. For example, P10 asked: “Should I say ‘yes’ if I am fully satisfied with it?” P13 asked: “ If I am okay with that, but if I need some new information, should I press yes or no?” Some participants also developed their own heuristics for giving feedback, even after the researcher’s explanation. P8 described her strategy: “One part is ‘yes,’ another part is ‘no,’ so I can say ‘yes.”’
Moreover, many participants wanted to give richer
explanations to the system about why they gave specific feedback
and what they want. For example, P5 gave this explanation:
“If someone is sitting and silent, maybe I want to know that
he is sitting, but if someone is speaking, I don’t want to know
whether he is sitting or standing.”
Concerns about inconsistent feedback. Participants were
concerned about providing inconsistent feedback, and took
responsibility for an unsatisfactory outcome if they had given
inconsistent feedback. As the training continued,
participants gradually learned more about their own preferences
and changed their mind about what they wanted, a form of
concept evolution [
Moreover, some participants thought it was their fault if they gave inconsistent feedback. For example, P10 commented: “I feel like I misled the system a little bit while I was still trying to figure out, during the training, what stuf I want.” Uncertainty about the personalisation ability of the system. Participants were unsure about the capability of the reinforcement learning model, and concerned about its outcome. The capability of the reinforcement learning model refers to what preferences the system is able to learn. Even though participants were only asked to give feedback to the system and did not need to worry about the underlying model, their understating of what the system is capable of can also afect how they give feedback. For example, P6 asked the researcher: “Maybe the second sentence came out, I will say ‘yes,’ but similar sentence came out at the fifth, I will just say ‘no,’ because at first I did not get too much information, but then I got more information, should I say ‘no’?” In this case, P6 was satisfied with more information about the first few attendees during the overview, but wanted less information when the system moved to the following attendees because the information started to be overwhelming. However, the order of the notifications was not implemented as a feature of the reinforcement learning model, and therefore the system is not able to learn this particular preference.
Moreover, during the training, some participants wondered about how the reinforcement learning model works and the outcome of the training. For example, P2 commented that she was focused on reasoning about how the system worked rather than the notifications themselves: “Maybe I think too much about - maybe trying to reason about what it is doing, rather focusing on the notifications themselves, and I found that distracting a little bit.” P9 asked the researcher about what will happen during the training: “Does the training, for the whole training period, give me all the options?” Participants were also curious about the outcome of the model. P10 asked, “Is my score gonna converge to a certain value?” P6 also asked the researcher: “So what is its conclusion?”
Advantages and disadvantages of the two approaches. Participants identified rule-based programming as fast and straightforward, and expressed feeling more in control of the system by entering the rules themselves. Six participants in particular described liking the functionality to be able to add, edit and delete the rules. However, two participants mentioned that it is dificult to come up with the rules from scratch, while other participants had clear rules in their head in the beginning. Three participants (all non-programmers) expressed concern if they had to use the rule interface themselves without the help of the researcher. For example, P3 said: “I would fail at the logic if I have to do that”. However, the majority of participants believed they could learn to use the interface easily if given a tutorial or by trial and error.
With reinforcement learning, participants thought it was easy to just give simple feedback to the system, and the variance of the outputs helped them figure out what they wanted. The disadvantage was that the training process was repetitive and took a considerable amount of time. Preference between approaches. The majority of the participants expressed preference for the rule-based programming approach over the reinforcement learning approach, due to the advantages described above. However, three participants (two of whom self-defined as non-programmers) said that they preferred the reinforcement learning approach because it learned their preferences and it was satisfying to automatically receive personalised notifications.
For this specific system and scenario, most participants thought the ideal approach would be a combination of the two approaches, while three participants (two of whom selfdefined as programmers) thought they would only need rulebased programming as it was an efective way to achieve well-defined preferences, but they could see the advantages of reinforcement learning for others. For people who wanted a combination of the two approaches, all of them expressed a similar idea of being able to control the system’s behaviour by entering or editing rules, while also using a pre-training phase with the reinforcement learning model to help them ifgure out what they want or to suggest rules. Then, the reinforcement learning model could also learn from how they use the system and suggest rules. For example, P13 mentioned that it would be helpful to detect what is a big versus a small meeting for him.
We found that participants were unsure about the system’s adaptation capability when using both techniques. With the rule based programming, they were unsure about what the rule grammar could express. For example, do users have to define only what they want or can they specify what they do not want? For the reinforcement learning model, they were unsure about what the system could learn. For example, can the system learn that the user wants to know the pose, or that the user only wants to know when the pose is standing?
We think issue is important because the first step towards personalisation is to form preferences. In some cases, the preferences are easy to form, but there are also cases where it is dificult for users to figure out what they really want. A clear understanding of the system’s personalisation capability can help them develop reasonable preferences. Rule-based programming was easy when users were sure about what they wanted and the interface supported them to achieve that. However, even when users were satisfied with the rules they created, there might not realise the system was more capable. For example, users could be satisfied with one rule for all meetings, but unaware that they could have set diferent rules for meetings with diferent sizes. The reinforcement learning approach has the advantage of exposing users to diferent actions that the system can take, therefore helping users figure out what they want. Then, a clear understanding of what the system is capable of learning can guide users to give feedback. A possible solution for this is to provide examples and explanations to users in the beginning.
Based on the results of our study, some participants struggled to express their preferences using the rule-based programming interface, and most struggled about how to give feedback to the system when using reinforcement learning. Rule-based programming can support users to edit/add/delete rules anytime, and therefore, the main issue for this technique centers on how to enter rules they have in mind, especially for users with little programming experience.
For the reinforcement learning model, the continuous training phase and binary feedback make it dificult for users to gradually develop their preferences and express them. For example, users cannot answer ‘yes’ or ‘no’ questions when they are not even sure what they want. Also, when participants did identify what information they wanted, there was no way for them to express their preference to avoid the lengthy process of reinforcement learning.
This suggests that systems should make clear to users what actions are needed achieve certain preferences. Possible solutions are developing interfaces to support users to give more rich feedback, for example, the feedback in the reinforcement learning scenario could be a scaled satisfaction score from 1 to 7. Or a hybrid interface could combine rule-based programming and reinforcement learning: users identify their preference through iterative feedback, but can later specify precise preferences through rules.
We also found out it was dificult for participants to develop a cognitive model of the personalised system. When using rulebased programming, participants became confused when dealing with multiple rules. For example, it was dificult for them to know which rules were in use in diferent scenarios. When using reinforcement learning, participants were concerned about the outcome of the personalised system. They did not know what the system learned, and thus the outcome of the system was unpredictable.
To help users develop their cognitive model, it is necessary to provide context. For example, if there are diferent rules for big and small meetings, users should be informed of the meeting size and hence which rule is in efect.
In this work, we studied a pure rule-based programming interface as well as a pure machine learning interface, in an identical setting, going beyond previous studies that have not compared the two alternatives in precisely the same end-user programming task. We found that for this specific system and task, the rule-based programming approach was perceived as more straightforward and easier to use than the reinforcement learning approach.
Our results suggest that an efective personalisation approach for this scenario and task would be a combination of both approaches. Reinforcement learning could help generate candidate rules, help users form preferences when they are unsure of thier preferences, detect contexts that are hard to express otherwise. Rule-based programming could help define context, explain behaviour learned by a machine learning model, and provide an interface for editing or specifying behaviour not captured by the model.
The attributes of meeting attendees used in personalised meeting overview are described in Table 1.
Previous work has found rule-based programming to be a viable approach for intelligent systems, but with limitations including dificulty of learning syntax and low motivation for end-users. Machine learning approaches, such as reinforcement learning, can alleviate issues, but only for part of the personalisation problem (such as for specifying the context).
Barkhuus and Dey [
The distinction between personalisation and active context awareness as defined by this study is essentially that in both cases, the system detects changes in contexts and acts automatically. However, in the personalisation scenario, the behaviour has been defined buy the user, whereas in the active context awareness scenario, the behaviour is defined by the system designers. When the system designers correctly anticipate behaviour that end-users prefer, this works well. However, this is not an assumption we could make for the meeting scenario.
It remains an open question as to how best to combine
rule-based end-user programming with machine learning
and automatic inference. Many systems, such as CASAS [
Several studies have looked at rule-based programming for
personalizing intelligent systems. This approach has also
been referred to as trigger-action [
Some challenges have also been identified with rule-based
personalisation. Brush et al. [
Attribute used to resolve ‘conflicts,’ i.e., where diferent users create diferent rules, or where the same user creates diferent rules over time.
Machine learning from user annotated examples has been
found to be efective for specifying contexts. The a
CAPpella system [
The idea that context should be inferred from sensor data
appears to be a central assumption of context-aware
computing research. A survey by Hong et al. [
In this research, we sought to identify design guidelines
for end-user personalisation of intelligent systems. As
previously mentioned, context-aware programming makes limited
reference to the end-user programming challenges of such
systems. However, some previous work has presented
design principles for smart systems, such as Davidof et al. [
C
All participants successfully completed the task. This section describes some general observations, and presents the analysis of cognitive load and satisfaction score questionnaires.
All participants were able to personalise the system by creating rules with the help of the researcher. While the rules created by the participants share some similarities (e.g., most participants wanted to know the names of all meeting attendees), participants also showed a diversity of individual preferences (e.g., some participants wanted to know the clothing information for attendees they did not know, but some participants thought the clothing information was too distracting and not useful). Seven participants defined diferent sets of rules based on the properties of the meeting. For example: When the total number of attendees > 5, tell me the name, location for all attendees; When the total number of attendees < 6, tell me the name, location, activity for all attendees; When there is an attendee whose activity is speaking, tell me the name, activity for attendees whose activity is speaking. The other 8 participants defined a single set of rules to apply to all meetings.
Figure 4 shows the binary feedback given by each participant and the resulting training loss of the model. A feedback value of “1” means that the participant responded ‘yes, I am satisfied with the notification,’ and “-1” means that the participant responded ‘no, I am not satisfied with the notification.’ Participants encountered between 45 to 160 notifications during their respective sessions. This amount varies due to the randomized length of notifications generated by the system and participants’ speed of giving feedback. We did not find any particular pattern for how participants give feedback. The training loss was calculated based on the ground truth (users’ feedback) and the model’s prediction on the training data. The training loss converged to values close to zero for most users, indicating that the reinforcement learning model would likely take actions consistent with the participants’ feedback given during training.
For the training process, participants reported a lower cognitive load when using the rule-based programming approach than the reinforcement learning approach. Based on the fact that the researcher helped the participants navigate the interface when using rule-based programming, and that the participants have to continuously give feedback to the system when using the reinforcement learning approach, it is not surprising that the participants reported a lower cognitive load when using rule-based programming. In particular, Wilcoxon Signed-rank tests show that there are significant efects on the temporal demand (the medians were 20 and 35 respectively, the TLX is within a 100-points range; z=-1.999, p=0.044, r =0.365; physical demand (the medians were 5 and 10 respectively, z=-2.120, p=0.047, r = 0.387); performance demand (the medians were 20 and 50 respectively, z=-2.684, p=0.005, r = 0.490); and frustration level (the medians were 10 and 40 respectively, z= -2.316, p=0.018, r =0.423). See Figure 5. There are no significant diferences for the mental demand and efort.
Wilcoxon Signed-rank tests show participants reported significantly higher satisfaction scores when using the rulebased programming approach than the reinforcement learning approach for all four questions: Q1 (the medians were 7 and 4 respectively, the satisfaction score is a 7-point scale; z=3.330, p<0.005, r =0.607); Q2 (the medians were 7 and 5 respectively, z=3.329, p<0.005, r =0.607); Q3 (the medians were 7 and 4 respectively, z=3.320, p<0.005, r =0.606); Q4 (the medians were 7 and 4 respectively, z=3.316, p<0.005, r =0.605). See Figure 6. Again, it is not surprising that participants reported a high satisfaction score when using rule-based programming. As P10 mentioned when filling out the satisfaction score for this technique: “Of course I am satisfied, because I chose the rules myself." This suggests that users feel more in control when using rule-based programming. 1 11 11 11 11 1 cka 111 dbe 11 fye 11 ran 11 iB 11 11 11 11 11 1 0 20 40 (b) Training loss for each participant, which converged to values close to zero in most cases.