e beyond-relevance objectives of recommender system are drawing more and more aention. For example, a diversity-enhanced interface has been shown to positively associate with overall levels of user satisfaction. However, lile is known about how a diversityenhanced interface can help users to accomplish various real-world tasks. In this paper, we present a visual diversity-enhanced interface that presents recommendations in a two-dimensional scaer plot. Our goal was to design a recommender system interface to explore the dierent relevance prospects of recommended items in parallel and to stress their diversity. A within-subject user study with real-life tasks was conducted to compare our visual interface to a standard ranked list interface. Our user study results show that the visual interface signicantly reduced exploration eorts required for explored tasks. Also, the users' subjective evaluation shows signicant improvement on many user-centric metrics. We show that the users explored a diverse set of recommended items while experiencing an improvement in overall user satisfaction.
Recommending people in a social system is a challenging task. e
user may look for other people for a range of reasons; for example,
they may wish to re-connect with an acquaintance or to nd a new
friend with similar interests [
A specic case in which a single ranked list might not work well
is in a parallel hybrid recommendation system that fuses several
recommendation sources. In this case, dierent sources might be
preferred for dierent needs (i.e., social similarity could work best
for nding known friends while content-based similarity could be
used to nd people with similar interests). Several authors argued
that the best approach in this situation is to oer users the ability
to control the fusion by choosing various algorithms [
When selecting a visual metaphor for the transparent fusion of
recommendation sources, we focused on beer informing users
about the diversity of the recommender results. It has been
demonstrated that a proper user interface could promote diversity in
information exploration. A diversity-enhancing interface evaluated
in [
In this paper, we aempted to use a scaer two-dimensional plot
visualization to present recommendations with several dimensions
of relevance. A scaer plot is an intuitive way to present
multidimensional data [
RELATED WORKS
e social recommender system should provide more diverse
content for the user to extend the social connection outside of the
personal bubble. However, not every user equally values the
diversity with the same standard [
Providing visual interface is an approach to improve the
recommendation diversity. ere are some previous works has been
conducted. For example, adopting a visual discovery interface can
increase click-through rate (CTR) across dierent item categories in
an e-commerce website [
ere is some design principle from the literature review. e
study of [
We propose a recommender system to help conference aendees to nd other relevant aendees to meet with a dual interface, which includes a ranking list and visual scaer plot components. e ranking list is a classic way of presenting recommended results in a single dimension, listed from high to low relevance. e scaer plot was added as a diversity-promoting interface to show the recommended result in two dimensions, with the second dimension used to reveal the overall diversity. Figure 1 illustrates the design of the dual interface.
Section A is the proposed scaer plot. e interface presents
each item (a conference aendee) on the canvas as a circle. e
user can move the mouse over the circle to highlight the selection.
Section B shows the control panel with which the user can interact.
e user can select the number of recommendations to display,
and both the major feature and the extra feature to visualize the
recommendations on the scaer plot. e major feature is used to
rank the results along the X axis and in the ranked list (section C),
while the extra feature shows the diversity of results in the selected
aspect along the Y axis. To further investigate the diversity of the
displayed recommendations, the user can also use a single aspect of
the data as a category to color-code the results. e default category
was Smart Balance, which color codes in the four quadrants with a
(1)
(2)
(3)
0.5 ratio. Section C is the standard ranking list. More exactly is a
combination of four ranked lists produced by four recommender
engines explained below. To make four dimensions more clear, a
normalized relevance of each user to the target user generated by
each recommender engine is shown on the right side of the ranked
list. Section D presents more detailed information about the person
selected in either the visualization or the ranked list. Among other
aspects, four of the six tabs visually explain how each recommender
engine calculates the relevance of the selected user to the target
user. e design detail of the explanation functions can be found
in the work of [
To rank other aendees by their relevance to the target user, the
system uses four separate recommender engines that rank other
aendees along four dimensions that we call features: text
similarity of their academic publications, social similarity through the
co-authorship network, current interests of CN3 activities, and the
distance of their aliated place to the target user. Each of the
features is dened as below:
(1) e Academic feature is determined by the degree of
publication similarity between two aendees using cosine similarity
[
SimAcademic ¹x; yº = ¹tx ty ºktx k kty k
where t is word vectors for user x and y.
(2) e Social feature approximates the social similarity between
the target and recommended users by combining co-authorship
network distance and common neighbor similarity from publication
data. We adopted the depth-rst search (DFS) method to calculate
the shortest path p [
SimSocial ¹x; yº = p + n (3) e Interest feature is determined by the the number of cobookmarked papers and co-connected authors within the experimental social system.e function is dened as
SimI nt er est ¹x; yº = ¹bx º \ ¹by º + ¹cx º \ ¹cy º
where bx ; by represent the paper bookmarking of user x and y;
cx ; cy represents the friend connection of user x and y.
(4) e Distance feature is simply a measure of geographic
distance between aendees. We retrieve the longitude and latitude
data based on aendees’ aliation information. We used the
Haversine formula to compute the geographic distance between any pair
of aendees [
SimDist ance ¹x; yº = Haversine¹Geox ; Geoy º (4) where Geo are pairs of latitude and longitude coordinates for user x and y.
Diversity Navigation Model e system determines the personalized relevance score for all conference aendees. Instead of ranking the recommended people by ensemble value, the user can lter the items based on multiple aspects of relevance through our system. ere are two kinds of diversication.
1) Feature diversication: the user can select any two pairs of proposed features and spot the recommended items from the relevance intersection. All of the proposed features were calculated on a dierent scale. For example, the distance feature is the physical distance in miles, while the academic feature is calculated as a percentage. To enable comparison of diverse features, we adopted a standard Z-score to normalize all the features to the same scale from 0 to 1. e function was dened as:
Z Score = xi
uj σj (5) where xi is ith recommended item and j represents the corresponding features from 1 to 4. en, we use the standard Z-table to convert the Z Score to the corresponding percentile pi j . Hence, we can list all the features on the same scale for presentation in a ranking list or scaer plot diagram.
2) Coverage diversication: a diversication model to help the
user select the recommended item from a dierent category. [
We can then measure the user selection diversity through the
two diversication model. We observe the user interaction with
items from dierent ”quadrants” (feature intersections) [
(6)
i=1
where pi is the probability for a particular quadrant (feature or
category) and the proportion of all of the user’s selections [
Data and Participants
e recommendations produced by all four engines are mostly
based on data collected by the Conference Navigator 3 (CN3) system
[
A total of 25 participants (13 female) were recruited for the user study. All of the participants were aendees at the 2017 Intelligent User Interfaces Conference (IUI 2017). Since the main goal of our system was to help junior scholars connect with other people in the eld, we specically selected junior scholars, such as graduate students or research assistants. e participants came from 15 dierent countries; their age ranged from 20 to 50. All of them could be considered as knowledgeable in the area of the intelligent interface for at least one academic publication from IUI 2017. To control for any prior experience with the recommender system, we included a question about in the background questionnaire. e average answer score was 3.28 on a ve-point scale. 4.2
To assess the value of the diversity visualization, we compared the dual interface with the scaer plot and the ranked list (SCATTER) with a baseline interface using only a ranked list (RANK) with part A removed. e study used a within-subjects design. All participants were asked to use each interface consecutively for three tasks and to ll out a post-stage questionnaire at the end of their work with each interface. At the end of the study, participants were asked to explicitly compare interfaces along of their preference. e order of using interfaces was randomized to control for the eect of ordering. In other words, half of the participants started the study with the SCATTER interface. To minimize the learning eect (geing familiar with data), we used data from two years of the same conference: the SCATTER interface used papers and aendees from IUI 2017, while the RANK interface used the same data from IUI 2016.
Participants were given the same three tasks for each interface.
T ask1 : Your Ph.D. adviser asked you to nd four Commiee Member candidates for the dissertation defense. You need to nd candidates with expertise close to your research eld while trying to lower the travel cost to the defense.
T ask2 : Your adviser asked you to meet four aending scholars, preferably from dierent regions across the world, with a close connection to your research group.
T ask3 : You want to nd four junior scholars (not yet faculty members) with reasonably similar interests among the conference aendees to establish your networking.
e participants were asked to pick suitable candidates among conference aendees based on their best judgment in each task. When designing the tasks, we aempted to make them realistic, yet focused on multiple aspects of relevance, as many real tasks are. We consider that task 1 is relevance-oriented and tasks 2 & 3 are diversity-oriented. For a relevance-oriented task, we expect to see if the proposed interface helps the user to lter the desired target eciently. For the diversity-oriented task, in contrast, we expect to see the user interact with the recommendation result diversely, compared to the baseline interface. 5 5.1
User’s Objective Evaluation e result of the users’ click paern is shown in Figure 2. e arc diagram shows a dierent click paern when the user is using the two interfaces. e users click to a more diverse recommendation through the scaer plot interface. is nding supports the design of dual interfaces can facilitate users to explore the recommendation result beyond the top rankings. Table 1 shows the system usage for two interfaces. e data indicate that participants extensively used both the control panel and explanation tabs to complete the tasks. ere is no signicant dierence between the interfaces, although, in the SCATTER interface, the users tend to use the explanation functions less.
RANK SCATTER RANK SCATTER Task 1 3.88 4.12 8.56 8.56 Task 2 2.88 2.88 6.56 4.8 Task 3 2.56 2.84 8.12 6.76
Overall 9.32 9.84 23.23 20.12 Table 1: Usage Analysis: control panel usage (the frequency of user change and submit the setting of control bar), explanation tab usage (the frequency of the user switch the tab on User Prole Page). Column 2 & 3 shows the comparison of user clicks between RANK / SCATTER interfaces.
Task 1 -37.16% -69.71%(*) +9.21% +161.7%(*) Task 2 -59.53%(*) -63.67%(*) -11.91% +115.2%(*) Task 3 -55.51%(*) -66.45%(*) +50.14% +179.6%(*) Overall -48.35%(*) -67.07%(*) +9.47% +134.8%(*) Table 2: Eciency Analysis: the frequency of hover, click, task time (seconds for nish each task) and engage time (seconds between each click). All columns show incremental changes between RANK and SCATTER interfaces. (*) indicates statistical signicance at the 0.05 level.
Table 3: Diversity Analysis: the test of diversity and coverage with two category variable. All columns show incremental changes between RANK and SCATTER interfaces.
Table 2 shows the work eciency comparison between the two interfaces. We counted how many mouseovers (hovering) and clicks the users made to complete each task and expressed the number of actions done in the SCATTER interface as a percentage increase or decrease from the RANK interface. e data shows that with the SCATTER interface, users completed the same tasks with 40-60% fewer mouseovers and about 66% fewer clicks. At the same time, we found no signicant dierence in the time spent on the tasks. e data hints that each action taken in the SCATTER interface delivered more interesting information to explore. Indeed, we found that with the SCATTER interface, the users spent signicantly more time engaged in analyzing results.
Table 3 shows the diversity analysis for each task and interface. We found that the diversity and coverage measurement shows the task dierence. All three tasks are with a signicant feature diversity dierence between two interfaces but in the dierent aspect of features. Task 1 (relevance-oriented) shows less diversity on academic/distance features and less coverage on the country and position variables. e SCATTER interface helped users to more accurately explore the aendees with multiple types of relevance. Tasks 2 & 3 (diversity-oriented) show more diversity in the interest/distance and social/distance features, respectively, as well as higher coverage in the country category. e result shows the user response to the same task with a dierent paern of exploration on diversity and coverage. 5.2
To compare subjective feedback, responses to the post-stage questions were analyzed using paired sample t-tests. e result of this analysis is shown in Figure 3. We compared the eight aspects of subjective feedback from the participants. Among them, the SCATTER interface received a signicantly higher rating for six aspects: Trust (Q4), Supportiveness (Q5), Interest (Q6), Satisfaction (Q8), Intention to Reuse (Q9), and Enjoyable (Q11). In two questions, facilitation (Q7) and the control-reversed Benet estion (Q12), the SCATTER interface scored higher, but not signicantly. It is interesting to see that the RANK interface scored a bit higher (though not signicantly) on explanation usefulness, which hints that the lack of visualization made explanations more important in the RANK interface. In the nal preference test, the SCATTER interface received much stronger support than the RANK interface in the user preference feedback (Figure 4). Most importantly, a majority of users (84%) considered the SCATTER interface to be a beer system for recommending aendees and a beer help in diversity-oriented tasks, as well as beer for recommending. 6
In this paper, we presented a dual visual interface for recommending aendees at a research conference. A research conference context introduces several dimensions of aendee relevance, such as social, academic, interest, and distance similarities. Due to these factors, a traditional ranked list makes it dicult to express the diversity of recommended items (aendees). By spreading ranking over two dimensions, the suggested interface helps users in exploring recommendations and recognizing their diversity in several aspects. Our approach can be applied to any recommender system with multiple relevance features and item categories. To assess the visual approach, we conducted a user study in a real conference environment to compare our interface (SCATTER) with a traditional ranked list (RANK) in three practical tasks.
Our experimental result shows the tangible incremental impact the metrics of system usage, eciency, and diversity. We found that the SCATTER interface benets more on the aspect of perceived tasks and helps enhance diversity tasks. Results from the nal preference survey show a strong preference for the SCATTER interface. Interestingly, we also found that users of the SCATTER interface beneted more from the feature diversity tasks. e user feedback suggests that it would be easier to nd and categorize variables through the RANK interface. However, even the user feedback indicates an ease of use for selecting and inspecting an item by category through the RANK interface. e users of the SCATTER interface still show signicantly higher coverage measurement between tasks.
e main contribution of this paper is to prove that the enhanced
diversity interface not only helps the user to perceive diversity [