This paper presents the results of a small field study to identify major barriers to adoption of a social software tool. Gleanr is a Web 2.0 application that can be used for group information management, social bookmarking, and personal research and branding. We present a brief overview of the software and its affordances, describe the study, and reflect on the results as we discuss lessons learned from our first pilot deployment.
Over the past decade, social computing has emerged
immensely as a phenomenon among distributed
communities. The benefits of social systems depend on
a large part on the existence of an active user
community who use it continuously to deploy and share
information. However, while certain systems have
enjoyed tremendous success (Facebook, twitter), others
have experienced modest adoption at best. It is not clear
what factors contribute to the rise and fall of these
systems. This paper is a report on our experience with
the deployment of a social software tool and our
attempts to identify the major barriers to its adoption.
We first introduce the system, Gleanr [
Gleanr is an online personal information management tool that allows users to control their web presence. The main idea behind Gleanr is that with today’s vast array of social tools (blogs, twitter, Facebook, etc.), modern knowledge professionals are faced with too many output streams to fill and too many input streams to follow. Gleanr provides means to automate one’s web flow through customized Gleanr channels. Here is how it works: Upon finding something interesting or relevant to their current activity, users can select the exact content and add it to their Gleanr account by clicking the Gleanr bookmarklet on their browser (figure 1). At the same time, they can edit, tag, and set access rights on the captured content. As a result, not only the content will be saved in Gleanr, but also depending on how the user has set his/her account, the newly captured content will also update his/her tweeter, Facebook, and/or LinkedIn status. Users can also subscribe to the information streams of others if they are interested. Gleanr then aggregates, indexes, and networks all of one’s captured or created information. Considering these functionalities, Gleanr can be used as a personal information management tool, a personal research assistant tool, a personal branding tool, or a collaborative tool. Figure 2 shows a screen shot of a series of “gleans” created by various users in Gleanr.
The first public dissemination of Gleanr happened in fall 2009. In order to further promote the software, a handful of acquaintances of the Gleanr design and development team were invited to one-to-one sessions where we introduced the software and it applicability to their daily information management practices as we saw fit for each individual. We also asked each of the attendees to introduce Gleanr to their personal and professional network if they find it useful for them. In total, 10 people accepted our invitation to use Gleanr for a certain period of time and participate in our study. The individuals’ ages ranged from 25 to 40 and they were all well-versed in the use of computers in general and social software systems in particular. While none of the participants were information technology professionals, for all of them use of the Internet and activities such as bookmarking, reading feeds, and publishing online content was part of their professional and/or personal life.
With their permission, we closely tracked and monitored the usage of the tool for this select group over a threemonth time period. After that, they were invited for a contextual inquiry in which each individual was asked to fill in a survey questionnaire and participate in a semistructured interview. The participants’ age ranged from 30 to 40 and they came from various technical and nontechnical backgrounds. While for some, the use of the tool had become a daily necessity, others had not warmed up to it. This gave us the opportunity to analyze the users’ reaction to the tool in different circumstances and to probe for specific success factors and usage barriers in each specific context.
In order to collect feedback on both benefits and challenges of using Gleanr for our user community, we organized our questions along various dimensions. First, for each of the main functionalities that Gleanr provides (i.e., channels, bookmarklet, RSS feed, broadcast, search, etc.) we asked each of our participants whether they have been using the feature and if not, whether this has been due to the lack of usefulness or ease of use. Second, we set to find out whether the tool provides enough cues as to how to start a task and its intermediate steps by asking our participants whether they can figure out how to use any of these system functionalities (i.e., whether it is easy to set up account, create/use channels, feed to LinkedIn and FaceBook, or use the bookmarklet). Next, we moved on to the more advanced features of the system such as privacy management and channel administration; and finally, we discussed the community effect in Gleanr by investigating whether our participants have taken advantage of others' presence in the system (by searching for other tags/channels/people, for example, or subscribing to someone else’s channels). We also asked each participant to name the most beneficial feature of Gleanr and whether there are other features that they would like to see added.
In the following sections, we present the findings of the
research and reflect on the lessons learned from Gleanr’s
first pilot deployment. The findings have been
categorized according to the six dimensions of the
Delone and McLean information systems success model
[
Like many other information systems, the main barrier in the adoption of Gleanr seemed to be lack of simplicity. We noticed that often times the reason behind not using a feature in the system was either the fact that users had not noticed it or that they had found it cumbersome to use. In various occasions, users reported to have stopped using a feature after making frequent mistakes, getting stuck, or seeing some unexpected behavior. This emphasizes the need for making main system functionalities simple, highly visible, and intuitive (i.e., what is the first thing to do upon entering the application?). This is especially important for adoption, since it is highly unlikely that users would be willing to take some mental effort to enter the system before they have been exposed to its benefits.
One important factor in the design of every social tool is providing the right balance in the cost/benefit equation. Cost is usually defined in terms of initial effort associated with learning or using a tool (whether there is need for training, for example). Benefit, on the other hand, comes from addressing a need or presenting an advantage over other similar tools or alternative methods of performing a task; such as increased speed, better availability, portability, or additional functionality. It is important that such benefit (or a tool’s value proposition) is clearly defined and communicated to users. Our results indicated that Gleanr has been successful in articulating its value proposition to users. Our participants named a number of benefits in using Gleanr, including control over one’s digital presence (e.g., what appears about them in search engines), fast indexing (how quickly Gleans appear in search engine results), ability to manage all contributions and activities that one personally makes in a tool, ease of tracking information of interest, and ease of modification. A major benefit mentioned was the availability of the usergenerated social metadata associated with the content (e.g., tags, comments, channels, ratings, etc.), which helped users to assess the relevance of information from various resources, and to arrive upon niches of high relevance and interest to them, thus adding to the overall quality of the information available to them through the tool.
Gleanr creates a pervasive technical infrastructure that
includes people, assets, relationships, and activities as
fundamental system components. Moreover, the
connection between content, its associated metadata (i.e.
tags), users, and their relationships creates opportunities
to exploit the “wisdom of the crowd”, and deeper
analysis of community structure helps identifying trends
and experts. However, the "wisdom of the crowd" model
works only if the system is successful in securing a
critical mass of members who use the system regularly
[
One interesting finding of our study, however, was that
usage depends more on how many people from the
personal network of the user are using Gleanr, as
opposed to how many users Gleanr has as a whole. In
this regard, the private invitation function in Gleanr
(supplemented with a video tour describing Gleanr
functionalities and how to start with it) proved to be a
suitable tool in improving adoption, as it enabled
satisfied users to inform others in their network of their
satisfaction and invite them to see a preview. Another
interesting finding was that awareness of the activities of
one’s network had a motivating effect on users to
participate more actively in Gleanr. Providing such
awareness, however, often presents a trade-off with
privacy as users need to be reassured that their data
won’t be exposed against their will [
Another important factor in shaping users’ willingness to use Gleanr was ease of integration with other tools and services. When asked how they would define Gleanr after using it for a while, our participants’ answers ranged from “a glorified bookmarking tool”, to “a personal publishing tool”, to “a centralized organization tool”, to “a collaborative tool for research”, which showed that the many affordances of the tool allow users to tune it to their particular context of use. One consistent aspect of users’ experiences, however, was that they all had tried to integrate it with the tools they were currently using for the same purpose, being it a blog (for personal publishing), a wiki (for collaborative research), or CiteULike (for bookmarking). One positive aspect of Gleanr that consistently showed up in users’ reports was ease of such integration (e.g., one can write a blog post in Gleanr and automatically feed it to the blogging tool of his choice). Also, the automatic feed from Gleanr to Facebook, Twitter, and LinkedIn proved to be quite popular with users, as it enabled them to use Gleanr as a portal to feed their various Web 2.0 applications.
Generally, users don’t want to be trained to use a tool, and they don’t want to have to change their behavior in order to be able to embed the tool in their daily activities. Furthermore, a fast Return of Investment (ROI) is often needed to secure their continuous use of a software application. While Gleanr seemed to perform relatively good with regard to the first two cases, one problem seemed to be the fact that users needed to make a considerable initial investment in the tool (by Gleaning content and creating and managing channels) before Gleanr’s main value proposition (personal branding) would materialize. This slow ROI (the differed benefit) was one of the major complaints users had about Gleanr. Our results indicated that users will not be satisfied with a system that could help some time in the future, but one that is of immediate value for their everyday professional and/or personal lives. As such, providing them with clear evidence of a immediate benefit that will make them better off seemed imperative.
One way to create incentive for initial usage is to prepopulate the application with valuable, relevant content in a simple and accessible format. While this will guarantee that the immediate value of the tool is obvious to users (even before they have used it extensively), there is also need for low cost/risk methods that allow users to test the waters and train behavior. One reason behind users’ reluctance to post content in social tools is confidence: users are often hesitant to post because they are not sure of their opinion or how the group will react to it. Providing anonymous and aggregated contribution methods (such as rating and ranking) can help alleviate this problem as these functionalities provide a sense of feedback from community in terms of value/relevance by showing users that their opinions matter while allowing them a non-threatening venue to test group reaction. Providing such anonymous and aggregated contribution methods is part of our future development plans for Gleanr.
Also, as a tool that claims to help users build credibility based on their surfing experience, users expected Gleanr to help them create an audience (i.e., by acting as an information broker) or somehow show their relevance in the area they were trying to identify themselves with (i.e., by facilitating between a user that Gleans about biking and companies that sell stuff related to biking). Implementing an information broker functionality is also part of our future plans.
While all the factors so far deal with the back-end design and information architecture of the system, the last factor deals with the front-end design and users experience of the tool. This is equally important, as poor user experience can cause the most solid systems to fail. To get an understanding of how users felt about the system (as opposed to what the design team perceived it), we gauged users’ feedback with regard to the usability of Gleanr by asking them about their ease of interaction with the system. Users’ comments showed that while Gleanr seems to be doing reasonably fine in terms of navigation, visualization, and staging (i.e., novice users can start by simply Gleaning, and then move towards the more advance features such as channeling, privacy management, automatic feed, etc., as they become more advanced), there are some usability problems that need to be addressed. Here we mention a few: • • • •
Learnability: Although all of our participants had been able to successfully use Gleanr after the initial introduction, most of them said that they wouldn’t have been able to do so if Gleanr was not introduced to them in a one-to-one session. This implies that Gleanr has a steep learning curve that needs to be improved if we want our users to rapidly begin working with the system.
Efficiency: Even after using Gleanr for a while, users claimed to sometimes get confused as to where to look for a certain functionality or do a certain task. This implies that better navigation, search, and help options are needed to enable users (who have already learned the system) to attain a high level of productivity with it.
Memorability: Related to the previous two issues is the issue of memorability: allowing the casual user to return to the system after a period of non-use without having to re-learn everything. Although we didn’t particularly test Gleanr for memorability, the two previous issues imply that this might also appear as a problem in the future.
Error Rate: It is important for a tool to ensure low error rate, so that users make fewer and easily rectifiable errors while using the system. Furthermore, catastrophic errors must be prevented. Gleanr didn’t fair very well in this regard, as most users reported having continuous problem with channel management and setting feeds.
Portability: Finally, integration across platforms (iPhone, desktop, etc.) was mentioned as a missing desirable functionality.
Although Gleanr has been designed with use cases beyond just educational, it is not hard to imagine it getting adopted and used as a PLE. In fact, some of our users had been using Gleanr as a personal research assistant; a usage which is very much inline with the goals and purposes of a PLE. As such, our findings, while situated in the general context of social software, are well applicable to PLEs as well. It is important to note, however, that considering the nature of the tool and the focus of our study, the identified adoption factors are rather addressed to the context of social software in general and that specific characteristics of the personal learning environments have not been factored in. It is logical to assume that conducting the same study in a tool specifically designed for eLearning or used for educational purposes might result in a revised set of factors or put emphasis on some factors and less on others.
While the weaknesses of Gleanr (such as complexity and
deferred benefit) can be considered as lack of fitness to
the environment, its strong features (such as ease of
information tracking and powerful control through
privacy management) can be identified as characteristics
that can ensure (or at least improve the chances of)
evolvability. The evolvability of a biological system has
been widely studied and shown to be dependent on
several properties [
In this paper, we reported on a diagnostic evaluation study on a social software system, Gleanr, to identify its positive as well as negative features and to evaluate its fitness for the purpose. By identifying main areas where users have difficulty with the system, we were able to probe major usability problems, obtain approximate measures for users' effectiveness, efficiency, and satisfaction, and gain an understanding of users’ perspective on the tool.
Social software systems are different from other software tools in terms of ubiquity, scale, collision, and exposure. To be successful, a social software system needs to recognize this difference. Based on the results of our subjective assessment, we proposed a set of criteria for the success of social software tools. Although our study was small-scale, the results seem to present a general view of factors that can potentially affect success or demise of a social software tool. We hope that these results can benefit other researchers and practitioners in creating social tools with better chances of mass adoption.