Web mashups are becoming increasingly popular. A mashup is a Web application that combines data from more than one source into a single integrated form. Several mashup creation tools, so-called mashup makers, exist and many of them are advertised as easy to use tools for casual users. These tools target Web users without programming background and promise that creating a Web mashup is just a matter of a few mouse clicks. However, no profound usability evaluations have been performed to justify those claims. The purpose of this PhD research work is twofold. Firstly, we want to investigate the usability of these mashup makers and verify if they indeed satisfy their promise of being easy to use. Secondly, and also for the purpose of accomplishing the first goal, we will develop a usability framework that can be used to evaluate the usability of existing and future Web mashup makers. In this paper, we discus the research objectives of the PhD work, the methodology used, related work, as well as the results achieved so far.
Mashup originally referred to the practice in pop music (notably hip-hop) of producing a new song by mixing two or more existing pieces [19]. In computer technology, a mashup is a Web application that integrates data from more than one source. A well-known example is the use of cartographic data from Google Maps to add location information to some costumer’s data, thereby creating a new service that was not originally provided by either source. The most common way to develop a mashup is by accessing content via a public interface or API. That data is made available by relevant Web protocols such as REST, RSS and Web services [20]. The data is extracted from the output of these APIs, and is then passed to the mashup site where the logic resides, it could be server-side (dynamic content aggregation) and/or client-side scripting or both of them. The application then is rendered graphically and transferred to the client’s Web browser.
Several mashup creation tools, so-called mashup makers, exist and many of them are advertised as easy to use tools for casual users. Our research objective is to investigate the usability of mashup makers for casual users, i.e. users without programming background. We believe that this research is very important from two perspectives. The first perspective is to check if those mashup makers indeed fulfill their promises and meet the needs of casual users (first major objective of the PhD). The second perspective concerns usability necessities for mashup makers for casual users. Our investigations and studies may reveal a considerable amount of hindrances and difficulties that casual users are facing when using mashup makers. One of the results of our usability study could be a set of minimal usability necessities for those kinds of tools. Therefore, it is also our aim to develop a set of usability criteria that can be used to measure usability of existing and future mashup makers. Together with a number of benchmarks that can be used to evaluate the usability of mashup makers and usability evaluation procedures, this will form the Usability Framework that we aim for as second major objective of the PhD.
As we want to investigate the usability of mashup makers, we first should define what we mean by mashup usability. Usability, as defined by ISO9241 part11 [15], is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”. Applying this definition of usability to mashup makers, we define mashup usability as the extent to which a mashup maker can be used by specified users (in our case casual users with no programming background) to achieve specific goals (in our case a web mashup maker) with effectiveness, efficiency and satisfaction in a specified context of use. In other word, mashup maker usability is about user aspects and measurements when learning, creating, developing, building, using and handling mashup creation tools. This should not be confused with web usability that usually refers to how well users can learn and use a Web site to achieve their goals.
The main contribution of our research work is the development of a Usability Framework for Web mashup makers for casual users. Challenges in this research can be summarized as follows. No usability evaluation framework for web mashup makers for casual users exists. Therefore, it is necessary to investigate which criteria should be considered for evaluating the usability of these tools. Next, the existing tools are very different in the type of mashups they allow to create. Therefore, it will not be obvious to compare them; a framework for this is also needed. 2
The work for this PhD is divided into 7 main steps. For each step, we will give the main objectives and the actions that we will take to reach the objectives.
Step 1. Objective: To obtain an overview of existing Web mashup makers, in order to discover the main issues related to Web mashup technology and to have a concrete understanding of the possibilities and limitations of Web mashup technology. Method: A literature study on Web mashup technologies, and reading related tutorials.
Step 2. Objective: To obtain an overview of mashup usability in order (1) to discover the main issues related to mashup usability and to obtain a good understanding of Web mashup usability; (2) to check related work in the context of measuring the usability of Web mashup technology.
Method: Literature study on usability, Web usability and Web mashups usability.
Step 3. Objective: To obtain a deeper understanding of the usability issues related to mashup makers for casual users Method: Selection of some Web mashup technologies for casual users and performing experiments with them in order to get practical knowledge and experiences on how these tools should be used. Here we intend to perform the experiments ourselves.
Step 4. Objective: To define a set of mashup usability criteria, i.e. usability measurement factors that can be used to evaluate the usability of Web mashup makers for the target audience (casual users).
Method: Critical analysis of the results of Step 2 and Step 3, and the identification of missing and/or irrelevant usability issues. Further investigation of the relationship between the usability criteria identified and the target audience.
Step 5. Objective: Development of an experimentation environment Method: Selection of a set of representative Web mashup makers to be used in the experiments; selection of a representative set of target users; preparation of the experiments that will be performed.
Step 6. Objective: To reach the first objective of the research: verifying the usability of existing mashup makers Method: Performing the usability experiments prepared in Step 5, analyzing the results by means of statistical methods, and summarizing the results.
Step 7. Objective: To reach the second objective of the research: development of a usability framework for mashup makers for casual users.
Method: Reinvestigation of recent research to keep track of new developments; evaluation of the approach used in Step 6; collecting and resuming guidelines, criteria’s and benchmarks for mashup usability into a coherent usability framework. 3
In this section we review some works related to usability of Web mashup makers. Our literature research and first investigations have found that there is no complete and comprehensive work about Web mashup usability at this moment.
usability guidelines and web usability guidelines have been used as a basis to create an outline of Rich Internet Application (RIA) usability guidelines. Most of their work is focused on a comparative study of general usability guidelines. In conclusion they only formulated some so-called start guidelines for developer in the field of RIA.
MIT Potluck Usability evaluation. Potluck [9] is a project that aims at the development of an easy to use tool to mashup data for casual users. They performed a usability evaluation study to ascertain whether people could learn how to use Potluck as well as to discover usability problems. We have learned a lot from their work and we will follow some of their notes related to the usability evaluation of mashups tools.
to create web mashups by browsing around, without needing to type, or plan in
advance what you want to do. The research team of MashMaker has performed a
usability evaluation of the tool following [7] and using the Cognitive Dimension of
notations (CDs) framework [
Oleg Beletski [
Marmite usability evaluation. Marmite [21] is an end-user programming tool for mashups that repurposes and combines existing Web content and services. Marmite is targeting users with programming backgrounds and spreadsheet skills. They have also performed a usability evaluation study. This usability evaluation has helped us understanding a new way of evaluating mashup development tools.
authors reviewed six mashup makers from the so-called End User Development
perspective. There reviewing methodology is a mix between some selected
dimensions of the CD´s framework [
Compared to this work, our research focus on a complete and robust methodology for evaluating the usability of web mashup makers. 4
At the moment of writing, Step 1, 2, 3, and a part of Step 4 have been performed.
Among different usability evaluation methods that we have reviewed, we currently
have selected the cognitive dimensions (CDs) of notations framework [
Furthermore, we have conducted a study on 8 general purpose mashup tools:
Yahoo Pipes YP [23], Microsoft Popfly MP [12], Intel Mashmaker IM [
Despite this limitation, the study itself was very interesting as we were able to detect which of the dimensions are useful to consider in further experiments and which should be omitted. This gives us useful information for the definition of the usability criteria. Furthermore, we found that it may also be necessary to divide the target users further into groups based on their computer skills, and background in English, because all considered mashup makers provide their interfaces in English. To allow for a better comparison between the different tools in further experiments, the experimentation environment should include a common example, and should also provide learning materials and some know-how tutorials. 7. Green, T., Petre, M., “Usability Analysis of Visual Programming Environments: a ‘cognitive dimensions’ framework”, Journal of Visual Languages & Computing, Volume 7, Issue 2, June 1996, Pages 131-174. 8. Gwardak, L., Påhlstorp, L., “Exploring Usability Guidelines for Rich Internet Applications”, Master thesis Lund University, 2007, available at: http://biblioteket.ehl.lu.se/olle/papers/0002774.pdf 9. Huynh, D., Miller, R., Karger, D., “Potluck: Data Mash-up Tool for Casual Users”, ISWC’2007 available at: http://people.csail.mit.edu/dfhuynh/research/papers/iswc2007potluck.pdf 10. IBM mashup center, http://services.alphaworks.ibm.com/graduated/mashupcenter.html 11. JackBe, http://www.jackbe.com 12. Microsoft Popfly, http://www.popfly.com 13. OpenKapow, http://openkapow.com/Default.aspx 14. Ort, E., Brydon, S., and Basler, M., “Mashup Styles”, 2007, available at: http://java.sun.com/developer/technicalArticles/J2EE/mashup_1/ 15. ISO 9241 Part 11, “International Standard for Ergonomic requirements for office work with visual display terminals (VDTs) – Guidance on Usability”, 1998. available at: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=16883 16. Programmable Web, http://www.programmableweb.com 17. Simon Peyton Jones, “Wearing the hair shirt: a retrospective on Haskell” (invited talk). In
ACM SIGPLAN Conference on Principles of Programming Languages (POPL’03), 2003. 18. Storey, M., Grammel, L., “An End User Perspective on Mashup Makers”, September 2008, available at: http://lars.grammel.googlepages.com/paper_mashup_makers.pdf. 19. Wikipedia, http://en.wikipedia.org 20. World Wide Web Consortium, http://www.w3c.org 21. Wong, J., and Hong, J., I., “Making mashups with marmite: towards end-user programming for the web”. In CHI ’07: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 1435 -1444, New York, NY, USA, 2007. ACM Press. 22. Wong, J., Hong, J., “Patterns in Mashups”, WEUSE IV 2008, ACM 1-58113-0000/00/0004 23. Yahoo Pipes, http://pipes.yahoo.com 24. Zillner, T., “Mashing It Up”, A talk at Annual WAAL Conference, April 2007, available at: http://www.wils.wisc.edu/events/waal07/Mashing_It_Up.webbed.ppt