Towards a Usability Evaluation Process for Model- Driven Web Development Adrian Fernandez Emilio Insfran Silvia Abrahão ISSI Research Group ISSI Research Group ISSI Research Group Department of Information Systems Department of Information Systems Department of Information Systems and Computation - Universidad and Computation - Universidad and Computation - Universidad Politécnica de Valencia, Camino de Politécnica de Valencia, Camino de Politécnica de Valencia, Camino de Vera, s/n, 46022, Valencia, Spain. Vera, s/n, 46022, Valencia, Spain. Vera, s/n, 46022, Valencia, Spain. +34 96 387 73 50 +34 96 387 73 50 +34 96 387 73 50 afernandez@dsic.upv.es einsfran@dsic.upv.es sabrahao@dsic.upv.es ABSTRACT Usability evaluations methods for Web applications can be This paper presents an approach to integrate usability supported by a quality model which defines usability as a evaluations into Model-Driven Web development processes. quality characteristic that is decomposed into specific attributes Our main motivation is to define a generic usability evaluation that are easier to measure. Although there are several proposes process which can be instantiated into any concrete Web in this field, most of these approaches [12],[13] only consider development process that follows a Model-Driven Development usability evaluation at final stages when the product is almost (MDD) approach. A preliminary version of a Web Usability completed where correcting its usability problems is more Model was defined in order to support this usability evaluation difficult. It is widely accepted that evaluations performed at process at several stages. This Web Usability Model each phase of Web applications development is a critical part of decomposes the usability sub-characteristics (from the Software ensuring that the product will actually be used and be effective Quality Model proposed in the ISO/IEC 25000 SQuaRE for its intended purpose. We argue that integrating usability standard) into other sub-characteristics and measurable issues into the MDD approach can be an effective way to reach attributes. Web metrics are intended to be associated to this objective since the quality evaluation of intermediate measurable attributes in order to quantify them. Our approach is artifacts (models that specify an entire Web application), is intended to perform usability evaluations at several stages of a applied in all steps of the process [2]. A Web development Web Development process. In this paper, we show how process that follows a MDD approach basically transforms usability evaluations at final user interface (UI) can provide models that are independent from implementation details feedback about changes in order to improve usability issues at (Platform-Independent Models - PIM) into other models that intermediate artifacts (Platform-Independent Models and contain specific aspects from a concrete platform (Platform- Platform-Specific Models) or at transformations rules among Specific Models - PSM). Transformation rules, which are these intermediate artifacts. applied at PSMs, are able to automatically generate the Web application source code (Code Model - CM). Categories and Subject Descriptors This paper presents an approach to integrate usability evaluation into any Model-Driven Web Development method D.2.9 [Management]: Software quality assurance, D.2.8 by defining a usability evaluation process. This Web Usability [Metrics]: product metrics. H5.2 [User Interfaces]: Model has been defined by decomposing the usability sub- Evaluation/methodology characteristics (from the Software Quality Model proposed in the ISO/IEC 25000 SQuaRE standard) into other sub- General Terms characteristics and measurable attributes taking into account Measurement, Design. ergonomic criteria proposed in Bastien and Scapin [4]. Although our approach is intended to perform usability Keywords evaluations at several stages of a Web development process, in this paper, we mainly focus on how evaluations at final user Web Usability Model, Usability Evaluation, Web Metrics, interface (Code Model) can provide feedback about changes in Model-Driven Development. order to improve usability issues at intermediate artifacts (PIM and PSM models) produced at early stages of the Web 1. INTRODUCTION development process and at transformations rules among these Usability in Web applications is a crucial factor since the ease intermediate artifacts. or difficulty that users experience with this kind of systems will This paper is organized as follows. Section 2 discusses related determine their success or failure. Web applications are work that report usability evaluation processes for Web increasing its importance in industrial domains; thereby, the applications. Section 3 presents our approach to integrate need for usability evaluation methods that are specifically usability evaluations into Model-Driven Web Development. crafted for the Web domain has become critical. Section 4 presents our Web Usability Model that supports our Permission to make digital or hard copies of all or part of this work for approach. Section 5 shows a brief example of how the usability personal or classroom use is granted without fee provided that copies are evaluation process can be instantiated into a concrete Web not made or distributed for profit or commercial advantage and that development method. We mainly focus on evaluations at final copies bear this notice and the full citation on the first page. To copy user interface. Finally, Section 6 presents discussions and otherwise, or republish, to post on servers or to redistribute to lists, further work. requires prior specific permission and/or a fee. I-USED ’09, August 24, 2009, Upssala, Sweeden. 2. RELATED WORK code can be automatically generated from the intermediate There are several approaches that deal with Web usability artifacts (PIM and PSM models). evaluation, for instance, Ivory [16], Olsina and Rossi [13], In previous work, Abrahão and Insfran [3] proposed a usability Calero et al. [5], Seffah et al. [15], and Moraga et al. [12]. model for early evaluation in model-driven architecture Ivory [16] presents a methodology for evaluating information- environments. Usability was decomposed into the same sub- centric Web sites. The methodology proposes five stages: characteristics as the ones in the ISO/IEC 9126 (learnability, identifying an exhaustive set of quantitative interface measures understandability, operability, and compliance), and then such as the amount of text on a page, color usage, consistency, decomposed again, into more detailed sub-characteristics and etc; computing measures for a large sample of rated interfaces; attributes. However, the model did not provide metrics for deriving statistical models from the measures and ratings; using measuring the model attributes and it was not proposed the models to predict ratings for new interfaces; and validating specifically for the Web domain. Panach et al. [14] presents an model prediction. adaptation from the previous model to the Web domain in order to evaluate usability at PIM models for a concrete and Olsina and Rossi [13] proposed the Web Quality Evaluation proprietary Model-Driven Web Development approach. Method (WebQEM) to define an evaluation process in four technical phases: Quality requirements definition and As far as we know, there is no proposal for a generic usability specification (specifying characteristics and attributes based on evaluation process supported by a usability model in the Model- the ISO/IEC 9126-1 [9]. such as usability, functionality, Driven Web Development context. reliability, and effectiveness and taking into account Web audience’s needs), elementary evaluation (applying metrics to 3. THE USABILITY EVALUATION quantify attributes), global evaluation (selecting aggregation PROCESS criteria and a scoring model), and conclusion (giving Since the adoption of Model-Driven Development (MDD) in recommendations). Nevertheless, evaluations take place mainly the industrial domain has increased recently, our approach is when the application is completed. intended to integrate usability issues into a Model-Driven Web Calero et al. [5] present the Web Quality Model (WQM), which Development. Web development methods that follow this is intended to evaluate a Web application according to three approach such as OO-H [7], WebML [6], or UWE [11] support dimensions: Web features (content, presentation, and the development of a Web application by defining different navigation); quality characteristics based on the ISO/IEC 9126- views (models), including at least one structural model, a 1 (functionality, reliability, usability, efficiency, portability, and navigational model, and an abstract presentation model. These maintainability); and lifecycle processes (development, methods also provide model transformations and automatic operation and maintenance) including organizational processes code generation. such as project management and reuse program management. The usability of a Web application obtained as a result of a WQM has been used to classify, according to these three MDD process can be assessed at different abstraction levels dimensions, a total of 326 Web metrics taken from the existing (PIM, PSM and CM). Our proposal is intended to use a Web literature. An evaluation process can be defined by selecting the Usability Model, which is a set of sub-characteristics most useful set of metrics to construct a “total Web quality” decomposed into measurable attributes that can be quantified by expression that could be used to quantify the quality of a given metrics. The Web Usability Model can be applied to assess the Web application. However, guidelines on how to define this models from each abstraction level (see Fig.1). However, not all process have not been provided. the measurable attributes can be evaluated at all the abstraction Seffah et al. [15] present the Quality in Use Integrated levels. The higher abstraction level, the less attributes can be Measurement (QUIM) as a consolidated model for usability considered. In addition, feedback that is obtained after the measurement in Web applications. An editor tool has presented artifact evaluation has different targets depending on the to define measurement plans collecting data from different abstraction level: combinations of metrics proposed in the model. QUIM 1. At the PIM level it is possible to assess models that combines existing models from ISO/IEC 9126-1 [9], ISO/IEC specify the Web application independently of platform 9241-11 [8], and others. It decomposes usability into factors, details such as: presentation models, navigational models, and then into criteria. In this approach, a criterion can belong to dialogue models, etc. (1 in Fig.1). The set of measurable different factors. Finally, these criteria are decomposed into attributes that can be evaluated at this level is mainly specific metrics that can quantify the criteria. related to how the information will be accessed by users Moraga et al. [12] present a usability model towards portlet and how this information will be presented by abstract UI evaluation. Portlets are pluggable user interface software patterns (i.e. navigability, information density, etc). components that are managed and displayed in a web portal. However, this set of attributes may differ depending on The portlet usability model is based on the sub-characteristics the PIM expressiveness from each Web development from ISO/IEC 9126 (understandability, learnability and method. This evaluation will generate a usability report in compliance), nevertheless, the operability sub-characteristic order to provide feedback about how to correct these PIM was replaced by customizability which is closer to the portlet models. context. The usability evaluation process proposed is based on a 2. At the PSM level it is possible to assess the concrete number of ranking with acceptance thresholds in order to interface models related to a specific platform (2 in quantify the sub-characteristics from the models. Fig.1). The set of measurable attributes that can be The majority of these approaches evaluate Web applications in evaluated at this level is wider since it includes attributes order to suggest changes at design or implementation stages. It related with specific software components (widgets) that implies that more efforts and resources must be invested into cannot be considered at PIM level (i.e. behavior of code maintenance. This fact does not occur in a MDD approach explore bars, visual feedback from radio buttons, where only the maintenance of models is required since source etc).This evaluation will generate a usability report in order to provide feedback to previous stages about how to 6. Analysis of changes suggested by usability reports and correct the PIM and PSM models, as well as the selection of the alternatives taking into account criteria transformation rules among them. such as level and priority of usability problems, resources 3. At the CM level it is possible to evaluate the final user needed to apply changes, etc. interface (3 in Fig.1). The set of measurable attributes that It should be noted that this process is defined to be instantiated can be evaluated at this level is the widest since more into any concrete Model-Driven Web Development method. aspects related to the end-user perspective can be The instantiation implies to know the modeling primitives of considered (i.e. browser compatibility, metaphor the concrete Model-Driven Web development method in order recognition, subjective appealing, etc). This evaluation to be able to specify the calculation formula of the metrics and will also generate a usability report in order to provide to understand the traceability between models. This traceability feedback to previous stages about how to correct the PIM helps the evaluator to establish the source of the usability and PSM models, as well as the transformation rules problems (PIMs, PSMs or transformations rules among them). among them, and code generation rules among PSM and CM. 4. THE WEB USABILITY MODEL The former evaluations can be applied in an iterative way until The SQuaRE standard [10] proposes three different views for a the models (PIM, PSM, and CM) have the required level of quality model. These views are related to the context where the usability. In order to integrate these evaluations into a model will be applied: Software Quality Model to evaluate a framework, a usability evaluation process should be defined as concrete software product; Data Quality Model to evaluate the an inspection method that guides evaluators on how the Web quality of the data managed in the product; and Quality in Use Usability Model can be applied. This inspection method could Model to evaluate how the stakeholders achieve their goals in a be defined in order to be compliant with the Quality Evaluation specific context of use. Division proposed in the ISO/IEC 2504n SQuaRE series [10]. Our Web Usability Model is an adaptation and extension from The main steps to be included are: the usability model for model-driven development presented in 1. Establish evaluation requirements such as the purpose of Abrahão and Insfran [3], specifically, the model was adapted to evaluation, identification of Web application type, and be compliant with the Software Quality Model proposed in the selection of the more relevant sub-characteristics of the SQuaRE. Web Usability Model taking into account the users’ The main quality characteristics of the software quality model needs. are: functionality, security, interoperability, reliability, 2. Specify the evaluation concerning with the establishment operability (usability) and efficiency. Although the term of the artifacts to be evaluated (PIM, PSM or CM); operability and ease of use have been proposed in SQuaRE to selection of metrics associated to the attributes selected rename usability and operability sub-characteristic, from the Web Usability Model; specification of the respectively, we prefer to use the term usability and operability calculation formulas of these metrics taking into account in this work to avoid misunderstandings in terminology. the abstraction level of the artifact and the modeling Usability can be decomposed into the five sub-characteristics primitives from the concrete Model-Driven Web proposed in SQuaRE [10]: learnability, understandability, ease development method; establishment of rating levels for of use (operability), attractiveness and compliance. The former these metrics; establishment of criteria for global three sub-characteristics are related to user performance and can assessment; and the definition of templates to report be quantified mainly using objective measures. The last two usability problems. sub-characteristics are related to the perception of the end-user 3. Design the evaluation plan describing the evaluator tasks or evaluator using the Web Application and can be quantified schedule. mainly using subjective measures. 4. Execute the evaluation by applying the selected Web The former three sub-characteristics were decomposed into metrics in order to detect usability problems. other sub-characteristics or measurable attributes, taking into account the ergonomic criteria proposed in Bastien and Scapin 5. Generate the usability reports providing feedback in order [4]: to improve the intermediate artifacts (PIM and PSM) or transformation rules. i. Learnability refers to the attributes of a Web application that facilitate learning: a) help facilities such as on-line Fig. 1 Integrating a Usability Evaluation Process into a Model-Driven Web development process help, contact section, etc; b) predictability, which refers to • Proportion of elements that show current user state: This the ease with which a user can determine the result of metric is defined as the ratio between the number of his/her future actions (i.e. significance of link/image titles); elements that show feedback about the current user state c) informative feedback in response to user actions; and d) and the total number of elements that are required to have memorability as a measure of how quickly and accurately this feedback capability. (Scale type: ratio between 0 and users can remember how to use a Web application that they 1). The interpretation is: values closer to 1 indicate that user have used before. can obtain feedback about his/her current state in the Web ii. Understandability refers to the attributes of a Web application. This metric is associated to the immediate application that facilitate understanding: a) optical legibility feedback attribute, which belongs to the user guidance sub- of texts and images (e.g., font size, text contrast); b) characteristic (ii. e). readability, which involves aspects of information-grouping Web metrics definitions from the Web Usability Model are cohesiveness and density; c) familiarity, the ease with generic, and their calculation formula must be instantiated by which a user recognizes the user interface components and identifying variables from this formula in the modeling views their interaction as natural; d) brevity, which is primitives of the concrete Web development method for each related to the reduction of user cognitive effort; and finally, abstraction level (PIM, PSM or CM). Not all the metrics can be e) user guidance, which is related to message quality, defined at all the abstraction levels, for instance, the former immediate feedback (to show the current user state), and metric can be applied at PIM level (if style properties are navigability (to guide the user and to improve the access to defined at the abstract UI) or at CM level (if style properties are the Web content). defined in Cascading Style Sheets files). However, the second iii. Operability refers to the attributes of a Web application that metric only can be defined at PSM or CM level since the facilitate user control and operation: a) execution facilities feedback depends on the widget behavior from the concrete such as compatible browsers, plug-ins needed, and update interface. frequency; b) data validity of the user inputs; c) controllability of the services execution such as cancel, 5. INSTANTIATION OF THE USABILITY undo and redo support; d) capability of adaptation which refers to the capacity of the Web application to be adapted EVALUATION PROCESS to the users’ needs and preferences and e) consistency in the In this section, we show an overview of how the previous execution of services and control behavior. usability process can be instantiated into a concrete Web development methodology. As an example, we selected the OO- The last two sub-characteristics are related to the perception of H [7] method. the end-user (attractiveness) or evaluator (compliance) using the Web Application: The OO-H method [7] provides designers with the semantics and notation for developing Web applications. The set of iv. Attractiveness refers to the attributes of a Web application conceptual models that represents the different concerns of a that are related to the aesthetic design. They can be Web application are: the specification of content requirements quantified by measuring the UI uniformity in terms of font (Class Model) and the specification of functional requirements style (color, face and size), background color, and position in terms of navigation needs (Navigation Model, NAD). A of elements. merge between the class and navigation models results in an v. Compliance can be measured by assessing the agreement of Abstract Presentation Diagram as an integrated PIM model, the proposed Web Usability Model with respect to the which presents an abstract user interface as a collection of standard SQuaRE and several Web design style guides. abstract pages. APD can be refined by a pattern catalog. Finally, Once the sub-characteristics have been identified, Web metrics platform-specific models (PSMs) are automatically obtained are associated to the measurable attributes in order to quantify from the APD, from which source code (CM) can be them. Values obtained from these Web metrics will allow us to automatically generated. interpret if measurable attributes contribute to achieving certain Next, we show as an example, a brief description about the usability level in the Web application. The metrics included in steps involved in our usability evaluation process. our model were mainly extracted from the survey presented in Calero et al. [5]. We only selected those metrics that were Step 1 (See Section 3): The purpose is to evaluate the usability theoretically and/or empirically validated. In addition, we of a Web application developed following the OO-H method. proposed new metrics for several measurable attributes that The selected Web application is a task management system were not appropriated covered by this survey. developed for a Web development company located in Alicante, Spain. Finally, the attributes chosen were font style uniformity As an example, we show some definitions of new proposed to evaluate the attractiveness sub-characteristic, and immediate metrics that can be associated to attributes of the Web Usability feedback to evaluate the user guidance sub-characteristic, at Model: least to some extent. • Number of different font styles for textual links: This metric Step 2 (See Section 3): The artifacts selected for this evaluation is defined as number of different font style combinations were the final UIs (Code Model). The metrics selected to (size, face, and color) for all textual links in the same evaluate the previous attributes were Number of different font navigation category. (Scale type: absolute value greater or styles for text links and Proportion of elements that show equal to 1). The interpretation is: more than one style current user state (see explanation of each metric in Section 4). combination in the same navigation category means that The rating level for the former metric was established at no UP font style uniformity is not insured. This metric is for values equal to 1; low UP for values equal to 2; medium UP associated to the font style uniformity attribute, which for values equal to 3; and critical UP for values greater than 3. belongs to the attractiveness sub-characteristic (iv). The rating level for the second metric was established at no UP for values equal to 1; low UP for values in the range [0.8, 1}; medium UP for values [0.5, 0.8} and critical UP for values [0, currently interacting, the value of the metric is 2/3=0.66. 0.5}. The usability report is defined as a list of usability The rating level of the metric (see Step 2) indicates the problems (UP) detected with the next fields: description of the existence of a medium usability problem (UP002). UP, level of the UP (critical, medium, or low), source of the Steps 5 and 6 (See Section 3): The usability problems detected problem (model), occurrences, and recommendations to correct after applying the previous metrics, can be explained in a it. More fields can be defined such as priority, impact, etc. usability report that contains the UP001 (See Table 1) and the Step 3 (See Section 3): In this case, the evaluator is the same UP002 (See Table 2). developer. The task assigned was the evaluation of all the user interfaces (CM) in order to present a usability report which will Table 1. Usability problem detected: UP001 contain the usability problems detected with all the proposed id UP001 fields filled in. The links Tasks, Reports, and Contacts Step 4 (See Section 3): As an example, we only show the are displayed in a font style that is execution of the evaluation of one user interface (CM). Figure 2 different from the font style of the Exit shows a user interface automatically generated (Code Model) link. The same problem occurs with the Description that represents the task management functionality of the Web all tasks link that is displayed in a font application. style that is different to the used in the links: New Folder, Pending tasks, Ended tasks, and Tasks out of date. Affected attribute Attractiveness / font style uniformity. Level Low (rating level: 2). Source of the Abstract Presentation Diagram (PIM problem model). Occurrences 2 occurrences (top menu and left menu) Change the font style properties for the links Tasks, Reports, Contacts and all Recommendations tasks in the Abstract Presentation Diagram. In this PIM model font style Fig.2 Example of a User interface automatically properties can be defined. generated (Code Model) Table 2. Usability problem detected: UP002 The selected metrics were applied: id UP002 1. Number of different font styles for textual links 1: The Tabs do not show the current user state in textual links that appears in the user interface (Fig. 2) are Description the Web application. Tasks, Reports, Contacts and Exit from the top menu; and Understandability/ User Guidance/ New Folder, All tasks, Pending tasks, Ended tasks, and Affected attribute Immediate feedback. Tasks out of date from the left menu. In the first navigation category (top menu), the value of the metric is Level Medium (rating level: 0.66) 2 since the links Tasks, Reports, Contacts are displayed in The transformation rule that maps the a different style from the Exit link, which is displayed in a representation of the tabs: Task, Reports different color and it is also underlined. In the second Source of the and Contacts (PIM level) with the navigation category (left menu), the value of the metric is problem specific widget of the platform that shows also 2 since the links New Folder, Pending tasks, Ended the tabs (PSM). tasks, and Tasks out of date Contacts are displayed in a 1 occurrence for each UI that shows these different style from the All tasks, which is displayed in a Occurrences tabs. different font face and font size. The rating level of the The widget target of the transformation metric (see Step 2) indicates the existence of a low rule should be changed for other widget usability problem (UP001) for each menu. Recommendations with a highlight feature when a tab is clicked. 2. Proportion of elements that show current user state 1: The user interface must show the current user state, it means, the current section and the current task that is After analyzing and applying the proposed recommendations, a being performed. There are three types of elements that more usable Web application can be obtained without to need show the current user state in the Web application: the maintenance of source code. tabs from the top menu (Tasks, Reports, and Contacts); the shape changes of the cursor when it is pointing on a textbox; and the highlight effects of a textbox when it has 6. DISCUSSIONS AND FUTHER WORK focus. Since the tabs are the only type of element that This paper has presented a proposal in progress to integrate a does not explicitly show the section in which the user is usability evaluation process into Model-Driven Web development processes. The purpose of our work is to give an outline of a generic usability evaluation process supported by a 1 Web Usability Model. A preliminary version of a usability It should be note that both metrics must be applied to all the evaluation process supported by a Web usability Model has user interfaces of the entire Web application. been presented. Our Web Usability Model decomposes the usability sub-characteristics (from the Software Quality Model 8. REFERENCES proposed in the ISO/IEC 25000 SQuaRE standard) into other [1] Abrahão, S., Condori-Fernández, N., Olsina, L., and sub-characteristics and measurable attributes taking into Pastor, O. 2003. Defining and Validating Metrics for account ergonomic criteria. Web metrics were associated to Navigational Models. Proc. of the 9th Inter. IEEE measurable attributes in order to quantify them. Finally, a brief Software Metrics Symposium, 200-210. example has been shown in order to illustrate how the usability evaluation process can be instantiated into a concrete Web [2] Abrahão, S., Iborra, E., and Vanderdonckt J. 2007. development method that follows the MDD approach. Although Usability Evaluation of User Interfaces Generated with a our example only shows a CM evaluation providing feedback to Model-Driven Architecture Tool. Maturing Usability. PIM models or transformations between PIM and PSM models, Springer HCI series, Vol. 10, 3-32. the usability evaluation process can evaluate intermediate [3] Abrahão, S. and Insfran, E. 2006. Early Usability artifacts (PIM and PSM models) by selecting metrics that their Evaluation in Model-Driven Architecture Environments. calculation formula has been defined to be applied to concrete Proc. of the 6th IEEE International Conference on Quality PIM and PSM models (i.e., depth and breadth of a navigational Software. IEEE Computer Society, 287-294. map [1] associated to the navigability attribute). [4] Bastien, J.M. and Scapin, D.L. 1993. Ergonomic criteria We believe that the inherent features of model-driven for the evaluation of human-computer interfaces. Tech. development processes (e.g., traceability between models by Rep. num.156. INRIA, Rocquencourt, France. means of model transformations) provide a suitable [5] Calero C., Ruiz J., and Piattini M. 2005. Classifying Web environment for performing usability evaluations. Specifically, metrics using the Web quality model. Emerald Group if the usability of an automatically generated user interface can Publishing Limited. Vol. 29(3), 227-248. be assessed, the usability of any future user interface produced by this approach could be predicted. In other words, we are [6] Ceri, S., Fraternali, P., and Bongio, A. 2000. Web talking about a user interface that is usable by construction [2], Modeling Language (WebML): A Modeling Language for at least to some extent. Designing Web Sites. Proc. of the 9th WWW Conference, 137-157. In this way, usability can be taken into account throughout the [7] Gómez, J., Cachero, C., and Pastor, O. 2001. Conceptual entire Web development process, enabling Web applications to Modeling of Device-Independent Web Applications. IEEE be developed with better quality thereby reducing effort at the MultiMedia, Vol. 8(2), 26-39. maintenance stage. [8] ISO/IEC. 1998. ISO/IEC 9241-11, Ergonomic Further work is intended to: Requirements for Office Work with Visual Display • Perform an entire instantiation of the usability Terminals (VDTs), Part 11: Guidance on Usability. evaluation process into the OO-H method. [9] ISO/IEC. 2001. ISO/IEC 9126-1 Standard, Software • Define guidelines in order to guide evaluators on how Engineering, Product Quality - Part 1: Quality Model. the Web Usability Model can be applied [10] ISO/IEC. 2005. ISO/IEC 25000 series, Software Engineering, Software Product Quality Requirements and • Explore aggregation mechanisms for aggregating Evaluation (SQuaRE). values obtained by individual metrics, and perform analyses of the impact on how the attributes affect [11] Kraus, A., Knapp, A., and Koch, N. 2006. Model-Driven (negatively or positively) other attributes of the Web Generation of Web Applications in UWE. 3rd Inter. Usability Model. Workshop on Model-Driven Web Engineering. • Instantiate the evaluation process into different [12] Moraga, M.A, Calero, C., Piattini, M., and Diaz, O. 2007. Model-Driven Web Development methods in order to Improving a portlet usability model. Software Quality improve our approach. Control, Vol. 15(2), 155-177. • Develop a tool to support the entire usability [13] Olsina, L. and Rossi, G. 2002. Measuring Web evaluation process. The tool will be able to manage Application Quality with WebQEM. IEEE Multimedia, the Web Usability Model by creating a repository of Vol. 9(4), 20-29. catalogued metrics following the SQuaRE patterns. [14] Panach, J., Condori-Fernández, N., Valverde, F., Aquino, N., and Pastor, O. 2007. Towards an Early Usability 7. ACKNOWLEDGMENTS Evaluation for Web Applications. International Conference This work is financed by META project (ref. TIN2006-15175- on Software Process and Product Measurement. LNCS C05-05), the Quality-driven Model Transformation Project Springer Vol. 4895, 32-45. from the Universidad Politécnica de Valencia. The authors [15] Seffah, A., Donyaee, M., Kline, R.B., and Padda, H.K. thank Jaime Gomez from Universidad de Alicante for his 2006. Usability Measurement and Metrics: A Consolidated valuable help in providing the generated Web application and Model. Software Quality Journal, Vol. 14(2), 159-178. its models used to illustrate our usability evaluation process. [16] Ivory, M.Y. 2001. An Empirical Foundation for Automated Web Interface Evaluation. PhD Thesis. University of California, Berkeley, Computer Science Division.