=Paper=
{{Paper
|id=Vol-490/paper-5
|storemode=property
|title=Towards a Usability Evaluation Process for Model-Driven Web Development
|pdfUrl=https://ceur-ws.org/Vol-490/paper_05.pdf
|volume=Vol-490
|dblpUrl=https://dblp.org/rec/conf/iused/FernandezIA09
}}
==Towards a Usability Evaluation Process for Model-Driven Web Development==
https://ceur-ws.org/Vol-490/paper_05.pdf
Towards a Usability Evaluation Process for Model-
Driven Web Development
Adrian Fernandez Emilio Insfran Silvia Abrahão
ISSI Research Group ISSI Research Group ISSI Research Group
Department of Information Systems Department of Information Systems Department of Information Systems
and Computation - Universidad and Computation - Universidad and Computation - Universidad
Politécnica de Valencia, Camino de Politécnica de Valencia, Camino de Politécnica de Valencia, Camino de
Vera, s/n, 46022, Valencia, Spain. Vera, s/n, 46022, Valencia, Spain. Vera, s/n, 46022, Valencia, Spain.
+34 96 387 73 50 +34 96 387 73 50 +34 96 387 73 50
afernandez@dsic.upv.es einsfran@dsic.upv.es sabrahao@dsic.upv.es
ABSTRACT Usability evaluations methods for Web applications can be
This paper presents an approach to integrate usability supported by a quality model which defines usability as a
evaluations into Model-Driven Web development processes. quality characteristic that is decomposed into specific attributes
Our main motivation is to define a generic usability evaluation that are easier to measure. Although there are several proposes
process which can be instantiated into any concrete Web in this field, most of these approaches [12],[13] only consider
development process that follows a Model-Driven Development usability evaluation at final stages when the product is almost
(MDD) approach. A preliminary version of a Web Usability completed where correcting its usability problems is more
Model was defined in order to support this usability evaluation difficult. It is widely accepted that evaluations performed at
process at several stages. This Web Usability Model each phase of Web applications development is a critical part of
decomposes the usability sub-characteristics (from the Software ensuring that the product will actually be used and be effective
Quality Model proposed in the ISO/IEC 25000 SQuaRE for its intended purpose. We argue that integrating usability
standard) into other sub-characteristics and measurable issues into the MDD approach can be an effective way to reach
attributes. Web metrics are intended to be associated to this objective since the quality evaluation of intermediate
measurable attributes in order to quantify them. Our approach is artifacts (models that specify an entire Web application), is
intended to perform usability evaluations at several stages of a applied in all steps of the process [2]. A Web development
Web Development process. In this paper, we show how process that follows a MDD approach basically transforms
usability evaluations at final user interface (UI) can provide models that are independent from implementation details
feedback about changes in order to improve usability issues at (Platform-Independent Models - PIM) into other models that
intermediate artifacts (Platform-Independent Models and contain specific aspects from a concrete platform (Platform-
Platform-Specific Models) or at transformations rules among Specific Models - PSM). Transformation rules, which are
these intermediate artifacts. applied at PSMs, are able to automatically generate the Web
application source code (Code Model - CM).
Categories and Subject Descriptors This paper presents an approach to integrate usability
evaluation into any Model-Driven Web Development method
D.2.9 [Management]: Software quality assurance, D.2.8
by defining a usability evaluation process. This Web Usability
[Metrics]: product metrics. H5.2 [User Interfaces]:
Model has been defined by decomposing the usability sub-
Evaluation/methodology
characteristics (from the Software Quality Model proposed in
the ISO/IEC 25000 SQuaRE standard) into other sub-
General Terms characteristics and measurable attributes taking into account
Measurement, Design. ergonomic criteria proposed in Bastien and Scapin [4].
Although our approach is intended to perform usability
Keywords evaluations at several stages of a Web development process, in
this paper, we mainly focus on how evaluations at final user
Web Usability Model, Usability Evaluation, Web Metrics,
interface (Code Model) can provide feedback about changes in
Model-Driven Development.
order to improve usability issues at intermediate artifacts (PIM
and PSM models) produced at early stages of the Web
1. INTRODUCTION development process and at transformations rules among these
Usability in Web applications is a crucial factor since the ease intermediate artifacts.
or difficulty that users experience with this kind of systems will
This paper is organized as follows. Section 2 discusses related
determine their success or failure. Web applications are
work that report usability evaluation processes for Web
increasing its importance in industrial domains; thereby, the
applications. Section 3 presents our approach to integrate
need for usability evaluation methods that are specifically
usability evaluations into Model-Driven Web Development.
crafted for the Web domain has become critical.
Section 4 presents our Web Usability Model that supports our
Permission to make digital or hard copies of all or part of this work for approach. Section 5 shows a brief example of how the usability
personal or classroom use is granted without fee provided that copies are evaluation process can be instantiated into a concrete Web
not made or distributed for profit or commercial advantage and that development method. We mainly focus on evaluations at final
copies bear this notice and the full citation on the first page. To copy user interface. Finally, Section 6 presents discussions and
otherwise, or republish, to post on servers or to redistribute to lists, further work.
requires prior specific permission and/or a fee.
I-USED ’09, August 24, 2009, Upssala, Sweeden.
2. RELATED WORK code can be automatically generated from the intermediate
There are several approaches that deal with Web usability artifacts (PIM and PSM models).
evaluation, for instance, Ivory [16], Olsina and Rossi [13], In previous work, Abrahão and Insfran [3] proposed a usability
Calero et al. [5], Seffah et al. [15], and Moraga et al. [12]. model for early evaluation in model-driven architecture
Ivory [16] presents a methodology for evaluating information- environments. Usability was decomposed into the same sub-
centric Web sites. The methodology proposes five stages: characteristics as the ones in the ISO/IEC 9126 (learnability,
identifying an exhaustive set of quantitative interface measures understandability, operability, and compliance), and then
such as the amount of text on a page, color usage, consistency, decomposed again, into more detailed sub-characteristics and
etc; computing measures for a large sample of rated interfaces; attributes. However, the model did not provide metrics for
deriving statistical models from the measures and ratings; using measuring the model attributes and it was not proposed
the models to predict ratings for new interfaces; and validating specifically for the Web domain. Panach et al. [14] presents an
model prediction. adaptation from the previous model to the Web domain in order
to evaluate usability at PIM models for a concrete and
Olsina and Rossi [13] proposed the Web Quality Evaluation proprietary Model-Driven Web Development approach.
Method (WebQEM) to define an evaluation process in four
technical phases: Quality requirements definition and As far as we know, there is no proposal for a generic usability
specification (specifying characteristics and attributes based on evaluation process supported by a usability model in the Model-
the ISO/IEC 9126-1 [9]. such as usability, functionality, Driven Web Development context.
reliability, and effectiveness and taking into account Web
audience’s needs), elementary evaluation (applying metrics to 3. THE USABILITY EVALUATION
quantify attributes), global evaluation (selecting aggregation PROCESS
criteria and a scoring model), and conclusion (giving Since the adoption of Model-Driven Development (MDD) in
recommendations). Nevertheless, evaluations take place mainly the industrial domain has increased recently, our approach is
when the application is completed. intended to integrate usability issues into a Model-Driven Web
Calero et al. [5] present the Web Quality Model (WQM), which Development. Web development methods that follow this
is intended to evaluate a Web application according to three approach such as OO-H [7], WebML [6], or UWE [11] support
dimensions: Web features (content, presentation, and the development of a Web application by defining different
navigation); quality characteristics based on the ISO/IEC 9126- views (models), including at least one structural model, a
1 (functionality, reliability, usability, efficiency, portability, and navigational model, and an abstract presentation model. These
maintainability); and lifecycle processes (development, methods also provide model transformations and automatic
operation and maintenance) including organizational processes code generation.
such as project management and reuse program management. The usability of a Web application obtained as a result of a
WQM has been used to classify, according to these three MDD process can be assessed at different abstraction levels
dimensions, a total of 326 Web metrics taken from the existing (PIM, PSM and CM). Our proposal is intended to use a Web
literature. An evaluation process can be defined by selecting the Usability Model, which is a set of sub-characteristics
most useful set of metrics to construct a “total Web quality” decomposed into measurable attributes that can be quantified by
expression that could be used to quantify the quality of a given metrics. The Web Usability Model can be applied to assess the
Web application. However, guidelines on how to define this models from each abstraction level (see Fig.1). However, not all
process have not been provided. the measurable attributes can be evaluated at all the abstraction
Seffah et al. [15] present the Quality in Use Integrated levels. The higher abstraction level, the less attributes can be
Measurement (QUIM) as a consolidated model for usability considered. In addition, feedback that is obtained after the
measurement in Web applications. An editor tool has presented artifact evaluation has different targets depending on the
to define measurement plans collecting data from different abstraction level:
combinations of metrics proposed in the model. QUIM 1. At the PIM level it is possible to assess models that
combines existing models from ISO/IEC 9126-1 [9], ISO/IEC specify the Web application independently of platform
9241-11 [8], and others. It decomposes usability into factors, details such as: presentation models, navigational models,
and then into criteria. In this approach, a criterion can belong to dialogue models, etc. (1 in Fig.1). The set of measurable
different factors. Finally, these criteria are decomposed into attributes that can be evaluated at this level is mainly
specific metrics that can quantify the criteria. related to how the information will be accessed by users
Moraga et al. [12] present a usability model towards portlet and how this information will be presented by abstract UI
evaluation. Portlets are pluggable user interface software patterns (i.e. navigability, information density, etc).
components that are managed and displayed in a web portal. However, this set of attributes may differ depending on
The portlet usability model is based on the sub-characteristics the PIM expressiveness from each Web development
from ISO/IEC 9126 (understandability, learnability and method. This evaluation will generate a usability report in
compliance), nevertheless, the operability sub-characteristic order to provide feedback about how to correct these PIM
was replaced by customizability which is closer to the portlet models.
context. The usability evaluation process proposed is based on a 2. At the PSM level it is possible to assess the concrete
number of ranking with acceptance thresholds in order to interface models related to a specific platform (2 in
quantify the sub-characteristics from the models. Fig.1). The set of measurable attributes that can be
The majority of these approaches evaluate Web applications in evaluated at this level is wider since it includes attributes
order to suggest changes at design or implementation stages. It related with specific software components (widgets) that
implies that more efforts and resources must be invested into cannot be considered at PIM level (i.e. behavior of
code maintenance. This fact does not occur in a MDD approach explore bars, visual feedback from radio buttons,
where only the maintenance of models is required since source etc).This evaluation will generate a usability report in
order to provide feedback to previous stages about how to 6. Analysis of changes suggested by usability reports and
correct the PIM and PSM models, as well as the selection of the alternatives taking into account criteria
transformation rules among them. such as level and priority of usability problems, resources
3. At the CM level it is possible to evaluate the final user needed to apply changes, etc.
interface (3 in Fig.1). The set of measurable attributes that It should be noted that this process is defined to be instantiated
can be evaluated at this level is the widest since more into any concrete Model-Driven Web Development method.
aspects related to the end-user perspective can be The instantiation implies to know the modeling primitives of
considered (i.e. browser compatibility, metaphor the concrete Model-Driven Web development method in order
recognition, subjective appealing, etc). This evaluation to be able to specify the calculation formula of the metrics and
will also generate a usability report in order to provide to understand the traceability between models. This traceability
feedback to previous stages about how to correct the PIM helps the evaluator to establish the source of the usability
and PSM models, as well as the transformation rules problems (PIMs, PSMs or transformations rules among them).
among them, and code generation rules among PSM and
CM. 4. THE WEB USABILITY MODEL
The former evaluations can be applied in an iterative way until The SQuaRE standard [10] proposes three different views for a
the models (PIM, PSM, and CM) have the required level of quality model. These views are related to the context where the
usability. In order to integrate these evaluations into a model will be applied: Software Quality Model to evaluate a
framework, a usability evaluation process should be defined as concrete software product; Data Quality Model to evaluate the
an inspection method that guides evaluators on how the Web quality of the data managed in the product; and Quality in Use
Usability Model can be applied. This inspection method could Model to evaluate how the stakeholders achieve their goals in a
be defined in order to be compliant with the Quality Evaluation specific context of use.
Division proposed in the ISO/IEC 2504n SQuaRE series [10]. Our Web Usability Model is an adaptation and extension from
The main steps to be included are: the usability model for model-driven development presented in
1. Establish evaluation requirements such as the purpose of Abrahão and Insfran [3], specifically, the model was adapted to
evaluation, identification of Web application type, and be compliant with the Software Quality Model proposed in the
selection of the more relevant sub-characteristics of the SQuaRE.
Web Usability Model taking into account the users’ The main quality characteristics of the software quality model
needs. are: functionality, security, interoperability, reliability,
2. Specify the evaluation concerning with the establishment operability (usability) and efficiency. Although the term
of the artifacts to be evaluated (PIM, PSM or CM); operability and ease of use have been proposed in SQuaRE to
selection of metrics associated to the attributes selected rename usability and operability sub-characteristic,
from the Web Usability Model; specification of the respectively, we prefer to use the term usability and operability
calculation formulas of these metrics taking into account in this work to avoid misunderstandings in terminology.
the abstraction level of the artifact and the modeling Usability can be decomposed into the five sub-characteristics
primitives from the concrete Model-Driven Web proposed in SQuaRE [10]: learnability, understandability, ease
development method; establishment of rating levels for of use (operability), attractiveness and compliance. The former
these metrics; establishment of criteria for global three sub-characteristics are related to user performance and can
assessment; and the definition of templates to report be quantified mainly using objective measures. The last two
usability problems. sub-characteristics are related to the perception of the end-user
3. Design the evaluation plan describing the evaluator tasks or evaluator using the Web Application and can be quantified
schedule. mainly using subjective measures.
4. Execute the evaluation by applying the selected Web The former three sub-characteristics were decomposed into
metrics in order to detect usability problems. other sub-characteristics or measurable attributes, taking into
account the ergonomic criteria proposed in Bastien and Scapin
5. Generate the usability reports providing feedback in order
[4]:
to improve the intermediate artifacts (PIM and PSM) or
transformation rules. i. Learnability refers to the attributes of a Web application
that facilitate learning: a) help facilities such as on-line
Fig. 1 Integrating a Usability Evaluation Process into a Model-Driven Web development process
help, contact section, etc; b) predictability, which refers to • Proportion of elements that show current user state: This
the ease with which a user can determine the result of metric is defined as the ratio between the number of
his/her future actions (i.e. significance of link/image titles); elements that show feedback about the current user state
c) informative feedback in response to user actions; and d) and the total number of elements that are required to have
memorability as a measure of how quickly and accurately this feedback capability. (Scale type: ratio between 0 and
users can remember how to use a Web application that they 1). The interpretation is: values closer to 1 indicate that user
have used before. can obtain feedback about his/her current state in the Web
ii. Understandability refers to the attributes of a Web application. This metric is associated to the immediate
application that facilitate understanding: a) optical legibility feedback attribute, which belongs to the user guidance sub-
of texts and images (e.g., font size, text contrast); b) characteristic (ii. e).
readability, which involves aspects of information-grouping Web metrics definitions from the Web Usability Model are
cohesiveness and density; c) familiarity, the ease with generic, and their calculation formula must be instantiated by
which a user recognizes the user interface components and identifying variables from this formula in the modeling
views their interaction as natural; d) brevity, which is primitives of the concrete Web development method for each
related to the reduction of user cognitive effort; and finally, abstraction level (PIM, PSM or CM). Not all the metrics can be
e) user guidance, which is related to message quality, defined at all the abstraction levels, for instance, the former
immediate feedback (to show the current user state), and metric can be applied at PIM level (if style properties are
navigability (to guide the user and to improve the access to defined at the abstract UI) or at CM level (if style properties are
the Web content). defined in Cascading Style Sheets files). However, the second
iii. Operability refers to the attributes of a Web application that metric only can be defined at PSM or CM level since the
facilitate user control and operation: a) execution facilities feedback depends on the widget behavior from the concrete
such as compatible browsers, plug-ins needed, and update interface.
frequency; b) data validity of the user inputs; c)
controllability of the services execution such as cancel, 5. INSTANTIATION OF THE USABILITY
undo and redo support; d) capability of adaptation which
refers to the capacity of the Web application to be adapted EVALUATION PROCESS
to the users’ needs and preferences and e) consistency in the In this section, we show an overview of how the previous
execution of services and control behavior. usability process can be instantiated into a concrete Web
development methodology. As an example, we selected the OO-
The last two sub-characteristics are related to the perception of H [7] method.
the end-user (attractiveness) or evaluator (compliance) using
the Web Application: The OO-H method [7] provides designers with the semantics
and notation for developing Web applications. The set of
iv. Attractiveness refers to the attributes of a Web application
conceptual models that represents the different concerns of a
that are related to the aesthetic design. They can be
Web application are: the specification of content requirements
quantified by measuring the UI uniformity in terms of font
(Class Model) and the specification of functional requirements
style (color, face and size), background color, and position
in terms of navigation needs (Navigation Model, NAD). A
of elements.
merge between the class and navigation models results in an
v. Compliance can be measured by assessing the agreement of Abstract Presentation Diagram as an integrated PIM model,
the proposed Web Usability Model with respect to the which presents an abstract user interface as a collection of
standard SQuaRE and several Web design style guides. abstract pages. APD can be refined by a pattern catalog. Finally,
Once the sub-characteristics have been identified, Web metrics platform-specific models (PSMs) are automatically obtained
are associated to the measurable attributes in order to quantify from the APD, from which source code (CM) can be
them. Values obtained from these Web metrics will allow us to automatically generated.
interpret if measurable attributes contribute to achieving certain Next, we show as an example, a brief description about the
usability level in the Web application. The metrics included in steps involved in our usability evaluation process.
our model were mainly extracted from the survey presented in
Calero et al. [5]. We only selected those metrics that were Step 1 (See Section 3): The purpose is to evaluate the usability
theoretically and/or empirically validated. In addition, we of a Web application developed following the OO-H method.
proposed new metrics for several measurable attributes that The selected Web application is a task management system
were not appropriated covered by this survey. developed for a Web development company located in Alicante,
Spain. Finally, the attributes chosen were font style uniformity
As an example, we show some definitions of new proposed to evaluate the attractiveness sub-characteristic, and immediate
metrics that can be associated to attributes of the Web Usability feedback to evaluate the user guidance sub-characteristic, at
Model: least to some extent.
• Number of different font styles for textual links: This metric Step 2 (See Section 3): The artifacts selected for this evaluation
is defined as number of different font style combinations were the final UIs (Code Model). The metrics selected to
(size, face, and color) for all textual links in the same evaluate the previous attributes were Number of different font
navigation category. (Scale type: absolute value greater or styles for text links and Proportion of elements that show
equal to 1). The interpretation is: more than one style current user state (see explanation of each metric in Section 4).
combination in the same navigation category means that The rating level for the former metric was established at no UP
font style uniformity is not insured. This metric is for values equal to 1; low UP for values equal to 2; medium UP
associated to the font style uniformity attribute, which for values equal to 3; and critical UP for values greater than 3.
belongs to the attractiveness sub-characteristic (iv). The rating level for the second metric was established at no UP
for values equal to 1; low UP for values in the range [0.8, 1};
medium UP for values [0.5, 0.8} and critical UP for values [0, currently interacting, the value of the metric is 2/3=0.66.
0.5}. The usability report is defined as a list of usability The rating level of the metric (see Step 2) indicates the
problems (UP) detected with the next fields: description of the existence of a medium usability problem (UP002).
UP, level of the UP (critical, medium, or low), source of the
Steps 5 and 6 (See Section 3): The usability problems detected
problem (model), occurrences, and recommendations to correct
after applying the previous metrics, can be explained in a
it. More fields can be defined such as priority, impact, etc.
usability report that contains the UP001 (See Table 1) and the
Step 3 (See Section 3): In this case, the evaluator is the same UP002 (See Table 2).
developer. The task assigned was the evaluation of all the user
interfaces (CM) in order to present a usability report which will Table 1. Usability problem detected: UP001
contain the usability problems detected with all the proposed id UP001
fields filled in.
The links Tasks, Reports, and Contacts
Step 4 (See Section 3): As an example, we only show the are displayed in a font style that is
execution of the evaluation of one user interface (CM). Figure 2 different from the font style of the Exit
shows a user interface automatically generated (Code Model) link. The same problem occurs with the
Description
that represents the task management functionality of the Web all tasks link that is displayed in a font
application. style that is different to the used in the
links: New Folder, Pending tasks, Ended
tasks, and Tasks out of date.
Affected attribute Attractiveness / font style uniformity.
Level Low (rating level: 2).
Source of the Abstract Presentation Diagram (PIM
problem model).
Occurrences 2 occurrences (top menu and left menu)
Change the font style properties for the
links Tasks, Reports, Contacts and all
Recommendations tasks in the Abstract Presentation
Diagram. In this PIM model font style
Fig.2 Example of a User interface automatically properties can be defined.
generated (Code Model)
Table 2. Usability problem detected: UP002
The selected metrics were applied:
id UP002
1. Number of different font styles for textual links 1: The Tabs do not show the current user state in
textual links that appears in the user interface (Fig. 2) are Description
the Web application.
Tasks, Reports, Contacts and Exit from the top menu; and Understandability/ User Guidance/
New Folder, All tasks, Pending tasks, Ended tasks, and Affected attribute
Immediate feedback.
Tasks out of date from the left menu. In the first
navigation category (top menu), the value of the metric is Level Medium (rating level: 0.66)
2 since the links Tasks, Reports, Contacts are displayed in The transformation rule that maps the
a different style from the Exit link, which is displayed in a representation of the tabs: Task, Reports
different color and it is also underlined. In the second Source of the
and Contacts (PIM level) with the
navigation category (left menu), the value of the metric is problem
specific widget of the platform that shows
also 2 since the links New Folder, Pending tasks, Ended the tabs (PSM).
tasks, and Tasks out of date Contacts are displayed in a 1 occurrence for each UI that shows these
different style from the All tasks, which is displayed in a Occurrences
tabs.
different font face and font size. The rating level of the The widget target of the transformation
metric (see Step 2) indicates the existence of a low rule should be changed for other widget
usability problem (UP001) for each menu. Recommendations
with a highlight feature when a tab is
clicked.
2. Proportion of elements that show current user state 1:
The user interface must show the current user state, it
means, the current section and the current task that is After analyzing and applying the proposed recommendations, a
being performed. There are three types of elements that more usable Web application can be obtained without to need
show the current user state in the Web application: the maintenance of source code.
tabs from the top menu (Tasks, Reports, and Contacts);
the shape changes of the cursor when it is pointing on a
textbox; and the highlight effects of a textbox when it has 6. DISCUSSIONS AND FUTHER WORK
focus. Since the tabs are the only type of element that This paper has presented a proposal in progress to integrate a
does not explicitly show the section in which the user is usability evaluation process into Model-Driven Web
development processes. The purpose of our work is to give an
outline of a generic usability evaluation process supported by a
1
Web Usability Model. A preliminary version of a usability
It should be note that both metrics must be applied to all the evaluation process supported by a Web usability Model has
user interfaces of the entire Web application. been presented. Our Web Usability Model decomposes the
usability sub-characteristics (from the Software Quality Model 8. REFERENCES
proposed in the ISO/IEC 25000 SQuaRE standard) into other [1] Abrahão, S., Condori-Fernández, N., Olsina, L., and
sub-characteristics and measurable attributes taking into Pastor, O. 2003. Defining and Validating Metrics for
account ergonomic criteria. Web metrics were associated to Navigational Models. Proc. of the 9th Inter. IEEE
measurable attributes in order to quantify them. Finally, a brief Software Metrics Symposium, 200-210.
example has been shown in order to illustrate how the usability
evaluation process can be instantiated into a concrete Web [2] Abrahão, S., Iborra, E., and Vanderdonckt J. 2007.
development method that follows the MDD approach. Although Usability Evaluation of User Interfaces Generated with a
our example only shows a CM evaluation providing feedback to Model-Driven Architecture Tool. Maturing Usability.
PIM models or transformations between PIM and PSM models, Springer HCI series, Vol. 10, 3-32.
the usability evaluation process can evaluate intermediate [3] Abrahão, S. and Insfran, E. 2006. Early Usability
artifacts (PIM and PSM models) by selecting metrics that their Evaluation in Model-Driven Architecture Environments.
calculation formula has been defined to be applied to concrete Proc. of the 6th IEEE International Conference on Quality
PIM and PSM models (i.e., depth and breadth of a navigational Software. IEEE Computer Society, 287-294.
map [1] associated to the navigability attribute).
[4] Bastien, J.M. and Scapin, D.L. 1993. Ergonomic criteria
We believe that the inherent features of model-driven for the evaluation of human-computer interfaces. Tech.
development processes (e.g., traceability between models by Rep. num.156. INRIA, Rocquencourt, France.
means of model transformations) provide a suitable [5] Calero C., Ruiz J., and Piattini M. 2005. Classifying Web
environment for performing usability evaluations. Specifically, metrics using the Web quality model. Emerald Group
if the usability of an automatically generated user interface can Publishing Limited. Vol. 29(3), 227-248.
be assessed, the usability of any future user interface produced
by this approach could be predicted. In other words, we are [6] Ceri, S., Fraternali, P., and Bongio, A. 2000. Web
talking about a user interface that is usable by construction [2], Modeling Language (WebML): A Modeling Language for
at least to some extent. Designing Web Sites. Proc. of the 9th WWW Conference,
137-157.
In this way, usability can be taken into account throughout the
[7] Gómez, J., Cachero, C., and Pastor, O. 2001. Conceptual
entire Web development process, enabling Web applications to
Modeling of Device-Independent Web Applications. IEEE
be developed with better quality thereby reducing effort at the
MultiMedia, Vol. 8(2), 26-39.
maintenance stage.
[8] ISO/IEC. 1998. ISO/IEC 9241-11, Ergonomic
Further work is intended to: Requirements for Office Work with Visual Display
• Perform an entire instantiation of the usability Terminals (VDTs), Part 11: Guidance on Usability.
evaluation process into the OO-H method. [9] ISO/IEC. 2001. ISO/IEC 9126-1 Standard, Software
• Define guidelines in order to guide evaluators on how Engineering, Product Quality - Part 1: Quality Model.
the Web Usability Model can be applied [10] ISO/IEC. 2005. ISO/IEC 25000 series, Software
Engineering, Software Product Quality Requirements and
• Explore aggregation mechanisms for aggregating
Evaluation (SQuaRE).
values obtained by individual metrics, and perform
analyses of the impact on how the attributes affect [11] Kraus, A., Knapp, A., and Koch, N. 2006. Model-Driven
(negatively or positively) other attributes of the Web Generation of Web Applications in UWE. 3rd Inter.
Usability Model. Workshop on Model-Driven Web Engineering.
• Instantiate the evaluation process into different [12] Moraga, M.A, Calero, C., Piattini, M., and Diaz, O. 2007.
Model-Driven Web Development methods in order to Improving a portlet usability model. Software Quality
improve our approach. Control, Vol. 15(2), 155-177.
• Develop a tool to support the entire usability [13] Olsina, L. and Rossi, G. 2002. Measuring Web
evaluation process. The tool will be able to manage Application Quality with WebQEM. IEEE Multimedia,
the Web Usability Model by creating a repository of Vol. 9(4), 20-29.
catalogued metrics following the SQuaRE patterns. [14] Panach, J., Condori-Fernández, N., Valverde, F., Aquino,
N., and Pastor, O. 2007. Towards an Early Usability
7. ACKNOWLEDGMENTS Evaluation for Web Applications. International Conference
This work is financed by META project (ref. TIN2006-15175- on Software Process and Product Measurement. LNCS
C05-05), the Quality-driven Model Transformation Project Springer Vol. 4895, 32-45.
from the Universidad Politécnica de Valencia. The authors [15] Seffah, A., Donyaee, M., Kline, R.B., and Padda, H.K.
thank Jaime Gomez from Universidad de Alicante for his 2006. Usability Measurement and Metrics: A Consolidated
valuable help in providing the generated Web application and Model. Software Quality Journal, Vol. 14(2), 159-178.
its models used to illustrate our usability evaluation process.
[16] Ivory, M.Y. 2001. An Empirical Foundation for
Automated Web Interface Evaluation. PhD Thesis.
University of California, Berkeley, Computer Science
Division.