One of the steps to evaluate quality is to define first quality requirements, for instance, by means of models. In order to specify quality models for Web 2.0 applications we propose an integrated approach which considers quality dimensions for contents and functionalities. In particular, we discuss how to model the different quality views reusing and extending the ISO 9126-1 quality models taking into account not only the software characteristics but also the own features of Web applications. To illustrate the added Content quality characteristic to quality models, excerpts of two evaluations are shown. The resulting models contribute towards a multidimensional integrated approach to evaluate Web applications of different eras -both Web 1.0 and 2.0 ones-, and lifecycle stages.
As the Web usage continues to grow, users expect Web applications (WebApps) to be more mature with regard to the functions and contents delivered for their intended goals and tasks at hand. Moreover, users expect these functions and contents be relevant, accurate, suitable, usable, secure, and ultimately with perceived quality in use. Despite the major breakthroughs in Web development methods and technologies there are some Web Engineering branches still in their infancy. Modelling quality requirements at different lifecycle stages is one of them. In this paper we propose an integrated approach to specify quality requirements for content quality to WebApps.
WebApps were conceived as content-oriented artefacts from the very beginning.
Few years later, websites started to provide not only contents but software-like
functionalities and services as well. Since then, WebApps have at a fast pace emerged
for many different sectors as e-commerce, e-learning, e-government, and so forth.
After that era named Web 1.0, a recent era that considers a set of strategies and
technologies focusing on social networking, collaboration, integration,
personalization, etc. is emerging –currently called Web 2.0 [
In the present work, by reusing and extending the ISO 9126-1 [
Later on, as a motivating example for evaluating the content quality dimension [
Line item information completeness Product description appropriateness
The rest of this paper is structured as follows. In Section 2, we give an overview of the Web eras as well as the unique features of WebApps compared to traditional software applications. A review of the ISO software quality models follows in Section 3. In Section 4, we discuss what is missing in these models with regard to the Web features; we also illustrate the proposed extension to ISO quality models in order to include the content quality dimension. Then, Section 5 illustrates an excerpt of a quantitative evaluation for a shopping cart of a Web 1.0 app regarding the content suitability characteristic; in addition, it describes the application of a qualitative evaluation by comparing three touristic apps related to Web 1.0 and 2.0 eras. Finally, in Section 6, we analyze our proposal in the light of related work, and then, in Section 7, we draw concluding remarks.
The first WebApps can be grouped in the Web 1.0 era, and they can be categorized
into static and dynamic; most recent WebApps can be grouped in the so-called Web
2.0 era as per O’Reilly [
In addition, Murugesan [
On the other hand, as said in Olsina et al [
They involve ethic, cultural, and legal aspects as well.
Most of the above features make a WebApp a particular artifact. However, like any software application, it also involves source and executable code, persistent structured data, architectural design and so on. Ultimately, many of the above features will influence the way quality requirements are modeled. We need to deal not only with usability, functionality, efficiency, reliability and maintainability, as in traditional software products but also with content quality, i.e. with content accuracy, suitability, accessibility, and legal compliance.
Quality is not a simple and atomic concept, but rather a multidimensional and relative
one. Common practice assesses quality by means of the quantification of lower
abstraction concepts, such as attributes of entities. An attribute can be defined as a
measurable property of an entity category. Therefore, quality –and its sub-dimensions,
called characteristics and sub-characteristics in the ISO 9126-1 standard-, is an
abstract relationship between attributes of an entity and a specific information need,
with regard to its purpose, context, and user’s viewpoint [
The ISO 9126-1 standard (and the ongoing SQuaRE project) distinguishes among
three different approaches to software product quality, viz. internal quality, external
quality, and quality in use. These views of quality can be summarized as follows [
The same quality models have been maintained both to internal and external views
(see Fig. 2). For instance, functionality characteristic is defined as “the capability of
the software product to provide functions which meet stated and implied needs when
the software is used under specified conditions” [
Besides, usability characteristic is defined as “the capability of the software
product to be understood, learned, used and attractive to the user, when used under
specified conditions” [
Lastly, the core aim in designing a WebApp is to provide users with degrees of excellence or quality in use by interacting with the application and by performing its tasks comfortably. Regarding the spirit of the ISO 9126-1 standard, quality in use is the end user’s view of the quality of a running system containing software, and it is measured and evaluated in terms of the result of using it rather than by properties of the product itself. Ultimately, taking into account meaningful software or WebApp attributes for internal quality are a prerequisite to achieve the required external behavior, and considering meaningful software attributes to external behavior are a prerequisite to achieve quality in use.
The assumption of our proposal is that the ISO 9126-1 software quality models
introduced in the previous section are also applicable to a great extent to intermediate
and final lifecycle WebApps. To justify this we are going to extend the discussion
made in [
Like any software production line, the Web lifecycle involves different stages of its products, whether in early phases as inception and development, or in late phases as deployment, operation, and evolution. To assure the quality of products, we can plan to do it by evaluating and controlling the quality from intermediate products to final products. Thus, to the general question, if we can apply to WebApps the same ISO internal and external quality, and quality in use models, the natural answer is yes. Nevertheless, to the more specific question whether we can use the same six prescribed quality characteristics (and their sub-characteristics) for internal and external quality requirements, and the four characteristics for quality in use requirements, our answer is yes for the latter, but some other considerations might be taken into account for the former. As highlighted in Section 2, the very nature of WebApps is a mixture of contents and functions. Therefore we argue that the set of six characteristics, i.e. functionality, usability, reliability, efficiency, maintainability and portability, are not well suited (or they were not intended) to specify requirements for information quality.
At this point, we would like to introduce a slight difference in meaning between data and information terms. A piece of data is raw material; even though it has some degree of information. Data come from attribute measurements, facts, formula calculations, etc. Structured data sets are often represented in databases. On the other hand, information has an added value over data. That is, information is the meaningful interpretation of data for a given purpose and context. Usually, a traditional software program is a mixture of functions and data. Very often a webpage is content oriented, i.e. is intended to deliver information (usually unstructured semantically). For example, this article can be hyperlinked and posted as content Web pages. Also a webpage component, e.g. a shopping cart, can edit an item quantity and recalculate prices (applying a function over data).
Therefore, to follow the thread of our argument, the central issue is how we can specify the content quality for WebApps from the internal and external quality viewpoints. We need to deal not only with usability, functionality, efficiency, reliability, and maintainability but also with the content quality characteristic, which in turn can be subdivided into content accuracy, suitability, accessibility, and legal compliance sub-characteristics, as shown in Fig. 2.
As a consequence, we propose to include the Content quality characteristic (or Infoquality, for short), and its sub-characteristics in the internal and external quality model of the ISO standard. A point worth mentioning is that in the spirit of the ISO 9126-1 standard is stated “evaluating product quality in practice requires characteristics beyond the set at hand”.
On the other hand, the quality in use definition may be rephrased as “the capability
of the software or WebApp product to enable specified users to achieve specified
goals with effectiveness, productivity, safety and satisfaction in specified context of
use”. Note that effectiveness, productivity, safety, and satisfaction are influenced not
only by the usability, functionality, reliability, efficiency, and content quality of a
WebApp, but also by two resource components of the context of use. The context of
use depends on both the infrastructure (i.e. the computer, network, or even the
physical working medium) and the user-oriented goals, i.e. the supported WebApp
tasks and the properties of the user type such as level of IT training, expertise, age,
and cultural issues as well. (See for example the quality in use case study for an
elearning WebApp [
The content sub-characteristics are: first, content accuracy, which addresses the
very intrinsic nature of the information quality; second, content suitability, which
addresses the contextual nature of the information quality; it emphasizes the
importance of conveying the appropriate information for user-oriented goals and
tasks; in other words, it highlights the quality requirement that content must be
considered within the context of use and the intended audience; third, content
accessibility, which emphasizes the importance of technical and representational
aspects in order to make Web contents more accessible for users with various
disabilities as regarded in [
Content Accuracy 1.1. Correctness, the extent to which information is reliable in the sense of being free of errors. 1.2. Believability (synonym: Credibility), the extent to which the information is reputable, objective, and verifiable. 1.2.1. Authority (synonym: Reputability), the extent to which the source of the information is trustworthy. 1.2.2. Objectivity, the extent to which the content (i.e., information or facts) is unbiased and impartial. 1.2.3. Verifiability (synonym: Traceability), the extent to which the owner and/or author of the content can be verified. 1.3. Currency (synonym: Up-to-dateness), the extent to which the information can be identified as updated.
Content Suitability 2.1. Value-added, the extent to which the information can be novel, beneficial, and contribute to react to a given user for the task at hand. 2.1.1. Novelty (synonym: Freshness), the extent to which the information is fresh and contributes to make new decisions for an intended user goal. 2.1.2. Beneficialness, the extent to which the information is advantageous and contributes to make new decisions for an intended user goal. 2.1.3. Reactiveness, the extent to which the information is compelling and contributes to react for an intended user goal. 2.2. Coverage, the extent to which the information is appropriate, complete but also concise for the task at hand to a given user. 2.2.1. Appropriateness, the extent to which the information coverage fits to an intended user goal. 2.2.2. Completeness, the extent to which the information coverage is the sufficient amount of information to an intended user goal. 2.2.3. Conciseness, the extent to which the information coverage is compactly represented without being overwhelming. 2.3. Consistency, the extent to which the content is consistent to the website’s piece of information or page with respect to the intended user goal.
In addition to the above content sub-characteristics, others to information architecture and organization can be addressed. Many of these sub-characteristics, such as global understandability, learnability, and also operability and attractiveness, can be related to the usability characteristic. Besides, other particular attributes to search and navigation functionalities can be specified in the functionality subcharacteristics; for example, are the basic and advanced search suitable for the end user? Or, are they tolerant of misspelled words and accurate in retrieving documents? In the same way, we can represent link and page maturity attributes, or attributes to deficiencies due to browsers’ compatibility into the reliability sub-characteristics.
Finally, from the quality in use perspective, we have proposed to use the same ISO model. However, for the satisfaction characteristic, specific (questionnaire) items for evaluating quality of content should be included.
The model for evaluating the content quality dimension shown above is a general schema adaptable to all kinds of WebApps. It means that the integrated model can be used with both the Web 1.0 and the Web 2.0 apps and different domain types. It is for this reason that when you decide to apply it to a given WebApp, it must be instantiated on the basis of some more specific parameters, e.g. measurable attributes and metrics, questionnaire items, and decision criteria (e.g. using indicators), considering also the specific domain, information need, and user viewpoint.
Next, for space reasons we show excerpts of two studies. One of them was a
quantitative evaluation for a shopping cart of a Web 1.0 app (performed twice, in
2004 [
Table 2 shows two attributes coded 1.1.1.1 and 1.1.1.2 (highlighted in Fig. 1), for
evaluating the coverage level of the basic information to the Cuspide shopping cart.
Note that Coverage is a sub-characteristic of Content Suitability as specified in Fig. 3.
The purpose of the evaluation was to understand the external quality level of Content
quality and Usability dimensions for the shopping cart entity, and then recommend
improvements in case low satisfaction of these requirements were reached. The
improvements were made applying Web Model Refactoring (refer to [
First, for each attribute of the requirement tree we selected a metric to quantify it. For example, the Line item information completeness attribute is defined as “the extent to which the line item information coverage is the sufficient amount of information to an intended user goal and task”; thus, final users expect to have suitable information as title, author, price, quantity, added on date, and availability in order to accomplish the task effectively. So we designed a direct metric named Degree of completeness to the line item information whose scale specifies three categories considering an ordinal scale type, namely: (0) Incomplete, it has less information than category 1, or no information at all; (1) Partially complete, it only has title, price, quantity, and sometimes availability fields; and (2) Complete, it has the whole required information.
After measurement it resulted 1, as shown in table 2. However, the measure by itself does not represent the elementary requirement’s satisfaction level, so we have to design an elementary indicator. Note that an elementary indicator interprets the level of satisfaction of an elementary nonfunctional requirement (i.e. an attribute). Therefore, a new scale transformation and decision criteria (in terms of acceptability ranges) should be defined. In our study, we used three agreed acceptability ranges in a percentage scale: a value within 40-70 (a marginal –yellow- range) indicates a need for improvement actions; a value within 0-40 (an unsatisfactory –red- range) means changes must take place with high priority; a score within 70-100 indicates a satisfactory level –green- for the analyzed attribute. Table 2 shows an elementary indicator value of 50% for the 1.1.1.1 attribute taking into account that a measure value of 1 mapped to 50%. In conclusion, this attribute would satisfy suitability totally if the measure is 2.
In our case, as mentioned in the introduction, for lack of information users are not
able to distinguish between two or more items with the same starting title, when they
want to add a specific item quantity or compare prices. The reader can refer to [
On the other hand, regarding the qualitative study, three WebApps to the tourism sector, namely www.opodo.com, www.tripadvisor.com, and www.wayn.com were evaluated by experts on a comparative basis. This sector, these WebApps, and the evaluation aim have been chosen for the following reasons: (i) the comparison is a method used to seize on correspondences and specific differences between the entities observed; (ii) the evaluation as aforementioned was conducted by experts in the tourism sector. The eTourism Research Group1 collaboration to which one of the authors belongs allows us to take advantage of its expertise and knowledge gained over the years in the tourism research field; and (iii) the selected WebApps represent to some extent the development of applications from Web 1.0 to Web 2.0 eras. We expected the model be useful with both Web 1.0 and Web 2.0 applications. Opodo has all the features to be classified as Web 1.0, Tripadvisor offers some Web 2.0 features, and finally Wayn can be entirely classified as a Web 2.0 application.
The evaluation considered the whole sub-characteristics/attributes of table 1 as well
as some attributes regarding content accessibility and content legal compliance. For
the former were included sub-characteristics as perceivable, operable, understandable,
robust, which are the principles of WCAG 2.0 [
The first relevant issue to underline is that in the process of content production in
Opodo (Web 1.0 app) a top-down logic is pursued, it means that only content
providers supply information to users. On the other hand, in TripAdvisor and Wayn
(Web 2.0 apps) the process of content production becomes bottom-up; i.e. mainly
users upload and update information. Moreover in Wayn the content is submitted to a
1 Research team working at the Department of Computer and Management Sciences, Faculty of Economics
at University of Trento; group site available at http://www.economia.unitn.it/eTourism/home.asp
social control mechanism, since with a Web 2.0 app (e.g. blog, wiki, social network) a
user can edit, add or comment content of other users. Besides, in the initial
observations it is quite evident how “non-structured” information can be considered
more accurate and suitable in ‘tripadvisor.com’ with respect to ‘opodo.co.uk’. Note
that when we talk about non-structured information we refer for example to hotel
review, location comment, etc. Conversely, for structured information –e.g. flight
timetables, holiday price lists, among other, it can be considered more accurate and
suitable in ‘opodo.co.uk’ (see more details in [
Lastly, in general terms we argue that the WebApp’s content quality does not depend on the kind of applications –whether Web 1.0 or Web 2.0; however, some kind of contents and services are more appropriate for Web 1.0 apps, while others for Web 2.0. Moreover, we can state the content sub-characteristics we have modeled for evaluation purposes can be applied to all WebApps, independently from which era they belong. However, some specific measurable attributes associated to content subcharacteristics could be considered for Web 2.0 and follow-on applications, as we discuss in a future work.
The model presented in this paper as an extension of the ISO 9126-1 quality models,
has been elaborated taken into account related researches about dimensions of data
and information quality. We describe here briefly the main information quality
models, often called frameworks in the literature. A broader reviews for WebApp
quality is given for instance in [
One of the first studies intended to categorize data quality dimensions was made
by Strong et al [
A different slightly way to model and evaluate the information quality for a
WebApp –both at page and site level- is proposed by Alexander et al [
The first published study about extending the ISO 9126 model has been made by
Zeist et al [
Most WebApps besides the increasing support to functionalities and services will continue aiming at showing and delivering content. This basic feature stemming from the early Web 1.0 applications is currently empowered by the Web 2.0 and follow-on applications. Web 2.0 applications rely strongly on actual users sharing, collaborating and performing content tasks in real contexts of use. So evaluating the quality of WebApps is still or even more challenging.
In the present work, we have proposed how to specify quality requirements for WebApps’ functions, services and contents employing a multidimensional and integrated approach. By reusing and extending the ISO 9126-1 quality models’ 2 Note that as part of the project the ISO 25012:2008 standard on data quality model was recently issued characteristics, we have discussed the need of modeling and adding the content quality characteristic for evaluating the quality of information. Specifically, we have argued that the internal and external quality models with the set of six characteristics, i.e. functionality, usability, reliability, efficiency, maintainability and portability, and their sub-characteristics respectively, are not sufficient to specify WebApps’ information quality. As a consequence, we have proposed to include in both models the Content quality (Infoquality) characteristic and its sub-characteristics. Besides, from the quality in use perspective, we have proposed to use the same ISO model.