=Paper= {{Paper |id=Vol-153/paper-5 |storemode=property |title=Instantiating Web Sites Quality Models: an Ontologies driven Approach |pdfUrl=https://ceur-ws.org/Vol-153/paper5.pdf |volume=Vol-153 |dblpUrl=https://dblp.org/rec/conf/caise/MichF05 }} ==Instantiating Web Sites Quality Models: an Ontologies driven Approach== https://ceur-ws.org/Vol-153/paper5.pdf
       Instantiating Web Sites Quality Models: an Ontologies driven Approach

                    Luisa Mich                                                 Mariangela Franch
           Department of Computer and                                     Department of Computer and
          Telecommunication Technology                                       Management Sciences
             University of Trento, Italy                                    University of Trento, Italy
               {luisa.mich}@unitn.it                                      {mariangela.franch}@unitn.it


                        Abstract                               purpose and domain-independent – it becomes necessary
                                                               to develop quality models that take also these features
    One of the most important steps in a Web site quality      into consideration (among models having a standard
evaluation project is the selection of which aspects to        version we can look at WebQual [2] and WebQEM [11];
consider. In terms of methodology, this means defining a       an extensive bibliography of models for Web site quality
model for the site. In some cases it is possible to use        is available at: http://www.economia.unitn.it/etourism/
standardized models, such as “syntactic” models, but this      risorseQualita.asp). Basically this means defining
is not possible when the evaluation must also consider         specialized models that can deal with the unique semantic
aspects that have to do with the domain and the specific       aspects of a site or sites that will undergo evaluation. The
aims of the site or more generally when the evaluation         process of definition and instantiation of a model takes
aims to consider the “semantics” of the site. The process      time and resources and also the input of experts in the
of identifying and adapting a quality model requires,          domain. In this paper we propose adopting an approach
apart from time and resources, the contribution of experts     based on the use of ontologies to support the definition of
in the domain of the site. In this paper we propose to use     detailed semantic models. This approach is an extension
ontologies to improve the efficiency of this “instantitation   of a methodology “the Quality Model Factory” described
processs”. To analyze the feasibility of the approach we       in [9] and successfully applied in the area of tourism. It
have looked at two applications in the tourism sector. The     was applied here to define modular quality models that
results, while preliminary, are encouraging. Moreover,         make it possible to take into account the characteristics of
some critical and delicate points were identified as           diverse types of tourist destinations. To do this the models
priorities for future research.                                were developed using a standard model called the 7Loci
                                                               meta-model [10]. There are two types of “modules”
                                                               specialized at two different levels of detail: the first,
1. Introduction                                                called the Common module, contains aspects that are
                                                               common to the sites of all types of tourist destinations,
    Quality in Web sites is determined by several diverse      while the second is comprised of Specialized modules
factors, some of which are general and therefore are           that contain specific aspects that are found at different
considered for all types of site and in all domains. Such      types of destinations. To analyze the feasibility of a
features include, for example, the correct functioning of      methodology based on the use of ontologies to define
the site, its conformity with standards of language use or     specialized models for Web site quality evaluation, we
of accessibility as described in normatives such as the        looked at the sites of accommodations and of tourist
Web Content Accessibility Guidelines of the W3C                destinations facilities. Both are within the tourism domain
(http://www.w3.org/WAI/GL/) or the U.S. Section 508            and are highly varied given the diversity of types of
Guidelines (http://www.section508.gov/), or standards          tourist destination as well as accommodations. Moreover,
introduced in Italy through the Stanca law in 2004             the decision to look at the tourist sector – a transversal
(http://www.pubbliaccesso.it/biblioteca/documentazione/s       sector that includes numerous actors and activities – made
tudio_lineeguida/), which requires these standards for         it possible to have a general idea of the difficulties and
government sites. Other factors or characteristics are         challenges, as well as the advantages, of using ontologies
more specific and depend on the type as well as the            to define quality evaluation models that are specialized
domain of the site. Therefore in cases where it is not         and modular.
possible to use “standard”, “syntactic” models - general-          Recent years have seen increased interest in the
                                                               development and application of ontologies. This has
52   Luisa Mich and Mariangela Franch




meant firstly the definition of languages and                    quality of Web sites becomes a vital strategic factor for
environments to set up ontologies. Examples of languages         all actors involved. Because tourism is a transversal
are DAML (Description Logic Markup Language), OIL                sector – or “umbrella industry” – it has contributions from
(Ontology Interchange Language), RDF (Resource                   other sectors, thus an analysis of the sector must have
Description Framework) and OWL (Ontology Web                     input from different fields such as transport, culture, and
Language), a semantic markup language for publishing             sport, to name a few. This fact explains the existence of
and sharing ontologies on the World Wide Web                     numerous ontologies for tourism. An exhaustive
(http://www.w3.org/TR/owl-ref/). Among the more                  classification can be found in [1].
numerous applications of ontologies are projects related            Existing ontologies are both general for the tourism
to the Semantic Web, to obtain “an extension of the              sector as well as specific, the latter referring to particular
current Web in which information is given well-defined           domains. In the first category we have the ontology
meaning, better enabling computers and people to work in         developed for the Harmonise project, whose goal is to
cooperation.” [3]. An obvious application of ontologies is       develop an ontology-mediated integration of tourist
in Web services; among the most recent works and most            systems following different standards so that
frequently downloaded (was in the fifth position of the          organizations can exchange information without changing
Top 10 Downloads from ACM’s Digital Library in                   their data structures (http://www.harmo-ten.info/). In
December 2004), we can cite [12]. Also important to note         addition there is the Mondeca´s tourism ontology, which
is the use of ontologies for the development of software         includes tourism concepts from the WTO (World
based on reuse [5] and for the management of multimedia          Tourism Organization) thesaurus. At this writing this
objects in a private and personal environment [7].               ontology      has     1000     concepts      that    describe
    The proposal described in this paper differs from these      accommodations and transportation and a few other
insomuch as it is applied at a different level: it is designed   secondary elements related to geography, health and
to use ontologies to support the development of quality          immigration (http://www.mondeca.com/).
models that are specialized for Web sites. As such it is a          There are over ten elements on the list of domain-
conceptual as well as methodological activity, and is            specific ontologies that can be useful for the tourist
applied at a meta-level with respect to the application of       sector, including geographic ontologies, means of
the models themselves.                                           transportation ontologies, gastronomy ontologies, etc. [1].
    The paper is structured as follows: the first section           General – or sometimes called upper – ontologies also
gives a general description of ontologies for tourism.           exist and aim to gather definitions and concepts that
Following this we look at the concept of tourist                 together make up what is known as unspecialized
destination, where we present the classification used to         common knowledge. One of the best known of these is
apply the “Quality Model Factory” methodology, which             WordNet – more appropriately referred to as a lexical
we are going to extend to include domain ontologies. The         reference system (http://www wordnet.princeton.edu/)
third section puts forth two possible applications for the       which was extended from solely English into other
development of specialized and modular evaluation                languages          through         the         EuroWordNet
schemes, respectively for hotel Web sites and for tourist        (http://www.globalwordnet.org/gwa/wordnet_table.htm).
destinations. The conclusion summarizes the preliminary             Looking at the ontologies for hotels and tourist
results of these applications, underlining the critical          destinations (the organizations examined for the
points emerging from the application of the methodology          feasibility analysis of the approach proposed here) the
and which require further study.                                 following facts emerged.
                                                                    Since the concept of hotel is part of common
2. Ontologies for tourism                                        knowledge, the notion is present in WordNet. More
                                                                 specifically, for each concept – in this case hotel –
   Tourism represents approximately 11% of worldwide             WordNet gives information on the generalization,
GDP, according to the World Travel & Tourism Council             specialization and also on “part of” relationships. A
(http://www.wttc.org/). Adding to this is the notable            description of the linked concepts is available at
growth in the number of tourism-related Internet                 Answer.com (www.answer.com). In short, in WordNet,
transactions in recent years (e-commerce). For example,          focusing to concepts that are directly linked to hotel, thus
in 2004, 40% of U.S. travelers who use the Internet              exploring only relationships represented by arcs going out
claimed to make all of their travel purchases online,            from the node of hotel, we obtain (see also figure 1):
versus 29% in 2003 (www.tia.org). In this context, the
                                                  Instantiating Web Sites Quality Models: an Ontologies driven Approach 53



hotel is a kind of:                                           projects. Answers.com also provides hypertext definitions
    • building, edifice — a structure that has a
      roof and walls and stands more or less                  (where they exist) found in other sources such as
      permanently in one place                                Wikipedia (http://www.wikipedia.org/), which can be
kinds of hotel:
    • hostel, hostelry, inn, lodge — a hotel
                                                              useful to integrate with those contained in WordNet, and
      providing overnight lodging for travelers               also their translation into diverse languages.
    • motel, motor hotel, motor inn, motor                       The more challenging problem is to identify an
      lodge, tourist court, court — a hotel for
      motorists; provides direct access from                  ontology for tourist destinations, although this difficulty
      rooms to parking area                                   is justified by the complex definition of destination, a
    • resort hotel, spa — a fashionable hotel                 concept which, albeit only recently, is assuming an
    • Ritz   —   (informal)   an   ostentatiously
      elegant hotel                                           increasingly important role in the tourist sector in general.
    • ski lodge — a hotel at a ski resort                     Basically, the features that identify a tourist destination
Parts of hotel:                                               and distinguish it from what is simply a local offering of a
    • hotel room — a bedroom (usually with bath)
      in a hotel                                              product or service that can be of interest to tourists are:
                                                                  - a well-defined geographic area with identifiable
   Furthering the analysis by using the concept of hotel            borders and a territorial identity;
room, WordNet gives other information:
                                                                  - the presence of numerous operators with different
kinds of hotel room:                                                prospectives and objectives that makes it necessary
    • adjoining room — a hotel room that shares                     to devise a shared strategy in presenting the
      a wall with an adjoining room but is not
      connected by a door                                           offering consisting of attractions and services
    • connecting room — a hotel room that shres                     specifically catering to tourists in the location;
      a wall with an adjoining room and is
      connected by a private door
hotel room is a kind of:                                          - an understanding of the nature of the potential
    • bedroom,     sleeping    room,     chamber,                   demand for the tourist products offered;
      bedchamber — a room used primarily for
      sleeping.                                                   - awareness of the need to balance tourism’s
hotel room is a part of:
    • hotel — a building where travelers can pay
                                                                    exploitation of resources with ecological,
      for lodging and meals and other services                      environmental and community stewardship.
                                                                 A classification of the destinations serving the leisure
                                                              tourist segment identifies eight distinct types of
                                                              destination based on the goals for the vacation and the
                                                              principal attractions present at the destination (table 1)
                                                              [8]. The table shows key information about the defining
                                                              features of a destination. Once established, these aspects
                                                              can then serve as input when determining the
                                                              requirements and the quality factors for the Web site of
                                                              the destination.
                                                                 The definition of tourist destination and the
                                                              classification of diverse types of destination show how
                                                              the necessary concepts belong to diverse domains.
                                                              WordNet gives no treatment to the concept of tourist
                                                              destination. Numerous ontologies were examined as part
                                                              of this research, but none proved able to cover all the
                                                              elements that characterize a destination: it is thus
                                                              necessary to use different ontologies to describe the
                                                              geographic area, the different attractions, sports,
    Figure 1 – The graph for hotel (WordNet browser           transport, etc. On the other hand, general ontologies
       Treebolic, http://treebolic.sourceforge.net/)          (upper ontologies) contain a lot of information that is not
   The quantity of information about the concept of hotel     useful because it is related to concepts that differ greatly
contained in WordNet is exhaustive, an important fact         from those used in the tourism context. In addition,
since WordNet is free and has been used in many               general ontologies for tourism cover only some of the
54       Luisa Mich and Mariangela Franch




necessary areas (for example, Mondeca gives good                                                    framework. The use of modules derives from the
treatment to accommodations and transportation) but not                                             application of reuse of artefacts [13] as a viable practice
for some indispensable aspects of a destination (examples                                           for definition of evaluation models. Scalability is obtained
being cultural, natural and artistic attractions or events).                                        thanks to the adoption of a general conceptual framework,
    For this reason we have used WordNet in our                                                     for Web site quality, the 7Loci meta-model
feasibility analysis, and the decision as to which concepts                                         (www.economia.unitn.it/etourism/pubblicazioni.asp).
to consider was informed by the definition of destination                                           This model introduces seven dimensions used to classify
itself and by the table. For example, for alpine                                                    the numerous features of a Web site that can then be
destinations we used the concepts of sport, landscape,                                              evaluated. The dimensions are Identity, Content, Services,
nature, etc. We were thus able to simulate the nucleus of                                           Location, Maintenance, Usability and Feasibility.
an ontology for tourist destinations.                                                                  The foundational procedure that serves as the starting
                                                                                                    point in developing a modular model for a given class of
     Table 1. Classification of destinations based on their                                         Web sites is outlined in the steps in table 2.
                      principal attractions

Type of                                            Well-known       Typical attractions found              Table 2. Procedure for the quality model factory
              Main reasons for visiting
destination                                        examples         at the destination
              Culture, art, architecture,                           Museums, historic
Urban                                              Capital cities
              shopping                                              buildings, shops
                                                                    Beaches, organized              {1ST PART: DEVELOPMENT OF COMMON AND SPECIALIZED MODULES}
              Relaxation, enjoyment,               Rimini,                                          IF no model for the class of sites currently exists
Beach/Sea                                                           activities, amusement
              socializing, sports, night-life      Ibiza, Miami                                     THEN FOR each of the 7Loci dimensions pertinent to the project
                                                                    parks, discos, bars, pubs
              Outdoor sports, landscape                                                                        Identify the requirements common to all sites in the class and convert them
                                                   Cortina,                                                    into a question; add the question to the Common module;
              and environment, nature,                              Nature trails, views, ski
Alpine                                             Chamonix,
              traditional events and                                trails and slopes, ski-lifts               Identify the specific requirements for the type of site under evaluation and
                                                   Aspen
              customs, folklore                                                                                convert them into a question; add the question to the Specialized module;
              Get back to nature, local                             Local food producers and        ELSE       FOR each dimension of the 7Loci:
                                                   Tuscany,
Rural         traditions in agriculture and                         agritours, visits to farms                    FOR each question of the existing model
                                                   Provence
              production                                            and vineyards                                   IF the question is applied to the type of sites in its current form
                                                                    Places equipped for health                      THEN Add the question to the Common module
                                                                    and therapeutic treatments,                     ELSE IF the question requires only a formal modification
              Health treatments,                   Fiuggi,          areas for complete                                      THEN Modify the question and add it to the Specialized
Wellness      relaxation, diet and exercise        Baden-           relaxation, medium- and                                 module;
              programmes, stress relief            Baden            high-level                                      IF the question is inapplicable to the type of site under evaluation
                                                                    accommodations facilities,                      THEN check whether there is an alternative question and add it to the
                                                                    fitness
                                                                                                                    Specialized module
              Renewal or deepening of
                                                                    Place of pilgrimage,            {2ND PART: COMPLETION OF COMMON AND SPECIALIZED MODULES}
              faith, symbolic value of the         Lourdes,
Religious                                                           religious practices and         FOR each requirement for the type of site under evaluation
              location, spiritual retreat and      Fatima
                                                                    celebrations                    Identify the 7Loci dimension it refers to
              introspection, solitude
              Adventure, discovery of                               Cities, historic places,        IF no question exists for it in the Common or Specialized module
              other cultures, understanding                         rites, customs,                 THEN IF the question regards all the sites in the class
Third                                              Yemen,                                                   THEN Add a question to the Common module
              of tribal life (rites, traditions,                    celebrations, guided tours,
World                                              Madagascar                                               ELSE Add a question to the Specialized module.
              lifestyle) anthropological                            contact with non-western
              investigation                                         local cultures
                                                                    Villages in traditional style
              Beautiful scenery, isolated
                                                                    but with all modern
Exotic and    locations, far from tourist          Maldives,
                                                                    conveniences, privacy,
Exclusive     trek, status symbol and              Seychelles                                          The procedure has two parts; the first is the
                                                                    untouched natural
              image
                                                                    environments
                                                                                                    instantiation of the 7Loci model, reusing where possible a
                                                                                                    model already defined for one of the types of site in a
                                                                                                    class. This step is necessary in order to define the model
3. The use of ontologies to define specialized                                                      using requirements as the starting point. In the second
models                                                                                              part the specific elements for the type of site are
                                                                                                    identified. When talking about the sites of tourist
3.1 The Quality Model Factory                                                                       destinations, this means translating these elements into
                                                                                                    points (characteristics) within the evaluation model; for
   In [9] we described a modular and scalable approach –                                            example, the unique features of the site for a seaside
the Quality Model Factory - to define specialized quality                                           destination will be converted into points in the model
models identifying the specific features of tourist                                                 used to evaluate that type of site, while features for
destination Web sites. Its goal was to introduce a                                                  another type of destination (religious, for example) may
systematic way to define a “personalized” evaluation                                                not contain those same points. In our previous work we
                                                                                                    looked at the aspects in table 1, which in general can be
                                                                 Instantiating Web Sites Quality Models: an Ontologies driven Approach 55




found in classification schemes of the sites of a specific                         hotels. The model consists of 18 questions: four for the
category. When describing the procedure, for the sake of                           dimension Identity, nine for Content, two for Services and
simplicity we refer to “questions” to insert in the                                one for Localization, Management and Usability. The
evaluation modules. In reality this is only one way of                             model is a useful reference to check the results that can be
formulating the points or factors of the evaluation model;                         obtained with the Quality Model Factory which is
besides interrogatives (e.g., in boolean questions) they                           strengthened through the use of ontologies. In this case
can also be described in declarative form.                                         we have used the concepts related to hotel in WordNet. In
   In this paper we want to broaden the approach to be                             addition, for the requirements of the owner and user, we
able to develop models for diverse types of sites, and such                        have put together information emerging from the research
generalization is conceptually based on the use of                                 conducted on the hotels [6]. In this case it meant
ontologies. The steps of the procedure where ontologies                            developing a model more similar to that constructed with
can be used are shown in table 2.                                                  the contribution of experts, and also applicable to all
   The logical architecture of the Quality Model Factory                           hotels. Thus we did not intend to construct different
is depicted in figure 2 (a Quality Factory to support                              models for different types of hotels.
“information quality” assessment is described in [4]),
where the database “Web site classification” is substituted                        3.3. Models for tourist destinations
with a database containing the ontologies.
                                                                                       Applying the Quality Model Factory approach to
                                                                                   tourist destination Web sites without using ontologies
                                                                                   produced a series of models, each containing about 100
                                  Web site evaluation
                                  project                                          factors that are a reference for evaluating and which can
                                                                                   be produced with ontologies [9]. Moreover, we thus
                                                                                   obtained important results for the adaption process, where
                                                           Quality model factory   ontologies are necessary to specialize the general-purpose
                 Web sites
                 classification
                                                                                   models. Also emerging, in particular, was how the first
                                     7Loci               Quality
                                     instantiation       evaluation                two dimensions of the 7Loci meta-model depend more on
                                                                                   the type of site (with an average of about 80% of
 Requirement
 elicitation
                 Quality models
                 repository
                                                                                   specialized factors); about one third are specialized for
                                     Quality
                                     requirements
                                                         Validation                Services, ten percent for Usability, Maintenance and
                                                                                   Localization (related principally to the presence of
                 Requirements
                 repository                                                        different target tourist groups). Development of a model
                                                                                   for the evaluation of Web site quality for tourist
                                                                                   destinations is underway as part of the joint project
                                    Quality evaluation
                                    Modules                                        IFITT/WTO (www.ifitt.org/)sito. The model foresees two
                                                                                   levels of assessment: the first looks at general aspects and
                                                                                   therefore at requirements that all tourist destination Web
               Figure 2. The Quality Model Factory
                                                                                   sites must satisfy; the second is called Strategy Based Full
                                                                                   Web Site Evaluation. A limitation of the IFITT/WTO
                                                                                   model is that there is a measurable gap between the two
    From among the different actors in the tourist sector
                                                                                   levels. Essentially the first level uses a general model for
present in the territory we focused on hotels and tourist
                                                                                   all destinations while the second must be defined ad hoc.
destinations when studying our approach. For both
                                                                                   We propose the use of ontologies to specialize the quality
categories tourism is the core business and the Web site is
                                                                                   models in a systematic way.
a strategic tool. For the tourist destination the Web site
serves to promote and commercialize the products as well
as to support the different actors.                                                4. Conclusions

3.2. Models for accommodations facilities                                             The project is still underway, the results obtained thus
                                                                                   far have revealed the following aspects:
  In a previous project we built a model for the
comparative evaluation of the Web sites of about 200                                  - The dimensions of the 7Loci meta-model where
                                                                                        the use of ontologies is more straightforward are
56    Luisa Mich and Mariangela Franch




        those which are more “semantic”, thus Content,                a version of at least two different types of file that
        and Services; for these dimensions it is possible to          even non-experts can understand.
        use the hierarchies contained in the ontologies as
        check-lists to identify aspects for which the site     10. References
        must give information or support for services.
     - For the dimension Identity it is necessary to           [1]. Bachlechner D., D10 v0.2 Ontology Collection in
                                                               view of an E-Tourism Portal, E-Tourism Working Draft
       integrate the ontologies with the aims of the site
                                                               5, Oct. 2004, http://www.deri.at/research/projects/e-
       and be able to connect them to the concepts in the
                                                               tourism/2004/d10/v0.2/20041005/
       ontology that mainly contribute to the creation of
       the image of the organization; these concepts are       [2]. Barnes S. J., Vidgen R. T., WebQual: An
       usually specializations of “father” concepts; in the    Exploration of Web Site Quality, H. Mahrer (ed), Proc. of
       case of hotels, their specialization is for specific    the 8th European Conf. on Information Systems, Vienna,
       target markets, for example motor hotels.               July 3-5, 2000, pp. 298-305
     - For all the dimensions of the 7Loci meta-model,
                                                               [3]. Berners-Lee T., Hendler J., Lassila O., The
       the combined use of ontologies, of the
                                                               Semantic Web, Scientific American, May 2001, 284(5):
       standardized version accompanied by a list of the
                                                               34-43
       main aims of the site (no more than five elements:
       e.g., target, business functions, general links), by    [4]. Capiello C., Francalanci C., Pernici B., A rule based
       people with limited experience working in Web           methodology to support information quality assessment
       site quality (undergraduates students), made it         and improvemnet, Studies in Communication Sciences,
       possible to develop “draft” specialized models in a     4(2): 137-154, 2004
       short time and which can be rapidly verified and
       completed by an expert. On the whole, the               [5]. Falbo R.A., Guizzardi G., Duarte K.C., Natali
       methodology proposed makes it possible to               A.C.C., Developing Software for and with Reuse: An
       notably improve the efficiency of the process of        Ontological Approach, Proc. CSITeA’2002, USA ACIS,
       defining specialized models.                            2002, pp. 311-316
     - Substantial initial effort is required to identify      [6]. Franch M., Martini U., Novi Inverardi P.L., Buffa
       ontologies for the different domains necessary to       F., Awareness and exploitation of the potential of the
       cover the tourist sector. Moreover, existing            Web by SMTEs: The case of alpine hotels in Italy and
       ontologies are heterogeneous with regard to the         France, Frew A.J. (ed.), Springer Verlag LNCS 2722:
       coverage of the domain they refer to. Nonetheless,      318-327, 2005
       they can be reused for numerous categories of
       operators and entities.                                 [7]. Hüsemann B., Vossen G., Ontology-Driven
                                                               Multimedia Object Management for Private Users –
     - Statistical analysis of the terms used in sites under
                                                               Overview and Research Issues, AIS SIGSEMIS Bulletin, 1
       analysis provides useful information for the choice
                                                               (1), April 2004
       of ontology, but principally to choose the concepts
       to use within the ontology to instantiate the quality   [8]. Martini U., Da luoghi a destinazioni turistiche. In
       models.                                                 Franch, M. (ed.), Destination management: Governare in
     - Most existing ontologies are in English language;       turismo tra locale e globale, Turin, Giappichelli, pp. 67-
       but it could be necessary to have ontologies in         111, 2002 (in Italian)
       other languages.
                                                               [9]. Mich L., Franch M., Martini U., A Modular
     - As for their implementation, most ontologies that       Approach to Quality Evaluation of Tourist Destination
       can be used in the tourism context are written in       Web Sites: The Quality Model Factory, Frew A.J. (ed.),
       DAML and some in OWL. This means that if we             Springer Verlag, LNCS 2722: 555-565, 2005
       want to create environments to support the
       “Quality Model Factory” it is necessary to extract      [10]. Mich L., Franch M., Novi Inverardi P., Marzani P.,
                                                               Choosing the "rightweight" model for Web site quality
                                                 Instantiating Web Sites Quality Models: an Ontologies driven Approach 57




evaluation, Frew A.J. (ed.), Springer Verlag, LNCS 2722:
334-337, 2003

[11]. Olsina L., Rossi G., Measuring Web Application
Quality with WebQEM, IEEE Multimedia 9(4): 20-29,
2002

[12]. Pahl C., Casey M., Ontology support for Web
Services Processes, Proc. 9th ESEC and 11th SIGSOFT
Symposium FSE, 2003, pp. 208-216

[13]. Pressman    R.S., Software Engineering:         A
Practitioner's Approach, 5th ed., McGraw-Hill, 2001