-Portals have evolved from being simple providers of Web page access and corporate databases into being sites that support intelligent management, integration of applications and collaborative processing. Portals can be considered as an evolution of datawarehouses by extending its application to Intranets and giving access to all information resources and knowledge of a firm. All these facts make it essential that developed portals are of good quality. Our aim is to define a portal quality model and, to achieve it, we have used the GQM method. As our model is in the definition phase now, only its two first phases have been used. That is the reason why our model must be considered as a first approach and not as a close and definitive model. Our model, called PQM, is composed of six dimensions: tangibles, reliability, responsiveness, assurance, empathy and data quality. These dimensions have been obtained by means of the SERVQUAL method. Moreover, we have done a survey of the workers of the portal of Castilla-La Mancha (castillalamancha.es) to obtain their opinion and assesment about different aspects of this portal in relation to PQM.
Portals prov ide a way to access to a local or remote network, to a company in the case of a corporate portal or to general-interest topics and services in the case of a public portal.
In general, portals provide [
- Personalization capabilities for individual users. - A set of “portlets” (components that integrate data, applications, content, and resources and present information to the portal user)
- A single sign -on to the set of applications accessed via the portal.
- Other features, such as search and collaboration.
Therefore, portals have evolved from being simple
providers of Web page access and corporate databases into
being sites that support intelligent management, integration of
applications and collaborative processing. Portals can be
considered as an evolution of datawarehouses by extending
its application to Intranets and giving access to all
information resources and knowledge of a firm [
According to [
- First generation: Access Portal: They were used to provide a set of links to other information and resources.
- Second generation: Aggregation Portals: They bring information back to the portal so that the user does not have to go anywhere else.
- Other generations: Workspace Portals (where the portal becomes the users’ work environment, including all the appropriate information, tools, and resources) and Adaptive Portals (where the portal experience, itself, dynamically depending on the user’s context and the ongoing process).
However, the research on portals is still beginning. Some
key points for its evolution are [
- The percentage of organizations that treat portals as core systems will increase from less than 10 percent in 2003 to 15 percent in 2004 to 35 percent by 2007.
- The portal market is only 40 percent introduced now; by the end of 2004, 85 percent of the Global 2000 will have implemented an enterprise portal framework. This fact will lead to a 161 percent growth in revenue in 2004.
- The portal market has contracted 26 percent since 2002; by the end of 2004, another 20 percent of portal players will exit the market.
The fact of achieving that a portal is of good quality is
fundamental because it is the only mechanism that makes users
return [
In [
In [
However, there is not a specific framework to control the quality of portals, probably due to the recent developing of portals.
In order to put an end to this lack, we present a first
proposal of a portal quality model. This proposal has been
made using, as a basis, the GQM method [
Also, we have done a survey of the workers of the Castilla-La Mancha portal (castillalamancha.es) to obtain their assessment about different aspects of the portal with the objective of evaluating the quality of the above mentioned portal in relation to each dimension of quality that we have defined.
Moreover, the model proposed by us is valid not only for corporate portals but also for public ones considering that nowadays public portals include public and private components, as corporate portals. The main difference between corporate and public portals is that in public portals the business is the portal by itself, while in corporate portals, the business is to be open to the public to present their products, services, image or beliefs. In spite of this difference, all portals are becoming multipurpose. Besides, nowadays, public portals include both public and private components like corporate portals.
At last, we can emphasize that we have only used the Goal and Question phases of the GQM method to achieve our aim (to define a portal quality model). So, we have put off the definition of metrics until later on (when the model will have been validated).
This paper is structured as follows: In section two, the GQM method is shown; in section three, the quality model that we have developed is explained and the phases that we have achieved are specified. In section four, the castillalamancha.es portal is described and in section five, it is explained the survey that has been done of the workers of the castillalamancha.es portal and the obtained results are analysed. Finally, in the last section, conclusions and future work are shown. 2
The GQM method [
This method starts top-down with the definition of an explicit measurement goal. The goal is refined into several questions that break down the issue into its major components. Then, each question is refined into metrics that should provide information to answer those questions.
The GQM method is composed of four phases [
2.- Definition phase. The measurement plan is defined and documented (goal, questions, hypotheses and metrics are defined).
3.- Data collection phase. In this phase, the data collection takes place, resulting in the collected data.
4.- Interpretation phase. During this phase, the collected data are processed with respect to the defined metrics into measurement results, that provide answers to the defined questions. With these answers, it will be possible to evaluate whether the goal has been achieved or not.
As we have said before, we have followed the GQM method to define a portal quality model, but we have only used its two first phases. The other two phases will be used when the model has been validated. In the next section, we will show how our PQM model has been obtained.
The PQM model (Portal Quality Model) has been developed using the two first phases defined in the GQM method.
Taking into account the fact that the quality of a portal is difficult both to define and to measure, the model that we present below must be considered as a first proposal of framework. This model can be used to measure the quality of a portal, considering that the quality of the portal can be defined as the degree which the portal facilitates services and relevant information to the customer with.
The activities carried out in the achieved phases are detailed below.
The first activity that we carried out in the planning phase was to establish a GQM team independent of the project team. Then, the area that we wanted to improve was selected. In our case, it was the quality of portals. The project team was formed by all the developers of castillalama ncha.es portal.
The second phase, called definition, is the most important one.
In the first activity of this phase, the goal was defined. In
our case, the goal was: “To improve the quality of portals”.
The next activity was to refine the goal into several
questions. We used the SERVQUAL model [
definition. The SERVQUAL model is composed of five di- which information is inputted, processed or presented to mensions: tangibles, reliability, responsiveness, assurance them? and empathy. These dimensions were adapted to portals. - Contextual DQ: To what degree does the informaMoreover, we added another one that is related to data qual- tion provided meet the needs of the users? ity. Once the portal quality model (PQM) was defined, we decided to do a survey of the workers of the portal of Castilla-La Mancha (castillalamancha.es) to obtain their opinion about different dimensions of the portal
However, we thought that the breakdown only to that level was not enough because it was too much generic, so we divided some of these dimensions into subdimensions.
Then, we show the six dimensions (questions) that make up our model (of quality of portals) together with their subdimensions (subquestions).
- Tangible: This dimension indicates if "the portal contains all the software and hardware infrastructures needed according to its functionality”.
- Reliability: It is the "ability of the portal to perform its functionality accurately". In addition, this dimension will be affected by: - Availability: The portal must be always operative.
- Search Quality: The results that the portal provides when making a search must be appropriate to the request made by the user.
- Responsiveness: It is the "willingness of the portal to help and to provide its functionality in an immediate form to the users ". In this dimension, we distinguish the following subdimensions:
- Scalability: It refers to the ability of the portal to
smoothly adapt to increasing workloads as a result of
additional users, an increase in traffic volume or the execution of
more complex transactions [
- Speed: It relates to the response times experienced
by portal users [
- Assurance: It is “the ability of the portal to convey trust and confidence”. This dimension will be affected by: - Confidentiality: Ability to keep the privacy of the
In this section, we will explain the aim and structure of the portal of Castilla-La Mancha, since we have done a survey of its workers to know which dimensions of the model should be improved and which dimensions are correct. Also, we have proved the reliability of the survey.
The Castilla-La Mancha portal (www.castillalamancha.es) is a corporate portal that is in operation since April, 2002.
Its main goal is to generate Internet projects in the region of Castilla-La Mancha and help to achieve their success.
Regarding the portal workers, they can be divided into two profiles according to the tasks they develop:
? Contents Responsible. They manage the contents that are published in the different channels of the castillalamancha.es portal.
? Technical Support. These workers can be divided into two groups:
- Programmers: they must maintain the technological aspects of the portal.
- Designers: they are in charge of the image of the portal.
This portal is aimed at providing, above all, information about Castilla-La Mancha.
We can distinguish several channels within this portal users. (each of them contains information related to a specific - Empathy: We define this dimension as the "ability of the topic): portal to provide caring and individual attention ". In this - Technology channel: it provides information about scidimension, we distinguish the following subdimensions: entific, technological, energy or astronomical issues among - Navigation: The portal must provide a simple and others. intuitive navigation when using it. - Sixth province channel: it establishes contact with castel- Presentation: The portal must have a clear and lano-manchegos (people from Castilla La Mancha) that, at uniform interface. present, are living outside.
- Integration: All the components of the portal must - Enterprise channel: You can find events related to enbe integrated into a coherent form. terprises among other things.
- Personalization: The portal must be capable of - Employment channel: People can find a new job. adapting to the user’s priorities. - Agro channel: It contains information related to agricul
Furthermore, due to the big amount of data that are han- ture.
dled in a portal and taking into account that it is fundamen- - Environment channel: it deals with human resources,
tal that these data are of good quality, we thought it is nec- pollution, waste, animals, vegetation, water, etc.
essary to add a new dimension: - Women and elderly channel: It contains interesting
is- Data quality (DQ): This dimension is defined as "quality sues for these groups of people.
of the data contained in the portal". According to [
- Intrinsic DQ: What degree of care was taken in the These channels are focused on Castilla-La Mancha region. creation and preparation of information? Moreover, there are e-commerce, links to others websites, e- Representation DQ: What degree of care was taken learning, and others interesting things. in the presentation and organization of information for users?
- Accessibility DQ: What degree of freedom do users have to use data, define and/or refine the manner in
⎛
r XX ' = 2*
⎜1⎜
⎝
s 2p + s i2 ⎟⎞
Questionnaires are, probably, the most commonly used
research method according to [
The questionnaire consists of forty two questions (see s i2 shows the variance of the marks obtained by the subAppendix). The items that compose each dimension (which jects in the odd items . is equivalent to indicate the dimension which each of the s X2 shows the variance of the global marks. questions asked in the questionnaire corresponds to) are: We have applied the above exposed formula to the results - In the Tangible dimension, we classify questions Q26, obtained by the survey done of the workers of castillalamanQ39. cha.es and we have obtained that the reliability of the survey is - In the Reliability dimension, we have questions Q11, 0.9. Therefore, we can state that our survey is reliable because Q15, Q29, Q41. the reliability coefficient can vary from 0 to 1, where 0 means - In the Responsiveness dimension, we have questions that there is no reliability and 1 means there is a maximum of Q1, Q3, Q4, Q12, Q13, Q14, Q38. reliability.
- In the Assurance dimension, we classify questions Q21, Q22, Q23. 5.2 Results
- In the Empathy dimension, we have questions Q2, Q5, Our following step was to analyse the results obtained by Q6, Q10, Q16, Q17, Q18, Q19, Q20, Q24, Q25, Q27, Q28, Q32, the survey.
Q33, Q34, Q35, Q36, Q37, Q40, Q42. Due to the different profiles of workers, not all the sub- In the Data Quality dimension, we classify questions Q7, jects answered all the questions that were raised. Therefore, Q8, Q9, Q30, Q31. the results of each question (table 1) have been obtained
Before the workers of the portal answered the question- depending on the subjects that answered it. naire, it was evaluated by a small group of users in order to In the next table, the average value for each of dimension verify its understandability. According to the results, some is shown. aesthetic modifications were made so as to obtain a better understanding.
It is necessary to prove the reliability of the survey to state that the results obtained by it are useful.
A survey is reliable if it is made several times by the same subjects and the result obtained is always the same.
Several methods to calculate the reliability of a survey
have been proposed. These methods try to calculate the
reliability coefficient (defined as the correlation among the
marks obtained by the subjects in two parallel forms of a
test). [
We tried to guarantee that the two halves that the test is divided into would be parallel. So, we considered that a half of the test was made up of even questions and the other half was made up of odd questions. This is better than considering that the first part of the test is a half and the second part is the other half because subjects are more tired when answering the second part of the test.
The reliability of the survey does not only depend on the characteristics of the survey but subjects that have answered the questionnaire are important as well
We calculate the reliability with the formula proposed by
Guttman-Flanagan, the equation is [
The dimension with the lowest value is assurance and therefore, it is necessary to improve it by carrying out different tasks that have been proposed to achieve this goal. In general, almost all the dimensions values are around the average. Hence, we have defined several tasks to improve the dimensions that have the lowest values. We can carry out all the tasks and later, make the test again to verify if the quality of the portal has been improved. The dimension that obtains a higher value on average is data quality. As a result, we can conclude that the information provided by the portal turns out to be correct so the portal fulfils its main goal, which is to offer classified information through different channels.
We know that portals have lately risen. Therefore, it is very important to assure its quality. Customers will choose portals that are more appropriate to their needs and objectives and this fact implies that if a portal is not of a good enough quality, it will disappear.
Our aim has been to define a portal quality model and to do so, we have used GQM method. As our model is in the definition phase, we have only used the two first phases. Moreover, this means that the model presented here must be considered as a first proposal but not as a close and definitive model.
Our model, called PQM, is composed of six dimensions: tangibles, reliability, responsiveness, assurance, empathy and data quality.
We must carry out several future works due to the initial state of our quality model. The validation of the model is our first and main objective and to achieve it, we are going to carry out several empirical validations from surveys of customers, experts, etc. to controlled experiment and study cases. We will make all these works with different kinds of portals in order to have the model completely validated. Once our model has been validated, we will study and classify, on the one hand, existing metrics and on the other, metrics of other environments but applicable to portals. If it is necessary, we will define new metrics, that must be formally and empirically validated, and will be used for the last phases of the GQM method. We are working in the preparation of a questionnaire for users of the portal.
This research is part of the PLATIA project supported by the Consejeria de Ciencia y Tecnología of Junta de Comunidades de Castilla-La Mancha (Spain) and the CALIPO project (TIC2003-07804-C05-03) supported by the Ministerio de Ciencia y Tecnología. Coral Calero is PhD in Computer Science. Associate Professor at the Escuela Superior de Informática of the University of Castilla-La Mancha in Ciudad Real (Spain). She is a member of the Alarcos Research Group, in the same University. Her research interests are: advanced databases design, database/datawarehouse quality, web/portal quality, software metrics and empirical software engineering. She is author of articles and papers in national and international conferences on these subjects. Her e-mail is: Coral.Calero@uclm.es Mario Piattini is PhD in Computer Science. Full Professor at the Escuela Superior de Informática of the University of Castilla-La Mancha. Author of several books and papers on databases, software engineering and information systems. He leads the ALARCOS research group of the Department of Computer Science at the University of Castilla-La Mancha, in Ciudad Real, Spain. His research interests are: advanced database design, database quality, software metrics, object oriented metrics, software maintenance. His e-mail is Mario.Piattini@uclm.es