- This paper examines concepts in the Quality in Use model proposed by SQuaRE standards. A more detailed differentiation is made about the results of the interaction with the system in a production environment and propositions are made for future revisions of the model in the standard. negative outcomes 4. Avoidance (consequences)
Keywords— SQuaRE, Quality in Use, Quality Models, Quality in Production Environments, ISO/IEC 25023, ISO/IEC 25022.
I.
INTRODUCTION
In many cases, Quality in Use model is highly, or only, associated with usability. This vision is at least partial.
This paper proposes re-examining one of the more important SQuaRE quality models, the Quality in Use model.
Basic concepts are reviewed, and several suggestions are offered to be considered on future revisions of the model.
When a system runs in a production environment, considerations about its quality could be based at least on:
Usability of the system (if a human user exists)
Fulfilling of expected positive outcomes (results) possible
If these conditions are met within acceptable values the system will be considered as a quality (enough) system.
Usually people describe desired solutions as required outcomes and an unrefined request about how these outcomes are obtained.
These elements, at a very early stage, are the first outline to the Quality in Use model for the system because they contain quality concepts expected and valuable for the customer.
This Quality in Use model will then depict the behaviour of the system in a production environment compared with target behaviour established at modelling stage.
It should be noted this paper is focused on the final user
but that, as stated by [
The ISO/IEC 25010:2011 defines Quality in Use as the degree to which a product or system can be used by specific users to meet their needs to achieve specific goals with effectiveness, efficiency, and satisfaction and freedom from risk in specific contexts of use.
Quality is modelled through five characteristics related to outcomes of interaction with a system: effectiveness, efficiency, satisfaction, freedom from risk and context coverage.
For Quality in Use model the term usability refers to the subset of quality in use composed of effectiveness, efficiency, satisfaction, and context of use coverage.
Figure 1 is a simplified graphical explanation of Quality in Use model and its relationship with a product in Usability
Outputs Things produced (virtual or real) operation.
Figure 1 Graphical explanation of quality in use model
System in a production environment sequence:
a. System started, with or without human interaction 1 Product execution: 2. Resultant outcomes: a. Outputs produced b. Usability attained c. Results achieved d. Consequences derived 3. System running in a specific context of use Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
III. REVIEWING COMPONENTS OF THE QUALITY IN USE
MODEL
Different measures are provided by the model for usability (effectiveness, efficiency, and satisfaction that includes user experience and ergonomics). All these measures apply if a human user, primary, secondary or indirect exists.
The model also considers consequences and results, explained later. Figure 2 shows the measures proposed for the production environment.
(*) Output is outside Quality in Use model proposition but is generated as an outcome during execution on a production environment; for that reason, measures from ISO/IEC 25023:2016, as functional suitability and appropriateness, and measures from ISO/IEC 25024:2015 for Data Quality could be used to measure quality of Output produced
Outcomes are goals, projected or attained, that are the main reason of the existence of the system.
Some clarification is required to distinguish between the different outcomes: outputs, results and consequences: • •
Outputs: Things, virtual or real, produced or changed as effect of the execution of the system. As mentioned above Output is outside Quality in Use model proposition but is generated as an outcome during execution on a production environment, for that reason quality measures from ISO/IEC 25023:2016 could be used.
Results: Short-term or long-term effects obtained from the use of the system.
fulfilled and are positive. The quality measures could compare the target value expected for each result and the actual value obtained during executions, one or many. •
Consequences: ISO/IEC 25022:2016 defines consequence as an outcome emerging from a negative risk (not an opportunity).
Enumerated risks include threats to economic status, human life, health, or the environment. The quality measures could compare the target value expected for each risk or consequence and the actual value obtained during executions.
Definitions for Outputs, Results, Consequences and
Outcomes concluded or obtained from [
Many existing systems operate exchanging activities with other systems, and subsequently all of them share responsibility for the results.
Organizations or individuals using the system will perceive quality without an exact discrimination of quality of each intervenient component.
Should Quality in Use model offer properties and measures for this kind of systems of systems?
Today we are developing systems that “see”, “hear” and “know” where they are located, making decisions and executing actions in actual environments based on incoming data and, for example, previously learned behaviours.
Will there be a quality measure for bias of the decisions of the algorithms? Which will be the target value? Will there be a quality measure for ethics of the solution? Which will be the target value?
Will it be acceptable to call these “interactions”?
ISO/IEC 25022:2016 defines context of use as users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a system, product or service is used. [SOURCE: ISO 9241‑11:1998, 3.5, modified — “product” replaced by “system, product or service”.]
The proposed quality property associated to context is Coverage. Completeness and Flexibility are the measures proposed for this property.
Such properties and measures are oriented towards changes in user skills or capabilities, type of users, and several target scenarios. These is correct for some types of systems where contexts can be enumerated.
E. Points to consider on context of use - Is Coverage as proposed by the Quality in Use model applicable to highly variable / unknow contexts? - How is quality measured in these cases? How are completeness and flexibility measured? - Context changes could require user actions or produce diverse outcomes. How could we measure the quality of the produced response to the new context? Quality in Use model is a very important model whose elements, as mentioned, are identified at the very beginning of the development process and completed throughout the product life cycle.
The paper rises several questions about meaning and content of the current Quality in Use model. These questions will require a deep discussion and opinions exchange, but some key ideas are briefed below.
• The model should be detached from the usability centred idea and the value of the model should be emphasised. Possibly a new name for the model should be considered showing the relationship between the model and the operation of the system in diverse and mutating production environments. A suggestion could be Production Quality Model. • ISO/IEC 25022:2016 establishes that the external measures can only be used during “the testing stages of the life cycle process and during any operational stages”. Included in this definition are environments identical to production used for testing process. For that reason, external measures that apply to production environment should be moved to Quality in Use model obtaining a complete picture of system execution. • Current and emerging systems should be considered by the Quality in Use model, as new forms of input, algorithms, processing and communications are constantly evolving and new ones appearing. Nevertheless, the correct level of abstraction should be maintained, and the new characteristics applicable to these kinds of systems must be considered. These characteristics could be allocated on a Technical Report suitable to the specific kind of ICT product without compromising the level of abstraction of the general model. • Measures applicable to Usability, Results and Consequences need to be analysed and/or defined in accordance to the kind of ICT product. • Current Service Quality model should be re-examined considering at least two possibilities. First, if it really is a different model or it is part of the Quality in Use model. Second, consider the Service Quality model as an specialization of the Quality in Use model. • The fundamental concept of Context should be reexamined. Current Quality in Use model includes two basic measures for context, Completeness and Flexibility. The measures embrace important ideas but with completely different meaning depending on the kind of systems. For this reason, context measures should be defined using for example a Technical Report mechanism as stated previously.
As a final thought, Quality in Use model appears at the very beginning as quality expectations about the future product and at the end as outcomes: outputs, results and consequences, of the use of the actual product in Production environments.
Between expectations and production, other models need to be developed, Product Quality Model and Data Quality Model, stating the quality characteristics required on the product. Fulfilment of first expectations and actual behaviour of product in Production conform the user perceived quality.