Quality Consideration for e-Learning System Based on ISO/IEC 25000 Quality Standard

Abstract

-e-Learning is defined as the use of information and communication technology (ICT) to support learning. eLearning has been used widely in the world. However, the extensive use of e-Learning requires a proper quality design approach. This paper investigates the use of quality standards and frameworks in the implementation of e-Learning system. Analysis is performed on quality evaluation effort based on standard which has been validated and accepted at International level. The analysis suggests that there is a possibility of an e-Learning project development to be assessed based on criteria from ISO/IEC standard in order to produce quality software product. Initially, the concept of quality and several existing standards for e-Learning system are discussed. In addition, an overview on ISO/IEC 25000 series is presented. The characteristics and sub-characteristics related to ISO/IEC 25000 are described. The research outlined in this paper shows that software product quality is essential in e-Learning project. The research outcome is to determine key quality evaluation criteria based on ISO/IEC 25000 for e-Learning systems. This standard supports building quality into software product.

Keywords

• e-Learning
• software product
• quality
• standard
• framework

-

INTRODUCTION (HEADING 1)

Quality is defined as fitness for purpose and it is important in ensuring the product compliance with its requirements. Among the quality standards and frameworks are ISO/IEC 19796-1, ISO/IEC 25010: 2011, ISO 9126, ISO/IEC 14598-1 and many more. ISO/IEC 19796-1: 2005 has been replaced with ISO/IEC 40180: 2017. ISO/IEC 40180 provides the fundamentals and reference framework for quality management and quality improvement in IT-enhanced learning, education and training (e-Learning). This standard contains Quality Reference Framework (QRF) for e-Learning, which is a common and generic framework to describe, specify and understand critical properties, characteristics and metrics of quality. ISO/IEC 40180 harmonizes existing approaches, concepts, specifications, terms and definitions related to quality for e-Learning, education and training. ISO/IEC 9126 is a standard that evaluate the software product quality, and now has been replaced by ISO/IEC 25000 series. The series is known as System and Software Quality Requirements and Evaluation (SQuaRE) and it provides framework for the evaluation of software product quality. This standard is a result of evolution of many standards such as ISO/IEC 9126 and ISO/IEC 14598-1: 1999.

II.

RELATED RESEARCH WORK

A. Software Product and e-Learning System

Software is a collection of separate computer programs and associated documentation. Software product may be developed for a particular customer (customised or bespoke product) or may be developed for a general market (generic product). Software systems are intangible and abstract.

An e-Learning is the use of Internet technologies to deliver educational solutions that enhance performance and knowledge [8]. The components of e-Learning are comprise of content, content management, content delivery and quality assurance. The content needs to be managed, delivered, deployed and standardised. One type of e-Learning system is the Learning Management System that is mainly used by universities. e-Learning is also known as online learning or web-based learning. e-Learning system is an example of software product.

The e-Learning system is computing solution that combines several tools and ensures education can be channelled to the intended users. The actors involved learner, tutors, course designer (subject matter expert) and administrator. The quality for an e-Learning system can be perceived as involving web technologies; learning content, pedagogy and cognitive (LCPC) approaches; and business. LCPC includes learning, teaching, learning environment and interaction quality aspects.

This section discusses on several standards related to software product and can be used to evaluate e-Learning system.

ISO/IEC 14598-1: 1999, Information Technology – Software Product Evaluation has been revised by ISO/IEC 25040: 2011. ISO/IEC 9126 is an international standard for evaluating the quality of software product [ 3 ]. It defines a quality model for software product evaluation. Software quality characteristics derived from the first part of the standard can be stated as follows: functionality, usability, efficiency, reliability, portability and maintainability. Each quality attributes consists of another level of sub-attributes. For instance, maintainability is a set of characteristics used for effort to be made on modifications. The attributes are stability, analysability, changeability and testability. ISO/IEC 9126 also gives different definition for defect and nonconformity. It defines defect as “the nonfulfilment of intended use of requirements”. Nonconformity is “the nonfulfilment of specified requirements”. ISO/IEC 9126-1 describes a model for software product quality consisting of six characteristics such as functionality, reliability, usability efficiency, maintainability and portability [ 3 ].

ISO/IEC 25010: 2011 emerges to replace ISO/IEC 91261:2001. It establishes a quality model for software quality products and software intensive systems. This standards serves as guidance for formulation of quality requirements and metrics to measure their satisfaction [ 5 ]. This standard describes two quality models which are “quality in use” and “product quality” models. Quality in use model relates the aspects of the system usage by a user in certain context. The five characteristics in this model are: effectiveness, efficiency, satisfaction, freedom from risk and context coverage. Product quality model relates to static properties of the software product and dynamic properties of the computer system. The eight characteristics in this model are: functional suitability, performance, efficiency, compatibility, usability, reliability, security, maintainability, and portability. 1) ISO/IEC 25000 and ISO/IEC 25001 – Quality Management Division reference model and guide. The standards that form this division define all common models, terms and definitions referred further by all other standards from SQuaRE series.

ISO/IEC 25000 - Guide to SQuaRE: Provides the SQuaRE architecture model, terminology, documents overview, intended users and associated parts of the series as well as reference models.

ISO/IEC 25001 - Planning and Management: Provides requirements and guidance for a supporting function which is responsible for the management of software product requirements specification and evaluation.

2) ISO/IEC 25010 - Quality Model Division reference model and guide. It consists of details of quality models for computer systems and software products, quality in use, and data.

 ISO/IEC 25010 - System and Software Quality Models: Describes the model, consisting of characteristics and sub-characteristics, for software product quality, and software quality in use.  ISO/IEC 25012 - Data Quality Model: defines a general data quality model for data retained in a structured format within a computer system. It focuses on the quality of the data as part of a computer system and defines quality characteristics for target data used by humans and systems.

3) ISO/IEC 25020 - Quality Measurement Division Measurement reference model and guide. It includes a software product quality measurement reference model, mathematical definitions of quality measures, and practical guidance for their application. Presented measures apply to software product quality and quality in use. Presents introductory explanation and a reference model that is common to quality measure elements, measures of software product quality and quality in use. It also provides guidance to users for selecting or developing, and applying measures.    

ISO/IEC 25021 - Quality Measure Elements: Defines a set of recommended base and derived measures, which are intended to be used during the whole software development life cycle. The document describes a set of measures that can be used as an input for the software product quality or software quality in use measurement.

ISO/IEC 25022 - Measurement of Quality in Use: Describes a set of measures and provides guidance for measuring quality in use.

ISO/IEC 25023 - Measurement of System and Software Product Quality: Describes a set of measures and provides guidance for measuring system and software product quality.

ISO/IEC 25024 - Measurement of Data Quality: Defines quality measures for quantitatively measuring data quality in terms of characteristics defined in ISO/IEC 25012.

5) ISO/IEC 25040 - Quality Evaluation Division Measurement reference model and guide. The standards that form this division provide requirements, recommendations and guidelines for software product evaluation.

4) ISO/IEC 25030 - Quality Requirements Division Measurement reference model and guide. It helps specifying quality requirements. These quality requirements can be used in the process of quality requirements elicitation for a software product to be developed or as input for an evaluation process.

ISO/IEC 25030 - Quality Requirements: Provides requirements and guidance for the process used to develop quality requirements, as well as requirements and recommendations for quality requirements.

ISO/IEC 25040 - Evaluation Reference Model and Guide: Contains general requirements for specification and evaluation of software quality. Provides a framework for evaluating quality of software product and states the requirements for methods of software product measurement and evaluation. It replaces ISO/IEC 14598-1: 1999.

ISO/IEC 25041 - Evaluation Guide for Developers, acquirers and independent evaluators: Provides requirements, recommendations and guidelines for developers, acquirers and independent evaluators of the system and software product.

ISO/IEC 25042 - Evaluation Modules: Defines the structure and content of the documentation to be used to describe an evaluation module. These evaluation modules contain the specification of the quality model (i.e. characteristics, subcharacteristics and corresponding product or quality in use measures), the associated data and information about its application.

ISO/IEC 25045 - Evaluation Module for Recoverability: Provides the specification to evaluate the sub-characteristic of recoverability defined under the characteristic of reliability of the quality model.

SQuaRE extension (ISO/IEC 25050 to ISO/IEC 25099) contains system or software product quality International Standards and/or Technical Reports that address specific application domains or that can be used to complement one or more SQuaRE International Standards.

C. Quality Standard and Framework of an e-Learning

System

There are various studies that have been conducted in evaluating current quality standard and framework in eLearning systems. These existing quality models have been discussed and compared [6, 7]. Chua and Dyson (2004) proposed ISO/IEC 9126 as the standard to evaluate the eLearning systems for educational administrators. Their study aims to support the academics at evaluating e-Learning systems by including frameworks to support decision making process by reviewing of existing systems and purchase of new system.

ISO/IEC 19796-1: 2005 is a quality management standard developed for learning, education and training in educational processes, activities and services [ 4 ]. It covers the needs analysis, design, development, realization and evaluation of any learning process including e-Learning. ISO/IEC 19796-1: 2005 Information Technology – Learning, Education and Training - Quality Management, Assurance and Metrics – Part 1 : General Approach defines the process model defining basic processes when managing quality in the field of ICTsupported learning, education and training.

e-Learning framework includes the Sharable Content Object Reference Model (SCORM) that supports content compatibility and reusability of learning objects [ 10 ]. Instructional Management Systems (IMS) defines technical specifications to ensure interoperability among e-Learning systems [ 2 ].

III. RESEARCH APPROACH

The research method focuses on the theoretical literature by critically reviewing the existing quality standard and framework. Minor mapping among the quality standards and frameworks is performed. The ISO/IEC 19796-1: 2005 focuses on the processes of e-learning systems therefore, a more suitable quality model selected for this paper is ISO/IEC 25000. Framework mapping produces the analysis among the characteristics, sub-characteristic and the interpretation. The characteristics in ISO/IEC 25010 quality model and the interpretation is given at the third column in italic text.

Further descriptions on each characteristics are elaborated in the next section.

IV. RESULT AND ANALYSIS

It is determined from recent research that ISO/IEC 9126 has weaknesses [ 9 ], and it is overcome with the new model ISO/IEC 25000. Secondly, the results from the comparison made in this paper on the existing quality models are used to determine the quality criteria to be assessed based on ISO/IEC 25000. This new ISO/IEC 25000 model has included the extension of the usability criteria, which can be considered for inclusion in e-Learning systems evaluation. Thirdly, Chua and Dyson (2004) in their research proposed ISO/IEC 9126 to assess e-Learning system. Thus, due to ISO/IEC 9126 is now replaced with ISO/IEC 25000, this research proposes ISO/IEC 25000 as a quality management tool to evaluate e-Learning systems. ISO/IEC 25000 provides a holistic view of software product evaluation.

One of the framework mapping result is determination of the most suitable quality measurement and component for use in evaluating the quality of e-Learning system from the aspect of usability. The Usability Characteristic is adopted from this standard, together with the sub-characteristics: Appropriateness Recognisability, Learnability, Operability, User Error Protection, User Interface Aesthetics and Accessibility. SQuaRE usability characteristic is defined as: the degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [ 5 ]. This characteristic is comprised of the following subcharacteristics:      

Appropriateness Recognisability – The degree to which users can recognise whether a product or system is appropriate for their needs.

Learnability – The degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use.

Operability – The degree to which a product or system has attributes that make it easy to operate and control. User Error Protection – The degree to which a system protects users against making errors.

User Interface Aesthetics – The degree to which a user interface enables pleasing and satisfying interaction for the user. It refers to properties of the e-Learning product or system that increase the satisfaction of the user (typography, visual design, multimedia usage through the use of color and nature of graphic design). Accessibility – The degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use. It refers to users’ ability requirements and also the ability to provide access for remote locations.

Systematic Usability Evaluation (SUE) Methodology defines general framework of usability evaluation [ 11 ]. This research indicates the importance of usability as an important criteria for assessing e-Learning system usability. In SUE Methodology, four dimensions are discussed: presentation, hypermediality, application proactivity and user activity. This strengthen the choice of usability characteristic as an important measures to be the first evaluated for this research. Other characteristics, sub-characteristics and criteria that are proposed to be included are described below. The description of interpretation in Table I are further described in the following paragraphs for the remaining seven characteristics. Characteristic 1: Functional Completeness

Functional Completeness – The degree to which the set of functions covers all the specified tasks and user objectives [ 5 ]. The elements include: forum, synchronous communication (chat), virtual classroom (webinar sessions), document sharing, schedule/calendar, awareness (online connectivity of users), assessment management (quiz, test, assignment, exam), collaboration (wiki, trello), learners ‘management (timeline, schedule, registration), customisation of platform, course percentage progression (task bar), management of resources.

Functional Correctness – The degree to which a product or system provides the correct results with the needed degree of precision [ 5 ]. The elements include: management of working time, assessment results and notes/learning materials, display of notes, course analytics/tracking statistics, assessment results report/ report book, glossary, frequency report/course usage. Functional Appropriateness – The degree to which the functions facilitate the accomplishment of specified tasks and objectives [ 5 ]. This sub-characteristics relates to suitability for task. The elements include: credential/certificate, messaging, plagiarism detecting tools (TurnItIn Software), file distribution (text, image, pdf, audio, video, etc)).

Time Behaviour – The degree to which the response and processing times and throughput rates of a product or system, when performing its functions, meet requirements [ 5 ]. The elements include: response time for a certain function, etc.

Resource Utilization – The degree to which the amounts and types of resources used by a product or system, when performing its functions, meet requirements [ 5 ]. The elements include: whether the platform meet the online distance learning requirements in terms of tools functionalities.

Capacity – The degree to which the maximum limits of a product or system parameter meet requirements [ 5 ]. The elements include: whether the platform meet the online distance learning requirements and assist quality, number of stored items, concurrent users, communication bandwidth, transactions throughput, and database size.

Co-existence – The degree to which a product can perform its required functions efficiently while sharing a common environment and resources with other products, without detrimental impact on another product [ 5 ]. The elements include: transfer of course from one platform to another, SCORM (allows creating unit size number of courses, reusable and interoperable), set of specifications allowing interoperability to test system), sets interoperability between platform and content.

Interoperability – The degree to which two or more systems, products or components can exchange

Characteristic 2: Performance Efficiency

Characteristic 3: Compatibility information and use the information that has been exchanged [ 5 ]. This sub-characteristics allows different features interact with each other. The elements include: sharing of information and application, resources import and export, plugins usage, and technology usage (PHP, ASP, etc). performed to determine whether those criteria have been met [ 5 ]. The elements include: system testing, maintenance, and updating.

Characteristic 6: Portability

Characteristic 5: Maintainability          

Confidentiality – The degree to which a product or system ensures that data are accessible only to those authorised to have access [ 5 ]. The elements include: security features using login/password; and assignment and allocation of access tight.

Integrity – The degree to which a system, product or component prevents unauthorised access to, or modification of, computer programs or data [ 5 ]. This sib-characteristic is related to protection from unauthorised access. The element include: SSL usage.

Non-repudiation – The degree to which actions or events can be proven to have taken place, so that the events or actions cannot be repudiated later [ 5 ]. The elements include: system backup and system tracking.

Accountability – The degree to which the actions of an entity can be traced uniquely to the entity [ 5 ]. The element include: security block.

Authenticity – The degree to which the identity of a subject or resource [ 5 ]. The elements include: Username and password assignment; and identity card/profile.

Modularity - The degree to which a system or computer program is composed of discrete components such that a change to one component has minimal impact on other components [ 5 ]. The element include: does platform have disconnected plugins.

Reusability - The degree to which an asset can be used in more than one system, or in building other assets [ 5 ]. The elements include: interactions with other operating system and other databases. Analysability - Degree of effectiveness and efficiency with which it is possible to assess the impact on a product or system of an intended change to one or more of its parts, or to diagnose a product for deficiencies or causes of failures, or to identify parts to be modified [ 5 ]. This sub-characteristic relates to the ability to identify the root cause of a failure within software. The elements include: selffault analysis mechanisms for product or system, provide reports before or after a failure/event. Modifiability - The degree to which a product or system can be effectively and efficiently modified without introducing defects or degrading existing product quality [ 5 ]. The element include: system modification without errors.

Testability - The degree of effectiveness and efficiency with which test criteria can be established for a system, product or component and tests can be

Adaptability – The degree to which a product or system can effectively and efficiently be adapted for different or evolving hardware, software or other operational or usage environments [ 5 ]. The elements include: adaptation to mobile platform, browsers compatibility, adaptation different training platform (developing MOOC, Micro Degrees, Micro Credential, etc).

Installability – The degree of effectiveness and efficiency with which a product or system can be successfully installed and/or uninstalled in a specified environment [ 5 ]. The elements include: error free platform installation.

Replaceability – The degree to which a product can replace another specified software product for the same purpose in the same environment [ 5 ]. The elements include: new platform version replaceability or upgrade to new version.

Maturity – The degree to which a system, product or component meets needs for reliability under normal operation [ 5 ]. This sub-characteristic relates to eLearning system capability to avoid failure as a result of defects in the software.

Availability – The degree to which a system, product or component is operational and accessible when required for use [ 5 ]. It refers to what extent the learning content is available for learners whenever the system is needed.

Fault tolerance – The degree to which a system, product or component operates as intended despite the presence of hardware or software faults [ 5 ]. Recoverability – The degree to which, in the event of an interruption or a failure, a product or system can recover the data directly affected and reestablish the desired state of the system [ 5 ].

Characteristic 7: Reliability

V. CONCLUSION

This paper has reviewed recent studies in quality and frameworks for software product quality and e-Learning processes and systems. The quality criteria from these models can be leveraged as a guidance to produce quality software product. Relevant quality criteria from ISO/IEC 25000 have been identified to assess e-Learning systems as basis for quality evaluation. Future works may consider to organize quality characteristics based on e-Learning systems actor and categories such as web technologies, pedagogical approaches and business aspects.

ACKNOWLEDGMENT The author wish to thank Asia e University (AeU) for the research support. REFERENCES

References

[1] B. Bee Chua and L. Evelyn Dyson , Applying the ISO 9126 Model to the evaluation of an e-Learning,” In Proc. 21st ASCILITE Conference , 2004 , pp. 184 - 190 .

[2] IMS, ' IMS Content Packaging Information Model - Version 1.1 .4 Final Specification', 2004 . [Online]. Available http://www.imsglobal.org/content/packaging. [ Accessed: September 27 , 2019 ]

[3] ISO , ISO/IEC 9126: 2001 , Information Technology - Software Product Quality - Part 1: Quality Model . London, UK, 2001 .

[4] ISO , ISO/IEC 2005: 19796 - 1 : Information Technology - Learning, Education and Training - Quality Management , Assurance and Metrics - Part 1: General Approach . Geneva, Switzerland, 2005 .

[5] ISO , ISO/IEC 25010: 2011 , Systems and Software Engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and Software Quality Models . London, UK, 2011 .

M. Abdullah & A. A. Aziz Ali , E-Learning Standards , in Communication, Management and Information Technology , London: Taylors & Francis Group, 2017 , pp. 639 - 645 .

M. Bari & R. Djouab, “ Quality Frameworks and Standards in ELearning Systems ,” International Journal of Computer , the Internet and Management, Technology, vol. 22 , no. 33 , pp. 1 - 7 .

M. Jeffrey Rosenberg , E-Learning: Strategies for Delivering Knowledge in the Digital Age . New York: McGraw-Hill , 2001 .

[9] R. E. Al-Qutaish , An investigation of the weaknesses of the ISO 9126 International Standard,” In Proc. 2nd International Conference on Computer and Electrical Engineering , 2009 , pp. 275 - 279 .

[10] SCORM, 'Shareable Content Object Reference Model', 2004 . [Online]. Available http://www.scormsoft.com/scorm. [ Accessed: Sept. 27 , 2019 ]

[11] C. Ardito , M. De Marsico , R. Lanzilotti , S. Levialdi , T. Roselli , V. Rossano , and M. Tersigni , Üsability of E-Learning Tools ,” In Proc. AVI , 2004 , pp. 80 - 84 .