-This paper presents the evaluation metrics for each SW test item for quantifying building energy management system quality. As the building energy management system market grows, many products are being instituted, but SW quality verification and validation are not conducted during installation or operation of the building energy management system. To solve these problems, in this paper, requirements were extracted by reflecting generalized software evaluation criteria, which are domestic and international standards. The standard-applied requirements are extracted, and the test cases are made through the extracted requirements. Based on this, we propose metrics to quantify the evaluation results. Using this metric, it is expected that the metric can be used for SW verification and validation activities required for building energy management system.
I. INTRODUCTION
Abnormal high temperatures and fossil fuel depletion are emerging as issues around the world. Accordingly, the paradigm of energy production and consumption is changing. The energy prosumer market is expanding through ICT-based technology, and the transition to eco-friendly energy such as wind power and solar power is accelerating. Moreover, as there is an international consensus on the need for greenhouse gas reduction, it is urgent to find countermeasures and strategies.
According to International Energy Agency’s 2018 Global
Status Report, building construction and operation accounted
for 36% of global final energy use and 39% of energy-related
carbon dioxide (CO2) emissions in 2017, as shown in Fig 1 [
Each country is establishing standards for energy saving and strengthening related policies. Accordingly, the building energy management system market is expanding, and the introduction of evaluation criterion to improve the reliability of building energy management system is emerging as an important task.
In this study, functional requirements are derived based on 'KS F 1800-1:2014' and 'KS F 1800-2:2021', which are Korean domestic standards reflecting international standards 'ISO 16484-1:2010 – Part 2: Hardware' and 'ISO 164843:2005 – Part 3: Functions'. Non-functional requirements apply 'ISO/IEC 25023' and 'ISO/IEC 25051', international standards related to SW quality used for generalized SW quality evaluation. Through this, we propose an evaluation criteria regardless of the type of building.
Requirements are extracted. The extracted requirements are used to select test cases. After this, we derive metrics from the selected Test case. To finish with, the metric is applied to a commercial building energy management systems to check the applicability of the metric.
Functional testing should be performed accord with installation purpose. In addition, tests suitable for the actual environment should be conducted. In order to perform a discriminatory evaluation, planned and systematic quality activities based on international standards must be carried out.
Section Ⅱ Method introduces the process of creating metrics for performing quality activities, focusing on the requirements presented in the standards.
ISO/IEC 25000 series, also known as SQuaRE (System
and Software Quality Requirements and Evaluation) series, it
is a set of standards covering quality model, quality measures
and quality evaluation of system, software, data and service
products. ISO/IEC 25023 provides a set of quality measures
to measure the quality of systems and software products[
No.
functions Data collection and display B. KS F 1800-1:2014
Building Energy Management System – Part 1 : Function
and data processing procedure provides nine building energy
management system functional requirements as TABLE I [
Building Energy Management System – Part 2 : Selection of data points, management and determination of energy savings provides six building energy management system function requirements needed to calculate energy saving as TABLE II [5].
For other quality characteristics, the requirements applicable to the building energy management system among the generalized software evaluation criteria were extracted, and the test cases were selected by reflecting the extracted requirements.
A. Extract functional and performance requirements
In the data collection and display function, 5 functional requirements and 2 performance requirements. In the case of FR-001-003, individual meter reading should be preceded. Some of the extracted requirements are as follows TABLE Ⅲ.
The performance requirements were based on performance efficiency, which is the characteristic of ISO/IEC 25023 SW Quality model, and were composed based on time behavior and resource utilization, which are sub- characteristics.
It was configured as a requirement to check whether BEMS properly utilizes system resources by measuring CPU, memory and response time while executing the designated functions according to functional requirements.
In monitoring function, 4 functional requirements and 6 performance requirements were extracted, and some of them are introduced to the TABLE Ⅳ.
First, based on the nine requirements of KSF 1800-1: 2014, functional and performance requirements were extracted, and the test cases were selected by reflecting the extracted requirements. Functional suitability and performance efficiency evaluation metrics were defined based on reference allocation and the pass rate of the selected test case. Resources mentioned in the performance requirements refer to CPU and memory.
ID FR-002-001 PR-002-002
ID FR-003-002 PR-003-003
In Analysis of energy consumption function, 5 functional requirements and 6 performance requirements like TABLE Ⅵ were extracted. In the case of FR-004-005, a baseline period should be set according to the calculation purpose. The baseline period can be set to 1 month, summer/winter, et cetera.
In data display function, 3 functional requirements and 4 performance requirements like TABLE Ⅴ were extracted. b. examples used in schools: the number of events, the number of teachers and students, home time
In analysis of facility performance and efficiency function, 4 functional requirements and 4 performance requirements like TABLE Ⅶ were extracted. In particular, in order to calculate the performance presented in FR-005-001 and FR005-002, the electric power quantity measurement function of the facility is required.
In provide indoor and outdoor environment conditions function, 2 functional requirements and 2 performance requirements like TABLE Ⅷ were extracted. In Prediction of energy consumption function, 3 functional requirements and 4 performance requirements like TABLE Ⅸ were extracted.
In display and analysis of energy costs function, 5 functional requirements and 10 performance requirements like TABLE Ⅹ were extracted.
ID FR-008-005 PR-008-010
In interworking with the control system function, 4 functional requirements and 6 performance requirements like TABLE XI were extracted. These functional requirements are essential for interworking with the control facilities. In addition, in FR-009-004, the Human-Machine Interface monitoring and control screen may be required to apply the control plan to the facility.
In section B, we derive test cases based on the requirements for each quality characteristic extracted from section A.
Functional suitability and performance efficiency test cases were extracted by functional and performance requirements in Section A, while other compatibility, usability, reliability, security, maintainability, portability, general requirements reflected 'ISO/IEC 25023' and 'ISO/IEC 25051' characteristics. Introduce some of the derived test cases by quality characteristics (TABLE XII to Table XIX). test result (P/F) installation time.
The test result was applied to the actual BEMS product, and it passed overall, but the resolution was not reflected after changing the environment setting, resulting in failure at Ma007-004.
The evaluation metric defined the test cases based on the functional and performance requirements of KS F 1800 series. The functional suitability quality score is calculated by multiplying the degree of function requirement achievement by the reference score. The performance efficiency quality score is calculated by multiplying the degree of performance requirement achievement by the reference score.
The degree of requirement achievement is calculated as the ratio of the total number of requirements and the number of test results pass. The evaluation grade is calculated by multiplying the degree of requirement achievement by allotment, as in: ( a b ) c= rn ()
Finally, the quality score for the functional suitability, which is the quality characteristics is calculated by summing all the calculated evaluation grades (TABLE XXIX). excluding functional suitability and performance efficiency is based on the requirements of the ISO/IEC 25000 series. The quality score is calculated by multiplying the degree of function requirement achievement.
Among the derived metrics, the compatibility quality evaluation metrics are shown in Table XXX.
In this paper, in order to perform a discerning evaluation, a metric was derived based on the requirements with the highest weight in Korean domestic standards. The test was conducted using metric for a commercial building energy management system, and although it was a product that already had a supplier, it was evaluated with a fairly low score (TABLE XXII). Maintainability
Portability = (100+0) / 2
Among them, the evaluation results of the functional suitability quality characteristic are described in detail as follow. a: The number of items from FR-005-001 to FR-005-004 whose performance result is Pass. b: The total number (FR-005-001 to FR-005-004). score evaluation
grade
Total score = ∑ =
In the case of Korean domestic authentication, there is no
problem in obtaining authentication even if only the lowest
weight is satisfied [6]. For this reason, it is believed that the cause
of the low evaluation score is that most of the development
companies are developing products with the lowest weight [
In order to proceed with the test, it seems that the weight of the score criterion needs to be modified according to the actual development and delivery market situation. Afterwards, when deriving the requirements on which the metric is based, the weight is changed for each requirement, and the metric expansion to the BAS field is improved.