Analysis for Blockchain Application Quality Jiajin Xu Huan Zhang Jiayu Gong Shanghai Key Laboratory of Computer Shanghai Key Laboratory of Computer Shanghai Key Laboratory of Computer Software Evaluation. Shanghai Software Evaluation. Shanghai Software Evaluation. Shanghai Computer Software Technology Computer Software Technology Computer Software Technology Development Center Development Center Development Center Shanghai, China Shanghai, China Shanghai, China xujj@sscenter.sh.cn zhangh@sscenter.sh.cn gjy@sscenter.sh.cn Abstract—With the maturity and popularity of blockchain technology, it has applied to many industries, such as financial System/Software Product Quality sector, logistics, notarization and so on, but there is no correct quality model to describe the brockchain software. This paper we combined with the general software quality Functional Performance Maintain- model(SQuaRE), formed a suitable quality model of the Suitability Efficiency Compatibility Usability Reliability Security ability Portability blockchain platform (Hyperledger Fabric) and expound its special quality measure. The research result of this paper can be applied to test or evaluate of blockchain application. Functional Time-behaviour Co-exstence Appropriateness Maturity Confidentiality Modularity Adaptability completeness Recognisability Integrity Resource Interoperability Availability Reusability Installability Keywords—Hyperledger Fabric, blockchain, software quality, Functional utilisation Learnability Non-repudiation correctness Operability Fault tolerance Analysability Replace ability software quality model Capacity Accountability Functional User error Recoverability Modifiability appropriateness protection Authenticity I. INTRODUCTION Testability User interface aesthetic Recently blockchains attract the interests of stakeholders across a wide span of industries: finance ，healthcare， real Accessibility estate, and government, because of its decentralization, non- tampering, traceability, collective maintenance, openness and Fig. 1. System/Software Product Quality Model—ISO/IEC 25010:2011 transparency. As a new infrastructure, blockchains application testing is quite different from the traditional ways. Based on the software product quality model, ISO/IEC General software quality models may not quite compatible 25023:2016 “Systems and software engineering - Systems with the blockchain applications since their different features and software Quality Requirements and Evaluation (SQuaRE) from other software. This paper introduces general software - Measurement of system and software product quality” model ISO/IEC 25010[2] and describes blockchain provides measures for quality characteristics. But ISO/IEC application’s features and its special quality measures. They 25023:2016 cannot cover the quality measurement for can be used as a supplement to ISO/IEC 25023:2016[1]. blockchain application, so more quality testing measures should be added. III. HYPERLEDGER FABRIC BLOCKCHAIN SOFTWARE II. GENERAL SOFTWARE QUALITY MODEL FEATURES Due to the different requirements, the stakeholders may In the blockchain application, Hyperledger is an excellent have different quality perspectives. For users, software implementation of the union chain, and Hyperledger Fabric quality is to meet the needs of users. For developers, is the kernel of Hyperledger project. software quality is consistent with requirements. In order to give a reasonable evaluation of the quality of software Hyperledger Fabric is essentially a distributed ledger, products, a software quality model is necessary using a high availability modularization as its design concept. The modules, such as MSP, Ordering Service, Ordering Last decade, ISO/IEC JTC1 SC7 has designed a new Service, Ordering Service are deployed separately. This way generation of software quality standards, namely ISO/IEC can configure the requirement flexibly. The key technologies 25000 SQuaRE or Software Product Quality Requirements of Hyperledger Fabric as follow. and Evaluation. ISO/IEC 25010 software product quality model is a general model for judging software’s quality, which includes eight quality characteristics, functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability and portability[2]. For each characteristic, it contains several sub characteristics. Fig.1 shows the details of System and software product quality model. This work was supported by National Key R&D Program of China (No.2018YFB1403404). Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Chain Chain Chain Coder Coder Org2 Chain Coder Org3 Order1 Org1 Peer0 Chain Peer0 Coder Org2 Peer1 Coder Org3 Chain Ledgers Ledgers Peer1 Ledgers Peer1 Coder Org1 Ledgers Peer0 Ledgers Order2 Order3 RAFT Ledgers chaincode Org1 Channel1:Org1、Org2、Org3 Query user Query image history information information Org1 CA Query image Channel2:Org1、Org2 details Org2 Org3 Up chain by Transfer by Org2 Org3 image image CA CA Channel3:Org1、Org3 Blockchain-network Fig. 2. Structure Chart A、 Ledger[3] then there are three channels (Channel1, Channel2 and A Fabric Ledger is a series of orderly and immutable Channel3) in the network, each channel contains different records which states has been transformed. Its inside organization, the organizations(Org1, Org2, Org3) have structure is been comprised of a block which stores the joined Chanel1, the organizations(Org1, Org2) have joined immutable, ordered records, and simultaneously contains a Chanel, the organizations(Org1, Org3) have joined Chanel3. state database to record the current state. The functions of chaincode are up chain by image, transfer by image, query user information, query history of image B、 Blockchain[3] and query detail of image. The Block chain is based on the local file system, and its IV. HYPERLEDGER FABRIC SOFTWARE QUALITY MODEL blocks are put in local hard disk which store the file system. The previous block hash is pointed to the current block hash, Based on the above characteristics of Hyperledger Fabric, so they can connect with a block chain. we tested with the application scenario of Hyperledger Fabric, and established a software quality requirement C、 Peer[3] system which is suitable for HyperLedger Fabric as the fig3. Peer is the main body to do the operation of trade It also contains eight quality characteristics which is the processing and maintain ledger, which is mainly responsible same as ISO/IEC 25010-2011. But many quality for the consensus process, and does ledger’s reading and characteristics have different sub characteristics, such as writing operation through chaincode. Peers can be divided functional suitability, performance efficiency; and many sub into Endorser Peer and Committer Peer, according to characteristics have different test measures, such as different functions. compatibility , reliability and security, the details as fig3. D、 Channel[3] Hyperledger Fabric Blockchain System Product Quality in A channel is a private space which is used to communicate information between two or multiple peers. Functional Performance Compatibility Usability Reliability Security Maintain-ability Portability The transaction data in the channel is separated from the data Suitability Efficiency outside the channel, so we can ensure that the data in the Time-behaviour Appropriateness Maturity Function of user Co-existence Confidentiality channel is safe enough. Transactions on the network must be application Resource Recognisability Availability Analysability Adaptability Interoperability Non-repudiation executed on the Channel. Each member of the transaction Function of Utilisation Learnability operability Fault tolerance distributed Authenticity needs authorization and certification, otherwise they cannot application Capacity User error Recoverability be allowed to trade on the Channel. Function of Transaction protection management time User interface E、 Orderer[3] application aesthetic Accessibility Orderer is sorting the transactions which is generated by different channels in the blockchain network, and at the same Fig. 3. Hyperledger Fabric Blockchain System Product Quality time broadcasting to the peer. Orderer is implemented as a pluggable component and has two kinds of algorithms A． Functional suitability (SOLO and Kafka). In ISO/IEC 25023-2016[1], there are two sub The fig2 is the structure chart of software which use the characteristics, functional completeness, functional technology of Hyperledger Fabric, we can know the correctness. But for Hyperledger Fabric, we add some new composition of this blockchain network clearly. In this sub characteristics to replace the normals’. The new sub blockchain network, there are three organizations(Org1, characteristics can be defined as function of user application Org2, Org3), and each organization maintains 2 peers(peer0, function, distributed application function and function of peer1) at the same time. The consensus algorithm is Raft, it management application [5], the detail definition as follow; consists of three orders (Order1 、Order2 、 Order3). And 1) Functiong of user application: It is the interaction information to application system that will automatically fill between the blockchain customer and the the encrypt attribute to the image. Then, the application will blockchain service, including two user operation send the basic information on image and encrypted mode, the command line interface and image information to block chain system after the user presses the interface. on chain button. The detailed interactions of this blockchain 2) Function of distributed application: It is the system as the fig4, and the interface specification of this distributed apply function of blockchain, including blockchain as the table1. selection or cancellation of blockchain service, Query detail of image ordering and unordering of blockchain services and X the business operation of ledger. Query history of image 3) Function of management application: It is the Up chain by image Image store management application function of blockchain, including manage by member and manage by Transfer by image Application system File system Blockchain sytem monitor. Fig. 4. Interaction of System B． Performance efficiency In ISO/IEC 25023-2016, there are three sub TABLE I. INTERFACE SPECIFICATION characteristics, time-behaviour, resource utilisation and Return Interfac Request Response capacity. But for Hyperledger Fabric, except these normal e Name System System Interface Specification ed sub characteristics, we add a additional indicator which Result called transaction time. It can be defined as time of sorting Method: EnrollAsset(ctx， the consensus algorithm, time of completing the block assetID， broadcast for transaction, time of completing verification and Application assetName， Up transact submission for transaction, and time of completing File System, assetUrlHash， ion submission for transaction, the detail definition as follow; chain by System Blockchain ownerID， metadata) result image system Parameter: 1) Time of sorting the consensus algorithm: The assetName, assetID, average time difference between WriteBlock and assetUrlHash, Propose in order’s log file; ownerID, Metadata 2) Time of completing the block broadcast for Method: Query Applicati QueryAssetsByUserI Image transaction: The average time difference between detail of on Blockchain D(ctx， userID) detail StoreBlock in peer’s log file and WriteBlock in image System system Parameter: order’s log file; userID 3) Time of completing verification and submission for Method: Query QueryAssetsHistoryB transaction: The average time difference between history Applicati Blockchain yAssetID(ctx， Image Validate and StoreBlock in peer’s log file; on history of system assetID) System 4) Time of completing submission for transaction: The image Parameter: average time difference between assetID Method: CommitWithPvtData and Validate in peer’s log file. ExchangeAsset(ctx， assetHistoryID， C． Compatibility assetID， In ISO/IEC 25023-2016, there are two sub-characteristics, Transfer Applicati Result Blockchain origenUserID， of co-existence and interoperability, interoperability is defined by on system currentUserID) transfer as data formats exchangeability, data exchange protocol image System Parameter: sufficiency and external interface adequacy. But for assetHistoryID, assetID, Hyperledger Fabric, interoperability has different test origenUserID, measures, it can be defined as the description of system currentUserID interaction interface[4]. The detail definition as follow; D． Reliability 1) Description of system interaction interface: It needs In ISO/IEC 25023-2016, there are four sub- to give a detailed description with the interface name, characteristics, maturity, availability, fault tolerance and request system, response system, method of the recoverability. For Hyperledger Fabric, it is a peer-to-peer interface be used , parameters of the interface and network, with equivalent nodes, each node of the block return results of the interface. should reach a consensus and then to go to the next operation, The example as follow: in the end all nodes make up a distributed ledger, complete a The image resource is stored in the file system while the command or result which cannot be changed. Because of this image encryption is stored in the application system. When consensus mechanism, the blockchain system allows partial the user wants to put image on chain, they must interact with node or ledger to be incorrect. As long as most of them are file system, application system and block chain system. correct, the system also can works normally. So the fault Firstly, the user fills in the basic information, chooses the tolerance in Hyperledger Fabric can be defined as node image which would be uploaded, and finally presses the abnormal tolerance and ledger abnormal tolerance, the detail upload button, enabling the file system to send the image definition as follow; 1） Node abnormal tolerance: Node tolerance rate of [5] CBD-Forum-001: Blockchain-Reference Architecture (Chinese), Dongcheng District, Beijing: China Academy of Electronic the block chain system, the purpose is to find out Technology Standardization, 2017. the maximum error number of endorsement nodes and verification nodes, while not affecting the normal use of the application; 2） Ledger abnormal tolerance: Ledger tolerance rate of the block chain system, the purpose is to find out the maximum number of the ledger file which is been deleted or tampered, while not affecting the normal use of the application; E． Security In ISO/IEC 25023-2016, confidentiality, non-repudiation and authenticity are all the security’s sub-characteristic. Confidentiality is defined as the correctness of data, access control; Non-repudiation is defined as data signature utilization quality metrics; Authenticity is defined as the adequacy of the identification mechanism and the conformity of the identification rules. But for Hyperledger Fabric, there are different test measures, confidentiality can be defined as unauthorized access restrictions, channel isolation, and node confidentiality; Non-repudiation can be defined as non- repudiation of operations on the chain and non-repudiation of log data; Authenticity can be defined as the identification mechanism, the detail definition as follow; 1） Unauthorized access restrictions: When there is unauthorized data access, the data privacy processing of the block chain; 2） Channel isolation: Isolation measures in block chain channels; 3） Node confidentiality: Confidentiality measures for block chain ledger data and node data; 4） Non-repudiation of operations on the chain: All chain operations can be logged and traced; 5） Non-repudiation of log data: All log data only can be viewed but cannot be tampered; 6） Identification mechanism: The identification mechanism adopted by block chain; V. CONCLUSION This paper analyzed the characteristics of Hyperledger Fabric, formed a software quality management system which is suitable for Hyperledger Fabric, and then explained its special quality measure. We hope that can provide a reference when testing with the blockchain software, and improve the blockchain’s software quality. REFERENCES [1] ISO/IEC 25023: Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) -- Measurement of system and software product quality, Geneva, Switzerland: International Organization for Standardization, 2016. [2] ISO/IEC 25010: Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) -- System and software quality models, Geneva, Switzerland: International Organization for Standardization, 2011. [3] Bellaj Badr, Richard Horrocks, Xun (Brian) Wu, “Blockchain By Example” England, Birmingham:Packt Publish, 2018. [4] CBTCA-001: Grading Evaluation Specification for Blockchain Distributed Ledger Information System (Chinese), Chaoyang District, Beijing: China Blockchain Evaluation Alliance, 2019.