5th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2017) Report on the 5th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2017) Horst Lichter Thanwadee Sunetnanta Toni Anwar RWTH Aachen University Mahidol University UTM Johor Bahru Germany Thailand Malaysia lichter@swc.rwth-aachen.de thanwadee.sun@mahidol.ac.th tonianwar@utm.my the exchange on the real world problems with quantitative I. INTRODUCTION approaches. For example, only limited research has been After the successful workshop QuASoQ 2016, which was held devoted to empirically evaluate risks, efficiency or limitations in Hamiltion, New Zealand, the organizers of the 4th workshop of different testing techniques in industrial settings. wanted to widen the scope of quantitative approaches to Hence, one main goal of the workshop was to exchange software quality. Therefore, the call for papers and the list of experience, present new promising approaches and to discuss topics of the workshop were adjusted in the direction of how to set up, organize, and maintain quantitative approaches to quantitative approaches in software testing. The topics of software quality. interest included • New approaches to measurement, evaluation, II. WORKSHOP FORMAT comparison and improvement of software quality Based on our former experience we wanted the workshop to be highly interactive. In order to have an interesting and interactive • Metrics and quantitative approaches in agile projects event sharing lots of experience, we organized the workshop • Case studies and industrial experience reports on presentations applying the author-discussant model. successful or failed application of quantitative Based on this workshop model, papers are presented by one approaches to software quality of the authors. After the presentation a discussant starts the • Tools, infrastructure and environments supporting discussion based on his or her pre-formulated questions. quantitative approaches Therefore the discussant had to prepare a set of questions and had to know the details of the presented paper. The general • Empirical studies, evaluation and comparison of structure of each talk was as follows: measurement techniques and models • The author of a paper presented the paper (15 minutes). • Quantitative approaches to test process improvement, test strategies or testability • After that, the discussant of the paper opened the discussion using his or her questions (5 minutes). • Empirical evaluations or comparisons of testing techniques in industrial settings • Finally, we moderated the discussion among the whole audience (10 minutes). Overall, the workshop aimed at gathering together researchers and practitioners to discuss experiences in the III. INVITED TALK application of state of the art approaches to measure, assess and This year we were happy to have Prof. Hironori Washizaki evaluate the quality of both software systems as well as software as an invited speaker. Hironori Washizaki is the Director and a development processes in general and software test processes in Professor with the Global Software Engineering Laboratory, particular. Waseda University, Japan. He is also a Visiting Professor with As software development organizations are always forced to the National Institute of Informatics, and an Outside Director develop software in the "right" quality, the quality specification with the SYSTEM INFORMATION CO., LTD. He was a and quality assurance are crucial. Although there are lots of Visiting Professor with the Ecole Polytechnique de Montreal, in approaches to deal with quantitative quality aspects, it is still 2015. He has long-term experience of researching and practicing challenging to choose a suitable set of techniques that best fit to software design, reuse, quality assurance, and education. the specific project and organizational constraints. Prof. Hironori Washizaki presented in his talk entitled Even though approaches, methods, and techniques are “Pitfalls and Countermeasures in Software Quality known for quite some time now, little effort has been spent on Measurements and Evaluations” important aspects that are Copyright © 2017 for this paper by its authors. 1 5th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2017) influencing the application of quality measurements in the C. Jan Thomas, Ana Nicolaescu and Horst Lichter: Static and context of software development. He identified a set of pitfalls Dynamic Architecture Conformance Checking: A and presented respective countermeasures. Essentially, Systematic, Case Study-Based Analysis on Tradeoffs and appropriate goals and strategies have to be defined and linked Synergies together to make a measurement program successful. In order to uncover architectural drift, a plethora of architecture IV. WORKSHOP CONTRIBUTIONS conformance checking tools has been proposed that mainly leverage two approaches: they extract architectural knowledge Altogether twelve papers were submitted. Finally, ten papers based on either source code artifacts (static approach) or run- were accepted by the program committee for presentation and time behavior (dynamic approach). Although both approaches publication covering very different topics. We grouped the have been evaluated separately, no up-to-date analysis of their papers into three sessions and added a final round-up slot to relative strengths and weaknesses, nor real-world comparative present and discuss the major findings of our workshop. In the case studies of the two were published. In this paper we address following we want to give a short overview of the accepted this issue by presenting the results of a direct comparison of both papers. approaches. We first identify and compare their strengths and A. Lov Kumar, Santanu Rath and Ashish Sureka: Estimating weaknesses on a theoretical level. We then evaluate these results Web Service Quality of Service Parameters using Source against our experiences gained in a large-scale industrial case Code Metrics and LSSVM study. As a result, we argue that the approaches cannot substitute each other as they differ in many key aspects. Hence, we We conduct an empirical analysis to investigate the relationship crystallize guidelines regarding how to combine these such that between thirty-seven different source code metrics with fifteen their strengths are emphasized while weaknesses mitigated. different Web Service QoS (Quality of Service) parameters. The source code metrics used in our experiments consists of nineteen D. Abdus Satter, Nadia Nahar and Kazi Sakib: Automatically Object-Oriented metrics, six Baski and Misra metrics, and Identifying Dead Fields in Test Code by Resolving Method twelve Harry M. Sneed metrics. We apply Principal Component Call and Field Dependency Analysis (PCA) and Rough Set Analysis for feature extraction Dead fields are the unused setup fields in the test code which and selection. The different sets of metrics are provided as input reduce the comprehensibility and maintainability of a software to the predictive model generated using Least Square Support system. A test class contains dead fields when developers Vector Machine (LSSVM) with three different types of kernel initialize setup fields without analyzing the usage of fields functions: RBF, Polynomial, and Linear. Our experimental properly. Manually identifying dead fields to remove from the results reveal that the prediction model developed using LSSVM code is a time consuming and error-prone task. In this paper, a method with RBF kernel function is more effective and accurate technique named Dead Field Detector (DFD) has been proposed for prediction of QoS parameters than the LSSVM method with to detect dead fields automatically. The technique constructs linear and polynomial kernel functions. Furthermore, we also Call Graph (CG) and Data Dependence Graph (DDG) from test observe that the predictive model created using object-oriented code to find method invocation and field dependency metrics achieves better results in comparison to other sets of relationships, respectively. It identifies the fields initialized in source code metrics. the setup method and its invoked methods from CG. It finds B. Sandhya Tarwani and Ashish Sureka: Investigating the setup fields by collecting the initialized fields and their Effectiveness of Greedy Algorithm on Open Source dependent fields from DG. To determine the usage of the setup Software Systems for Determining Refactoring Sequence fields, it checks the bodies of the test methods and their invoked methods obtained from CG. All the unused setup fields are The deeper problem in the source code are the bad smells that separated from the used fields and considered as dead fields. In indicates something is wrong and if they are not detected timely, order to evaluate DFD, an open source project named eGit was then they lead towards the complete deterioration of the working used. The result analysis shows that DFD has identified 14.03% software causing major financial and productivity loss. more setup fields and 60.98% more dead fields than an existing Refactoring helps in removing these bad smells by improving technique named TestHound for eGit. internal quality attributes of the software without affecting its external behaviour. However refactoring needs to be applied in E. Yuichiro Senzaki, Siyuan Liu, Hironori Washizaki, a controlled manner. In this study an approach has been propose Yoshiaki Fukazawa, Hiroshi Kobayashi and Masaharu for determining an optimal refactoring sequence that will Adachi: A Web Application to Manage and Improve maximize the source-code maintainability using greedy Software Development Projects by SEMAT Essence algorithm. The proposed approach selects the most optimum As part of the rapid advances in software engineering, each year sequence at every step-in hope of finding the global optimum a vast amount of new knowledge and ideas are proposed. solution. We conduct an empirical analysis on four open-source However, a gap often arises between new ideas and current software and select those classes that have bad smells greater methods due to a lack of fundamental theory. To bridge this gap, than or equal to four. Further filtration is done by selecting those SEMAT (Software Engineering Methods and Theory) Essence classes from the group that have high value of source code lines. has been proposed as the common ground in software We demonstrate the effectiveness of our approach using engineering. Using SEMAT Essence, developers can track the concrete examples of the experimental dataset and presenting progress and health of a project more efficiently from various summary results. viewpoints. However, SEMAT Essence has some limitations. In practice, only a few tools implement SEMAT Essence. Most of Copyright © 2017 for this paper by its authors. 2 5th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2017) these tools are problematic and do not sufficiently satisfy the analysis of where the problems of practice implementation lie, requirements for practical developments. Therefore, we develop i.e. at process modeling, at process enactment, or at process a tool called OCMS (Online Checklist Management System), implementation. which improves existing tools. An experiment where students manage an ET robot contest project using OCMS confirms its I. Tachanun Kangwantrakool and Thanaruk Theeramunkong: effectiveness and demonstrates that OCMS can help developers Towards the Re-engineering of Readiness Review Process improve efficiency. with R2P2 Lifecycle Model As a lesson learned, the readiness review process of SCAMPI F. Felix Timm, Simon Hacks, Felix Thiede and Daniel appraisal is very complicated and effort and cost consuming for Hintzpeter: Towards a Quality Framework for Enterprise novice organizations who need to know their status of CMMI Architecture Models practices classification. In SCAMPI appraisal, the readiness While Enterprise Architecture Management is an established review process is a single process that runs from start to the end. and widely discussed field of interest in the context of During the readiness review process, nobody collected database information systems research, we identify a lack of work to improve the process’s performance. Our research towards the regarding quality assessment of enterprise architecture models re-engineering of readiness review process aims to enhance the in general and frameworks or methods on that account in process performance and reduce effort/cost of the readiness particular. By analyzing related work by dint of a literature review process implementation. We design our R2P2 Lifecycle review in a design science research setting, we provide twofold Model to provide the benchmark of readiness review process contributions. We (i) suggest an Enterprise Architecture Model relative to Capability Maturity Model Integration (CMMI) Quality Framework (EAQF) and (ii) apply it to a real world practices implementation. The R2P2 Lifecycle Model describes scenario. the requirements, activities, and practices associated with the readiness review process that composes the model. The total of G. Ke Dai and Philippe Kruchten: Detecting Technical Debt 49 appraisals data are used to establish, evaluate, and enhance through Issue Trackers the performance and efficiency of the R2P2 Lifecycle Model. Managing technical debt effectively to prevent it from This paper presents our conceptual view of the R2P2 Lifecycle accumulating too quickly is of great concern to software Model and a lesson learned from the preliminary development stakeholders. To pay off technical debt regularly, software of the model. We are at the first stage of the R2P2 Lifecycle developers must be conscious of the existence of technical debt Model development by using 30 historical appraisals cases for items. The first step is to make technical debt explicit; that is the root causes analyzing of the weakness of readiness review identification of technical debt. Although there exist many kinds process. Therefore, we will use this lesson learned to enhance of static source code analysis tools to identify code-level the R2P2 Lifecycle Model in next stage. technical debt, identifying non-code-level technical debt is very challenging and needs deep exploration. This paper proposed an V. SUMMARY OF THE DISCUSSIONS approach to identifying non-code-level technical debt through About 25 researchers attended the workshop and participated in issue tracking data sets using natural language processing and the discussions. The author-discussant model was well received machine learning techniques and validated the feasibility and by the participants and led to intensive discussions among them. performance of this approach using an issue tracking data set recorded in Chinese from a commercial software project. We To conclude, in the course of this workshop the participants found that there are actually some common words that can be proposed and discussed different approaches to measure and used as indicators of technical debt. Based on these key words, quantify relevant aspects of software development. For we achieved the precision of 0.72 and the recall of 0.81 for Especially the discussions led to constructive feedback, deeper identifying technical debt items using machine learning insights, and hopefully some take-aways for all participants. techniques respectively. VI. ACKNOWLEDGMENTS H. Suppasit Roongsangjan, Thanwadee Sunetnanta and Many people contributed to the success of this workshop. First, Pattanasak Mongkolwat: Multi-Level Compliance we want to give thanks to our invited speaker and the authors Measurements for Software Process Appraisal and presenters of the accepted papers. Furthermore, we want to Software process appraisal is to assess whether an implemented express our gratitude to the APSEC 2017 organizers; they did a software process complies with a process reference model. To perfect job. Finally, we are glad that these people served on the conduct the appraisal, the appraisal team will request an program committee (some of them for many years) and organization to provide objective evidence reflecting practice supported the workshop by soliciting papers and by writing peer implementation. Then such evidence will be examined, verified, reviews: and validated to generate appraisal results. This evidence collection process is done after a process is implemented. To • Matthias Vianden, Aspera GmbH, Aachen, Germany better prepare for software process appraisal, we argued that the • Wan M.N. Wan Kadir, UTM Johor Bahru, Malaysia compliance of a process can be measured prior to its implementation. In the light of that, we proposed multi-level • Maria Spichkova, RMIT University, Melbourne, Australia compliance measurements to determine process reference model • Taratip Suwannasart, Chulalongkorn Univiversity, Thailand compliance, in terms of Process Model Readiness Score, Process Enactment Score, and Process Implementation • Tachanun Kangwantrakool, ISEM, Thailand Readiness Score. These measurements help provide an insight Copyright © 2017 for this paper by its authors. 3 5th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2017) • Jinhua Li, Qingdao University, China • Sansiri Tanachutiwat, Thai German Graduate School of Engineering, TGGS, Thailand • Apinporn Methawachananont, NECTEC, Thailand • Hironori Washizaki, Waseda University, Japan • Nasir Mehmood Minhas, PMAS - AAUR Rawalpindi Pakistan • Hongyu Zhang, The University of Newcastle, Australia • Chayakorn Piyabunditkul, NSTDA, Thailand • Ashish Sureka, Ashoka University, India Copyright © 2017 for this paper by its authors. 4