Report on the 9th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2021) Horst Lichtera , Selin Aydina , Thanwadee Sunetnantab and Toni Anwarc a Research Group Software Construction, RWTH Aachen University, Aachen, Germany b Computer Science Academic Group, Faculty of Information And Communication Technology, Mahidol University, Bangkok, Thailand c Faculty of Science and Information Technology, Chair Computer & Information Systems, Universiti Teknologi Petronas: Bandar Seri Iskandar, Perak, MY 1. Introduction quality specification and quality assurance are crucial. Although there are lots of approaches to deal with quan- After a successful 8th QuASoQ workshop we slightly titative quality aspects, it is still challenging to choose adjusted the list of topics for the workshop. The topics a suitable set of techniques that best fit to the specific of interest included project and organizational constraints. Even though approaches, methods, and techniques are • New approaches to measurement, evaluation, known for quite some time now, little effort has been comparison and improvement of software quality spent on the exchange on the real-world problems with • Application of metrics and quantitative ap- quantitative approaches. For example, only limited re- proaches in agile projects search has been devoted to empirically evaluate risks, • Case studies and industrial experience reports efficiency or limitations of different testing techniques on successful or failed application of quantitative in industrial settings. approach-es to software quality Hence, one main goal of the workshop was to exchange • Tools, infrastructure and environments support- experience, present new promising approaches and to ing quantitative approaches discuss how to set up, organize, and maintain quantitative • Empirical studies, evaluation and comparison of approaches to software quality. measurement techniques and models • Quantitative approaches to test process improve- ment, test strategies or testability 2. Workshop History • Empirical evaluations or comparisons of testing techniques in industrial settings The QuASoQ workshop series has been started in 2013. Since then, the workshop is always organized as a col- • Mining software repositories located event of the Asia-Pacific Software Engineering Overall, the workshop aimed at gathering together Conference (APSEC). researchers and practitioners to discuss experiences in These are the past workshop editions: the application of state of the art approaches to measure, • 8th QuASoQ 2020 assess and evaluate the quality of both software systems Singapore | CEUR Vol-2767 as well as software development processes in general and th QuASoQ 2019 software test processes in particular. • 7 As software development organizations are always Putrayaya, Malaysia | CEUR Vol-2511 forced to develop software in the ”right” quality, the • 6th QuASoQ 2018 Nara, Japan | CEUR Vol-2273 QuASoQ 2021: 9th International Workshop on Quantitative • 5th QuASoQ 2017 Approaches to Software Quality, December 06, 2021, Taipei, Taiwan Envelope-Open lichter@swc.rwth-aachen.de (H. Lichter); Nanjing, China | CEUR Vol-2017 aydin@swc.rwth-aachen.de (S. Aydin); • 4th QuASoQ 2016 thanwadee.sun@mahidol.ac.th (T. Sunetnanta); Hamilton, New Zealand | CEUR Vol-1771 toni.anwar@utp.edu.my (T. Anwar) • 3rd QuASoQ 2015 GLOBE https://www.swc.rwth-aachen.de (H. Lichter); https://www.swc.rwth-aachen.de (S. Aydin); New Delhi, India |CEUR Vol-1519 https://www.ict.mahidol.ac.th (T. Sunetnanta); • 2nd QuASoQ 2014 https://www.utp.edu.my (T. Anwar) Jeju, Korea | IEEE Xplore Orcid 0000-0002-3440-1238 (H. Lichter); 0000-0002-0390-8749 • 1st QuASoQ 2013 (T. Anwar) © 2021 Copyright for this paper by its authors. Use permitted under Creative Bangkok, Thailand | IEEE Xplore Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR http://ceur-ws.org Workshop ISSN 1613-0073 Proceedings 38 Taipei, so that presenters don’t have to attend at night- time. The order of presenters were also determined by their respective timezone. 4. Workshop Contributions Altogether 6 papers were submitted. Finally, the follow- ing 4 papers were accepted by the program committee for presentation and publication covering very different topics. • Ruhaya Ab. Aziz The impact of Requirements Relationships knowl- edge on Requirements Quality and Software Devel- opment Project success • Anıl Holat and Ayse Tosun Predicting Requirements Volatility: An Industry Figure 1: Origin of QuASoQ authors Case Study • Lukas Liss, Henrik Kämmerling, Peter Alexander and Horst Lichter Towards a Catalog of Refactoring Solutions for En- Since the first edition 62 papers have been presented; terprise Architecture Smells the average acceptance rate is 75 %. The chart shown • Derya Yeliz Ulutaş and Ayşe Tosun in figure 1 depicts where the authors of accepted papers A Condition Coverage-Based Black Hole Inspired come from. Meta-Heuristic for Test Data Generation 3. Workshop Format 5. Summary of the Discussions Because of the covid-19 pandemic, the workshop was ex- About 10 researchers attended the workshop and partici- ecuted digitally using the video conferencing tool Zoom. pated in the discussions. The author-discussant model Based on our former experience we wanted the work- was well received by the participants and led to inten- shop to be highly interactive. In order to have an inter- sive discussions among them. Hereby, other participants, esting and interactive event sharing lots of experience, apart from the discussant, also joined the resulting dis- we organized the workshop presentations applying the cussions. author-discussant model. The first discussion was on the paper by Ab Aziz on Based on this workshop model, papers are presented requirements relationship knowledge (RRK). Participants by one of the authors. After the presentation, a discussant were particularly interested in the value of this knowl- starts the discussion based on his or her pre-formulated edge in agile project management approaches compared questions. Therefore, the discussant had to prepare a set to more sequential ones. In addition, methods for as- of questions and had to know the details of the presented sessing the current understanding of RRK in a software paper. The general structure of each talk was as follows: development team were explored. • The author of a paper presented the paper (20 The paper by Holat et. al. lead to a discussion of how minutes). their approach of predicting highly volatile requirements • After that, the discussant of the paper opened can be integrated into the software development process. the discussion using his or her questions. Finally, The presenter of the paper explained that the approach we moderated the discussion among the whole can be applied both in the early phase of a project and audience (10 minutes). after it is completed. Especially in the early phase the predicted volatility can be translated into additional story The presentations were divided into two sessions with points for the corresponding issue. After doing this, more a ten minute break in-between. Each session was ac- experienced developers/reviewer can be assigned to is- companied by a moderator who tried to ensure that the sues which refer to highly volatile requirements. schedule was kept to. A particular challenge were the In the discussion of the paper by Liss et al. it became different time zones of the participants. We decided to clear that the authors have a particular interest in future hold the workshop in the afternoon of the timezone in collaboration in their research area. Not only have they 39 made the resulting catalog available as a public reposi- • Apinporn Methawachananont tory that allows merge requests, but they also explain NECTEC, Thailand how to contribute. One participant was interested in the • Ana Nicolaescu differences between code and EA refactoring. Here, the Daimler AG, Germany presenter explained that the goals are fundamentally dif- • Maria Spichkova ferent. While code refactoring focuses on the technical RMIT University, Melbourne, Australia level to improve e.g. maintainability or readability, EA • Minxue Pan refactoring affects major entities in the IT landscape to Nanjing University, China improve efficiency of the business. • Lov Kumar Finally, the paper by Ulutaş et al. led to a discussion BITS-PILANI, Hyderabad, India of existing approaches, particularly combinatorial test- ing, and how they might compare with the approach presented. The author suggested that perhaps a study could be conducted to evaluate whether a combination with combinatorial testing approaches is more effective. The discussions show, that empirical studies and the results of experiments are of high value and lead to a deeper understanding of the subject that has been inves- tigated. To conclude, in the course of this workshop the par- ticipants proposed and discussed different approaches to quantify relevant aspects of software development. Es- pecially the discussions led to new ideas, insights, and take-aways for all participants. 6. Acknowledgments Many people contributed to the success of this workshop. First, we want to give thanks to the authors and presen- ters of the accepted papers. Furthermore, we want to express our gratitude to the APSEC 2021 workshop orga- nizers; they did a perfect job and gave us the freedom to conduct the workshop virtually based on our experience. Finally, we are glad that these people served on the program committee (some of them for many years) and supported the workshop by soliciting papers and by writ- ing peer reviews: • Hironori Washizaki Waseda University, Japan • Nasir Mehmood Minhas BTH Karlskrona, Sweden • Jin-Hua Li Qingdao University, China • Hongyu Zhang University of Newcastle, Australia • Taratip Suwannasart Chulalongkorn University, Thailand • Wan Mohd Nasir Wan-Kadir UTM Johor Bahru, Malaysia • Sansiri Tanachutiwat Thai German Graduate School of Engineering, TGGS, Thailand 40