=Paper=
{{Paper
|id=Vol-1771/paper1
|storemode=property
|title=Report on the 4th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2016)
|pdfUrl=https://ceur-ws.org/Vol-1771/paper1.pdf
|volume=Vol-1771
|authors=Horst Lichter,Konrad Fögen,Thanwadee Sunetnanta,Toni Anwar
|dblpUrl=https://dblp.org/rec/conf/apsec/LichterFSA16
}}
==Report on the 4th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2016)==
https://ceur-ws.org/Vol-1771/paper1.pdf
4th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2016)
Report on the 4th International Workshop on
Quantitative Approaches to Software Quality
(QuASoQ 2016)
Horst Lichter Konrad Fögen Thanwadee Sunetnanta Toni Anwar
RWTH Aachen University RWTH Aachen University Mahidol University UTM Johor Bahru
Germany Germany Thailand Malaysia
lichter@swc.rwth-aachen.de foegen@swc.rwth-aachen.de thanwadee.sun@mahidol.ac.th tonianwar@utm.my
approaches. For example, only limited research has been
I. INTRODUCTION devoted to empirically evaluate risks, efficiency or limitations
After the successful workshop QuASoQ 2015, which was held of different testing techniques in industrial settings.
in New Delhi, India, the organizers of the 4th workshop wanted Hence, one main goal of the workshop was to exchange
to widen the scope of quantitative approaches to software experience, present new promising approaches and to discuss
quality. Therefore, the call for papers and the list of topics of the how to set up, organize, and maintain quantitative approaches to
workshop were adjusted in the direction of quantitative software quality.
approaches in software testing. The topics of interest included
II. WORKSHOP FORMAT
• New approaches to measurement, evaluation,
comparison and improvement of software quality Based on our former experience we wanted the workshop to be
highly interactive. In order to have an interesting and interactive
• Metrics and quantitative approaches in agile projects event sharing lots of experience, we organized the workshop
• Case studies and industrial experience reports on presentations applying the author-discussant model.
successful or failed application of quantitative Based on this workshop model, papers are presented by one
approaches to software quality of the authors. After the presentation a discussant starts the
• Tools, infrastructure and environments supporting discussion based on his or her pre-formulated questions.
quantitative approaches Therefore the discussant had to prepare a set of questions and
had to know the details of the presented paper. The general
• Empirical studies, evaluation and comparison of structure of each talk was as follows:
measurement techniques and models
• The author of a paper presented the paper (15 minutes).
• Quantitative approaches to test process improvement,
test strategies or testability • After that, the discussant of the paper opened the
discussion using his or her questions (5 minutes).
• Empirical evaluations or comparisons of testing
techniques in industrial settings • Finally, we moderated the discussion among the whole
audience (10 minutes).
Overall, the workshop aimed at gathering together
researchers and practitioners to discuss experiences in the Again, this format was very successful as it led to more
application of state of the art approaches to measure, assess and intensive discussions among the participants.
evaluate the quality of both software systems as well as software III. WORKSHOP CONTRIBUTIONS
development processes in general and software test processes in
particular. Altogether nine papers were submitted. Finally, seven papers
were accepted by the program committee for presentation and
As software development organizations are always forced to publication covering very different topics. We grouped the
develop software in the "right" quality, the quality specification papers into three sessions and added a final round-up slot to
and quality assurance are crucial. Although there are lots of present and discuss the major findings of our workshop. In the
approaches to deal with quantitative quality aspects, it is still following we want to give a short overview of the accepted
challenging to choose a suitable set of techniques that best fit to papers.
the specific project and organizational constraints.
Even though approaches, methods, and techniques are
known for quite some time now, little effort has been spent on
the exchange on the real world problems with quantitative
1
4th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2016)
A. Hirohisa Aman, Sousuke Amasaki, Tomoyuki Yokogawa contributions to industry and contribute towards improving their
and Minoru Kawahara: Local Variables with Compound adoption.
Names and Comments as Signs of Fault-Prone Java D. Lov Kumar, Santanu Rath and Ashish Sureka: Predicting
Methods Quality of Service (QoS) Parameters using Extreme
This paper focuses on local variables and comments in methods Learning Machines with Various Kernel Methods
of Java applications. Both of them are usually used at the
Web services which are language and platform independent self-
programmer’s discretion. Thus, naming local variables and
contained web-based distributed application components
commenting code can vary among individuals, and such an
represented by their interfaces can have different Quality of
individual difference may cause a dispersion in quality.
Service (QoS) characteristics such as performance, reliability
The authors conducted an empirical analysis on the fault- and scalability. One of the major objectives of a web service
proneness of Java methods which are collected from nine provider and implementer is to be able to estimate and improve
popular open source products. The results report the following the QoS parameters of their web service as its clients application
three findings: (1) Methods having local variables with are dependent on the overall quality of the service.
compound names are more likely to be faulty than the others; (2)
In this paper the authors hypothesized that the QoS
Methods having local variables with simple and short names are
parameters have a correlation with several source code metrics
unlikely to be faulty, but their positive effects tend to be decayed
and hence can be estimated by analyzing the source code. They
as their scopes get wider; (3) The presence of comments within
investigated the predictive power of 37 different software
a method body can also be useful sign of fault-prone method.
metrics to estimate 15 QoS attributes. Furthermore, they
B. Ahmed Alharthi, Maria Spichkova and Margaret developed QoS prediction models using Extreme Learning
Hamilton: Sustainability Profiling of Long-living Software Machines (ELM) with various kernel methods. Since the
Systems performance of the classifiers depends on the software metrics
that are used to build the prediction model, the authors also
In this paper the authors introduce a framework for software
examined two different feature selection techniques i.e.,
sustainability profiling. The goal of the framework is to analyse
Principal Component Analysis (PCA), and Rough Set Analysis
sustainability requirements for long-living software systems,
(RSA) for dimensionality reduction and removing irrelevant
focusing on usability and readability of the sustainability
features. The performance of QoS prediction models are
profiles. To achieve this goal, the authors applied a quantitate
compared using three different types of performance parameters
approach such as fuzzy rating scale-based questionnaires to rank
i.e., MAE, MMRE, RMSE. The obtained experimental results
the sustainability requirements, and the Technique for Order
demonstrate that the model developed by extreme learning
Preference by Similarity to Ideal Solution (TOPSIS) to analyse
machine with RBF kernel achieves better results as compared to
the results of questionnaires and to provide a basis for system
the other models in terms of the predictive accuracy.
profiling.
E. Abdus Satter and Kazi Sakib: Improving Recall in Code
The core profiling elements provided by our framework are
(1) a sustainability five-star rating, (2) visualisation of the five Search by Indexing Similar Codes under Proper Terms
sustainability dimensions as a pentagon graph detailing The recall of a code search engine is reduced, if feature-wise
combination for individual, social, technical, economic and similar code fragments are not indexed under common terms. In
environmental dimensions, and (3) a bar graph of overall this paper, a technique named Similarity Based Method Finder
sustainability level for each requirement. To ensure (SBMF) is proposed to alleviate this problem. The technique
sustainability, the proposed profiling framework covers the five extracts all the methods from a source code corpus and converts
dimensions of sustainability to quantify the sustainability of any these into reusable methods (i.e., program slice) through
software system not only during the requirement gathering resolving data dependency. Later, it finds similar methods by
phase but also during maintenance phase of software system checking signature (i.e., input and output types) and executing
lifecycle. methods for a randomly generated set of input values. Methods
are considered as feature-wise similar if these produce the same
C. Richa Awasthy, Shayne Flint and Ramesh output set. In order to index these methods against common and
Sankaranarayana: Towards improved Adoption: proper terms, SBMF selects the terms that are found in most of
Effectiveness of Research Tools in Real World the methods. Finally, query expansion is performed before
One of the challenges in the area of software engineering searching the index to solve the vocabulary mismatch problem.
research has been the low rate of adoption by industry of the
In order to evaluate SBMF, fifty open source projects
tools and methods produced by university researchers. In this
implementing nine different functionalities or features were
paper the authors present a model to improve the situation by
used. The results were compared with two types of techniques -
providing tangible evidence that demonstrates the real-world
Keyword BasedCode Search (KBCS) and Interface Driven
effectiveness of such tools and methods. A survey of practising
Code Search (IDCS). On an average, SBMF retrieves 38% and
software engineers indicates that the approach in the model is
58% more relevant methods than KBCS and IDCS, respectively.
valid and applicable. The authors applied and tested the model
Moreover, it is successful for all the features by retrieving at
for providing such evidence and demonstrated its effectiveness
least one relevant method representing each feature whereas
in the context of static analysis using FindBugs. This model can
IDCS and KBCS are successful for 3 and 7 features out of 9
be used to analyse the effectiveness of academic research
respectively.
2
4th International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2016)
F. Eun-Hye Choi, Osamu Mizuno and Yifan Hu: Code As another example, the discussion of paper C (Awasthy et
Coverage Analysis of Combinatorial Testing al.) focused on issues regarding the adoption of research tools in
Combinatorial t-way testing with small t is known as an efficient practice. It was pointed out that there is a mismatch between the
black-box testing technique to detect parameter interaction researcher’s focus when developing a tool and the practitioner’s
failures. So far, several empirical studies have reported the expectations when actually considering to use that tool. As a
effectiveness of t-way testing on fault detection abilities. result, the proposed model could be adjusted to obtain feedback
However, few studies have investigated the effectiveness of t- by the practitioner’s earlier and more frequently.
way testing on code coverage, which is one of the most The last discussion of the workshop was about an empirical
important coverage criteria widely used for software testing. study (Gren & Goldman) regarding a connection between agility
This paper presents a quantitative analysis to evaluate the and the maturity of a group of software developer’s. The
code-coverage effectiveness of t-way testing. Using three open findings of this research were negative and thus no connection
source utility programs, the authors compared t-way testing with could be approved or disproved. However, this led to interesting
exhaustive (all combination) testing w. r. t. code coverage and discussions about the reasons and about any other yet
test suite sizes. unconsidered factors which might be involved.
To conclude, in the course of this workshop the participants
G. Lucas Gren and Alfredo Goldman: Trying to Increase the
proposed and discussed different approaches to quantify
Mature Use of Agile Practices by Group Development
relevant aspects of software development. Especially the
Psychology Training - An Experiment discussions led to new ideas, insights, and take-aways for all
There has been some evidence that agility is connected to the participants.
group maturity of software development teams. This study aims
at conducting group development psychology training with V. ACKNOWLEDGMENTS
student teams, participating in a project course at university, and Many people contributed to the success of this workshop. First
compare their group effectiveness score to their agility usage of all, we want to give thanks to the authors and presenters of
over time in a longitudinal design. Seven XP student teams were the accepted papers. Furthermore, we want to express our
measured twice (43+40), which means 83 data points divided gratitude to the APSEC 2016 organizers; they did a perfect job.
into two groups (an experimental group and one control group). Finally, we are glad that these people served on the program
The results showed that the agility measurement was not committee (some of them for many years) and supported the
possible to increase by giving a 1.5-hour of group psychology workshop by soliciting papers and by writing peer reviews:
lecture and discussion over a two-month period. The non- • Matthias Vianden, Aspera GmbH, Aachen, Germany
significant result was probably due to the fact that 1.5 hours of
training were not enough to change the work methods of these • Wan M.N. Wan Kadir, UTM Johor Bahru, Malaysia
student teams, or, a causal relationship does not exist between
the two concepts. A third option could be that the experiential • Maria Spichkova, RMIT University, Melbourne,
setting of real teams, even at a university, has many more Australia
variables not taken into account in this experiment that affect the • Taratip Suwannasart, Chulalongkorn Univiversity,
two concepts. The authors therefore had no conclusions to draw Thailand
based on the expected effects. However, they believed these
concepts have to be connected since agile software development • Tachanun Kangwantrakool, ISEM, Thailand
is based on teamwork to a large extent, but there are probable • Jinhua Li, Qingdao University, China
many more confounding or mediating factors.
• Apinporn Methawachananont, NECTEC, Thailand
IV. SUMMARY OF THE DISCUSSIONS
• Jarernsri L. Mitrpanont, Mahidol University, Thailand
In total 10 researchers attended the workshop and participated in
the discussions. The author-discussant model was well received • Nasir Mehmood Minhas, PMAS - AAUR Rawalpindi
by the participants and led to intensive discussions among them. Pakistan
For instance, the discussion of paper A (Aman et al.) about • Chayakorn Piyabunditkul, NSTDA, Thailand
compound names and comments as signs for faults has
encountered great interest among the audience as many of them • Sansiri Tanachutiwat, Thai German Graduate School of
reported similar experiences. The discussion also led to new Engineering, TGGS, Thailand
ideas as it was revealed that there may also be cultural aspects • Hironori Washizaki, Waseda University, Japan
worth to be considered in future work. As an example,
styleguides of a company which enforce certain conventions for • Hongyu Zhang, Microsoft Research, China
variable names and comments or even the tongue in which the
source code is written as some tongues may have an impact on
the use of compound naming or comments.
3