Quality of large-scale mission-critical software systems depends on software system architecture. Although we design and create software system architecture we are still unable to evaluate how architectural decisions influence software behavior. This problem is becoming even more important in the context of future networks which assume autonomic software creation and interconnection guided by special needs and purposes of its creation where software architectures are created autonomously. One of the approaches to this problem is in development of software structure metrics that can be used as characterization metrics for architecture comparison and evaluation of its properties. Our previous work proposed to use of motifs for such purposes. However, its application is limited because of computational eficiency and we were able to investigate only 3-node motifs. Here in this paper, we provide a preliminary study investigating how higher dimensional substructures within software architecture may be used as software metrics and if these higher dimensional structures bring benefit to software characterization. Here, we search for higher dimensional substructures and aim to investigate their growth and change trends in the context of evolving software systems. We find out that structure behavior we may only understand by having a multidimensional structure view. Furthermore, we observed that the initial project size may correlate to structure stabilization across the product evolution. More precisely, projects with larger initial sizes may be slower in stabilizing the structure of internal dependencies while projects with smaller initial sizes may stabilize the structure of internal dependencies in just a few project releases.
One of the key research goals within the software engineering community was to understand
the mechanisms of software dynamics and its relations to the software static attributes which
we can measure during the software creation [
The software architecture is one of the main artifacts we design and create during the software
design phase. Usually, the modeling of software structure and architecture is based on static
software attributes or based on human subjective decisions [
One large group of research is devoted to developing various models for Software Defect
Prediction, SDP experimenting with various statistical approaches, machine learning, and
artiifcial intelligence that aims to build models based on aforementioned static software metrics
measured on software components such as software size, development efort, detected during
the design phase, and various software complexity metrics. However, these approaches have
almost neglected the existence of the software communication structure and focus solely on
static software metrics on files, modules, and classes (we will call them modules further on), and
correlate these static metrics with a number of failures or faults identified during the software
operation. These approaches rely on the module clustering principle based solely on data
similarity measured on those modules. However, many researchers have recognized the influence
of communication patterns within the software system on the fault and failure behavior and
operational software characteristics [
On the other hand, many approaches have been investigated for software structure analysis.
Previous research eforts have mainly focused on applying complex network science based on
the software dependency or call graph, [
Our previous study has been motivated by the aim to find appropriate software structure
characterization metrics that would enable us to further improve software defect prediction
modes. We found that information obtained from software structure expressed as the frequency
of the three-node subgraphs may be in correlation with software defects, [
The paper is structured as follows.
In Sect. 2 we provide an overview of the software structure analysis on the evolving software systems by introducing the concept of software structure, providing details of Eclipse dataset which is frequently used for SDP research, providing an overview of related works on software structures. Furthermore, in Sect. 3 we explain how we extracted the higher dimensions and the results we obtained. In Sect. 4 we discuss the issues that may represent threats to the results and conclusions we derive based on the selected study case. Finally, in the Sect. 5 we conclude the paper.
In this section, we aim to provide an overview of the research in finding appropriate models
for delivering best-efort software quality by doing appropriate actions during the design of
the software. One of the research directions is to predict and detect as many as possible
defects early, already during the design time, and to additionally invest in quality assurance
activities on selected-prediction supported faulty parts. In this line of research belong numerous
software defect prediction studies aiming to predict faulty software parts needed additional
attention with the main benefit to save on focused quality activities that are invested into
high-risk defective modules. The majority of such modules are employing machine learning and
artificial intelligence tools to find high-risk modules based on historical data and fining relations
between faulty modules’ incidence and their static attributes. There are numerous metrics
investigated to measure static code attributes and therefore a lot of studies reported challenges
of dimensionality reduction and feature selection procedures, [
Software architecture is an abstract term that we usually use in the software design phase.
Mostly it is used to represent a software structure as a set of software components i.e. modules,
ifles, or classes and their interactions [
In our previous works, we analyzed the software structure from open-source software repositories from the Eclipse community. Eclipse is an open-source community that stores its software versions on the open GIT repository (https://git.eclipse.org/c/). For the purposes of our study, we selected Java Development Tools (JDT) and Plug-in development environment (PDE) and Business Intelligence and Reporting Tools (BIRT) which is an Eclipse Platform-based reporting system used to create data visualizations and reports that can be embedded into rich client and web applications. These projects were selected because they are often analyzed in various research papers so our selection was motivated with the aim to understand the obtained results and their meaning when integrated into the existing knowledge base. Our study involves 14 JDT, 13 PDE, and 9 BIRT sequential releases. Most of the other studies analyzes only the first 3 releases of each of these projects. Here we wanted to analyze changes over the software evolution and therefore we undertake data collection for all available releases in the respective projects. The projects are developed during a longer period of time, BIRT in a period of 7 years, JDT 12 years of development and PDE is developed in 11 years.
The data collection process consists of the following steps. Firstly we downloaded from the
GIT repository all class files related to each project release. Then we extracted call graphs with
rFind tool that we developed within our research group and used in our previous studies for
software structure analytics [
Here, in this paper, we will base our analysis solely on the call graphs collected with rFind tool. But our future plans are to extend the analysis also for the context of SDP.
Firstly, in [
In our previous study [
Based on this analysis we may conclude that with the help of subgraph defectiveness, we may capture the code liveness property that other static code metrics may not be able to ofer. Furthermore, we found that subgraph defectiveness is correlated to system release defectiveness and this finding opens new opportunities to further investigations within the Software Defect Prediction community.
Furthermore, we were analyzing the software structure with the help of network science
models [
Here in this paper, we aim to further investigate higher-order structures in graphs that may bring some positive results in developing new models for smart software architecting. We believe that particular software graph structures may have bad influences on software behavior and should be avoided by proper software design decisions. Some earlier works have analyzed software modularity but we are not aware of studies analyzing this impact directly in relation to code defectiveness. Also, previous studies were analyzing the level of modularity between software classes as a number of communicating links but here we aim to investigate the behavior of higher-order software structures within the software evolution.
The main goal of the study was to understand how software structures evolve with respect to the higher dimensional subgraph structures that are represented within the software structure. In our study, as explained in the previous sections, we observe software structure from the code dependency viewpoint and the main source of our analysis is the call graphs. As we have already explained, the call graphs are software structures formed from the nodes that represent Java classes from the Eclipse software and edges between these nodes that are represented by method calls between these classes.
Within the call graphs, the goal is to find how higher dimensional substructures are
represented as follows:
• Zero-dimensional subgraphs (which we refer to as 0D) are the substructures present
within the call graphs represented by nodes, i.e., these are just the classes present in the
software.
• One-dimensional subgraphs (which we refer to as 1D) ) are the substructures present
within the call graphs represented by two nodes and an edge connecting these two nodes,
i.e., these are just the method calls present in the software.
• Two-dimensional subgraphs (which we refer to as 2D) are the substructures present
within the call graphs represented by three nodes fully connected by edges among them,
i.e., these are the complete subgraphs with three nodes. These were studied as part of our
research on motifs in the software [
The dimension of these substructures refers to their geometric realization. The OD substructures are nodes, represented by points, the 1D substructures are edges, represented by segments, the 2D substructures can be viewed geometrically as triangles, the 3D substructures as tetrahedra, and so on. The higher dimensional substructure of dimension can be viewed geometrically as the polyhedron with + 1 vertices.
Figure 3 represents the trend of the software growth over the project releases a) in the zero dimension 0D represented by a number of nodes, and b) in the first dimension 1D represented by a number of edges. The figure provides also the results of linear regression analysis performed on the three Eclipse projects. The obtained regression parameters (a, b) are provided in Table 1 for each analyzed dimension and Eclipse product. From the figure, we can observe that there is bigger growth in 0D and 1D (the number of classes and method calls) at the beginning of the evolution, and then, in the middle of the evolution, this growth is gradually decreasing or if we imagine infinite evolution we can say that size measured in 0D and 1D dimension has an asymptote towards some finite software size. This is also evident from the table1 where we can observe a reductions of a shape parameter (a) in higher dimensions.
From the figures and tables provided for 0D and 1D we may observe that size of the slope parameter may be in correlation with the initial software size, software size at the first release. The initial size of the BIRT project and all its subsequent releases is more then doubled with respect to the JDT and PDE projects and the same may be observed with the slope parameter in the respective projects.
Moreover, variations in growth size between releases are unstable in the first several releases, and then after some point, these variations stabilize as software evolves for JDT and PDE projects and tend to converge to some asymptote. Projects with smaller initial sizes have smaller growth variations compared to projects with larger initial sizes.
In Figure 4 we present the trend of software growth with respect to the 2D subgraphs structures along with the regression lines for each analyzed project. From the figure we may observe that in the case of smaller Eclipse projects (JDT and PDE) the software evolution has similar behavior as in lower dimensional substructures. However, the behavior of the largest project BIRT is somewhat diferent. In 2D we may observe that bigger projects have weaker linear regression fit, and we may observe larger deviation of the measurements from the linear regression line. Software growth, expressed as a number of triangles in the 2D dimension, at the beginning of the evolution is much higher than in the second part of the analyzed evolution cycle. Like in 0D and 1D dimensions, we can observe that the slope parameter in the linear regression curve may be correlated with software size in the initial project release.
Variations in software change between the project releases, as a measure of structural change during the evolution, are more evident in the first half of the observed evolution cycle while in the second part of the evolution cycle, these structural changes tend to stabilize. It is interesting to observe the diference in software structure change trends for the three observed projects during the evolution. We can observe that the slope parameter in the linear regression curve of structure change is very low and resulting in an almost horizontal regression line while the slope parameter of the 2D structure change between releases is very high for BIRT in comparison with other two projects, JDT and PDE. For the BIRT project, we may also observe a larger deviation of the measured curve from the linear regression line than for the other two projects. It seems that larger projects have larger variations in structure change between the project releases than smaller projects.
Figure 5 presents the growth in the number of tetrahedral found in software structure and the change in the number of tetrahedral in software evolution. We can observe from the figure there is a significantly diferent trend in the number of tetrahedral over software evolution. The smallest project in the 0D dimensions, PDE, has a constant and very low number of tetrahedral and no growth during the evolution can be observed. Then, a project with a bit larger initial size, JDT, in the 0D dimension has a trend of almost linear growth in this 3D dimension. While, the project with the largest initial size in the 0D, BIRT, has rapid growth in the 3D dimension in the first three evolution cycles, then we may observe to stabilize and almost constant trend in the middle four versions of evolution and finally we may observe a high drop down almost to the initial size during the last three releases of evolution. It is interesting to observe how smaller projects JDT and PDE had relatively small or no variations in a growth change during the evolution while in the largest project, we may observe significant variations over the evolution.
The current open-source development practice is still not matured to perform and develop
clear measurement standards. In Software code analytics there has been numerous attempts to
systematically collect source code metrics however the datasets are often criticised in sense of
lack of systematic procedure. Besides numerous criticism, the SDP community have succeeded
to develop some research results that improved not only the SDP research but also found
applications in other areas of research and practice. However, recent studies have indicated on
low reports from industrial case studies [
There are numerous issues one should address: remove test source files, identify relation of bug report and bug source (usually it is hard to match the exact location where the bug is corrected in which code file and there exist numerous approaches to establish unique standards to address that problem.), remove duplicates (which may be hard).
Furthermore, in our study, we used the call graphs that are measuring dependencies among
software parts as a source for grounding conclusions. The call graphs have been widely
explored within the software engineering community aiming to develop better software design
and analysis tools that would be able to perform and predict impact or vulnerability detection,
analysis, risk assessment, etc. Using call graphs as a source to bring structural and quality
conclusions have several threats. First of all, there are also numerous tools developed for
call graph extraction from the source codes. One comparative study on the capabilities and
efectiveness of various call graph extraction tools has been performed in [
Despite numerous criticisms, some datasets become standard for various machine learning, deep learning and artificial intelligence approaches to software defect prediction. It turns positive to have at least some dataset, although not fully correct, to compare various algorithms on the unique base. The prerequisite for such an action is to have open datasets so the community can easily approach them and experiment. The conclusion is that it is more important to have a standard metric or dataset to bring valuable conclusions than to have a very precise and complex procedure. In that sense, we have developed tools for data collection and have systematically collected the data for all the datasets in the case study. Since we collected the data on the same Eclipse projects as the open datasets (PROMISE) we have the possibility to compare our results and analyze the diferences. Moreover, our tools go beyond the PROMISE dataset because it connects two communities of software structure analysis and software defect prediction.
Here, we do not claim to have a completely ideal dataset. However, we do our best to collect thorough data and report on all possible misunderstandings. We did not use the usual datasets (PROMISE) because we do not have full control over the data in analysis. Although we have performed systematic comparisons with these datasets and reported on diferences we identified.
From our analysis, it is obvious that we can observe diferent trends in structure change when we observe the same software structure in diferent dimensions. Moreover, we may observe that this trend is somehow related to the project size of its initial release.
Projects that are initially smaller tend to stabilize already in lower-order dimensions and in fewer evolution steps. While for the projects with larger initial sizes in the zero and one dimensions, we may capture size growth stabilization at higher-order dimensions. Therefore, from the analyzed case we observed that initial project size may be in relation to trends observed at various structural dimensions.
All these imply that the evolution of structure growth has to be analyzed across various dimensions and a complete overview of structure growth behavior we may only understand by having such a multidimensional approach. One dimension of structural analysis may not be enough to understand structure evolution.
Furthermore, we concluded that the project’s initial size is very important for its further evolution. Projects with initially larger sizes may have greater opportunities for future growth. If we observe this conclusion in the context of call graphs that are representing code dependencies we may say that projects with higher initial code complexity may have more evolution cycles before they stabilize. Also, it is interesting to observe the meaning of stabilization in higherorder dimensions. Projects with higher initial sizes have more fluctuations in higher dimensions and have a slower trend to stabilize. Also, we observe higher rising trends in higher dimensions. It is interesting to observe a significant downtrend in the third dimension of the largest project in our analysis. Probable, due to unpredictable complexity there was some redesign of software structure that might explain such a downtrend.
In analysing Eclipse projects we were able to find representations up to three-dimensional software structures. This implies that software structures are not much represented in higher dimensional structures and thus finding higher dimensional structures may not be a computationaly demanding task. However, we analysed just Eclipse projects and our further explorative analysis would involve a much wider study case.
This paper acknowledges the support of the Erasmus+ Key Action 2 (Strategic partnership for higher education) project No. 2020–1–PT01–KA203–078646: “SusTrainable - Promoting Sustainability as a Fundamental Driver in Software Development Training and Education” and the support of the Croatian Science Foundation under the project HRZZ-IP-2019-04-4216. The information and views set out in this paper are those of the author(s) and do not necessarily reflect the oficial opinion of the European Union. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein.