In recent years, topological data analysis has become an important aspect of modern data science. Persistent homology is one of its popular features, and there are several specialized tools for its calculation. On the other hand, software reliability can be studied as a time-series of discovered software faults and/or occurrences of software failures. Topological data analysis of such time-series reduces to calculation of the persistent homology of certain point-clouds attached to time-series in suficiently high dimension. The aim of this paper is to investigate which of the tools for computing persistent homology is the most appropriate for applications in software reliability analysis.
Software becomes a cornerstone of digital society, where software systems are interconnecting infrastructure, transportation, healthcare, and many other domains. Failures in such systems can have far-reaching, even catastrophic, consequences. Software reliability is a crucial system quality attribute, especially for mission-critical systems where failures can have severe consequences for safety, human lives, and the operation of vital infrastructure. Moreover, reliability is also becoming increasingly important for all other domains due to economic, business, and sustainability constraints that may be afected by system failures.
Reliability modeling plays a pivotal role in managing such systems, enabling engineers to
quantitatively analyze past failures, predict future reliability, and help risk management to make informed
decisions about further management decisions like investments, release readiness, and quality assurance.
The current state of the art in software reliability modeling focuses on advanced software reliability
growth models (SRGM) that are essential tools for quantitatively assessing and predicting the reliability
of software systems throughout their lifecycle [
In our previous studies [
In particular, the increasing frequency of software releases in complex software systems and the
common practice of overlapping multiple projects that present significant challenges for fundamental
assumptions underlying traditionally used SRGMs. When development and maintenance activities for
diferent versions of a product occur simultaneously, the fault detection rates from previous revisions
can interfere with current reliability measurements, leading to inaccurate predictions if traditional
models are applied. Research has shown that in overlapping project environments, a single reliability
growth model often fails to fit the observed data accurately. Thus, combining multiple distributions
or adapting models becomes necessary to achieve reliable estimations and support efective project
management and planning [
Some further studies, such as [
A comprehensive examination of research trends, model selection practices, and enhancement techniques in SRGMs is provided in [12]. The authors conducted a systematic mapping and literature review covering 142 primary studies published between 1992 and 2020. The study categorizes SRGM selection criteria into model assumptions, application context, and evaluation metrics, and it identifies various enhancement methods such as hybrid modeling, incorporation of testing efort functions, and parameter estimation using metaheuristics (e.g. genetic algorithms and particle swarm optimization). The review reveals a notable shift in research toward more adaptable and context-aware models, while also highlighting gaps in empirical validation and standardization of evaluation practices. The authors conclude by recommending more rigorous experimental setups, broader datasets, and consistent benchmarking to support SRGM reliability and industrial relevance.
Our long term goal is to study software reliability using topological data analysis, as already announced in [13]. Geometry, and in particular topology, has gained in importance in computer science in recent years. Among the most prominent research direction are the geometric approaches to artificial intelligence [14], [15], [16], and the topological approach to quantum computers [17], [18]. The seed of all these new developments and applications of topology lies in topological data analysis [19], [20]. It relies on the toolbox of algebraic topology [21], [22], to extract hidden patterns in data that are out of reach by other, in particular quantitative, methods. Topological data analysis has found several important applications, see the expository papers [23], [24] for an overview of these.
Our previous work [13] is a preliminary study of the application of topological data analysis to software failure time-series, using persistent homology to extract qualitative patterns that enhance the understanding of reliability behaviors and support SRGM selection. Rather than focusing solely on quantitative measures traditionally used in Software Reliability Growth Models (SRGMs), we explore the qualitative, shape-based features of failure data to uncover hidden structural patterns across software versions. Using datasets from two open-source Eclipse projects (JDT and PDE) and an industrial telecom system (MSC), we demonstrate how topological features in failure data can reveal diferences and similarities in reliability evolution. The analysis suggests that these topological insights can support early identification of reliability behaviors and inform the selection of appropriate SRGMs, ofering a promising direction for future research into modeling complex, evolving software systems.
As the first step in that endeavor, we explore in this paper diferent tools available for computing the persistent homology of time-series using the Vietoris–Rips simplicial complexes obtained by the sliding window approach to construct the point-clouds in higher dimensions from a given time-series. Although a detailed analysis of the most important tools has already been reported in [25], [26], we undertake here an assessment of computation time of these tools in view of the application in software reliability analysis. The motivation is that the general results reported in the literature may not be appropriate for the special type of time-series arising from the software reliability analysis.
The assessment of the persistent homology tools is performed on 60 simulated software fault timeseries. The simulations are made following four diferent classical reliability growth models, and 120 point clouds are constructed using two diferent widths of the sliding window. The size of the dataset and the variability in models and sliding window, allows us to evaluate the considered tools in regard to the software reliability analysis.
The paper is organized as follows. In Section 2, the reliability growth models are recalled and the simulation of software fault time-series is explained. A brief and rough overview of persistent homology and tools for its calculations are the subject of Section 3. The conducted experiment is described and the results are reported in Section 4. Section 5 concludes the paper.
In this section we recall the classical reliability growth models for software reliability modeling. The
main reference for these matters are still the books [
We use the standard notation, that is, () represents the number of faults discovered in a software system from the beginning of testing or operation up to time ≥ when the testing or operation starts. The four SRGMs considered in this paper are the same as in the papers [27], [28]. These are recalled in the following list with the formula for () and the bounds for 0. Here = 0 represents the moment their parameters: • Goel–Okumoto or basic Musa model [29]
where > 0, > 0, • Delayed S-shaped model [30]
where > 0, > 0, • Gompertz model [31]
where > 0, 0 < < 1, 0 < < 1, • Yamada model [32] where > 0, > 0, > 0.
GO() = 1 ︁(
− − )︁ , DS() = 1 [︁
− (1 + ) − ]︁ , () = ︁(
)︁ , () = 1 [︁ − − (1− − )]︁ , All these models are based on the nonhomogeneous Poisson process with diferent distributions of inter-arrival times of faults. The Goel–Okumoto and Yamada models are concave, that is, the curve () is concave. The delayed S-shaped and Gompertz model are S-shaped, which means that the curve () exhibits an inflection point at which it changes from convex to concave.
The parameter represents the total number of faults in the system. In all the models except the Yamada model, this is the same as the number of faults that would be detected as the testing proceeds infinitely. The Yamada model assumes that even if the testing proceeds forever, there would still be some faults left in the system. This fact is taken into account when making simulations of fault counts.
There is an additional subtlety with the Gompertz model, as the faults can be detected at negative time, i.e., for < 0. This happens because the Gompertz model assumes that some faults are detected before the actual testing started.
The simulated time-series following the considered SRGMs are constructed from a uniformly distributed random numbers in the unit interval using the transformations arising from the formulas for (). The problem with possibly negative values of in the case of the Gompertz model is resolved in three diferent ways, thus producing three diferent types of time-series for the Gompertz model.
To ensure that the simulated time-series exhibit the fault behavior of real world software, and thus increase the generalizability of the findings, the parameters of SRGMs that are used in simulations are estimated from fault data of five industrial large-scale software development projects in the telecommunication domain.
In summary, we constructed the input for the experiment with persistent homology tools of Section 3. It is a dataset with 60 simulated time-series of software fault counts. There are 10 time-series that follow each of the considered SRGMs, including three diferent types of the Gompertz model.
This section begins with a very rough overview of persistent homology. The overview is included for completeness, and the reader interested only in applications to software engineering can safely skip this part and consider it as a black box. Due to limited space and the complexity of the subject, we are not in position to provide a complete self-contained account of the topic with examples and/or simple cases to demonstrate the theory. An example of topological analysis of higher dimensional structures in software systems is already published in this conference [33]. For the interested reader, we refer to the comprehensive standard reference [21], in which persistent homology is discussed in detail in Chapter VII. An excellent source for a beginner in algebraic topology and its applications are the lecture notes [34].
Persistent homology is an algebraic object attached to a filtration of simplicial complices. It captures how homology classes, in diferent degrees, appear and vanish when computing homology of simplicial complices in the filtration. These are called the birth and death times of homology classes (also called topological features from the machine learning point of view), where time refers to the position in the ifltration. The results are encoded in either persistent barcodes or persistent diagrams. Both represent the birth and death times of homology classes.
In the application to point-clouds that represent some data, the filtration of simplicial complices arises from the Vietoris–Rips complices associated with the growing radius of balls around points in the cloud. Since point-clouds are finite, there are only finitely many radii at which the Vietoris–Rips complex changes. This gives a required finite filtration.
In our case, the time-series simulated in Section 2 are transformed in point-clouds using a sliding window approach. For each time-series the sliding windows of width five and ten are applied. These produce point-clouds in dimensions five and ten that are the input for persistent homology tools in the experiment described in Section 4. Thus, we have in total 120 point-clouds, two arising from each of the 60 simulated time-series. Among them, 60 point clouds are in dimension five, and 60 in dimension ten.
The tools for computing persistent homology considered in this paper are the following: • Dionysus version 2.0.10 in Python [35], • DIPHA version 2.1.0 in C++ [36], [37], • Eirene version 1.3.6 in Julia, [38], [39], • GUDHI version 3.11.0 in Python, [40], [41], • Ripser version 1.2.1 in C++, [25], [42].
These are the open source tools that all implement the calculation of Vietoris–Rips complex from a point-cloud with respect to growing radius and the calculation of the persistent homology of the ifltration of simplicial complices so obtained. There are, however, certain diferences in their approach. Dionysus takes advantage of the fact that cohomology is easier to compute than homology, and exploits the duality between homology and cohomology. DIPHA is making the calculation using distributed computing, which is very beneficial for large and high-dimensional data. Eirene is relying on matroid
All
Std. dev.
Std. dev.
Std. dev.
Min.
Max. theory and discrete Morse theory. GUDHI employs new data structures for simplicial complices and their boundary matrices. Ripser is known as one of the best-performing tools due to its special optimizations. For a detailed account of existing tools for calculation of persistent homology see [26].
The experiment is made on the dataset of 120 point-clouds constructed in Section 3 from 60 time-series simulated as in Section 2. The calculations of persistent homology was done for degrees 0, 1, 2, 3 in homology. Higher degrees are computationally more demanding and would take quite long, at least on our equipment.
The tools for computation of persistent homology listed in Section 3 were deployed on a laptop running Linux Mint 21.2 (kernel 6.8.0), equipped with an AMD Ryzen 7 4800H processor (8 cores / 16 threads), 32 GB of DDR4 RAM at 3200 MHz, and two graphic cards (integrated AMD Radeon RX Vega 6 and a discrete NVIDIA GeForce RTX 3050 Mobile). All tools were executed via the Kitty terminal using the Zsh shell. The GPU was not utilized, as none of the tools support GPU acceleration. Each computation completed within a few seconds, and 32 GB of RAM was suficient for all processed point-clouds.
For each of 120 point-clouds the computational time in seconds for each of the five tools is recorded. The summary of the descriptive statistics of these times is given in Table 1, for point-clouds in all dimensions, and separately for points-clouds in dimension 5 and dimension 10. In Table 2, the summary of the descriptive statistics is given separately for point-clouds obtained from diferent SRGMs.
GUDHI — — — — — — — — — — — — — — — — — — — — — — — — — —
The descriptive statistics of computational times for all point-clouds in Table 1 reveals substantial diferences between considered tools. All statistics consistently indicate that Eirene is the best performing tool in terms of computational time, followed by GUDHI and then Ripser, while Dionysus is slightly better than DIPHA in the last place. The same conclusion also holds for results in the case of point-clouds in dimension 5 in Table 1. The results of Table 1 in dimension 10 show very similar behavior, except that DIPHA seems to be better than Dionysus in that case.
It is interesting to observe how all the means in Table 1 are considerably greater than medians, maxima are pretty large, and there is high variability in terms of standard deviation. This shows that there is a small portion of point-clouds with very high computational times, and the remaining large portion of point-clouds that are very quickly processed. Checking by hand which of the point-clouds have larger computational times than average, we discovered that all tools perform badly on the same point-clouds.
To explore this observation further, we compare in Table 2 the descriptive statistics separately for point-clouds arising from diferent SRGMs. According to these statistics, the computational times of all tools combined strongly depends on the SRGM from which a point-cloud is constructed. The tools perform best on point-clouds from Gompertz model, closely followed by delayed S-shaped, while they perform significantly slower in the case of Yamada model and the last placed Goel–Okumoto model. This is consistent with the worst point-clouds from computational time point of view observe above.
It is worth noticing in Table 1 that all the tools except DIPHA are faster on point-clouds of dimension 5 than on those of dimension 10. This is as expected, since computational complexity rises with growth of dimension, that is, with the width of the sliding window used in the construction of the point-cloud from a time-series. The reason why DIPHA exhibits the opposite behavior may lie in the fact that it is designed for distributed computing.
To support the conclusions made above based solely on descriptive statistics, we performed the -test of equal means between diferent samples of computational times. The -statistic and the corresponding -value for compared samples of five considered tools applied to all point-clouds are reported in Table 3, applied to point-clouds in dimension 5 in Table 4 and in dimension 10 in Table 5. On the other hand, the results of -test for samples of the same tool applied to point-clouds in dimension 5 and 10 are reported in Table 6. Finally, the -statistic and -value for samples arising from the application of all tools to point-clouds obtained from time-series of diferent SRGMs are reported in Table 7. In all these tables, if the null hypothesis of equal means is not rejected at significance level = 0.5, i.e., if the -value > 0.05, then the -value is written in boldface for the convenience of the reader.
Browsing through these tables shows that the only cases in which the null hypothesis is not rejected are in the case of comparison of Dionysus and DIPHA tool applied to all point-clouds, and the comparison of all tools applied to point-clouds arising from Goel–Okumoto and Yamada model. In both cases these are the worst performing models. For all other comparisons the null hypothesis can be rejected, thus providing evidence for our conclusions based on descriptive statistics.
One of the key aspects of software quality, especially in mission-critical systems, is its reliability. Topological data analysis through persistent homology is one of the modern approaches to the analysis of time-series. There are several tools for computing persistent homology of time-series. We analyzed ifve of them (Dionysus, DIPHA, Eirene, GUDHI, Ripser) from the point of view of software reliability.
More precisely, we analyzed computational time of diferent tools when applied to time-series datasets simulated to follow some of the classical software reliability growth models (Goel–Okumoto, Delayed S-shaped, Gompertz, Yamada) with parameters estimated from five industrial large-scale software development projects. The main conclusion is that the best performing tool is the latest version of Eirene, followed by GUDHI, and then Ripser. Dionysus and DIPHA performed substantially slower. However, we also observed high variability in computation time and a small portion of input data with very long computational times, so that the conclusions are not decisive.
It is interesting that the computational times of all tools strongly depend on the underlying software reliability growth model which the time-series follows. We also confirmed the expectation that computational times grow with higher dimensions.
This work was supported by the Croatian Science Foundation under the project number HRZZ-2022-104615.
During the preparation of this work, the author(s) used GPT-4o in order to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content. [11] M. Camilli, B. Russo, Modeling performance of microservices systems with growth theory, Empirical Software Engineering 27 (2022). URL: https://doi.org/10.1007/s10664-021-10088-0. doi:10. 1007/s10664-021-10088-0. [12] R. Rana, M. Staron, C. Berger, J. Hansson, M. Nilsson, F. Törner, W. Meding, C. Höglund, Selecting software reliability growth models and improving their predictive accuracy using historical projects data, Journal of Systems and Software 98 (2014). doi:10.1016/j.jss.2014.08.033. [13] J. Pita Costa, T. Galinac Grbac, The topological data analysis of time series failure data in software evolution, in: W. Binder, V. Cortellessa, A. Koziolek, E. Smirni, M. Poess (Eds.), Companion Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering, ICPE 2017, L’Aquila, Italy, April 22-26, 2017, ACM, 2017, pp. 25–30. URL: https://doi.org/10.1145/3053600. 3053604. doi:10.1145/3053600.3053604. [14] A. Micheletti, A new paradigm for artificial intelligence based on group equivariant non-expansive operators, European Mathematical Society Magazine (2023) 4–12. [15] R. Balestriero, A. Imtiaz Humayun, R. G. Baraniuk, On the geometry of deep learning, Notices of the American Mathematical Society 72 (2025) 374–385. [16] Erlangen AI Hub Team, Erlangen AI hub – mathematical foundations of intelligence, 2025. URL: https://erlangenhub.ox.ac.uk/. [17] S. H. Simon, Topological Quantum, Oxford University Press, Oxford, 2023. URL: https://doi.org/10.
1093/oso/9780198886723.001.0001. doi:10.1093/oso/9780198886723.001.0001. [18] C. Edwards, Tales of topological qubits, Communications of the ACM 66 (2023) 8–10. URL: https://doi.org/10.1145/3624436. doi:10.1145/3624436. [19] G. Carlsson, Topology and data, Bulletin of the American Mathematical Society (New Series) 46 (2009) 255–308. URL: https://doi.org/10.1090/S0273-0979-09-01249-X. doi:10.1090/ S0273-0979-09-01249-X. [20] P. Dłotko, On the shape that matters—topology and geometry in data science, European Mathematical Society Magazine (2024) 5–13. [21] H. Edelsbrunner, J. L. Harer, Computational Topology, American Mathematical Society, Providence,
RI, 2010. URL: https://doi.org/10.1090/mbk/069. doi:10.1090/mbk/069, an introduction. [22] J. R. Munkres, Elements of Algebraic Topology, Addison-Wesley Publishing Company, Menlo Park,
CA, 1984. [23] G. Carlsson, Persistent homology and applied homotopy theory, in: H. Miller (Ed.), Handbook of Homotopy Theory, Chapman and Hall/CRC, New York, 2020, pp. 297–329. doi:10.1201/ 9781351251624. [24] J. A. Perea, Topological times series analysis, Notices of the American Mathematical Society 66 (2019) 686–694. [25] U. Bauer, Ripser: eficient computation of Vietoris-Rips persistence barcodes, Journal of Applied and Computational Topology 5 (2021) 391–423. URL: https://doi.org/10.1007/s41468-021-00071-5. doi:10.1007/s41468-021-00071-5. [26] R. Ceccaroni, L. Di Rocco, U. Ferraro Petrillo, P. Brutti, A distributed approach for persistent homology computation on a large scale, The Journal of Supercomputing 80 (2024) 25510–25532. doi:10.1007/s11227-024-06374-5. [27] C. Stringfellow, A. A. Andrews, An empirical method for selecting software reliability growth models, Empirical Software Engineering 7 (2002) 319–343. doi:10.1023/A:1020515105175. [28] C. Andersson, A replicated empirical study of a selection method for software reliability growth models, Empirical Software Engineering 12 (2007) 161–182. doi:10.1007/s10664-006-9018-0. [29] A. L. Goel, K. Okumoto, Time-dependent error-detection rate model for software reliability and other performance measures, IEEE Transactions on Reliability 28 (1979) 206–211. [30] S. Yamada, M. Ohba, S. Osaki, S-shaped reliability growth modeling for software error detection,
IEEE Transactions on Reliability R-32 (1983) 475–484. doi:10.1109/TR.1983.5221735. [31] D. B. Kececioglu, Reliability Engineering Handbook, volume 2, Prentice Hall, Englewood Clifs, NJ, 1991. [32] S. Yamada, H. Ohtera, H. Narihisa, Software reliability growth models with testing-efort, IEEE
Transactions on Reliability 35 (1986) 19–23. doi:10.1109/TR.1986.4335332. [33] E. Puh, T. Galinac Grbac, N. Grbac, Preliminary study of higher dimensional software structures, in: Z. Budimac, V. Vranić, J. Lang (Eds.), Proceedings of the 10th Workshop on Software Quality Analysis, Monitoring, Improvement, and Applications, SQAMIA 2023, CEUR Workshop Proceedings, volume 3588, 2003, pp. 13–25. [34] T. Galinac Grbac, E. Puh, N. Grbac, Algorithms for sustainable system topologies (lecture notes, SusTrainable Summer School 2022, Univ. of Rijeka), Lecture Notes in Computer Science, vol. 15670, Springer, Cham, to appear, 2025. URL: http://tania.unipu.hr/$\sim$negrbac/2025_Sustrainable_SS_ 2022_LNCS_15670_to_appear.pdf. [35] D. Albrecht, Topology and data analysis with Dionysus, 2017. URL: https://github.com/nonabelian/ tda_dionysus. [36] H. Wagner, C. Chen, E. Vuçini, Eficient computation of persistent homology for cubical data, in: R. Peikert, H. Hauser, H. Carr, R. Fuchs (Eds.), Topological Methods in Data Analysis and Visualization II: Theory, Algorithms, and Applications, Springer, Berlin, Heidelberg, 2012, pp. 91– 106. URL: https://doi.org/10.1007/978-3-642-23175-9_7. doi:10.1007/978-3-642-23175-9_7. [37] DIPHA team, DIPHA tool, 2017. URL: https://github.com/DIPHA. [38] G. Henselman, R. Ghrist, Matroid filtrations and computational persistent homology, preprint (2017) 16pp. URL: https://arxiv.org/pdf/1606.00199. doi:10.48550/arXiv.1606.00199. [39] G. Henselman-Petrusek, Eirene: Julia library for homological persistence, 2016–2021. URL: https: //github.com/henselman-petrusek/Eirene.jl. [40] C. Maria, J.-D. Boissonnat, M. Glisse, M. Yvinec, The Gudhi library: Simplicial complexes and persistent homology, in: H. Hong, C. Yap (Eds.), Mathematical Software – ICMS 2014, Springer, Berlin, Heidelberg, 2014, pp. 167–174. [41] J.-D. Boissonnat, M. Carrière, M. Glisse, C. Maria, V. Rouvreau, A. Stamm (eds.), GUDHI – Geometry
Understanding in Higher Dimensions, 2014–2025. URL: https://gudhi.inria.fr/. [42] U. Bauer, Ripser: tool for eficient computation of Vietoris–Rips persistence barcodes, 2021. URL: https://github.com/Ripser.