=Paper=
{{Paper
|id=Vol-2763/CPT2020_paper_s6-4
|storemode=property
|title=Generalized computational experiment in the problems of numerical methods verification
|pdfUrl=https://ceur-ws.org/Vol-2763/CPT2020_paper_s6-4.pdf
|volume=Vol-2763
|authors=A.K. Alekseev,A.E. Bondarev,A.E. Kuvshinnikov
}}
==Generalized computational experiment in the problems of numerical methods verification==
https://ceur-ws.org/Vol-2763/CPT2020_paper_s6-4.pdf
Generalized computational experiment in the problems of numerical
methods verification
A.K. Alekseev1, A.E. Bondarev1, A.E. Kuvshinnikov1
aleksey.k.alekseev@gmail.com | bond@keldysh.ru | kuvsh90@yandex.com
1
Keldysh Institute of Applied Mathematics RAS, Moscow, Russia;
This work is devoted to the application of a generalized computational experiment for a comparative assessment of numerical
methods accuracy. The construction of a generalized computational experiment is based on the simultaneous solution using parallel
computations in a multitasking mode of a basic problem with different input parameters, obtaining results in the form of multidimensional
data volumes and their visual analysis. This approach can be effective in problems of verification of numerical methods. A comparative
assessment of the accuracy for solvers of the open software package OpenFOAM is carried out. The classic inviscid problem of oblique
shock wave is used as a basic task. Variations of the key parameters of the problem — the Mach number and angle of attack — are
considered. An example of constructing error surfaces is given when the solvers of the OpenFOAM software package are compared.
The concept of an error index is introduced as an integral characteristic of deviations from the exact solution for each solver in the class
of problems under consideration. The surfaces of deviations from the exact solution in the L2 norm, constructed for each solver, together
with the calculated error indices, make it possible to obtain a complete picture of the accuracy of the solvers under consideration for
the class of problems defined by the ranges of variation of the Mach number and angle of attack.
Keywords: generalized computational experiment, numerical methods, verification problems, error index
The concept, basic methods and approaches of a
1. Introduction generalized computational experiment, as well as a
The tasks of numerical methods verification have number of software tools for its implementation were
always been of paramount importance throughout the developed in Keldysh Institute of Applied Mathematics
history of the development of computational mathematics. RAS. The main aspects of constructing a generalized
Today, verification problems are of particular importance computational experiment and examples of its
in the problems of computational gas dynamics. In implementation are described in detail in [4–11].
mathematical modeling of practical problems in This work is devoted to the application of a generalized
aerodynamics, the researcher must be sure of the accuracy computational experiment for a comparative assessment of
of the numerical method used. numerical methods accuracy.
A comparative assessment of numerical methods
2. Generalized computational experiment
accuracy is of particular importance at present. This is due
to the wide distribution of software packages, both open The emergence of the concept of a generalized
and commercial, allowing to solve a wide range of computing experiment is associated with the development
problems. As a rule, a large number of numerical methods of high-performance computing clusters and parallel
implemented in various solvers are integrated into such technologies. In problems of computational aerodynamics,
packages. When solving practical problems, it is not easy parallel technologies usually provide the ability to quickly
for the user to choose the most suitable solver for the calculate on detailed grids. However, parallel technologies
studied class of problems. provide us with another important opportunity. This is the
The relevance of verification tasks is confirmed by the ability to simultaneously calculate on different nodes the
introduction in 2018 of the Federal Standard for the same task with different input data. As a rule, such a
numerical simulation of supersonic inviscid gas flows and calculation is performed in multitasking mode.
software verification [1]. Similar foreign standards have This opens up the possibility of implementing a
been around for quite some time [2,3]. Such standards will generalized computational experiment. The key
determine the direction of research in this area over the parameters of the problem under consideration are divided
next decade. However, all these methodological in certain ranges with a certain step, forming a grid
documents are focused on verification in relation to a partition of a multidimensional box in a multidimensional
specific task with fixed values of key parameters. space of key parameters. The basic problem is solved
At the present stage, researchers need more using parallel technologies at each point of the grid
comprehensive estimates of the accuracy of numerical partition. The obtained results represent multidimensional
methods. For example, in assessing accuracy, not for a data volumes. Processing, analysis and visual presentation
single task, but for a class of tasks. By a class of tasks is of this data is carried out using methods of visual analytics
meant a basic task considered in the ranges of change in and scientific visualization. This computing technology is
the set of key parameters. Such parameters in the most general description of a generalized computing
computational aerodynamics can serve characteristic experiment.
numbers that determine flow velocity, viscosity, Obviously, such a concept can be applied to a wide
thermophysical properties of the medium, geometric range of tasks. This range includes parametric studies,
parameters, etc. An opportunity of getting solution for a optimization problems. A generalized computational
class of problems is provided by the construction of a experiment is an effective tool for solving inverse
generalized computational experiment. problems.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY
4.0)
A large number of different applications of a cases, approaches to lowering the dimension of the
generalized computational experiment are described in multidimensional space of key parameters under
detail in [4-14]. The concept of a generalized consideration are useful.
computational experiment was applied to a wide range of A separate problem is the estimation of the accuracy of
both model and practical problems. numerical methods in the absence of a reference solution.
These tasks include the analysis of the interaction of a Here, to assess the accuracy, foreign standards recommend
viscous supersonic flow with a jet barrier, the flows in the to apply Richardson method [2,3]. However, for practical
wake of the body, the problems of the interaction of jets, problems of computational aerodynamics this is very
the problem of flowing around a cone at an angle of attack, difficult due to the enormous computational costs. The
the problem of oblique shock waves, and many others. The computational costs are due to the fact that the
approach to constructing a generalized computational implementation of Richardson method requires multiple
experiment was applied to the problem of finding the calculations with a decrease in the step of the spatial grid
optimal three-dimensional shape of the blades assembly decomposition. One of the alternatives in this case is the
for a power plant in terms of power loads. estimation of accuracy on the ensemble of solutions. The
Also, this approach was applied to the problems of ensemble of solutions obtained by various numerical
verification of numerical methods. A comprehensive methods on the same grid allows us to estimate the
comparative analysis of a number of solvers of the location of the exact solution and to divide the obtained
OpenFOAM open software package [15] was carried out numerical solutions into clusters of different levels of
in [12–14, 20, 21]. As basic tasks, we used problems that accuracy. This direction is being actively developed at
have a reference solution (exact solution or experimental present and is presented in [16-19]. A natural drawback of
data). These tasks include the problem of a supersonic this approach is the need for researcher to have at his
inviscid flow around a cone at an angle of attack and the disposal a certain number of solvers that implement
problem of an oblique shock wave formation. In both numerical methods with different computational
cases, a class of problems was considered, formed by key properties.
parameters variations of the problem in question.
4. The example of verification problem
3. The problems of numerical methods
This section provides an example of constructing a
verification
generalized computational experiment for a comparative
As already mentioned above, verification problems assessment of numerical methods accuracy. As an
have been an important section throughout the history of example, we use the numerical results described in detail
the development of computational mathematics and in the authors' works [20,21]. In these papers, a class of
mathematical modeling. As a rule, a comparison of the computational gas dynamics problems is considered that
numerical results was carried out with some reference describe the incidence of an inviscid supersonic gas flow
solution, in the role of which the exact solution was used onto a flat plate at an angle of attack.
if available or the available experimental data. With such an incidence, an oblique shock wave is
If there is a reference solution, the accuracy of the formed. The Mach number and angle of attack are used as
numerical method can be estimated for the solution key parameters. These values vary in certain ranges. This
element or for the entire calculation domain. For example, problem has an exact solution. With the exact solution, a
for problems containing discontinuities (shock waves), comparison is made at each point of the calculation
previously, the width of the “smearing” of the solution at domain, and for each combination of key parameters, an
discontinuity was traditionally considered as a error is evaluated in the norm of L1 and L2. The results
characteristic of the numerical method accuracy. In this obtained make it possible to construct an error surface as
case, a comparison with the reference solution over the an error function of two key parameters of the problem.
entire flow field was also applied. For an objective Carrying out similar calculations for several numerical
assessment of numerical method accuracy, it seems methods implemented in the solvers of the open software
appropriate and reliable to apply both approaches. In the package OpenFOAM, makes it possible to build several
presence of a reference solution, the construction of a such surfaces on one drawing. This opens up the
generalized computational experiment allows us to possibility of a deep and clear comparative analysis of the
compare not only for one problem with fixed key accuracy of the studied numerical methods. The
parameters, but also for problems in the entire field of construction of such a generalized computational
variation of key parameters. experiment involves the creation of computational
If the class of problems is determined by two key technology from solving a direct problem up to visual
parameters, then for each numerical method involved in analysis of the results. One of the most expressive and
the comparison, the dependence of the error on these visual forms of visualization is the construction of stereo
parameters is constructed in the form of an error surface. animations of the results of numerical studies. A similar
In the case of three key parameters, scientific visualization construction of stereo images for this task was carried out
methods are used to analyze a three-dimensional figure and described in [22].
representing the dependence of the error on key Fig. 1 presents the results of constructing error surfaces
parameters. In the case when the number of key for four OpenFOAM solvers with variations in the Mach
parameters is more than three, then methods of visual number from 2 to 4 and variations in the angle of attack
analytics should be used to analyze the results. In some from 6 to 20 degrees [21]. It should be noted that error
surfaces for the class of problems for the comparative that the best accuracy in the class of problems is provided
analysis of the accuracy of numerical methods were by the rCF and pCF solvers, for which the error surfaces
constructed in [21] for the first time. Four error surfaces almost coincide.
for OpenFOAM solvers are presented - rhoCentralFoam Thus, the construction of a generalized computational
(rCF), pisoCentralFoam (pCF), sonicFoam (sF) and experiment allows us to conduct a full-fledged
QGDFoam (QGDF). comparative accuracy assessment for four solvers of the
These surfaces allow a thorough visual comparison of OpenFOAM software package in the class of problems.
deviations from the exact solution in the class of problems The class of tasks in this particular case is determined by
under consideration. It can be seen that all 4 surfaces the basic task (oblique shock wave) and the ranges of
behave in a very similar way. The deviation from the exact variation of the key parameters of the problem - the Mach
solution increases with the growth of key parameters - the number and angle of attack.
angle of attack and the Mach number. Fig. 1 also shows
Fig. 1. Image of the surface deviation from the exact solution for 4 OpenFOAM solvers with variation of the Mach number and angle
of attack for the oblique shock wave [21]
The image of error surfaces presented in Figure 1 gives the accuracy of numerical methods is considered. An
a fairly clear idea of the comparative accuracy of example of constructing a generalized computational
OpenFOAM solvers in the class of problems. However, experiment for a class of problems is described. The class
for a more complete assessment, an integral characteristic of tasks is formed on the basis of the basic problem (the
for each surface can be introduced. oblique shock wave) and variations of the determining
We call this characteristic the Error Index (EI). parameters of the problem - the Mach number and angle
The error index is defined as follows. Let i = 1, M and of attack. An example of constructing error surfaces is
j = 1, N be the grid partitions of key parameters, and Aij - given. The concept of a numerical method error index for
the deviation from the exact solution at each point of the a class of problems is introduced.
grid partition. Then the error index is defined as: The construction of a generalized computational
𝐸𝐸𝐸𝐸 = � 𝐴𝐴𝑖𝑖𝑖𝑖 �(𝑀𝑀 ∗ 𝑁𝑁). experiment can serve as an effective tool for verification
problems.
𝑖𝑖𝑖𝑖
Then the error index values for the surfaces shown in Acknowledgments
Fig. 1 can be calculated and written in the form of table 1.
This work was supported by RFBR grants 19-01-
Table 1. Error Index Values for 4 OpenFOAM Solvers 00402 and 20-01-00358.
Solver rCF pCF QGDFOAM sF
Error Index 0.037734 0.038751 0.0453406 0.058216 References
[1] Federal standard Р 57700.12–2018. Numerical
Table 1 shows that the values of the error index EI simulation of supersonic flows for an inviscid gas.
completely correspond to the relative positions of the Software verification / National standard of the
surfaces in Figure 1. Therefore, the calculated error index Russian Federation for numerical modeling of
can serve as a characteristic of the accuracy of numerical physical processes. 2018, 20 p.
methods in the selected class of problems. [2] Guide for the Verification and Validation of
Computational Fluid Dynamics Simulations,
5. Conclusion
American Institute of Aeronautics and Astronautics,
The application of a generalized computational AIAA-G-077-1998, Reston, VA, 1998.
experiment to the problems of comparative estimation of
[3] Standard for Verification and Validation in "Mathematica Montisnigri", Vol. XXXVIII, 2017, p.
Computational Fluid Dynamics and Heat Transfer, 63-77.
ASME V&V 20-2009, 2009. [17] A.K. Alekseev, A.E. Bondarev, A.E. Kuvshinnikov.
[4] Bondarev A.E. Analysis of Space-Time Flow Verification on the Ensemble of Independent
Structures by Optimization and Visualization Numerical Solutions // Lecture Notes in Computer
Methods // Transactions on Computational Science Science, Vol. 11540, pp. 315–324, 2019. DOI:
XIX, LNCS 7870, pp. 158-168. Springer, Heidelberg 10.1007/978-3-030-22750-0_25
(2013) [18] Alekseev A.K., Bondarev A.E. Estimation of the
[5] Bondarev A.E, Galaktionov V.A. Parametric Distance between True and Numerical Solutions //
Optimizing Analysis of Unsteady Structures and Computational mathematics and mathematical
Visualization of Multidimensional Data // physics, 2019, V. 59, № 6, с. 857-863 DOI:
International Journal of Modeling, Simulation and 10.1134/S0965542519060034
Scientific Computing, 2013, V.04, N supp01, 13 p., [19] A.K. Alekseev, A.E. Bondarev, A.E. Kuvshinnikov.
DOI 10.1142/S1793962313410043 On uncertainty quantification via the ensemble of
[6] Bondarev A.E. On the Construction of the independent numerical solutions // Journal of
Generalized Numerical Experiment in Fluid Computational Science 42 (2020) 101114, DOI:
Dynamics // Mathematica Montisnigri, Vol. XLII, 10.1016/j.jocs.2020.101114
2018, p. 52-64 [20] Alekseev A.K., Bondarev A.E., Kuvshinnikov A.E.
[7] A.E. Bondarev. On visualization problems in a Comparative analysis of the accuracy of OpenFOAM
generalized computational experiment (2019). solvers for the oblique shock wave problem //
Scientific Visualization 11.2: 156 - 162, DOI: Matematica Montisnigri, 2019, vol. XLV, p. 95-105
10.26583/sv.11.2.12 DOI: 10.20948/mathmontis-2019-45-8
[8] Alexander E. Bondarev and Artem E. Kuvshinnikov. [21] Bondarev A., Kuvshinnikov A. Parametric Study of
Analysis of the Accuracy of OpenFOAM Solvers for the Accuracy of OpenFOAM Solvers for the Oblique
the Problem of Supersonic Flow Around a Cone // Shock Wave Problem // IEEE The Proceedings of the
Lecture Notes in Computer Science 10862, pp. 221– 2019 Ivannikov ISPRAS Open Conference 2019. P.
230, 2018. https://doi.org/10.1007/978-3-319-93713- 108-112, DOI:10.1109/ISPRAS47671.2019.00023
7_18 [22] S.V. Andreev, A.E. Bondarev, N.A. Bondareva.
[9] A.E. Bondarev. On the Estimation of the Accuracy of Stereo images of error surfaces in problems of
Numerical Solutions in CFD Problems // Lecture numerical methods verification (2020). Scientific
Notes in Computer Science, Vol. 11540, pp. 325–333, Visualization 12.2: 151 - 157, DOI:
2019. DOI: 10.1007/978-3-030-22750-0_26 10.26583/sv.12.2.12
[10] A.E. Bondarev, V.A. Galaktionov. Construction of a
generalized computational experiment and visual About the authors
analysis of multidimensional data // CEUR Workshop Alekseev Aleksey K., Doctor in physics and mathematics,
Proceedings, V. 2485, 2019, p. 117-121. DOI: senior researcher, Keldysh Institute of Applied Mathematics
10.30987/graphicon-2019-2-117-121 RAS. E-mail: aleksey.k.alekseev@gmail.com.
[11] A.E. Bondarev, V.A. Galaktionov. Generalized Bondarev Alexander E., PhD, senior researcher, Keldysh
Computational Experiment and Visual Analysis of Institute of Applied Mathematics RAS. E-mail:
Multidimensional Data (2019). Scientific bond@keldysh.ru.
Visualization 11.4: 102 - 114, DOI: Kuvshinnikov Artem E., junior researcher, Keldysh Institute
of Applied Mathematics RAS. E-mail: kuvsh90@yandex.ru
10.26583/sv.11.4.09
[12] Bondarev A.E., Kuvshinnikov A.E. Comparative
study of the accuracy for OpenFOAM solvers //
Proceedings of Ivannikov ISPRAS Open Conference
(ISPRAS), 2017, IEEE, IEEE Xplore, p.132-136
DOI: 10.1109/ISPRAS.2017.00028
[13] Bondarev A.E., Kuvshinnikov A.E. Analysis of the
accuracy of OpenFOAM solvers for the problem of
supersonic flow around a cone // Lecture Notes in
Computer Science, vol. 10862, pp. 221–230, 2018.
https://doi.org/10.1007/978-3-319-93713-7_18.
[14] Bondarev A., Kuvshinnikov A. Comparative
Estimation of QGDFoam Solver Accuracy for
Inviscid Flow Around a Cone // IEEE The
Proceedings of the 2018 Ivannikov ISPRAS Open
Conference (ISPRAS-2018). P. 82-87,
https://doi.org/10.1109/ISPRAS.2018.00019
[15] OpenFOAM Foundation. [Online]. Available:
http://www.openfoam.org.
[16] Alekseev A.K., Bondarev A.E. On exact solution
enclosure on ensemble of numerical simulations //