=Paper=
{{Paper
|id=Vol-3027/paper26
|storemode=property
|title=Visualizing Generalized Computational Experiment State
|pdfUrl=https://ceur-ws.org/Vol-3027/paper26.pdf
|volume=Vol-3027
|authors=Alena Zakharova,Dmitriy Korostelyov,Aleksandr Podvesovskii,Vladimir Galaktionov
}}
==Visualizing Generalized Computational Experiment State==
https://ceur-ws.org/Vol-3027/paper26.pdf
Visualizing Generalized Computational Experiment State
Alena Zakharova 1,2, Dmitriy Korostelyov1,3, Aleksandr Podvesovskii1,3 and Vladimir
Galaktionov1
1
Keldysh Institute of Applied Mathematics Russian Academy of Sciences, Miusskaya sq., 4, Moscow, 125047,
Russian Federation
2
Institute of Control Sciences of Russian Academy of Sciences, 65, Profsoyuznaya st., Moscow, 117997, Russian
Federation
3
Bryansk State Technical University, 7, 50 let Oktyabrya blvd., Bryansk, 241035, Russian Federation
Abstract
The paper considers the problem of evaluating the state of a generalized computational
experiment in the context of a general problem of creating methods for adaptive planning and
control of a generalized computational experiment in mathematical modeling of real physical
processes. A generalized computational experiment implies multiple solution of the numerical
simulation problem for various sets of values of defining model parameters. As a method for
assessing a generalized computational experiment state, it is proposed to visualize a set of
experimental data specifying this state then followed by analysis of the resulting visual image.
An approach to visualization of a generalized computational experiment state is proposed
based on the sequential applying of two methods: visualization of a series of dependencies of
the output simulation parameters on the input ones for a given set of approximating functions
and visualization of approximation accuracy parameters for different ranges of values of the
input parameters. A description of each of these methods is given. Examples of their
application are considered when assessing the accuracy of numerical models of the
OpenFOAM software platform for a three-dimensional problem of inviscid flow around a cone.
Keywords1
Generalized computational experiment, generalized computational experiment state,
multidimensional data, visualization, visual analytics, approximation, problem of flow around
a cone, OpenFOAM.
1. Introduction
The study of mathematical models of physical processes, usually, involves a series of computational
experiments the purpose of which is to investigate the state of the model and its behavior in different
ranges of variation of simulation parameters. In this regard, the greatest interest often lies in
simultaneously investigating the influence of several parameters on relevant model characteristics,
including investigating their joint influence in various combinations of variation ranges. For this
purpose, construction of the so-called generalized computational experiment (GCE) is performed [1],
the main idea of which is to repeatedly solve a direct or inverse problem of numerical simulation for
different sets of values of simulation parameters. As noted in [1], such an approach allows for obtaining
a solution not for one but for a certain class of problems given in a multidimensional space of simulation
parameters.
Currently, there are examples of successful construction and application of a GCE in solving
problems of computational fluid dynamics [1-2], gas dynamics [3-4], power plant design
automation [5].
GraphiCon 2021: 31st International Conference on Computer Graphics and Vision, September 27-30, 2021, Nizhny Novgorod, Russia
EMAIL: zaawmail@gmail.com (A. Zakharova); nigm85@mail.ru (D. Korostelyov); apodv@tu-bryansk.ru (A. Podvesovskii);
vlgal@gin.keldysh.ru (V. Galaktionov)
ORCID: 0000-0003-4221-7710 (A. Zakharova); 0000-0002-0853-7940 (D. Korostelyov); 0000-0002-1118-3266 (A. Podvesovskii);
0000-0001-6460-7539 (V. Galaktionov)
©️ 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
From the point of view of representing the results, a generalized computational experiment is
characterized by a multidimensional array the elements of which specify the distribution of values of
simulation output parameters for a given set of input parameters. In this case, some generalized
indicators often act as output parameters, which are the results of processing primary experimental data,
contain information about general patterns and relationships inherent in the object of modeling, and are
used for interpretation, search for patterns, formation and testing of hypotheses. Examples of
generalized indicators are the principal components in dimension reduction problems [6], L1 and L2
error vector norms in problems of estimating the accuracy of various numerical methods with varying
the key simulation parameters [3-4], etc.
It is obvious that both conducting a GCE and processing and interpreting its results are very
resource-intensive tasks. Moreover, it is not possible to conduct an experiment with all allowable
combinations of models and simulation parameters. Therefore, it is necessary to resort to GCE planning
choosing a specific scenario for its implementation. At the same time, it is advisable to build a GCE on
the basis of not a static, predetermined, but a dynamic, adaptively changing plan. The principle of
constructing such a plan can be as follows: based on the results of a series of experiments for a certain
set of values of input parameters and processing of its results in conjunction with the results of previous
series of experiments, the current GCE state is recorded. This state must be evaluated and analyzed in
order to determine or adjust the plan for the subsequent series of experiments. To do this, it is necessary
to establish dependencies of the output simulation parameters on the input ones and on this basis to
select the value ranges of the input parameters for which more detailed studies are required with new
or refined sets of parameters. Such situations may arise, for example, when new patterns are discovered
that require confirmation and refinement, or if the results of some already conducted experiments do
not correspond to the expected patterns and thus require rechecking.
As a method for evaluating a GCE state, it is proposed to visualize the experimental data that define
it, followed by analysis of the resulting visual image. The approach based on visualization of
multidimensional data has proven itself well in the tasks of exploratory analysis and hypothesis
formation [7]. Visual analysis of a GCE state helps to visually and quickly enough detect and highlight
problematic or promising value ranges of input parameters that are subject to more detailed study.
A number of papers are devoted to the study of visualization problems in a generalized
computational experiment, among which, for example, [2, 3, 5] can be noted. At the same time, these
studies are mainly aimed at application of visualization methods for the analysis and interpretation of
experimental results. Visualization methods that could be used to evaluate a GCE state in order to clarify
the scenario for its implementation are currently out of consideration. This paper attempts to fill this
gap. Several methods of visualization of a GCE state are proposed, the application of which is
considered by the example of evaluating the accuracy of numerical models of the OpenFOAM software
platform [8] for a three-dimensional problem of inviscid flow around a cone.
2. Description of Visualization Methods for a Generalized Computational
Experiment State
We believe that within the framework of a GCE on a set of models M = {m1, m2, …, mNm), where
Nm is the number of models, Nk computational experiments were carried out within which the input
simulation parameters P = {p1, p2, …, pNp} were varied, where Np is the number of input simulation
parameters. As a result of the computational experiments, the output parameters S = {s1, s2, …, sNs}
were determined, where Ns is the number of output parameters. Each computational experiment k was
carried out for a fixed set of input simulation parameters Pk = {pk,1, pk,2, …, pk,Nk}. In this case, situations
are possible when, for a fixed set of simulation parameters, computational experiments were not
performed on all models. The results of the performed computational experiments form a set
R = {rk,m,v}, where k is the number of the computational experiment (1 ≤ k ≤ Nk ), m is the number of the
model (1 ≤ m ≤ Nm), v is the number of the output parameter (1 ≤ v ≤ Ns).
Let us visualize the state of the GCE using a series of two-dimensional approximating curves of the
results of computational experiments. Considering the fact that there can be several input simulation
parameters, let us apply the following algorithm (Fig. 1) to construct two-dimensional approximating
curves for one model m.
Start
Np, P, Ns, S, R, m
i := 1; v := 1
Definition of the set T and count Nti
d := 1
Fi,d,v, Nfi,d,v
j := 1
Selection of the corresponding results of computational experiments from the set R
and the values of the input parameter i
Formation of pairs of points (x; y) for approximation
Approximation of pairs of points (x; y) using the function fi,d,v,j
Determining the accuracy of the approximation – ei,d,v,j
j := j + 1
Yes No
d := d + 1 j > Nfi,d,v?
Yes No
v := v + 1 d > Nti?
No
v > Ns?
Yes
Visual analytics of graphs of approximating curves
i := i + 1; v := 1
No
i > Np?
Yes
End
Figure 1: Flowchart of visualization algorithm for a GCE state using a series of approximating graphs
1. Fix sequentially each input parameter pi. Values of this parameter determine the values of x
pairs (x; y) for which we will further carry out approximation.
2. Determine set T = {td} of all possible combinations of the remaining input parameters pj (p ≠ pi),
where 1 ≤ d ≤ Nti, Nti is the number of possible combinations of the remaining input parameters for
a fixed parameter pi.
3. For each such combination td, obtain Nsd dependences of the output parameters sv on the
parameter pi (it is assumed that not all combinations of input values could be obtained for all output
parameters).
4. Carry out construction of the functional dependence using approximating functions. To do this,
first determine the number – Nfi,d,v and the form of possible approximating functions
𝐹𝑖,𝑑,𝑣 = {𝑓𝑖,𝑑,𝑣,1 , 𝑓𝑖,𝑑,𝑣,2 , … , 𝑓𝑖,𝑑,𝑣,𝑁𝑓𝑖,𝑑,𝑣 } for each resulting parameter sv, a combination of the input
parameters d and a fixed input parameter i. The values of the resulting parameters sv
(the corresponding rk,m,v are selected from the set R) specify the values of y in pairs (x; y) for which
we will carry out the approximation.
5. For each approximating function fi,d,v,j for the resulting parameter sv and each combination td of
input parameters, a graph of functional dependence on the parameter pi construct and approximation
accuracy is determined ei,d,v,j.
6. Visually compare of graph shapes is carried out for a fixed parameter pi for different
approximating functions, and deviations and patterns are revealed. Among other things, shapes of
the curves obtained for different resulting parameters are compared.
7. Choose the following fixed input parameter: i = i + 1 and go to step 2.
8. If all the input parameters are exhausted, then the algorithm is completed.
The presented algorithm is repeated for each model.
The method based on application of this algorithm makes it possible to visually identify deviations
in the results of experiments and determine patterns; however, it does not reflect a GCE state as a whole,
since it forms not a single visual image but a series of visual images. To form a single visual image, it
is proposed to use the following method.
The obtained characteristics of approximation accuracy ei,t,v,j are summarized in tables: for each pair
of a fixed input parameter i and an output parameter v, we obtain one table, the columns of which are
approximating functions fj,v, and the rows are specific values of the remaining parameters. The cells of
this table are the corresponding values of approximation accuracy – ei,t,v,j. For each row of these tables,
let us define characterizing values by the following methods:
1. Minimum accuracy of approximation (one parameter).
2. Maximum accuracy of approximation (one parameter).
3. Average accuracy of approximation and root-mean-square deviation of different methods of
approximation from the mean (two parameters).
Visualization of the obtained characteristics will be carried out using two-dimensional dot plots. On
this graph, for each resulting parameter v, the corresponding points will be of the same color. For
different resulting parameters v, the colors will be different. We will also visualize different input
parameters i using different colors (different types of markers can be also used). The ordinal number of
the table rows will be used as the value of the point along the abscissa axis in a two-dimensional
visualization. The characterizing values will be used as values along the ordinate axis. When using the
3rd method for determining the characteristic values, the average accuracy will determine the value
along the ordinate axis, and the root-mean-square deviation will determine the size of the point.
Thus, on one two-dimensional dot plot, we can reflect a GCE state for one specific model. Analyzing
the relative position (and the size of the points for the third method of determining the characteristic
values) of the points, it is possible to visually determine for which input parameters there are problems
with the approximation accuracy, and as a consequence, errors in carrying out computational
experiments are possible, and for which it is not yet possible to determine the best approximation
methods and, consequently, determine the value ranges of the input parameters for which it makes sense
to carry out additional computational experiments in intermediate values.
3. Experiment Description
Let us consider application of the proposed visualization methods for a GCE carried out within
evaluating the accuracy of solvers of the OpenFOAM platform (in OpenFOAM terminology, solvers
are software modules in which various numerical models of mechanics of continua are
implemented [8]) for the three-dimensional problem of inviscid flow around a cone [4, 9] (Fig. 2).
Solvers rhoCentralFoam, pisoCentralFoam, sonicFoam will be considered as models M.
Figure 2: The density distribution and the streamline on the cone surface in a supersonic flow angle
of attack
The simulation input parameters (P) are Mach number (Ma), cone half-angle (Betta, in degrees), and
angle of attack (Angle, in degrees). The output parameters (S) of computational experiments are: the
results of calculating deviation norms L1 and L2 of the numerical calculation from the analytical
solution.
As approximating functions, we will use the following:
linear (y = ax + b);
exponential (y = aebx);
logarithmic (y = aln(x) + b;
quadratic (y = ax2 + bx + c).
As the approximation accuracy, we will use the value of the approximation reliability R2 [10].
As a result of the approximation of the obtained data, 178 approximating functions were constructed
for each solver, considering the fact that some types of approximations in specific cases could not be
carried out. For example, for the quadratic approximation, in the case of only two points, the graph was
expressed in a line and therefore was not taken into account. Also, for the logarithmic function, it was
impossible to determine approximating functions for the cases when the value of the parameter x could
be equal to 0. The visualization results of the obtained approximating functions are partially shown in
Fig. 3-5. For convenience, one graph displays the curves for all output parameters (norms L1 and L2).
Figure 3: An example of visualization of approximating functions for rhoCentralFoam solver with
a fixed input parameter – angle of attack
Figure 4: An example of visualization of approximating functions for psioCentralFoam solver with
a fixed input parameter – cone half-angle
Figure 5: An example of visualization of approximating functions for sonicFoam solver with a fixed
input parameter – Mach number
After constructing a series of diagrams and determining the approximation accuracy, their mean
values and root-mean-square deviations were calculated. Bubble charts were used for visualization, a
different bubble color means belonging to different data series (a combination of a fixed input parameter
and an output parameter), and the size of a point (bubble) characterizes the root-mean-square deviation
of the approximation accuracy by different types of curves. Fig. 6 shows thus obtained visual GCE state
diagram for sonicFoam solver. This diagram summarizes the results of evaluating the accuracy of the
given solver by representing them as a single visual image.
Figure 6: Visualization of the GCE state of sonicFoam solver
4. Discussion of Experimental Results
Carrying out a visual analysis of the constructed approximating functions, it can be noted that, in
most cases, the curve shapes for norms L1 and L2 are similar for a fixed value of the input parameter
and the corresponding combinations of the remaining input parameters. However, in some cases
deviations are observed. In particular, for sonicFoam solver (Fig. 5) with a fixed input parameter –
Mach number and cone half-angle – 10° and angle of attack – 5°, a significantly greater curvature of the
approximating curve for the L1 norm is observed, as well as the osculation of curves for the L1 and L2
norms. Similar anomalies were also observed for the same solver and the L1 norm in two other cases:
1. Fixed input parameter – angle of attack, Mach number – 5, half-angle – 10°.
2. Fixed input parameter – half-angle, Mach number – 5, and angle of attack – 5°.
These anomalies suggest that at Mach number of 5, half-angle of 10°, and angle of attack of 5° for
sonicFoam solver, an error could have been made in the computational experiment or in the calculation
of the L1 norm. Further clarification of this fact by the authors of the computational experiment
confirmed the presence of a technical error related to tabulating the results of the computational
experiments for the given combination of input parameters – the corresponding value of the L2 norm
fell into the table cell for the L1 norm by mistake.
Analyzing the diagram of the GCE state visualization for sonicFoam solver (Fig. 6), it can also be
noticed that for 3 cases the applied approximation methods give rather different results in terms of flow
(3 large bubbles) and all of them are for the L1 norm:
1. Fixed input parameter – Mach number, half-angle – 10°, angle of attack – 5°.
2. Fixed input parameter – half-angle, Mach number – 5, angle of attack – 5°.
3. Fixed input parameter – angle of attack, Mach number – 7, half-angle – 20°.
The first two cases correspond to an already discovered problem by visualizing the approximating
curves. The third case is typical for other computational experiments and may indicate the need for an
additional series of experiments for intermediate values of the corresponding parameters in order to
clarify the nature of the dependence and, possibly, to correct the list of types of approximating curves
for sonicFoam solver.
5. Conclusion
The paper considers the problem of evaluating the state of a generalized computational experiment
and the method for its solution using visualization of a set of experimental data that determines the GCE
state, followed by analysis of the resulting visual image. An approach is proposed to visualize the GCE
state based on the sequential applying of two methods: visualization of a series of dependencies of the
output simulation parameters on the input ones for a given set of approximating functions and
visualization of approximation parameters for different value ranges of the input parameters.
Due to the use of the proposed visualization methods, it was been possible to identify experiments
that have signs of errors as well as the value ranges of the input parameters for which it is advisable to
conduct additional experiments for intermediate values. These circumstances make it possible to correct
the further plan of conducting computational experiments.
In combination with other methods for verifying GCE data [11], the presented methods can be of
great help to researchers in planning and performing computational experiments. Their software
implementation will allow creating a reliable and efficient visualization tool that can be used in a wide
range GCEs.
It should also be noted that application of the above visualization methods can only be possible with
a sufficient number of already performed computational experiments within a GCE, since to construct
the approximating curves, at least 2 points are required with a fixed input parameter.
6. Acknowledgements
The research has been supported by Russian Science Foundation (project No. 18-11-00215).
7. References
[1] A.E. Bondarev, On the Construction of the Generalized Numerical Experiment in Fluid Dynamics.
Mathematica Montisnigri XLII, 52–64.
[2] A.E. Bondarev, V.A. Galaktionov, Generalized Computational Experiment and Visual Analysis of
Multidimensional Data. Scientific Visualization, 11 (4), 102–114 (2019). doi: 10.26583/sv.11.4.09
[3] A. Alekseev, A. Bondarev, V. Galaktionov, A. Kuvshinnikov, L. Shapiro, On Applying of
Generalized Computational Experiment to Numerical Methods Verification, in: CEUR Workshop
Proceedings of GraphiCon 2020, vol. 2744 (2020). doi: 10.51130/graphicon-2020-2-3-19
[4] A.E. Bondarev, A.E. Kuvshinnikov, Analysis of the Accuracy of OpenFOAM Solvers for the
Problem of Supersonic Flow Around a Cone. in: Shi, Y. et al. (eds.) ICCS 2018, LNCS, vol. 10862,
Springer, Cham, 2018, pp. 221–230. doi: 10.1007/978-3-319-93713-7_18
[5] S.V. Andreev, et al., A Computational Technology for Constructing the Optimal Shape of a Power
Plant Blade Assembly Taking into Account Structural Constraints. Programming and Computer
Software, 43(6), 345–352 (2017). doi: 10.1134/S0361768817060020
[6] A.N. Gorban, B. Kegl, D. Wunsch, A.Y. Zinovyev (eds.), Principal Manifolds for Data
Visualisation and Dimension Reduction, Springer-Verlag, Berlin Heidelberg, 2007. doi:
10.1007/978-3-540-73750-6
[7] A.A. Zakharova, A.G. Podvesovskii, A.V. Shklyar, Visual and Cognitive Interpretation of
Heterogeneous Data. in: Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-2/W12,
2019, pp. 243-247. doi: 10.5194/isprs-archives-XLII-2-W12-243-2019
[8] OpenFOAM. Free CFD Software. The OpenFOAM Foundation. URL: https://openfoam.org.
[9] S.V. Andreev, A.E. Bondarev, N.A. Bondareva, Stereoscopic Construction of Textual Information
in Presentations of Research Results on an Autostereoscopic Monitor. Scientific Visualization,
12(1), 132–139 (2020). doi: 10.26583/sv.12.1.12
[10] B.H. Baltagi, Econometrics. 5th ed., Springer-Verlag, Berlin Heidelberg, 2011. doi: 10.1007/978-
3-642-20059-5
[11] A.A. Zakharova, D.A. Korostelyov, Application of Visual Analytics Methods to Reduce the
Dimensionality of Decision-making Problems. Scientific Visualization, 12(4), 23–32 (2020). doi:
10.26583/sv.12.4.03