<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Simulation of Thermal Processes in Permafrost: Parallel Implementation on Multicore CPU</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Elena N. Akimova</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mikhail Yu. Filimonov</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Vladimir E. Misilov</string-name>
          <email>v.e.misilov@urfu.ru</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nataliia A. Vaganova</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Krasovskii Institute of Mathematics and Mechanics, Ural Branch of RAS</institution>
          ,
          <addr-line>Ekaterinburg</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Ural Federal University</institution>
          ,
          <addr-line>Ekaterinburg</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The processes of seasonal changes in upper layers of permafrost soil are described. A thermal conductivity equation is considered and algorithm is presented for solving it. An approach of computational optimization is suggested. The proposed parallel algorithm is implemented for multicore processor using OpenMP technology.</p>
      </abstract>
      <kwd-group>
        <kwd>thermal conductivity</kwd>
        <kwd>numerical simulation</kwd>
        <kwd>permafrost</kwd>
        <kwd>parallel computing</kwd>
        <kwd>MPI</kwd>
        <kwd>OpenMP</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Permafrost occupies 35 million km2 (about 25% of the total land area of the
world [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ]. In Russia, more than 60% of its territory is occupied by permafrost,
and 93% natural gas and 75% oil are produced in these regions. Human
activity, production and transportation of oil and gas have a signi cant e ect on
permafrost. The warm oil heated the pipes in wells and pipelines and other
processes may lead to permafrost degradation. Therefore, the problem of reducing
the intensity of thermal interactions in the \heat source | permafrost" zones
has a particular importance for solving problems of energy saving,
environmental protection, safety, cost savings, and enhance operational reliability of various
engineering structures [
        <xref ref-type="bibr" rid="ref3 ref4 ref5 ref6">3, 5, 4, 6</xref>
        ].
      </p>
      <p>
        We consider new three-dimensional model which allows one to describe the
heat distribution in upper layers in permafrost soils with taking into
consideration not only most signi cant climatic factors (seasonal changes in temperature
and intensity of solar radiation due to geographic location of the eld) and
physical factors (di erent thermal characteristics of non-uniform ground that
change over the time), but also engineering construction features of the
production wells, and other types of technical systems such as ripraps, tanks, pipelines,
are systems, and others [7{10]. Considered numerical algorithms are justi ed
and approved on the base of data for 12 Russian oil and gas elds. When the
source of heat in the frozen ground was a well, a comparison was made between
numerical data on the distribution of the boundary of the melting of frozen
ground (zero isotherm) and experimental data. The detailed calculations of the
long-term prediction of the permafrost boundary changes demand considerable
computing power, the parallel approach to solving such problems is also
required [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. The complete simulation of all technical systems located in a well
pad makes it necessary to solve such problems in a signi cally larger area with a
three-dimensional computational grid. This leads to an essential encreasing time
of computations (up to 100 hours).
      </p>
      <p>In numerical simulations with using nite-di erence methods, an original
problem is often reduced to numerical solution of systems of linear algebraic
equations (SLAE). In papers [12{15], for solving SLAE with the block-three- and
block- vediagonal matrices, direct and iterative parallel algorithms are designed.
Those are: the conjugate gradient method with preconditioner, the conjugate
gradient method with regularization, the square root method, and the parallel
matrix sweep algorithm. In papers [16{18], the iterative gradient methods were
used for solving SLAE with dense matrices arising in inverse gravity problems.
These algorithms are implemented on multi-core processor with good e ciency.</p>
      <p>THERMAL EXCHANGE</p>
      <p>SOLAR RADIATION</p>
      <p>EMISSIVITY
THERMAL CONDUCTIVITY</p>
      <p>PHASE TRANSITION</p>
      <p>However, the considered model of heat distribution is assumed a nonlinear
boundary condition of heat uxes balance at the soil{air boundary that increases
the computational comlexity and does not allow the direct application of the
parallel methods of SLAE solving. In this paper, an appoach of e ective code
implementation of a splitting method by spatial variables is presented.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Statement of problem</title>
      <p>
        Simulation of processes of heat distribution in the permafrost soil is reduced to
solution of a three-dimensional di usivity equation with non-uniform coe cients.
The equation includes the localized heat of phase transition; i.e., an approach to
solve the problem of the Stefan type without the explicit separation of the phase
transition in 3D area (Fig. 1) [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. The equation for the temperature T (t; x; y; z)
at the instant t at the point (x; y; z) has the form
c (T ) + k (T
      </p>
      <p>T )
with initial condition</p>
      <p>T (0; x; y; z) = T0(x; y; z):</p>
      <p>Here, is the density [kg/m3], T is the temperature of the phase
transition [K],
(1)
(2)
c (T ) =
(T ) =
c1(x; y; z); T &lt; T ;
c2(x; y; z); T &gt; T
1(x; y; z); T &lt; T ;
2(x; y; z); T &gt; T
is the speci c heat [J/(kg K)];
is the thermal conductivity coe cient [W/(m K)];
k = k(x; y; z) is the speci c heat of phase transition, is the Dirac delta function.</p>
      <p>Balance of the heat uxes at the surface z = 0 de nes the corresponding
nonlinear boundary conditions
q + b(Tair</p>
      <p>T (x; y; 0; t)) = " (T 4(x; y; 0; t)</p>
      <p>Ta4ir) +
:
(3)</p>
      <p>
        To determine the parameters in the boundary condition (3), an iterative
algorithm is developed that takes into account the geographic coordinates of the
area, lithology of soil, and other features of the considered location [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Method for solving the problem</title>
      <p>
        An e ective scheme of through computations with smoothing the discontinuous
coe cients was developed for the equation of thermal conductivity by
temperature in the neighborhood of the phase transformation. This scheme is applied
to numerical simulations. On the basis of ideas in [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], a nite di erence method
is used to solve the problem with splitting by the spatial variables in
threedimensional domain. We construct an orthogonal grid, uniform, or condensing
near the ground surface or to the surfaces of internal boundaries.
      </p>
      <p>
        The original equation for each spatial direction is approximated by a locally
additive implicit pattern, and to solve SLAE, a combination of the sweep and
Newton method is used. At the upper boundary z = 0, there is an algebraic
equation of the fourth degree, which is solved by the Newton method. Solvability
of the same di erence problems approximating (1){(3) is proved in [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <sec id="sec-3-1">
        <title>INPUT</title>
        <sec id="sec-3-1-1">
          <title>Preparation of parameters of the domain, constructions, and soil</title>
        </sec>
        <sec id="sec-3-1-2">
          <title>Computation of the temperature</title>
          <p>field at the instant t at the mesh
nodes: comp_on_mesh (59,92%)</p>
        </sec>
      </sec>
      <sec id="sec-3-2">
        <title>OUTPUT</title>
        <p>A basic scheme of the computational algorithm is presented in Fig. 2</p>
        <p>Let us consider an approach to construction of the parallel algorithm for
multicore CPU using OpenMP technology.</p>
        <p>To analyse the serial program, the Visual Studio 2017 Performance Pro ler
tool was used. The results presented in Fig. 3 show that the most time-costly
procedures are computation of the temperature eld `comp on mesh()` and
preparation of the domain parameters ` zad teplo z grunt ()` and
` zad teplo z skv ()`.</p>
        <p>The temperature computation procedure consists of three steps of forming
and solving SLAEs for each spatial direction. We can solve each SLAE
independently within one direction. For example, when forming and solving SLAEs
along the X axis, we can split the Y Z domain into a number of horizontal
bars and distribute them between a number of threads. We do this by using
`#pragma omp parallel for` for outer of two nested loops. After all SLAEs are
solved for one direction, we should synchronize threads by
`#pragma omp barrier` to avoid update con icts.</p>
        <p>Figs. 4 and 5 show the serial algorithm and the parallel variant, respectively.</p>
        <p>The procedures for domain parameters consist of three nested loops; so, we
can just use ` parallel for` for the outer one.</p>
        <p>Pro ling of parallel program shows that the proportion of the parallel code
is 75%.</p>
        <p>er DATA
u
ttereapm tttsan XYZ(((iii,,,jjj,,,kkk)))</p>
        <p>n
teh ehi T(t,x,y,z)
fo t
n ta parameters
ito ld
au ife
t
p
m
o
C
for k
for j
n
o
it
rce 1) forming SLAE
id- 2) solving SLAE
x 3) updating T
for i
for k
n
o
it
ce 1) forming SLAE
r
id- 2) solving SLAE
y 3) updating T
for i
for j
n
o
it
ce 1) forming SLAE
ird 2) solving SLAE
z 3) updating T
As a model problem, let consider a seasonal freezing{thawing of the upper
layers of a permafrost soil. The permafrost temperature lower than the area of
in uence of the seasonal changes (lower than 10 meters) is -0.7 C. The basic
thermal parameters of the soil are in the following: the thermal conductivity is
1.82 and 1.58 W/(m K), the volumetric heat is 2130 and 3140 kJ/(m3 K) for
frozen and melted soil, respectively, the volumetric heat of phase transition is
1.384 105 kJ/(m3 K).</p>
        <p>Table 1 shows the data for the considered area. The other parameters in
equation (1) and conditions (2) and (3) are determined as a result of the geophysical
research.</p>
        <p>The experiments are carried out using the six-cores AMD Ryzen 5 1600X
CPU. The grid size for problem is 91 91 51. The time interval was 10 days
with the step of 1 day.</p>
        <p>
          Table 2 shows the computing times Tp, speedup Sp = Tp=T1, and e ciency
Ep = Sp=p for solving the xed problem size by various numbers p of OpenMP
threads (strong scaling). It also contains the theoretical speedup calculated using
the Amdahl law [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]:
        </p>
        <p>Sp =
is proportion of the serial code (25% for our program).
(4)</p>
        <p>By extrapolating these calculation times, we can say that the serial program
would take 6.3 hours to solve the problem for 1 year interval and 188 hours (8
days) for 30 years interval. Using the parallel computing, we can cut this time
to approximately 2 days.
6</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Conclusion</title>
      <p>Thus, the developed mathematical model and software allow one to carry out
detailed numerical calculations on long-term forecasting the temperature eld
changes from di erent technical systems in the near-surface layer of soil in the
permafrost zone. E ective and quite simple OpenMP approach for a multicore
system allows one to reach approximately 90% of the theoretical speedup. The
results of numerical experiments show that by using the six-core processor, the
computation time is reduced up to 2.5 times.</p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <p>The work was partially supported by the Russian Foundation for Basic Research
16{01{00401 and by the Center of Excellence \Geoinformation technologies and
geophysical data complex interpretation" of the Ural Federal University
Program.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1. Zhang,
          <string-name>
            <given-names>T.</given-names>
            ,
            <surname>Barry</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. G.</given-names>
            ,
            <surname>Knowles</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            ,
            <surname>Heginbottom</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. A.</given-names>
            ,
            <surname>Brown</surname>
          </string-name>
          , J.:
          <article-title>Statistics and characteristics of permafrost and ground ice distribution in the Northern Hemisphere</article-title>
          .
          <source>Polar Geography</source>
          , Vol.
          <volume>23</volume>
          (
          <issue>2</issue>
          ),
          <volume>132</volume>
          {
          <fpage>154</fpage>
          (
          <year>1999</year>
          ) https://doi.org/10.1080/ 10889379909377670
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Jorgenson</surname>
            ,
            <given-names>M. T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Romanovsky</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harden</surname>
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shur</surname>
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>O'Donnell</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schuur</surname>
            ,
            <given-names>E. A. G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kanevskiy</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Resilience and Vulnerability of Permafrost to Climate Change</article-title>
          .
          <source>Canadian J. Forest Research</source>
          , Vol.
          <volume>40</volume>
          (
          <issue>7</issue>
          ),
          <volume>1219</volume>
          {
          <fpage>1236</fpage>
          (
          <year>2010</year>
          ) https://doi. org/10.1139/X10-060
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Filimonov</surname>
            ,
            <given-names>M. Yu.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vaganova</surname>
            ,
            <given-names>N. A.</given-names>
          </string-name>
          :
          <article-title>Simulation of thermal stabilization of soil around various technical systems operating in permafrost</article-title>
          .
          <source>Applied Mathematical Sciences</source>
          ,
          <volume>7</volume>
          (
          <issue>144</issue>
          ),
          <volume>7151</volume>
          {
          <fpage>7160</fpage>
          , (
          <year>2013</year>
          ) https://doi.org/10.12988/ams.
          <year>2013</year>
          . 311669
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Filimonov</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Yu</surname>
          </string-name>
          .,
          <string-name>
            <surname>Vaganova</surname>
            <given-names>N. A.</given-names>
          </string-name>
          :
          <source>Simulation of Technogenic and Climatic Inuences in Permafrost.: Lecture Notes in Computer Science</source>
          . Vol.
          <volume>9045</volume>
          . 178{
          <fpage>185</fpage>
          . (
          <year>2015</year>
          ) https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -20239-6_
          <fpage>18</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Vaganova</surname>
            ,
            <given-names>N. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Filimonov</surname>
            .,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Yu</surname>
          </string-name>
          .: Simulation of Engineering Systems in Permafrost.
          <source>Siberian Journal of Pure and Applied Mathematics</source>
          . (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Filimonov</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vaganova</surname>
          </string-name>
          , N.:
          <article-title>Permafrost thawing from di erent technical systems in Arctic regions</article-title>
          .
          <source>IOP Conf. Ser.: Earth Environ. Sci</source>
          . Vol.
          <volume>72</volume>
          . No. art.
          <volume>012006</volume>
          (
          <year>2017</year>
          ) https://doi.org/10.1088/
          <fpage>1755</fpage>
          -1315/72/1/012006
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Vaganova</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Filimonov</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Simulation of freezing and thawing of soil in Arctic regions</article-title>
          .
          <source>IOP Conf. Ser.: Earth Environ. Sci</source>
          . Vol.
          <volume>72</volume>
          . No. art.
          <volume>012005</volume>
          (
          <year>2017</year>
          ) https: //doi.org/10.1088/
          <fpage>1755</fpage>
          -1315/72/1/012005
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Vaganova</surname>
            ,
            <given-names>N. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Filimonov</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Yu</surname>
          </string-name>
          .:
          <article-title>Computer simulation of nonstationary thermal elds in design and operation of northern oil and gas elds</article-title>
          .
          <source>AIP Conf. Proc</source>
          . Vol.
          <volume>1690</volume>
          , No.
          <volume>020016</volume>
          (
          <year>2015</year>
          ) https://doi.org/10.1063/1.4936694
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Filimonov</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vaganova</surname>
          </string-name>
          , N.:
          <article-title>Short and Long Scale Regimes of Horizontal Flare System Exploitation in Permafrost</article-title>
          .
          <source>CEUR Workshop Proceedings</source>
          . Vol.
          <volume>1662</volume>
          ,
          <issue>253</issue>
          {
          <fpage>260</fpage>
          (
          <year>2016</year>
          ) http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1662</volume>
          /mod3.pdf
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Filimonov</surname>
            ,
            <given-names>M. Yu.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vaganova</surname>
            ,
            <given-names>N. A.</given-names>
          </string-name>
          :
          <article-title>On Boundary Conditions Setting for Numerical Simulation of Thermal Fields Propagation in Permafrost Soils</article-title>
          ,
          <source>CEUR Workshop Proceedings</source>
          , Vol.
          <volume>2109</volume>
          ,
          <issue>18</issue>
          {
          <fpage>24</fpage>
          , (
          <year>2018</year>
          ) http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>2109</volume>
          / paper-04.pdf
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Filimonov</surname>
            ,
            <given-names>M. Yu.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vaganova</surname>
            ,
            <given-names>N. A.</given-names>
          </string-name>
          :
          <article-title>Parallel splitting and decomposition method for computations of heat distribution in permafrost</article-title>
          ,
          <source>CEUR Workshop Proceedings</source>
          , Vol.
          <volume>1513</volume>
          ,
          <issue>42</issue>
          {
          <fpage>49</fpage>
          , (
          <year>2015</year>
          ) http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1513</volume>
          /paper-05.pdf
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Akimova</surname>
            ,
            <given-names>E. N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Belousov</surname>
            ,
            <given-names>D. V.</given-names>
          </string-name>
          :
          <article-title>Parallel algorithms for solving lin- ear systems with block-tridiagonal matrices on multi-core CPU with GPU</article-title>
          ,
          <source>Journal of Computational Science</source>
          . Vol.
          <volume>3</volume>
          , No.
          <volume>6</volume>
          ,
          <issue>445449</issue>
          (
          <year>2012</year>
          ) https://doi.org/10.1016/j.jocs.
          <year>2012</year>
          .
          <volume>08</volume>
          .004
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Akimova</surname>
            ,
            <given-names>E. N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Belousov</surname>
            ,
            <given-names>D. V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Misilov</surname>
            ,
            <given-names>V. E.</given-names>
          </string-name>
          :
          <article-title>Algorithms for solving inverse geophysical problems on parallel computing systems</article-title>
          .
          <source>Numerical Analysis and Applications</source>
          , Vol.
          <volume>6</volume>
          (
          <issue>2</issue>
          ),
          <volume>98</volume>
          {
          <fpage>110</fpage>
          (
          <year>2013</year>
          ) https://doi.org/10.1134/S199542391302002X
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Akimova</surname>
            ,
            <given-names>E. N.:</given-names>
          </string-name>
          <article-title>A parallel matrix sweep algorithm for solving linear system with block- vediagonal matrices</article-title>
          .
          <source>AIP Conf. Proc.</source>
          , Vol.
          <volume>1648</volume>
          ,
          <fpage>850028</fpage>
          . (
          <year>2015</year>
          ) https: //doi.org/10.1063/1.4913083
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Akimova</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Belousov</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Parallel algorithms for solving linear systems with block- vediagonal matrices on multi-core CPU</article-title>
          .
          <source>CEUR Workshop Proceedings</source>
          , Vol.
          <volume>1729</volume>
          ,
          <issue>38</issue>
          {
          <fpage>48</fpage>
          (
          <year>2016</year>
          ) http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1729</volume>
          /paper-06.pdf
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Martyshko</surname>
            ,
            <given-names>P. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pyankov</surname>
            ,
            <given-names>V. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Akimova</surname>
            ,
            <given-names>E. N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vasin</surname>
            ,
            <given-names>V. V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Misilov</surname>
          </string-name>
          , V. E.:
          <article-title>On solving a structural gravimetry problem on supercomputer \Uran" for the Bashkir Predural's area</article-title>
          ,
          <source>GeoInformatics 2013 - 12th International Conference on Geoinformatics: Theoretical and Applied Aspects</source>
          (
          <year>2013</year>
          ) http://www.earthdoc. org/publication/publicationdetails/?publication=
          <fpage>68121</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Akimova</surname>
            ,
            <given-names>E. N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Martyshko</surname>
            ,
            <given-names>P. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Misilov</surname>
            ,
            <given-names>V. E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kosivets</surname>
            ,
            <given-names>R.A.</given-names>
          </string-name>
          :
          <article-title>An e cient numerical technique for solving the inverse gravity problem of nding a lateral density</article-title>
          .
          <source>Applied Mathematics and Information Sciences</source>
          , Vol.
          <volume>10</volume>
          (
          <issue>5</issue>
          ),
          <volume>1681</volume>
          {
          <fpage>1688</fpage>
          (
          <year>2016</year>
          ) http://doi.org/10.1063/1.4992206
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Akimova</surname>
            ,
            <given-names>E. N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Martyshko</surname>
            ,
            <given-names>P. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Misilov</surname>
          </string-name>
          , V. E.:
          <article-title>On nding a density in a curvilinear layer by biconjugate gradient type methods</article-title>
          ,
          <source>AIP Conference Proceedings</source>
          , Vol.
          <year>1863</year>
          ,
          <volume>050009</volume>
          , (
          <year>2017</year>
          ) http://doi.org/10.1063/1.4992206
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Voevodin</surname>
            ,
            <given-names>V. V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Voevodin</surname>
            ,
            <given-names>V. V.: Parallel</given-names>
          </string-name>
          <string-name>
            <surname>Computing. BHV-Petersburg</surname>
          </string-name>
          , St.
          <source>Petersburg</source>
          (
          <year>2002</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>