We present an analysis of high performance computational method for solving the problem of crystal grows. The method uses PETSc and PETIGA C-language based libraries and supports parallel computing. The evolution of calculation process was studied in series of special computations are obtained on innovative mobile cluster platform, which provides exclusive system tuning abilities. The results of research conrm the high e ciency of the proposed algorithm on multi-core computer systems and allow us to recommend the use of PETSc and PETIGA for solving high order di erential equations.
crystal
Problems involving di erential operators of order more than two have not
historically lent themselves well to nite element analysis[
Two di erent approaches have been used to describe phase transition
phenomena: sharp interface models and phase- eld(di use-interface) models[
Originally, the PFC model has been formulated in a parabolic form. For now it
has been extended to allow faster degrees of freedom consistent with inertia in
correspondence to transformation propagative regimes. Particularly, a modi ed
or hyperbolic PFC model which includes an inertial term was introduced, and
therefore, gives a possibility of the description of both fast and slow dynamics
of transition [
F [ ; r ; r2 ] =
Z f ( )
1 jr j2 + 2 (r2 )2 d ; associated to the domain . The chemical potential is can be obtained as the variational derivative of the free-energy functional F , namely ( ) =
F = f 0( ) + 2r2 + r4 : The function f represents the homogeneous part of the free energy density. It takes on the form f ( ) = 1 Here, = (Tc T )=Tc is the undercooling, where T and Tc are the temperature and critical temperature of transition, respectively. is a coe cient which means a measure of metastability. 2
The modi ed PFC equation is a hyperbolic di erential equation of the sixth
order. The solution of this equation from the computational point of view is not
an easy task. Therefore, we developed special numerical algorithm using a
Clanguage code based on the PETIGA library[
To assess the performance of the computational program, we developed a series of experimental tasks. We made 3 types of experiments for homogeneous HPC cluster. For correct analysis, the con guration of hardware and software computer system must be optimized in order to minimize possible errors. Thus, the researchers had the task to form an experimental computational cluster in a short time with exclusive access to hardware and software. Such an approach could quickly adjust the computer system for each experiment with the requirements to minimize measurement uncertain. The use of traditional largescale computer systems for this task seems impractical and required a signi cant amount of e ort and time. At the same time, the use of personal workstations is not possible because of their insu cient performance. In this situation, we able to nd a solution with innovative Immers technology. Such solutions make it possible to build compact HPC autonomous mobile computational clusters with a form factor close to the personal workstation. In that conditions we got the ability to use the optimum set of system software and hardware settings. Due to this potential sources of measurement error were xed.
The cluster con guration includes 5 computational nodes connected by Inniband QDR network of 40 GB/sec. Each compute node consists of two 14-core processor Intel E5-2697 v3 and 64GB of DDR4 RAM. All computational nodes were running under the managment of SLES 11.3 OS. No extra optimization software were installed. In all experiments, the program calculated the task for 1 time step.
Descriptions and the results of experimental calculations are presented below. The purpose of these calculations was to evaluate the amount and speci city of computational resources, to assess the balancing of the program parallelization and estimate the amount of overhead. 2.1
The purpose of the rst experiment is evaluation of dependence the complexity of the problem from the amount of nite elements. Fixed parameters for this experiment are: computational domain size 160x160x160 and using of 5 nodes including 28 processor cores on each node. Variable parameter is a grid size: 10x10x10, 20x20x20, 40x40x40, 80x80x80, 160x160x160. Results of the experiment are presented in the following gures:
Percentage of MPIBarrier call time in the maximum computation time for a single core in cases of di erent grid size.
[Blue]The ratio of the maximum time for a single core to a minimum. [Red]The ratio of the maximum memory usage to a minimum during the calculation. [Green]The ratio of the maximum ops indicator to a minimum during the calculation. Diagrams for di erent grid size.
Percentage of MPI messages in maximum computation time for a single core in cases of di erent grid size. 2.2
The purpose of the second experiment is the assessment of the e ciency of the algorithm parallelization by increasing the number of computing nodes. Fixed parameters for this task are: computational domain size 160x160x160 and the grid size 50x50x50. Variable parameter is the number of nodes: from 1 to 5 nodes included 28 processor cores on each node. Results of the experiment are presented in the following gures:
PETSc and PETIGA in Solving the Problem of Crystal Growth
Maximum computational time for a single core in cases of di erent amount of nodes, sec.
Percentage of MPIBarrier call time in the maximum computation time for a single core in cases of di erent amount of nodes.
[Red]The ratio of the maximum time for a single core to a minimum. [Green]The ratio of the maximum memory usage to a minimum during the calculation. [Blue]The ratio of the maximum ops indicator to a minimum during the calculation.
Percentage of MPI messages in maximum computation time for a single core in cases of di erent amount of nodes. 2.3
The purpose of of the third experiment is estimation of e ciency of the algorithm parallelization by increasing the number of cores on single node. Fixed parameters for this task are: only one node, computational domain domain size 160x160x160 and the grid size 50x50x50. Variable parameter is the number of cores on single node: from 1 to 28. Results of the experiment are presented in the following gures:
Total memory used for di erent computation cores amount, bytes.
Percentage of MPIBarrier call time in the maximum computation time for a single core in cases of di erent computation cores amount.
[Red]The ratio of the maximum time for a single core to a minimum. [Green]The ratio of the maximum memory usage to a minimum during the calculation. [Blue]The ratio of the maximum ops indicator to a minimum during the calculation.
Percentage of MPI messages in maximum computation time for a single core in cases of di erent computation cores amount. 3
During computing the tasks the cluster Immers showed the best performance 4.45E+09 Flops/sec, 3.68E+09 Flops/sec and 8,03E+08 Flops/sec for the rst, second and third experiments, respectively.The experiments results suggest the following conclusions: 3.1
In the rst experiment, the total memory usage, the total number of operations and the total size of messages MPI grow exponentially. These indicators are rising at roughly the same speed. In the second experiment, the total amount of memory used and the total size of MPI messages almost unchanged. The total number of operations is slightly reduced, and this process requires further study. The computation time is also reduced with a logarithmic rate as expected. In the third experiment, there is a large variation in the total amount of memory used - this e ect requires further study. Variations in the total number of operations and the total amount of MPI message, apparently associated to variations from memory. Computational time is change expected. 3.2
In the rst experiment, variation in the memory up to 40%, the number of operations up to 60%. In the second experiment, variation in the memory up to 20%, the number of operations up to 35%. In the third experiment, variation in the memory up to 28%, the number of operations up to 55%. The indicators seem to be unexpectedly large and require further research.
The overhead of synchronization, as expected, increases with increasing amounts of computing nodes and decreasing the size of the grid. Meanwhile, the proportion of execution time MPI Barrier in total computation time is less than 0.001%, which is quite a bit. 4
Current work shows that the solution of the crystal growth problem in the hyperbolic statement allows to simulate the structural transformation of matter at the supercooling. These studies have a big practical importance in the materials science applications. The computational complexity of the hyperbolic PFC equation is forcing scientists to develop special high-performance algorithms. One of the algorithms, which is used PETSC and PETIGA libraries, discussed in the article. This algorithm has already shown e ectiveness, but the important issues are the conditions of its applicability and the possibility of it's improvement in terms of parallel programming. An important step in the matter of such issues is to study the e ciency of use of computing resources. To investigate this question, the authors have set speci c numerical experiments. To reduce the error in the results due to the peculiarities of the computing cluster setup researchers have used homogeneous autonomous mobile computer. This approach has proved successful in the sense that the used hardware con guration eliminates the in uence of unaccounted factors in the work of the computational algorithm.
The calculation results have showed a good balance of parallel computing. Authors have evaluated the overall computing resources. We can say that the amount of consumed memory and CPU time is linearly dependent on the complexity of the problem in terms of the number of computational cells. Also, in all the experiments overheads for synchronization tasks were low and even not up-to-date QDR network system was mostly idle. We believe that these results are a consequence of the high-quality implementation of computational algorithm. Some issues are deviations in performance of computational algorithm in the third experiment. Apparently, they are caused by the peculiarities of the CPU and data exchange on the nodes. It should be noted that these variations are generally not signi cant e ect on the calculations e ciency.
The continuation of this work would be conducting similar experiments on a heterogeneous HPC cluster. Comparison of the results with the results of this paper will be useful for choosing the best HPC con guration with capability of solving of hyperbolic PFC equations. For researchers,who conduct a big series of experiments, these conclusions can be especially interesting.