Introduction

The behavior of multilayer magnetic structures has become of great technological importance due to the applications in magnetic storage devices. The ab initio calculations are widely used to calculate some characteristics of solids [1] and multilayer magnetic structures [2]. The main advantage of ab initio approach is independence from experimental data. Unlike the case of semi-empirical methods, there is no need for calibration or fitting parameters. Thus, ab initio methods can also be used to calculate the characteristics of perspective systems, i.e., for prediction of properties of materials that have not yet been developed. Mainly used packages that can perform ab initio calculations are VASP [3][4][5], Quantum Espresso [6], ABINIT [7], Wien2K [8].

Effective application of ab initio calculations requires the scalability of the code for novel high performance systems (HPS) with different hardware architectures. In this work we focused on the Vienna Ab initio Simulation Package (VASP). VASP is a complex package for performing ab initio quantum-mechanical molecular dynamics simulations using pseudopotentials or the projector-augmented wave method and a plane wave basis set [4]. Now it is one of the most popular parallel code for quantum chemistry and solid-state calculations of electronic structure. So the estimation of VASP code performance for HPS is an actual and non-trivial task. In this paper, we compare time of execution on POWER8 CPU and PASCAL P100 GPU with NVLink interconnection. We apply adjusting parameters in VASP INCAR file to increase performance on GPU.

2

The Basics of the Density Functional Method

We calculate energy and magnetic characteristics of Fe/Cu(111)/Fe and Co/Cu(100)/Co multilayer magnetic nanostructures. The central idea of density functional theory (DFT) [3] is to consider the electron density n(r) instead of the full many-body wave functions ) ,..., ( 1 N r r  . To ensure the possibility of calculating the magnetic properties, the energy of the system is written in the form of a functional not only of the electron density n(r), but also of the magnetization density m(r), see formula (1). The Kohn-Sham wave functions are replaced by two-component Pauli wave functions Ψν (r), capable of representing both the electron density and the magnetization density. Index ν here denotes spin states.

, ) ( ) ( 1  −  =  = N i i r r r n   =   = N v v v r r r m 1 * ) ( ) ( ) (  (1)

From the variational principle, the Kohn-Sham equations are obtained:

, 0 ) ( ) ( 2 2 =        −  + +  − r r B V m v v xc eff      ) ( ) ( ), ( r m r m r E B xc xc   =  (2)

where Bxc is the effective magnetic field arising from the exchange-correlation energy.

The main problem associated with the density functional theory method is that exact analytical expressions for exchange and correlation functionals are known only for the particular case of a gas of free electrons. Nevertheless, the existing approximations allow us to calculate a number of physical quantities with sufficient accuracy.

In this work we used GGA (generalized gradient approximations) approximations in terms of Perdew-Burke-Ernzerhof (PBE) [9]:

    r d r n r n r n r n r n r n E GGA xc                 = ) ( ), ( ), ( ), ( ) ( ), ( (3)

The essence of the projection augmented wave (PAW) method is to transform the pseudowave functions, obtained in the pseudopotential method into all-electron wave functions, thereby restoring the information lost when considering pseudowave functions. The number of plane wave components is limited by the Cut-off Energy. To describe the first Brillouin zone we used standard method Monkhorst-Pack with the parameter K-points characterizing regular grid in k-space [10].

Compiling the Parallel Version of VASP Code for Openpower and Intel Architectures

Official support of the GPU calculations appeared in VASP from version 5. 4

Hardware Information

IBM Power System S822LC is two-socket HPC system with two POWER8 CPUs with 20 cores running at 4 GHz and interconnected with two Nvidia Pascal P100 GPUs with a high bandwidth (80GByte in and 80GByte out) NVLink 1.0 interface (Fig. 1). It is very important for exchange data between multiple GPUs and fast load data from CPU. The major goal of this system is efficiently use GPU units and accelerating calculations. A large part of HPC resources installed during the last decade are based on Intel CPUs. Novel generations of Intel CPUs present a wide spectrum of multicore processors [5]. Intel Core i7 4770 is desktop processor, but it is "tock" model in Intel extensive strategy of microprocessor developmentit is mostly complete 22 nm architecture. We compared IBM POWER8 with Intel Haswell because both architectures introduced in 2013 year and had 22 nm technical processes.

Model and Simulation Parameters

In this work, the results of numerical first-principles calculations of the energy and magnetic characteristics for cobalt and iron films on a copper surface obtained by using VASP software package by means of the Projector Augmented Wave (PAW) method are presented. The values of the total energy of collinear spin configuration, the total magnetic moment and the magnetic moments of Co and Fe atoms are calculated. We investigated a system consisting of a copper slab and adsorbed on it from both sides by a ferromagnetic film with the thickness of the films in three monoatomic layers. The multilayer structure was simulated using a periodic 2×2 36-atom supercell with the lattice constant corresponding to the copper substrate a = 3.6367 (5) Å, which we obtained as a result of calculations taking into account the optimization of the lattice parameters. The surface face orientation is (100) for Co/Cu system and (111) for Fe/Cu system. There is an opinion that for GPU NSIM parameter needs to be increased while we have free memory on the GPU. − LPLANE is a useful parameter for optimization which can reduce intercommunication time, but it is actual firstly for massively parallel systems, according to VASP documentation.

Estimation of Performance and Accuracy of Calculations

For CoCu system we used three configurations to compare VASP calculation times with similar INCAR parameters for IBM POWER8 CPU and Intel Haswell CPU. We set LREAL=.TRUE. as described at VASP official documentation and use NCORE=1 with one MPI thread for GPU calculations. We do not use NVIDIA MPS system and do not set NSIM parameter clearly, but we know that it is important for large tasks especially. Core i7 has only 4 real cores and we run VASP with 4 processes only. On POWER8 we run VASP on 8 cores for using most of one CPU. The results of magnetization and free energy calculations (Table 3) are well correlated. The calculations with the GPU provide less accuracy, but the value of the error is not so sufficient. The times of calculations are different for POWER8 system and Intel Core i7 (Table 4). The one POWER8 thread was more efficient than one Intel thread for the VASP calculations. If we use GPU only with one MPI thread we have much better performance ( (Fig. 4). As we can see in Table 5 antiferromagnet spin configuration need sufficient more memory than a ferromagnet. Execution time on GPU highly depends on spin configuration too. It makes sense to note, that GPU utilization is not full and floating from about 20% to 70% during calculation for both systems.

Conclusions

VASP is widely used by researchers to get characteristics of solids and multilayer magnetic structures. NCORE and NSIM parameters can be very useful to maximize performance on GPU. The values of acquired quantities and accuracy of GPU calculations are in a good agreement with CPU results. To use VASP efficiently with GPUs more memory and calculation time is required in comparing with calculations on CPU, especially for antiferromagnet spin configurations.