A population annealing method is a universal algorithm applicable to statistical mechanics systems and optimization problems. It is potentially scalable on any parallel architecture. We review recent developments in the area, emphasizing the implementation of the algorithm on a hybrid parallel program architecture combining CUDA and MPI. The problem is to keep all general-purpose graphics processing unit devices as busy as possible by efficiently redistributing replicas. We provide testing details on hardware-based Intel Skylake/Nvidia V100, running more than two million replicas of the Ising model samples in parallel. As the complexity of the simulated system increases, the acceleration grows toward perfect scalability. This work was done under Grant No. 19-11-00286 from the Russian Science Foundation and was supported in part through computational resources of HPC facilities at HSE University.[1].
Population annealing algorithm proposed by Hukushima and Iba [2] to simulate statistical mechanics systems with the complex energy landscape. The central idea is to simulate the enormous number of system replicas, splitting simulation into two steps, resampling replicas at each cooling step, and equilibrating replicas independently at the current temperature. The exciting feature of the algorithm is that it estimates the free energy at each cooling step using the average over the number of replicas. It was shown by Machta [3] that it is possible to estimate the behavior of statistical errors and systematic errors using weighted averages as a function of the number of replicas $R$. The statistical errors decay as $1/R1/2$, and the systematic errors decay as $1/R$ for a large enough number of replicas $R$.
There are successful applications of the method to the spin glasses [4,5], molecular dynamics [6], first-order phase transitions [7], and optimization problems [8]. A detailed description of the method and the analysis of the accuracy dependence on the essential parameters of the simulation can be found in the recent publication [9].
The algorithm was successfully implemented using CUDA [10], and it was found that the optimal number of replicas per one GPU V100 node should be about ten times larger than the number of threads. It gives the possibility to run 20 thousand replicas on one GPU node.
Recently, we extended the range of population annealing (PA) simulations up to more than
two million replicas running in parallel on the HSE University supercomputer cHARISMa with 104
GPU Nvidia V100 [
The main question in the multi-node realization of the PA algorithm is how to keep the distribution number of replicas at each node as flat as possible? It can happen that at some nodes, the number of replicas will grow while cooling, and at some nodes, the number of replicas can become very small. In such a case, the computing time at nodes becomes very different, and simulation could be inefficient.
The possible solving of the problem consists of the grouping of replicas in the blocks with the
moderate size 1024 and using twenty blocks per GPU [
The number of replicas during the simulations is relatively flat with the block algorithm and
fluctuates within the allowed window, which is one or two excess/shortage blocks [
The block PA algorithm was tested in [
Nowadays, the HSE supercomputer cHARISMa has been upgraded with the 184 GPU available. We plan to use the block PA algorithm to simulate the statistical mechanic's complex behavior at low temperatures.
In addition, there is an extension of the PA algorithm in which the variable parameter temperature is replaced with the variable parameter energy [12]. This algorithm is based on the program code for GPU from paper [10] and demonstrates the possibility of catching the models' nonequilibrium properties. We are working on the combined GPU/MPI implementation with the spirit of the block PA algorithm.