This paper describes the design of a simulation model of the infrastructure for data processing from the Super Charm-Tau factory class "megasience" electron-positron collider. The model simulates the behavior of the main subsystems of a supercomputer, such as the task scheduler, computing clusters, data storage system, etc. Using simulation modeling allows for the maximally reliable representation of the exact characteristics and volume of the needed equipment for developing the desired HPC system. The simulation model accounts for all the aspects of operation of this system from parallel data storage system to arrangement of the parallel launch of tasks. The developed system for processing software errors and equipment failures, as well as the system for ensuring energy efficiency make it possible to estimate the needed equipment with account for all possible emergency situat ions. This model allows calculating the parameters of the computing system necessary for processing and storing the results of operation of the Super Charm-Tau factory after its commissioning.
The «Super Charm-Tau Factory» (SCTF) project, which is a symmetric electron-positron
collider of ultrahigh luminosity with a beam energy at the mass center system from 3 to 7 GeV, is
developed at the BINP SB RAS. This project comprises a unique accelerating-storage complex with a
luminosity of 1035 cm-2s-1 and a universal elementary particle detector. The main goal of experiments
carried out on the SCTF is to study the properties of tau lepton and charmed particles, subject the
existing theory of elementary particles, the Standard Model, to high-precision verification, and to
search for phenomena not described within the framework of this theory [
An important role in the project is played by the system for data processing and storage. This system should ensure recoding and reliable storage of «RAW» (approx. 100 petabytes/year) and calibration data (approx. 100 gigabytes/year) from the particles detector of the SCTF. This data should be constantly available for analysis and processing using of specialized software. Specialized software should include tools for solving reconstruction and modeling tasks, as well as for comparing model ed and reconstructed events. The computational complexity of reconstruction and modeling tasks is 0.5GFLOPS/event and 5GFLOPS/event. Specialized software should allow one to analyze the accumulated data by a collective of about thousands physicists. Parallel execution of all user tasks should be processed by the SCTF computational infrastructure. For simplicity, we will call the integration of the computational infrastructure and the storage system a High-Performance Computing (HPC) system [fig. 1].
It is impossible to calculate characteristics of the required HPC system due to the assumed vast volume of primary data and a large number of computationally complex tasks running simultaneously, Thus, it is proposed to solve the problem of designing this HPC system using simulation modeling.
The AGNES multi-agent system [
All types of objects given in the scheme are intelligent agents possessing their own set of
parameters and behavior algorithms. A detailed description of the algorithms of operation and
interaction of these intelligent agents to simulate the operation of the system is described in this article
[
The scheme of interaction of agents in the process of data flow in data storage system of the HPC system is shown in the figure 4. There are 6 controller agents in the data storage and processing system that works only with data from a specific source. This separation is provided in order to be able to estimate how much data and with what intensity each subsystem will be processed.
A test simulation of the HTC system operation in the data processing mode with SCTF was
carried out to approve the developed model. For this purpose, the infrastructure of the NKS-1P cluster
was modeled. The NKS-1P cluster is installed in the Siberian supercomputer center [
During the simulation, data from the detector (with a given frequency), as well as tasks from users (with a given probability) were received to the model of the HTC system. When a certain number of events from the detector accumulated, reconstruction tasks were automatically started. In different modes of operation of the detector, events are received with a frequency from 100 to 300 kHz. The computational complexity of the reconstruction task of single event is 0.5 GFlops/s, the modeling task is 5 Gflops/s. The main purpose was to determine whether the NKS-1P infrastructure could handle such a flow of tasks and data. Therefore, the following parameters were considered: the average waiting time for the task execution and the amount of saved and lost data at the event frequency of 100 and 300 kHz [fig. 4].
The simulation showed that: ● The cluster's storage system copes with a 100kHz event stream. As the frequency increases, the volume of lost data (events) increases; ● The 300Tb data storage system will be filled within 30 hours with RAW and processed data from the detector with event frequency of 100kHz. Significantly large amounts of memory are required to store events from the detector; ● The computing infrastructure copes with processing packets of events with a frequency of 100 kHz. The waiting time for task execution increases significantly with an increase in the frequency; ● Increasing the probability of receiving simulation tasks also increases the waiting time. An increase in the number of computing nodes is required.
It is necessary to estimate the computational infrastructure parameters for storing and processing data of the physical experiment even at the stage of design SCTF project. Using simulation modeling allows to do this most accurately. The developed simulation model allows calculating the parameters of the computing system required for processing and storing the operation results of the Super Charm-Tau factory after its commissioning.
A full-scale simulation is planned to assess the necessary and sufficient amount of computing resources to ensure the operation of the SСTF. Full-scale modeling means the launch of a model in which the SСTF will not be limited by either disk space or computing nodes. The number of resources required at the moment of the model time will be added automatically. Thus, at a certain point in time, the number of computing nodes will reach not only the necessary number for processing the expected task flow, but also sufficient for a given probability of failures. Of course, the required amount of memory will constantly increase, but it will be possible to accurately assess the trend of this growth. Certainly, for modeling will be used the characteristics of the most modern server equipment at the current time.
This work was financially supported by the Russian Science Foundation (Grant No. 19-7220114) using the resources of the Siberian Supercomputer Center cluster (ICM&MG SB RAS, Novosibirsk). SSCC
SB
RAS.