A critical challenge of high-luminosity Large Hadron Collider (HL-LHC), the next phase in LHC operation, is the increased computing requirements to process the experiment data. Coping with this demand with today's computing model would exceed a realistic funding level by an order of magnitude. Many architectural, organizational and technical changes are being investigated to address this challenge. This paper describes the prototype of a WLCG data lake, a storage service of geographically distributed data centers connected by a low-latency network. The architecture of an EOS data lake is presented, showing how it leverages economy of scale to decrease cost. The paper discusses first experiences with the prototype and first test computing jobs reading data from the lake.
High-luminosity Large Hadron Collider (HL-LHC) is the next step in increasing LHC
luminosity. With more raw data produced by the ATLAS detector, computing resource requirements
are predicted to exceed the limits expected to be available under the current computing model [
Data storage and management is one of the areas where critical improvements to efficiency can be made. With more granular and explicit Quality of Service management for both replication and access latency, redundancy and storage cost may be decreased dramatically while still providing data availability.
Another way to decrease cost of WLCG data storage is to reduce human effort engaged in data center administration. Using large-scale distributed grid sites that consolidate individual institutional resources into a “data lake” is one possible path to that scenario.
CPU and storage resources are no longer always co-located in the GRID. Desktop grids,
clouds and HPC are example of CPU-only resources. Caching is being increasingly used instead of
pre-staging on individual sites with new software system being developed specifically for this mode of
access [
To study the feasibility of using EOS to create such a highly distributed data lake a prot otype with several collaborating sites was created. Support for granular parity-based redundancy of stored data was tested with realistic access patterns.
The prototype data lake is built as a distributed EOS storage system. In EOS metadata is
stored separately from data, with MGM and MQ services responsible for the metadata and multiple
FST servers providing data storage [
The prototype resources are commissioned, the services are deployed, and basic transfer tests had been performed. Prototype transfer monitoring plots are presented in Figure 2.
QoS management with replicated and striped storage support was tested. Automatic replication of uploaded files based on namespace directory attributes, as well as manual conversion on command was observed to work. Some compatibility issues with several remote FST servers were observed.
The prototype was integrated into the ATLAS data management system Rucio[
As can be seen on Figure 5, if a processing job does not require a lot of input data, execution time is not strongly affected by storage, as can be expected. On Figure 6 results for more I/O demanding job are shown. Noticeable degradation of service for striped storage can be attributed to compatibility and configuration problems on some of the storage servers with regards to striped storage support, as well as striped storage requiring data from at least 4 locations, which lowers effective throughput if one of the 4 is bottlenecked.
A distributed storage instance based on EOS was set up. This data lake prototype is small in terms of resources, but very geographically distributed. Different EOS deployment options are used by member sites. The lake is integrated with Rucio and Hammercloud. Performance of different data placement scenarios with different access patterns can be measured. CMS integration is the logical next step. The deployment and testing experience on the data lake will help prepare for HL-LHC.