KCDC, the 'KASCADE Cosmic-ray Data Centre', is a webbased interface where data from the astroparticle physics experiment KASCADE-Grande is made available for the scienti c community as well as for the interested public. Over the past 5 years, we have continuously extended the KASCADE DataShop with various releases and increased both the number of detector components from the KASCADE-Grande experiment and the data sets. With the latest release we added a new and independent DataShop for a speci c KASCADE-Grande event selection and by that created the technology for integrating further DataShops and data of other experiments in KCDC. We present a `brief history of data - from KASCADE to KCDC' and will discuss future plans partly related to GRADLCI, the German-Russian Astroparticle Data Life Cycle Initiative.
With KCDC [
With regard to a more general and global data and analysis centre, we are working on a KCDC extension to include scienti c data from other experiments, which nally will allow immanent multi-experiment or multi-messenger data analyses. 2 2.1
Open access as described by the Berlin Declaration [
The principles also demand the publication of meta information and documentation, which have to provide all information to understand, to work with and to process the data. This includes a thorough and transparent description of the detector, the data taking process and the physics background the analyses are based on. 2.2
The current web portal provides several options to access the physics data from KASCADE-Grande and the matching simulations provided. The user can produce his own data selections from the complete data sets available by means of quantity selections and applied cuts. These quantity selection and cuts are transmitted and processed in the backend on our servers. Moreover, we provide interesting parts of the complete data set as preselections for direct download.
With KCDC, we rely exclusively on non-commercial and state of the art open source software. For managing the webpages, data streams, databases and communicating with the backend, we focus on the web framework Django and Python 3 as well as several other open source libraries. Web pages are rendered in HTML by Django's template engine enriched with our very own JavaScript (e.g. the jQuery library) and CSS additions. Interfaces to our data sources like a MongoDB database for physics data are realized within Python.
The experimental data are stored in a NoSQL database, which enables us to expand easily the number of events or of detector components without the restraint of a x database scheme. We are using MongoDB at a sharded cluster for better performance. The full KCDC system runs on an Nginx server and communicates with the database server and a worker node. On each worker node, managed and monitored through Django via the Celery extension, Python tools process the user selections.
The data packages reside on an FTP server, where they can be retrieved by the users via an HTML link, provided when processing of their job has been successfully nished. Preselections and simulations can be directly accessed by a registered user via FTP.
The web portal software and all accompanying tools are designed a vast set of tools to con gure and manage the web portal directly via a web browser interface without directly connecting to the server. The KCDC software is also structured in a basic software package called KAOS (Karlsruhe Astroparticle Open-data Software). It is planned to publish the KAOS software in order to make it usable for other experiments. A plug-in system makes it easy to add functionality to the basic KAOS package overwriting the KAOS default settings. 2.3
The air showers measured by the KASCADE-Grande detectors are analysed
using the reconstruction program KRETA (KASCADE Reconstruction for
ExTensiveAir showers). Starting from the energy deposits and the individual time
stamps KRETA determines physical quantities like the total number of electrons,
muons and hadrons, the shower core and the shower direction. The KASCADE
data acquisition and the data analysis is described in detail in [
Analysing experimental data of air showers in terms of parameters of the primary particle or nucleus entering the Earth's atmosphere requires a detailed theoretical modelling of the entire cosmic ray shower. This can only be achieved by Monte-Carlo calculations taking into account all knowledge of particle interactions and decays.
At KASCADE, the entire simulation chain consists of three parts: (1) air shower simulation performed by CORSIKA; (2) detector simulation performed by CRES (Cosmic Ray Event Simulation); (3) data reconstruction performed by KRETA. Figure 1 illustrates the parallel work ow of measurements and simulations as applied in KASCADE-Grande.
With KASCADE-Grande, we have not only reconstructed energy spectra for
ve mass groups using seven di erent high-energy hadronic interaction models
from three model families, but also tested the validity of these models by
studying correlations of various individual observables to help the model builders to
improve their models. All the models are implemented in the CORSIKA
simulation package. CORSIKA [
CORSIKA is a detailed Monte Carlo program to study the evolution and properties of extensive air showers in the atmosphere. Primaries (Protons, light nuclei up to iron, photons etc.) are tracked through the atmosphere down to the observation level until they undergo reactions with the air nuclei or { in the case of non-stable secondaries { decay.
CRES is a code package to simulate signals and energy deposits in all
detectors of KASCADE-Grande as response to an extensive air shower as simulated
with CORSIKA. CRES is based on the GEANT3 [
KASCADE measurements and simulations have the same data structure, so both can be analysed using the same reconstruction program KRETA. The complete work ow for measured and simulated data is outlined in g. 1.
Unlike for measured data where we have event speci c information like event
time and calibration parameters (like air pressure and temperature), we have
some additional information in simulations from CORSIKA and CRES. Shower
properties like true primary energy, particle ID and shower direction are derived
from the CORSIKA data banks, while the shower core locations were randomly
thrown on a prede ned detector area in CRES and stored in KRETA [
As we have entered new territory with the establishment of a public data centre for high-energy astroparticle physics, there were no completed concepts available, how the data should be treated and prepared that they could be used reasonably outside the collaboration. From the rst release in November 2013, the following requirements were ful lled and are meanwhile implemented and released as a series of KCDC versions: { KCDC as data provider : free and unlimited open access to KASCADE cosmic ray data { KCDC as information platform : a detailed experiment description and sufcient meta information on the data and the analysis is provided { KCDC as long-term digital data archive : KCDC serves not only as a software and data archive for the collaboration, but also for the public.
Since the rst release (Open Beta Version Wolf 359), KCDC was further developed and improved. Subsequently, the versions VULCAN, MERIDIAN and NABOO were published, where as communication platforms serve an Email list for the KCDC subscribers as well as social networks, like at Twitter (https://twitter.com/KCDC KIT).
Workflow for KASCADE Measurement
and Simulation Data
Measurements - data acquisition - event building - event storage
Simulations air shower simulation
CORSIKA Input primary energy E particle ID A shower direction Q , F ... high energy models
QGSjet EPOS
Sibyll low energy model
FLUKA detector simulation
CRES based on GEANT 3 data reconstruction using KRETA - data calibration and data correction - reconstruction of EAS variables like Ne, Nµ, shower direction, core postion, densities .. - data storage in root files
Data Analysis compare measurements with distributions achieved from extensive air shower simulations
In 2013, we published 158 Mio events with seven reconstructed parameters called quantities, based on data analyses of the KASCADE detectors only. With every major release we added more data sets and/or more KASCADE-Grande detector components.
With the release VULCAN we switched from SQL to NoSQL data base (mongodb) and added two more quantities.
With MERIDIAN, directly measured detector data were published for the rst time, the `energy densities' and the `arrival times' per station, which enlarged the database by a factor of 100 up to 3 TB. Two new Django plugins handling the KCDC `publications' and the `spectra' data of related cosmic ray experiments were included.
The biggest changes so far have been introduced with the release of NABOO. In addition to the newly added GRANDE detector component, all the data from KASCADE and GRANDE were published (1998-2013). This brings the number of events to 433 million. The back-end programming has been completely rebuilt to handle this amount of data. Furthermore, the matching CORSIKA simulations for KASCADE and GRANDE as well as 88 published spectra of 21 cosmic-ray experiments are now accessible. The data arrays `energy density' has been replaced by the `energy deposits' per station which is closer to the measured data and o ers more analysis options for the users.
With the release OCEANUS we moved the DataShop database to a sharded
cluster speeding up the processing time by roughly a factor of 50. Furthermore,
the data of the Radio LOPES detector [
From the very beginning, a large interest from the community was given,
proved, e.g. by our anonymous monitoring of the access to the portal. Up to
now about 250 users registered from more than 30 countries distributed over 5
continents. We track page views and downloads with MAMOTO analytics [
In the latest release published in May 2020 we added a second DataShop called `COMBINED', where the data from the joint analysis of the KASCADE and the GRANDE detector arrays were made publicly available. Thus 'COMBINED' is subsample of the KASCADE DataShop, analysed in a completely di erent way. This made it necessary to o er the data in a new DataShop. The combined detector output has the quality of a stand-alone experiment, not of an additional component.
Until now, data taken by KASCADE and its extension GRANDE have been analysed more or less independently of each other. The aim of the combined analysis was to utilize an improved reconstruction to get one single, consistent spectrum in the energy range of 1015 eV to 1018 eV. The focus is on the mass composition, which is one of the most important sources of information needed to restrict astrophysical models on the origin and propagation of cosmic rays. With the improved reconstruction of the extensive air showers, a study of the elemental composition of high-energy cosmic rays is possible in a more detailed way.
Adding a new DataShop to KCDC had a lot of implications for the existing web portal: { the back-end programming was extended to host more DataShops, { detector components and quantities for the DataShop were de ned in the `administrator interface', { a new mongodb was lled with the `COMBINED' data sets, { matching CORSIKA simulations were generated for `COMBINED', { new `Preselections' were added, { documentations for `COMBINERD' and `COMBINED-Simulations' were provided { most of the static and dynamic web pages needed refurbishing, { a new menu item called `Materials' was added to host the manuals and the programming tools for all DataShops, { the `Simulations' and `Preselections' pages were completely redesigned. All of these changes have been made to allow us to include more data from other experiments by incorporating more independent DataShops into the KCDC system. Some interesting numbers of the two DataShops published until now are displayed in table 1. The comparatively small number of events included in the `COMBINED' DataShop is due to the fact that both, the KASCADE and the GRANDE detector arrays, require enough stations with data to be able to perform a joint data analysis.
Fig. 2 shows on a timeline the data acquisition times of the data published in KCDC for the four detector components of `KASCADE' and the `COMBINED' analysis. 5
Emerged from the requests of the GRADLC Initiative and our experience on how to handle big data in science, we have several tasks and ideas on our to-do list for the further development of KCDC.
In the next release, which is planned for end of 2020, a complete upgrade of all software components will be carried out. Most of all, a replacement for the ftp download server is necessary because ftp is deprecated in modern browsers.
Furthermore, we intend to expand our range of open access with data of other
cosmic rays experiments to protect the data from being lost. Presently the data sets from
the MAKET-ANI ([
In the framework of an envisaged global analysis and data centre, the
GermanRussian Astroparticle Data Life Cycle Initiative [
To include KCDC within the project the provided data have been modi ed. A
Universally unique identi er (UUID [
One step further to an easy-to-use `Data Lake' [
With the partnerships to astroparticle.online [