NetIs is a service used to monitor the Data Acquisition network of the ATLAS experiment. The first version was developed at CERN in 2010. Over the years, the need to replace NetIs with an improved service emerged. Indeed, the effort to maintain NetIs has considerably increased together with the size and complexity of the network system; additionally, the Round Robin Database used to store the data results in a loss of granularity over time that makes the tool unsuitable for retrieving accurate values from the past. The graphs produced by NetIs are generated by the backend server and they are quite static, though the GUI is familiar to many users. The main idea was to exploit the recent advancements in time series databases and visualization tools like Grafana in order to present data to the users in a more dynamic way. The Persistent Back-End for the ATLAS Information System, developed in ATLAS for permanent storage of operational data, was already integrated with Grafana and successfully collecting network monitoring statistics. Grafana, despite being a very popular visualization web application, does not support some GUI elements that are used in NetIs such as a tree or position of drop-down. Javascript code integrated with Grafana was used to overcome these limitations.
The ATLAS experiment is one of four LHC accelerator experiments at CERN [
The common structure of the legacy NetIs monitoring had two data sources: the first for
creating dashboards and the second for obtaining the network topology (Figure 2). In this version, the
monitoring data was retrieved from switches using the Simple Network Management Protocol
(SNMP) and was stored in a MySQL database to be later converted into RRD files by a separate
service. The Matplotlib [
The tree is the main GUI navigator in NetIs, but as mentioned Grafana does not support the Tree element. It is possible to add a tree to Grafana in the following ways: create a custom plugin or inject JavaScript code dashboards using the Grafana text panel widget. Both approaches have their pros and cons.
Creating a new panel is the main way to add new functionality to Grafana. This path requires the presence or acquisition of skills to improve Grafana itself. The plugin must be compiled as part of the Grafana developer workflow and installed on the server side. Within this path, to add any new function, for example, in our case we need to set the position for the dropdown menu, a new plugin is required. Unfortunately, this path does not guarantee compatibility with new versions. Sometimes some plugin does not work with new version.
The most effective way to add a tree into Grafana is to add JavaScript code to the text panel. This approach does not require a Grafana developer workflow because all the code will be placed in the Grafana text panel. The tree is not an element of Grafana itself and cannot use the data retrieval methods. It should directly use the JavaScript objects of the Grafana library to interact with other Grafana elements. This method can be easily adopted for any HTML/JavaScript object, including a dropdown menu.
The new version of NetIs uses JavaScript code in a text panel to implement the tree (Figure 3) and some other elements. The text panel is a basic element of Grafana. It supports HTML code and JavaScript inside it. This external JavaScript tree interacts with the just using the Grafana JavaScript code downloaded by the browser, which is included in the main HTML page. Using this library, the user can receive data from other panels and update dashboards. The NetIs uses tree as the navigation panel.
The initial data to build the tree for NetIs is stored in the CDB MySQL database. Grafana has a plugin for MySQL, but as already mentioned, the tree is not an element of Grafana so it cannot directly interact with a Grafana data source. Instead, the tree receives data using the Grafana template element. The template stores the data as a list of strings that needs to be created first out of the data retrieved from several tables in the MySQL database. The strings have the following structure: Function:Device::Linecard::Interface. Some devices do not have a linecard component, so in this case when converting data for such devices a special token, PORTS or LAG, needs to be generated.
The NetIs uses standard edition of the dhtmlxTree library for the tree implementation in HTML. The Initial method gets all strings of the template, parses them and generates the tree. NetIs uses special ID of tree node for detection the level of tree and getting the data required for the dashboards. The ID has the following format: ● ● ● ●
Root level (function in CDB): _RR_FunctionName Level 1 (device in CDB): DeviceId Level 2 (linecard in CDB): _LL_DeviceId:LinecardId
Level 3 (interface in CDB): _II_DeviceId:LinecardId:InterfaceId Different levels of the NetIs tree have different context structures, with the exception of the tree area, which is always present in the upper left area. The tree uses the ID of selected node to generate content of the page using Grafana‘s JavaScript library. This approach allows the developers to make deeper changes in the structure of the monitoring page than those that can be done using standard Grafana tools.
The linecard view (level 2) has the same structure but uses the metrics gathered for the selected linecard and its interfaces. The interface view (level 3) does not display aggregated data (Figure 5), but instead it has a text area with additional information about the interface itself.
The new version of NetIs was successfully implemented, tested and put into production. These services are based on Grafana and P-BEAST. The use the P-BEAST time-series database avoided the loss of data over time of the stored samples and does not degrade the resolution of the dashboards. Grafana makes the NetIs page navigation more dynamic and flexible. Maintaining the new system should be easier than before because only knowledge of Web and JavaScript technologies is required to support the NetIs service. The monitoring system will be evolved and updated following operational experience.