Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019) Budva, Becici, Montenegro, September 30 – October 4, 2019 DRIFT-TUBE SYSTEM ELECTRONICS UPGRADE IN CONTEXT OF HIGH-LUMINOSITY LHC D. Eliseev 1 on behalf of the CMS Muon Group 1 RWTH Aachen University, Physics Institute IIIA E-mail: eliseev@physik.rwth-aachen.de The Drift Tube (DT) system is the primary detector in the barrel region of the CMS experiment dedicated to the measurement of muon tracks. The signals from about 172 000 DT cells must be fast and synchronously acquired to deliver the information about hits. In the context of the increasing luminosity of the LHC, in preparation for Phase 2, the DT system is being upgraded. The main focus of this upgrade is the development of a new generation of read-out electronics based on FPGA technology. The new on-chamber electronics will provide higher acquisition rates, increased radiation tolerance, and improved flexibility of the trigger settings for the DT system. The DT chambers will be equipped, depending on chamber type, with 3 to 5 unified boards called On-Board electronics for Drift Tubes (OBDTs). Along with better read-out characteristics, the OBDTs ensure fewer intermediate elements in the read-out chain. This contribution presents an overview of the new read-out chain and OBDT architecture. Keywords: LHC upgrade, CMS upgrade, DT upgrade, Muon Detector, OBDT Dmitry Eliseev Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 453 Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019) Budva, Becici, Montenegro, September 30 – October 4, 2019 1. Introduction During Long Shutdown 2 (LS 2) of the Large Hadron Collider (LHC), many upgrade activities on different LHC sites are planned. In the barrel region of the Compact Muon Solenoid experiment (CMS), the muon detection system is composed of 250 Drift-Tube (DT) chambers. Each of the DT chambers consists of a number of oblong gas detector cells. Depending on the location, the particular DT chamber can have a different size and therefore have a different number of the gas cells, which can vary between 700 and 1200 per DT chamber. Each cell is operating as a classic gas detector filled with a gas mixture (Ar:CO2 85%:15%) and equipped with high-voltage electrodes. The DT chamber also contains the front-end circuits to capture the hit events as a muon flies through a particular cell. The electric output of a front-end circuit of a DT cell is a Low-Voltage Differential Signal (LVDS) pulse. This pulse is transferred through a pair of wires to the DT chamber’s acquisition electronics. It is important to synchronously acquire the hit information from all cells of all DT chambers in order to be able to reconstruct the track information properly. For this purpose, one of the ends of each DT chamber is equipped with a compartment, referred to as a DT minicrate. The minicrate contains the first level read-out electronics which digitizes the exact hit times from all front-end circuits of all cells of the DT chamber. The read-out electronics of the minicrate is the beginning of a read-out chain. Currently this read-out chain includes a number of components and ends on the other side with specialized boards stacked into the Micro Telecommunications Computing Architecture (µTCA) crates. These crates are located in the underground service cavern (USC) of CMS. The read-out chain is the focus of the DT upgrade activities. No upgrade is planned for the front-end circuits of the DT cells. The subjects considered below are the structure of the read-out chain and, in particular, the new electronics of the DT minicrate that performs the read-out of the front-end circuits. 2. Description of the existing and new read-out chains 2.1 DT legacy chain The read-out chain which is in operation till the end of Phase 1 is often referred to as the legacy chain. One can subdivide the legacy chain into the following hierarchical stages with their respective components: • Minicrate Read-Out Boards (ROBs) [1]: these boards are located in the minicrate compartments of the DT chambers. The ROBs are connected directly to the LVDS output of the cell’s front-end circuits. The principal components of the ROB boards are customly developed ASICs, which monitor all channels of the respective DT chamber and perform Time-to-Digital Conversion (TDC). The ROB also carries a Field-Programmble Gate Array (FPGA) chip, which recognizes simple primitives based on a limited number of channels, connected to a particular ROB. The information about the recognized trigger events as well as the respective hit times are transferred from the DT chambers over copper cables. • Copper to Optic Fiber converters (CuOF) [2]: these units perform the translation between two transmission media. The data packages coming via the copper cables from the ROBs are converted for further transfer via fiber optics. The CuOF units are located in the Underground eXperimental Cavern (UXC), next to the CMS detector. • µROS and TwinMUX boards [3]: these are the slot boards inserted in the µTCA crates in the USC. These boards are the back-end of the DT read-out chain. These slot boards handle the data, which is coming from the CuOF translators. Both the µROS and TwinMUX boards are based on the same hardware platform, TM7 [3]. The µROS board processes information about the hit times, whereas the TwinMUX boards acquire trigger events. The µROS and TwinMUX 454 Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019) Budva, Becici, Montenegro, September 30 – October 4, 2019 perform higher level processing and forward the processed data further to the Central Data Acquisition (cDAQ) and Barrel Muon Track Finder (BMTF) systems respectively. 2.2 New read-out Chain for DT The new read-out chain contains two stages: • New DT electronics On-Board electronics of the Drift Tubes (OBDT): this board is a highly integrated FPGA based solution for fast parallel digitization of the DT chamber front ends. The TDC blocks, which were previously implemented in the dedicated ASICs, are now implemented directly in the FPGA of the OBDT. Together with read-out routines, the FPGA chip drives the uplink communication of the OBDT. This communication includes streaming of the hit data as well as transfer of clock data and slow control data. The communication links connected to the OBDT are fiberglass based. This makes the use of any intermediate components (like CuOFs previously) unnecessary. A more detailed description of the OBDT and its components is given in section 3. • AB7 and MoCo boards: these are slot boards hosted inside particular µTCA crates. Each AB7 board is a back-end system that handles the hit data streams from 3 to 5 OBDTs installed on a particular DT chamber. Uplink connection from OBDTs is provided in the way, that the complete data stream of all OBDTs of one DT chamber is covered by a single AB7 board. The routines for extracting the trigger primitives are implemented in the firmware of the AB7. The AB7 then sends data about the recognized trigger patterns to the BMTF. The AB7 also transfers the filtered hit data to the cDAQ system. The MoCo board implements the slow- control communication as well as providing the clocking data packages for up to 12 OBDTs. Both the AB7 and MoCo boards are based on the same TM7 hardware platform. This platform is identical to the one used by the back-end boards in the legacy chain (i.e. by µROS and TwinMUX boards). Particular specialization of a TM7 board is provided entirely by the firmware. The universality of the back-end boards at the hardware level contributes to the better serviceability. 2.3 Comparison of the new and legacy read-out chains Figure 1 shows the architectures of the Phase 1 and Phase 2 read-out chains. The architecture of the Phase 2 read-out chain has two important advantages. The first advantage is the OBDT board eliminates the need of using the CuOF translators. This is particularly important because the higher luminosity during Phase 2 leads to higher irradiation of the electronic components residing in the UXC. Figure 1. Read-out chain architectures: Phase 1 legacy chain (left), Phase 2 chain (right) The reduced number of electronic units in the UXC ensures higher reliability of the whole read-out chain. The second architectural advantage is the whole trigger logic moves completely to the 455 Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019) Budva, Becici, Montenegro, September 30 – October 4, 2019 back-end AB7 boards located in the USC. Since each of the AB7 boards acquires the whole stream of hit data of one particular chamber, the trigger primitives are now recognized based on the whole scope of cells within a particular DT chamber. This step also ensures a better maintenance of the read-out chain because the USC is accessible during the run time of the LHC. 3. New On-chamber electronics (OBDT) 3.1 Structure of OBDT The OBDT is an FPGA based board for the read out of the DT chamber’s front-end circuits. Since the OBDT is the only unit of the read-out chain which will reside in the UXC, radiation tolerant components were selected for the design of this board. The central part of the OBDT is the FPGA PolarFire MPF300T [4] from Microsemi. The OBDT can read out as many as 240 cells of a DT chamber. In order to cover all front-end channels of a particular DT chamber, 3 to 5 OBDTs are installed inside the chamber’s minicrate compartment. The particular number depends on the size of the DT chamber (i.e. the number of chamber’s cells). The LVDS signals coming to the OBDT are grouped in ribbon cables connected to 8 connectors with 34 LVDS signal pairs each. The LVDS pairs are then routed directly to the FPGA. If the front-end circuit of a cell sends a pulse on the respective LVDS line, the arrival time of this pulse is registered with high precision (below 1 ns). This precise time acquisition is performed by the TDC blocks, which are implemented inside the FPGA. The basic functional blocks of TDCs are the deserializers commonly present in modern FPGAs. In particular, for the PolarFire family, the deserializers used are called SERDES blocks [4]. Arrival times of all registered pulses (related to the clock of the LHC) as well as the respective channel numbers are multiplexed inside the FPGA. The resulting data packages are sent via the 4 optical transceivers driven by the same FPGA. These optical transceivers are present inside the QSFP+ cage of the OBDT. As described previously in section 2.2, the fiber optic links are connected on the other side to the AB7 board. Next to the FPGA, is the GBTx [5] chip. Like the FPGA itself, this chip also drives a fiber optic communication link dedicated to a separate SFP+ cage of the OBDT. This link is connected on the other end to the MoCo board (see section 2.2). This communication hosts the up and downstream transfers of the slow-control and clock data packages. The GBTx chip extracts the LHC clock signal and forwards it to the FPGA. Along with the FPGA, the GBTx chip also communicates with the SCA chip [5] on the OBDT. The SCA is a proprietary CERN development, which acquires slow analog signals (temperature sensors, measurement of voltage rails) and controls a few I2C links. The data link between the GBTx and the SCA is based on the E-Link interface standard [6]. As a reserve solution, one E-link interface is also provided between the GBTx and FPGA chips. A block-scheme of the OBDT board with its fiberglass links is sketched in Figure 2. Figure 2. OBDT Block structure and external interfaces 456 Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019) Budva, Becici, Montenegro, September 30 – October 4, 2019 Figure 3. Assembled OBDT prototype 3.2 Production status and tests A set of 14 OBDT prototypes have been produced for a series of tests described below. One of the assembled OBDT prototypes is shown in Figure 3. A subset of 12 OBDTs are mounted on the neighboring DT chambers of the second wheel of the CMS detector (positive side). Since all the OBDTs are in the same wheel-sector, the series of tests is often referred to as a Slice Test. The OBDTs participating in the slice tests are subdivided into two groups: • Group for integration tests: 6 OBDTs were mounted in the minicrate compartments of the MB1 and MB2 DT chambers. All possible integration activities are provided for these boards in order to test the final configuration of the cooling, mechanics, and cabling. A further important aspect here is to obtain the understanding of time and material resources needed for the replacement of the considered components. • Group for read-out validation tests: 6 OBDTs were mounted on the MB3 (2 boards) and MB4 (4 boards) DT chambers. The OBDTs of this group are working in parallel with the legacy chain. This is made possible with help of additional splitter boards, inserted directly after the front-end circuits. These splitters provide analog splitting of all front-end signals, ensuring that the signals from every front-end channel can be simultaneously acquired by both legacy and Phase 2 read-out chains. This is needed to validate the correctness of the hit data, acquired by the new read-out chain. The plot in Figure 4 demonstrates the time coincidence of the registered hits acquired by both Phase 2 and legacy read-out chains. Figure 4. Time coincidence demonstration of the Phase 2 and legacy chains. Plot: Lourdes Urda, Jonas Roemer 457 Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019) Budva, Becici, Montenegro, September 30 – October 4, 2019 4. Conclusion A new read-out chain was developed for the CMS DT system in preparation for LHC Phase 2. The new read-out chain offers improved architecture using fewer standalone functional blocks. The design of the new on-chamber electronics (OBDT) ensures a precision for registering the hit events below 1 ns. The first set of OBDT prototypes was produced. Ongoing tests show the expected performance of the whole read-out chain. The final version of the DT chamber electronics will be developed based on the considered OBDT prototype. References [1] BEDOYA, C. Fernandez, et al. Electronics for the CMS muon drift tube chambers: the read-out minicrate // IEEE transactions on nuclear science, 2005, 52. Jg., Nr. 4, S. 944-949. DOI:10.1109/TNS.2005.852698 [2] DE REMIGIS, P., et al. The CUOF optical link for the electronic system upgrade of the CMS Muon DT // Journal of Instrumentation, 2014, 9. Jg., Nr. 02, S. C02020. [3] NAVARRO TOBAR, A. et al. CMS Drift Tubes readout Phase 1 upgrade // TWEPP Conference Report 2018 (No. CMS-CR-2018-298). [4] MICROSEMI. PolarFire FPGA // Datasheet DS0141. Available at Microsemi document portal https://www.microsemi.com/product-directory/fpgas/3854-polarfire-fpgas#documentation, accessed on 10.11.2019 [5] CARATELLI, A., et al. The GBT-SCA, a radiation tolerant ASIC for detector control and monitoring applications in HEP experiments. // Journal of Instrumentation, 2015, 10. Jg., Nr. 03, S. C03034. [6] BONACINI, S.; MOREIRA, P.; KLOUKINAS, K. E-link: A radiation-hard low-power electrical link for chip-to-chip communication. // TWEPP Conference Report 2009. DOI:10.5170/CERN-2009- 006.422 458