=Paper=
{{Paper
|id=Vol-2507/453-458-paper-84
|storemode=property
|title=Drift-Tube System Electronics Upgrade in Context of High-Luminosity LHC
|pdfUrl=https://ceur-ws.org/Vol-2507/453-458-paper-84.pdf
|volume=Vol-2507
|authors=Dmitry Eliseev
}}
==Drift-Tube System Electronics Upgrade in Context of High-Luminosity LHC==
https://ceur-ws.org/Vol-2507/453-458-paper-84.pdf
Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019)
Budva, Becici, Montenegro, September 30 – October 4, 2019
DRIFT-TUBE SYSTEM ELECTRONICS UPGRADE IN
CONTEXT OF HIGH-LUMINOSITY LHC
D. Eliseev 1 on behalf of the CMS Muon Group
1
RWTH Aachen University, Physics Institute IIIA
E-mail: eliseev@physik.rwth-aachen.de
The Drift Tube (DT) system is the primary detector in the barrel region of the CMS experiment
dedicated to the measurement of muon tracks. The signals from about 172 000 DT cells must be fast
and synchronously acquired to deliver the information about hits. In the context of the increasing
luminosity of the LHC, in preparation for Phase 2, the DT system is being upgraded. The main focus
of this upgrade is the development of a new generation of read-out electronics based on FPGA
technology. The new on-chamber electronics will provide higher acquisition rates, increased radiation
tolerance, and improved flexibility of the trigger settings for the DT system. The DT chambers will be
equipped, depending on chamber type, with 3 to 5 unified boards called On-Board electronics for Drift
Tubes (OBDTs). Along with better read-out characteristics, the OBDTs ensure fewer intermediate
elements in the read-out chain. This contribution presents an overview of the new read-out chain and
OBDT architecture.
Keywords: LHC upgrade, CMS upgrade, DT upgrade, Muon Detector, OBDT
Dmitry Eliseev
Copyright © 2019 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
453
Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019)
Budva, Becici, Montenegro, September 30 – October 4, 2019
1. Introduction
During Long Shutdown 2 (LS 2) of the Large Hadron Collider (LHC), many upgrade activities
on different LHC sites are planned. In the barrel region of the Compact Muon Solenoid experiment
(CMS), the muon detection system is composed of 250 Drift-Tube (DT) chambers. Each of the DT
chambers consists of a number of oblong gas detector cells. Depending on the location, the particular
DT chamber can have a different size and therefore have a different number of the gas cells, which can
vary between 700 and 1200 per DT chamber. Each cell is operating as a classic gas detector filled with
a gas mixture (Ar:CO2 85%:15%) and equipped with high-voltage electrodes. The DT chamber also
contains the front-end circuits to capture the hit events as a muon flies through a particular cell. The
electric output of a front-end circuit of a DT cell is a Low-Voltage Differential Signal (LVDS) pulse.
This pulse is transferred through a pair of wires to the DT chamber’s acquisition electronics. It is
important to synchronously acquire the hit information from all cells of all DT chambers in order to be
able to reconstruct the track information properly. For this purpose, one of the ends of each DT
chamber is equipped with a compartment, referred to as a DT minicrate. The minicrate contains the
first level read-out electronics which digitizes the exact hit times from all front-end circuits of all cells
of the DT chamber. The read-out electronics of the minicrate is the beginning of a read-out chain.
Currently this read-out chain includes a number of components and ends on the other side with
specialized boards stacked into the Micro Telecommunications Computing Architecture (µTCA)
crates. These crates are located in the underground service cavern (USC) of CMS. The read-out chain
is the focus of the DT upgrade activities. No upgrade is planned for the front-end circuits of the DT
cells. The subjects considered below are the structure of the read-out chain and, in particular, the new
electronics of the DT minicrate that performs the read-out of the front-end circuits.
2. Description of the existing and new read-out chains
2.1 DT legacy chain
The read-out chain which is in operation till the end of Phase 1 is often referred to as the
legacy chain. One can subdivide the legacy chain into the following hierarchical stages with their
respective components:
• Minicrate Read-Out Boards (ROBs) [1]: these boards are located in the minicrate
compartments of the DT chambers. The ROBs are connected directly to the LVDS output of
the cell’s front-end circuits. The principal components of the ROB boards are customly
developed ASICs, which monitor all channels of the respective DT chamber and perform
Time-to-Digital Conversion (TDC). The ROB also carries a Field-Programmble Gate Array
(FPGA) chip, which recognizes simple primitives based on a limited number of channels,
connected to a particular ROB. The information about the recognized trigger events as well as
the respective hit times are transferred from the DT chambers over copper cables.
• Copper to Optic Fiber converters (CuOF) [2]: these units perform the translation between
two transmission media. The data packages coming via the copper cables from the ROBs are
converted for further transfer via fiber optics. The CuOF units are located in the Underground
eXperimental Cavern (UXC), next to the CMS detector.
• µROS and TwinMUX boards [3]: these are the slot boards inserted in the µTCA crates in the
USC. These boards are the back-end of the DT read-out chain. These slot boards handle the
data, which is coming from the CuOF translators. Both the µROS and TwinMUX boards are
based on the same hardware platform, TM7 [3]. The µROS board processes information about
the hit times, whereas the TwinMUX boards acquire trigger events. The µROS and TwinMUX
454
Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019)
Budva, Becici, Montenegro, September 30 – October 4, 2019
perform higher level processing and forward the processed data further to the Central Data
Acquisition (cDAQ) and Barrel Muon Track Finder (BMTF) systems respectively.
2.2 New read-out Chain for DT
The new read-out chain contains two stages:
• New DT electronics On-Board electronics of the Drift Tubes (OBDT): this board is a highly
integrated FPGA based solution for fast parallel digitization of the DT chamber front ends.
The TDC blocks, which were previously implemented in the dedicated ASICs, are now
implemented directly in the FPGA of the OBDT. Together with read-out routines, the FPGA
chip drives the uplink communication of the OBDT. This communication includes streaming
of the hit data as well as transfer of clock data and slow control data. The communication links
connected to the OBDT are fiberglass based. This makes the use of any intermediate
components (like CuOFs previously) unnecessary. A more detailed description of the OBDT
and its components is given in section 3.
• AB7 and MoCo boards: these are slot boards hosted inside particular µTCA crates. Each
AB7 board is a back-end system that handles the hit data streams from 3 to 5 OBDTs installed
on a particular DT chamber. Uplink connection from OBDTs is provided in the way, that the
complete data stream of all OBDTs of one DT chamber is covered by a single AB7 board. The
routines for extracting the trigger primitives are implemented in the firmware of the AB7. The
AB7 then sends data about the recognized trigger patterns to the BMTF. The AB7 also
transfers the filtered hit data to the cDAQ system. The MoCo board implements the slow-
control communication as well as providing the clocking data packages for up to 12 OBDTs.
Both the AB7 and MoCo boards are based on the same TM7 hardware platform. This platform
is identical to the one used by the back-end boards in the legacy chain (i.e. by µROS and
TwinMUX boards). Particular specialization of a TM7 board is provided entirely by the
firmware. The universality of the back-end boards at the hardware level contributes to the
better serviceability.
2.3 Comparison of the new and legacy read-out chains
Figure 1 shows the architectures of the Phase 1 and Phase 2 read-out chains. The architecture
of the Phase 2 read-out chain has two important advantages. The first advantage is the OBDT board
eliminates the need of using the CuOF translators. This is particularly important because the higher
luminosity during Phase 2 leads to higher irradiation of the electronic components residing in the
UXC.
Figure 1. Read-out chain architectures: Phase 1 legacy chain (left), Phase 2 chain (right)
The reduced number of electronic units in the UXC ensures higher reliability of the whole
read-out chain. The second architectural advantage is the whole trigger logic moves completely to the
455
Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019)
Budva, Becici, Montenegro, September 30 – October 4, 2019
back-end AB7 boards located in the USC. Since each of the AB7 boards acquires the whole stream of
hit data of one particular chamber, the trigger primitives are now recognized based on the whole scope
of cells within a particular DT chamber. This step also ensures a better maintenance of the read-out
chain because the USC is accessible during the run time of the LHC.
3. New On-chamber electronics (OBDT)
3.1 Structure of OBDT
The OBDT is an FPGA based board for the read out of the DT chamber’s front-end circuits.
Since the OBDT is the only unit of the read-out chain which will reside in the UXC, radiation tolerant
components were selected for the design of this board. The central part of the OBDT is the FPGA
PolarFire MPF300T [4] from Microsemi. The OBDT can read out as many as 240 cells of a DT
chamber. In order to cover all front-end channels of a particular DT chamber, 3 to 5 OBDTs are
installed inside the chamber’s minicrate compartment. The particular number depends on the size of
the DT chamber (i.e. the number of chamber’s cells). The LVDS signals coming to the OBDT are
grouped in ribbon cables connected to 8 connectors with 34 LVDS signal pairs each. The LVDS pairs
are then routed directly to the FPGA. If the front-end circuit of a cell sends a pulse on the respective
LVDS line, the arrival time of this pulse is registered with high precision (below 1 ns).
This precise time acquisition is performed by the TDC blocks, which are implemented inside
the FPGA. The basic functional blocks of TDCs are the deserializers commonly present in modern
FPGAs. In particular, for the PolarFire family, the deserializers used are called SERDES blocks [4].
Arrival times of all registered pulses (related to the clock of the LHC) as well as the respective channel
numbers are multiplexed inside the FPGA. The resulting data packages are sent via the 4 optical
transceivers driven by the same FPGA. These optical transceivers are present inside the QSFP+ cage
of the OBDT. As described previously in section 2.2, the fiber optic links are connected on the other
side to the AB7 board.
Next to the FPGA, is the GBTx [5] chip. Like the FPGA itself, this chip also drives a fiber
optic communication link dedicated to a separate SFP+ cage of the OBDT. This link is connected on
the other end to the MoCo board (see section 2.2). This communication hosts the up and downstream
transfers of the slow-control and clock data packages. The GBTx chip extracts the LHC clock signal
and forwards it to the FPGA. Along with the FPGA, the GBTx chip also communicates with the SCA
chip [5] on the OBDT. The SCA is a proprietary CERN development, which acquires slow analog
signals (temperature sensors, measurement of voltage rails) and controls a few I2C links. The data link
between the GBTx and the SCA is based on the E-Link interface standard [6]. As a reserve solution,
one E-link interface is also provided between the GBTx and FPGA chips. A block-scheme of the
OBDT board with its fiberglass links is sketched in Figure 2.
Figure 2. OBDT Block structure and external interfaces
456
Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019)
Budva, Becici, Montenegro, September 30 – October 4, 2019
Figure 3. Assembled OBDT prototype
3.2 Production status and tests
A set of 14 OBDT prototypes have been produced for a series of tests described below. One of
the assembled OBDT prototypes is shown in Figure 3. A subset of 12 OBDTs are mounted on the
neighboring DT chambers of the second wheel of the CMS detector (positive side). Since all the
OBDTs are in the same wheel-sector, the series of tests is often referred to as a Slice Test. The OBDTs
participating in the slice tests are subdivided into two groups:
• Group for integration tests: 6 OBDTs were mounted in the minicrate compartments of the
MB1 and MB2 DT chambers. All possible integration activities are provided for these boards
in order to test the final configuration of the cooling, mechanics, and cabling. A further
important aspect here is to obtain the understanding of time and material resources needed for
the replacement of the considered components.
• Group for read-out validation tests: 6 OBDTs were mounted on the MB3 (2 boards) and
MB4 (4 boards) DT chambers. The OBDTs of this group are working in parallel with the
legacy chain. This is made possible with help of additional splitter boards, inserted directly
after the front-end circuits. These splitters provide analog splitting of all front-end signals,
ensuring that the signals from every front-end channel can be simultaneously acquired by both
legacy and Phase 2 read-out chains. This is needed to validate the correctness of the hit data,
acquired by the new read-out chain. The plot in Figure 4 demonstrates the time coincidence of
the registered hits acquired by both Phase 2 and legacy read-out chains.
Figure 4. Time coincidence demonstration of the Phase 2 and legacy chains.
Plot: Lourdes Urda, Jonas Roemer
457
Proceedings of the 27th International Symposium Nuclear Electronics and Computing (NEC’2019)
Budva, Becici, Montenegro, September 30 – October 4, 2019
4. Conclusion
A new read-out chain was developed for the CMS DT system in preparation for LHC
Phase 2. The new read-out chain offers improved architecture using fewer standalone
functional blocks. The design of the new on-chamber electronics (OBDT) ensures a precision
for registering the hit events below 1 ns. The first set of OBDT prototypes was produced.
Ongoing tests show the expected performance of the whole read-out chain. The final version
of the DT chamber electronics will be developed based on the considered OBDT prototype.
References
[1] BEDOYA, C. Fernandez, et al. Electronics for the CMS muon drift tube chambers: the read-out
minicrate // IEEE transactions on nuclear science, 2005, 52. Jg., Nr. 4, S. 944-949.
DOI:10.1109/TNS.2005.852698
[2] DE REMIGIS, P., et al. The CUOF optical link for the electronic system upgrade of the CMS
Muon DT // Journal of Instrumentation, 2014, 9. Jg., Nr. 02, S. C02020.
[3] NAVARRO TOBAR, A. et al. CMS Drift Tubes readout Phase 1 upgrade // TWEPP Conference
Report 2018 (No. CMS-CR-2018-298).
[4] MICROSEMI. PolarFire FPGA // Datasheet DS0141. Available at Microsemi document portal
https://www.microsemi.com/product-directory/fpgas/3854-polarfire-fpgas#documentation,
accessed on 10.11.2019
[5] CARATELLI, A., et al. The GBT-SCA, a radiation tolerant ASIC for detector control and
monitoring applications in HEP experiments. // Journal of Instrumentation, 2015, 10. Jg., Nr. 03, S.
C03034.
[6] BONACINI, S.; MOREIRA, P.; KLOUKINAS, K. E-link: A radiation-hard low-power electrical
link for chip-to-chip communication. // TWEPP Conference Report 2009. DOI:10.5170/CERN-2009-
006.422
458