With an ever-increasing number of connected devices (e.g., IoT), cloud computing faces eficiency challenges due to complex infrastructure, high communication costs, and privacy. Fog and edge computing enable computing closer to data sources, ofering alternatives to the limitations of relying exclusively on the cloud. When combined with high-performance cloud platforms, fog, and edge devices form a computing continuum. However, the continuum challenges designers who need to compile and deploy on distributed and heterogeneous devices and optimize for a diverse set of nonfunctional requirements. To ease the usage and ensure the full potential of the continuum, a Design and Programming Environment (DPE) that is interoperable, reusable, portable, and cross-layer is needed. In this context, the Multi-Level Intermediate Representation (MLIR) becomes vital since it provides an extensible and reusable compiler infrastructure. The project development of a continuum-oriented DPE leveraging the MLIR infrastructure is discussed in this paper as a work in progress.
Cloud computing has emerged as a critical technology in the industry over the past years
due to its flexibility in managing information and resources across the Internet. It has also
relieved users from the burden of configuring their working environments, allowing them to
reduce infrastructural costs. However, in recent years, the rise of Artificial Intelligence (AI)
related technologies and the Internet-of-Things (IoT) has made relying solely on cloud-based
computing increasingly challenging. This is due to the significant energy consumption and
communication costs associated with real-time interactions between the cloud and devices.
New computing paradigms, such as fog computing and edge computing, have been introduced
as extensions. These approaches aim to address the limitations of cloud-based computing
by distributing computational tasks closer to the data source. Cloud, edge, and fog form a
computing continuum [
The MYRTUS [
Specific Languages (DSLs) to enhance developer eficiency and ease of use; • Support diferent architectures with a focus on accelerators for eficient processing, such as Coarse-Grained Reconfigurable Architectures (CGRAs) and Field-Programmable Gate Arrays (FPGAs); • Provide automatic insertion of adaptivity knobs for runtime adaptation.
The MLIR project will be leveraged to develop the features above in a unified compilation lfow. MLIR ofers us a framework in which we can extend our needs for the continuum, reusing state-of-the-art compilation tools like Low-Level Virtual Machine (LLVM)’s backends and optimizers, supporting hardware heterogeneity, and integrating external tools that will facilitate the construction of the DPE’s NLOP.
This section presents a brief background on the concepts and tools that will serve to develop the
NLOP. This includes, a brief introduction to MLIR, initial MLIR-based infrastructure developed
in a previous EU project, and fundamentals of the adaptable models of computation.
2.1. MLIR
MLIR is a promising framework for constructing reusable and extensible compiler infrastructure.
It aims to tackle software fragmentation, enhance compilation for diverse hardware systems,
considerably lower the expenses associated with developing domain-specific compilers, and
facilitate the integration of existing compilers. It extends the monolithic LLVM IR into multi-level
abstractions, each of which serves its own purpose and has specific functionalities, such as
arith for arithmetic operations and linalg for linear algebra. The infrastructure of MLIR
makes it possible to seamlessly transition from abstract, high-level representations to concrete
executable code. This makes MLIR an enabler for the eficient implementation of DSLs and other
programming constructs [
In MLIR, we can roughly understand that everything is an operation. These operations exist in diferent dialects, each serving its own abstraction level. MLIR allows users to define custom dialects, along with custom types, interfaces, etc. One of the most important infrastructure within MLIR is Pass, with which we can lower or convert one dialect to another by giving several rewrite patterns.
External frontends onnx torch tosa
quant EVEREST frontends
cfdlang base2
ekl Condrust/ohua
Entry dialects jabbah esn teil cyclic
ub MLIR dialects tensor affine linalg buffer dfg evp olympus Coordination, integration, backend
bit Arithmetic support External backends gpu hw fsm A compelling example of how the MLIR multi-level abstractions can be leveraged is demonstrated in the Software Development Kit (SDK) of the EVEREST project.
EVEREST is a H2020 EU project
that aims to simplify the
development of complex big data
applications for FPGA-based data
centers [
The main dialects and their relations are shown in Figure 1. Machine Learning (ML)
applications from tvm can be translated into the jabbah dialect. The SDK also includes
dialects for kernel language frontend (ekl), the coordination dialect dfg-mlir, and a DSL
cfdlang. ekl and cfdlang can be converted to an MLIR implementation of the intermediate
tensor language teil [
The dfg-mlir of the EVEREST SDK will be extended to cater for requirements in MYRTUS. In the dfg-mlir dialect, a user can define an Homogeneous Synchronous Data-Flow (HSDF) node using a custom operation dfg.operator (see Figure 2a). Users can define input and output ports and perform any operations with them. The definition of ports is followed by an MLIR region with only one block. Users can use any MLIR operation inside this region such as arith.addi from the arith dialect. The returned result is an MLIR Value that can be used in other operations as operands. An Output operation indicates which values should be output. The input/output ports are connected to channels, which are implicitly pulled/pushed from/to at the beginning and end of the region.
For broader modelling of a Data-Flow Graph (DFG), dfg-mlir also provides a Process operation. This operation has a similar syntax to an Operator but is capable of describing a Kahn Process Network (KPN) node, which means that users can pull/push from/to the channels multiple times. There is a Pass inside dfg-mlir, which can convert every Operator to the equivalent Process operation, as shown in Figure 2b.
dfg-mlir supports diferent hardware platforms, enabling parallel execution of DFGs. The
CPU backend, for instnace, lowers to OpenMP. For hardware generation, dfg-mlir can be
lowered to Olympus [
For DSE, tools like Mocasin can be utilized. Mocasin [
Dataflow lacks reactive behavior to inputs from the environment which is key in the context of Cyber-Physical System (CPS). Recently, LinguaFranca [17] emerged as a coordination language for CPSs, extending dataflow with time semantics using the discrete event model with explicit semantics of time [18]. LinguaFranca adopts the reactor model [19] and supports various runtimes capable of concurrent and distributed execution. The reactor model also supports topological changes to the underlying dataflow graph for adaptable execution. In MYRTUS, we will borrow ideas from LinguaFranca to enable reactive and adaptive execution in the computing continuum.
This section gives an overview of the ongoing works and plans for the MYRTUS DPE’s NLOP . First, a general overview of the NLOP , including its main components, inputs, and outputs, is given. Next, we detail two work fronts currently taking place for extending the dfg-mlir dialect for the NLOP.
While the PyTorch models can be translated into torch-mlir dialect, all inputs (C/C++ or PyTorch) are converted into the MLIR domain. Additionally, NLOP will support techniques such as in [21] in MLIR to further optimize the computation of PyTorch models. These techniques are applied at the PyTorch level to save execution time and energy.
Subsequently, passes will be implemented to translate MLIR programs from the previous step into custom dialects (see Figure 3), including dfg-mlir. This process includes automatic DFG recognition and generation, converting them to the custom dialects while maintaining the same semantics. Once the program in custom dialects is generated, several analyses and optimizations will be performed to obtain the quasi-optimal DFG based on a cost function or similar technology. To that end, Mocasin will be used to run simulations and DSE to identify the best mapping and partitioning for deployment on heterogeneous nodes. Finally, the NLOP generates low-level Intermediate Representation (IR) or code for the supported FPGA platforms with MDC [22] and CGRAs with STRELA [23]. 3.2. dfg-mlir with Mocasin We assume that all the nodes in the generated dfg-mlir program will be an Opeartor, which means from/to each port, we only pull/push one data in each iteration. However, there is a limitation with the Operator operation shown in Figure 2a: it lacks the ability to take values from the previous iteration. With the syntax of Single Static Assignment (SSA), it is illegal to directly use the result value as operand. This limits our ability to translate a wider range of applications into an Opeartor in dfg-mlir (e.g., a Multiply Accumulate (MAC) operation). dfg.operator @mac To address this issue, we introduced the iteration inputs(%in0: i32, %in1: i32) arguments syntax to Operator. As shown in Figure 4, outputs(%out: i32) an iter_args list can be added after defining the i n i tiiatleirz_ear g{s(%sum: i32) input and output ports. If this list is present, an %0 = arith.constant 0 : i32 initialize region must be appended to provide the dfg.yield %0 : i32 initial values for each iteration argument. In the } {%0 = arith.muli %in0, %in1 : i32 body region, these iteration arguments can be used %1 = arith.addi %0, %sum : i32 like any other values in any operation. To pass the } ddffgg..oyuitepludt %1%1 :: i3i232 ruesseudlttoofupcudarrteenttheitmeraattiothneteondthoef nOepxeta, ratoYri.eld is
To integrate with Mocasin we implemented a YAML Figure 4: Iteration arguments support reader as well as a new CGRA platform. Within LLVM, we developed a transformation pass that outputs the internal dataflow of an Operator to a YAML file. This file contains the information on each node, their ports and the channels connecting them. If there is iteration argument, an initial token will be added in the channel, representing a backedge in the loop graphically.
As mentioned in Section 2, we will expand the semantics of the underlying MoC to account for runtime adaptivity and reactive behavior. 3.3. dfg-mlir atop CIRCT Currently, CIRCT relies on Polygeist to read in C/C++ programs into MLIR. Each function is then turned into CIRCT’s entry dialect, called handshake. Passes are available to lower handshake into low-level dialects within CIRCT, ultimately generating System Verilog code. However, handshake can only describe HSDFs, as it assumes that users will only pull/push one data from/to the ports by default. More expressive computational graphs coming from high-level DSLs, e.g., using Synchronous Data-Flow (SDF) or KPN semantics, cannot use CIRCT as backend at the moment.
As discussed in Section 2, dfg-mlir supports more expressive MoCs (ProcessOp for KPNs). Within dfg-mlir, we have also implemented passes that can directly generate low-level dialects in CIRCT before generating the System Verilog code, such as fsm for Finite State Machine (FSM) generation and sv for SystemVerilog syntax. dfg-mlir also uses elastic circuit for each port, the same as handshake, meaning each port will be converted into three signals: valid, data, and ready. For the multiple pulls/pushes behavior in a KPN, we will generate a FSM to correctly handle the elastic circuit signals.
Another consideration is that handshake only inserts a buffer operation, which has a capacity of two elements between two ports for synchronizing diferent modules. In contrast, we implement a more flexible channel inspired by Chisel [24]. Currently, the channel’s capacity is manually controlled. By utilizing the work of Josipović et al. [25], we aim to improve the sizing of channels automatically. With all these features, our approach allows users to have a more powerful way to describe a wider range of DFGs. In the project, this will be extended to support reactive and adaptive execution.
In this paper, we introduced current eforts to implement the NLOP phase of the MYRTUS DPE. Naturally, some challenges remain to be tackled throughout the NLOP development , including: • System integration: This involves connecting application components, utilizing various abstractions, navigating among transformations and trade-ofs in heterogeneous and distributed computing environments, with custom MLIR dialects code generation being the ifnal step. • CGRA mapping: The integration of dfg-mlir and Mocasin for CGRA mapping exploration will not be limited to supporting specific architectures such as STRELA in Figure 3. The approach will support diferent CGRAs by accepting architecture properties. • CIRCT extension: CIRCT ofers an alternative approach to HLS but has some limitations that must be addressed. For instance, the handshake dialect can adopt the dfg-mlir semantics. Another critical extension is the support for pipelining, which is typically available in HLS tools as pragmas. To avoid vendor-locking, how to automatically apply diferent optimization pragmas will also be explored. • Adaptive MoC execution: Applying HAM at the MLIR level and taking LinguaFranca’s time semantics could also be explored.
By addressing these points, the DPE’s NLOP can fully leverage the resources of the continuum.
This work was funded by the EU Horizon Europe Programme under grant agreement No 101135183 (MYRTUS). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. Mocasin—rapid prototyping of rapid prototyping tools: A framework for exploring new approaches in mapping software to heterogeneous multi-cores, in: Proceedings of the 2021 Drone Systems Engineering and Rapid Simulation and Performance Evaluation: Methods and Tools Proceedings, 2021, pp. 66–73. [17] C. Menard, M. Lohstroh, S. Bateni, M. Chorlian, A. Deng, P. Donovan, C. Fournier, S. Lin, F. Suchert, T. Tanneberger, H. Kim, J. Castrillon, E. A. Lee, High-performance deterministic concurrency using lingua franca, ACM Transactions on Architecture and Code Optimization (TACO) 20 (2023) 1–29. URL: https://doi.org/10.1145/3617687. doi:10.1145/3617687. [18] M. Lohstroh, C. Menard, A. Schulz-Rosengarten, M. Weber, J. Castrillon, E. A. Lee, A language for deterministic coordination across multiple timelines, in: 2020 Forum for Specification and Design Languages (FDL), 2020, pp. 1–8. URL: https://ieeexplore.ieee. org/document/9232939. doi:10.1109/FDL50818.2020.9232939. [19] M. Lohstroh, Í. Í. Romero, A. Goens, P. Derler, J. Castrillon, E. A. Lee, A. SangiovanniVincentelli, Reactors: A deterministic model for composable reactive systems, in: R. Chamberlain, M. Edin Grimheden, W. Taha (Eds.), Cyber Physical Systems. ModelBased Design – Proceedings of the 9th Workshop on Design, Modeling and Evaluation of Cyber Physical Systems (CyPhy 2019) and the Workshop on Embedded and CyberPhysical Systems Education (WESE 2019), Springer International Publishing, Cham, 2020, pp. 59–85. URL: https://link.springer.com/chapter/10.1007/978-3-030-41131-2_4. doi:10.1007/978-3-030-41131-2_4. [20] W. S. Moses, L. Chelini, R. Zhao, O. Zinenko, Polygeist: Raising c to polyhedral mlir, in: 2021 30th International Conference on Parallel Architectures and Compilation Techniques (PACT), IEEE, 2021, pp. 45–59. [21] G. Korol, M. G. Jordan, M. B. Rutzig, J. Castrillon, A. C. S. Beck, Pruning and early-exit co-optimization for cnn acceleration on fpgas, in: 2023 Design, Automation and Test in Europe Conference and Exhibition (DATE), 2023, pp. 1–6. doi:10.23919/DATE56975.2023. 10137244. [22] F. Manca, F. Ratto, F. Palumbo, Onnx-to-hardware design flow for adaptive neural-network inference on fpgas, arXiv preprint arXiv:2406.09078 (2024). [23] D. Vazquez, J. Miranda, A. Rodriguez, A. Otero, P. D. Schiavone, D. Atienza, Strela: Streaming elastic cgra accelerator for embedded systems, arXiv preprint arXiv:2404.12503 (2024). [24] J. Bachrach, H. Vo, B. Richards, Y. Lee, A. Waterman, R. Avižienis, J. Wawrzynek, K. Asanović, Chisel: constructing hardware in a scala embedded language, in: Proceedings of the 49th Annual Design Automation Conference, 2012, pp. 1216–1225. [25] L. Josipović, S. Sheikhha, A. Guerrieri, P. Ienne, J. Cortadella, Bufer placement and sizing for high-performance dataflow circuits, ACM Transactions on Reconfigurable Technology and Systems (TRETS) 15 (2021) 1–32.