The evolution of programming languages and their runtime systems has been a continuous journey toward eficiency and performance. One such significant milestone in this journey is the introduction of Just-In-Time (JIT) compilation in the Erlang/Open-Telecom-Platform (OTP). This paper presents an in-depth investigation into the impact of JIT compilation in Erlang/OTP applications. We demonstrate our cross-platform measurement framework and evaluate a few problems on Windows and Linux with diferent BEAM versions. The findings emphasise the critical role of both proficient algorithm design and runtime optimisations in minimizing energy consumption and enhancing performance.
(2) How does the input size and implementation behave when used with and without JIT compilation? This question focuses on the performance implications of varying input sizes and implementations under both JIT and non-JIT compilation conditions. By analyzing how diferent factors influence performance, we aim to identify any significant variations and understand the conditions under which JIT provides the most benefit.
Besides the analysis of the BEAM JIT, we are presenting a cross-platform measurement framework for
Erlang built as an extension of our Scaphandre-based [
The rest of the paper is structured as follows. We start with context and related work presented in 2. In Section 3 we summarise our previous results on Erlang green coding measurements. In Section 4 we show the extension of our measurement framework originally developed for Windows. Section 5 details the performed measurement on Windows and Linux and discusses our main findings. Finally, Section 6 concludes the paper.
The concept of JIT compilation is not exclusive to Erlang/OTP but is also prevalent in other programming
languages, notably Java. The Java Virtual Machine (JVM) has been a subject of extensive research and
development with regards to JIT compilation and its impact on energy consumption [
In the context of JVM, JIT compilation has been found to significantly reduce energy consumption
by reducing instruction counts. However, it is worth noting that certain components such as garbage
collection, which is an integral part of JVM, can incur runtime overhead that consumes more energy.
Despite this, both JIT optimisation and garbage collection have been found to decrease the average
power dissipated by a program[
A study evaluating the impact of diferent JVM distributions and parameters on energy consumption found that some JVM platforms can exhibit up to 100% more energy consumption [7]. Interestingly, the default configuration of the garbage collector was found to be energy eficient in only 50% of the experiments. This highlights the importance of carefully selecting and configuring the JVM distribution for energy eficiency.
In another study, the energy consumption of diferent Java I/O libraries and methods was compared, to reduce the overall energy consumption by substituting default I/O methods with ones that reported better energy eficiency [8].
These studies demonstrate that similar to Erlang/OTP, JIT compilation and careful configuration of the runtime environment play a crucial role in the energy eficiency of Java applications. This underlines the importance of continued research and development in this area to further optimise the energy consumption of software systems.
Compiler optimisation is another critical area that significantly impacts the performance of a
program [9, 10, 11]. Optimizing compilers apply a series of transformations to the intermediate code to
produce a semantically equivalent output program that uses fewer resources or executes faster. The
latest OTP has seen enhancements in type-based optimisations and other performance improvements
in the compiler and JIT [
In their comparative analysis, Sayed Najmuddin et al [12] investigated power and battery consumption in Windows operating systems and modern Linux distributions (such as Ubuntu, Fedora, Debian, Red Hat, and others). They highlighted the critical role of driver quality in Linux power management. Often, poorly written or incompatible drivers hinder eficiency. The researchers also pointed out that some Linux kernel versions are suboptimal for mobile systems, as they primarily target desktop platforms. They recommended that users should carefully choose an eficient kernel to enhance power and battery performance on Linux. These insights underscore the significant diferences between Windows and Linux in terms of power management.
Several Erlang applications are based on continuously running servers that consume lots of energy.
Therefore, analysing the energy behaviour of Erlang applications has a significant impact on
sustainability. Providing tools to measure the energy usage and introducing patterns to follow when creating
energy aware applications are needed by the community. In our previous research [
In our previous research, we aimed to extend the GreenErl [13] framework by integrating the
Scaphandre [
The integration of the Scaphandre tool into the GreenErl framework represents a significant milestone in enabling precise energy measurements for Erlang functions on Windows. This advancement bridges the gap left by the absence of traditional energy measurement tools like RAPL on Windows. By providing consistent and reliable data collection, the Scaphandre-based GreenErl framework serves as a vital entry point for our broader research into energy eficiency. It facilitates detailed energy profiling of various Erlang constructs. This work not only enhances our understanding of energy dynamics in software but also lays the groundwork for future research aimed at developing a cross-platform framework.
Understanding the energy eficiency of diferent programming languages, especially in the context of JIT compilation in Erlang, is critical for developing sustainable software. Our previous work [17] utilised a subset of the Computer Language Benchmarks Game (CLBG)1 and introduced a server implementation specifically for Erlang. These studies provide a foundation for our current research and underscore the importance of energy-eficient programming.
Building on the work from our study [17], our current research aims to delve deeper into the impact of JIT compilation in Erlang on energy eficiency and performance. By comparing results across diferent operating systems, we seek to provide a comprehensive understanding of how JIT optimizations can enhance energy eficiency in Erlang applications.
The integration of the Scaphandre tool into the GreenErl framework marked a significant advancement in our ability to measure energy consumption and runtime of Erlang functions not only on Windows but for enabling a cross-platform GreenErl framework. This new framework provides a robust solution for overcoming the limitations of traditional energy measurement tools on Windows, such as the absence of 1https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html RAPL support. It enables precise and reliable energy consumption data collection for Erlang functions, allowing for a comprehensive analysis of their performance across multiple operating systems.
For this benchmark evaluation, we utilized a Lenovo ThinkPad X13 Gen 1 laptop, which is equipped with an Intel® Core™ i7-10610U processor featuring 8 cores and 16 threads. The system is configured with 32 GB of memory. The tests were conducted on two operating systems: Windows 11 Pro and Debian GNU/Linux 12 (Bookworm), both running in their 64-bit versions.
Scaphandre’s integration ensures that energy measurements can be consistently collected on both Linux and Windows. This compatibility is crucial for conducting comparative studies and for developing energy-eficient software that performs optimally across various environments.
Additionally, the new framework allows for in-depth energy profiling of diferent Erlang constructs, we can obtain granular insights into the energy usage of Erlang programs, which is essential for optimising performance and reducing energy consumption.
The new GreenErl framework consists of several integral components, working in harmony to measure and analyze energy consumption in Erlang functions. At its core is a Python script that orchestrates the entire process. This script handles critical tasks. At first, function spawning creates instances of Erlang functions for measurement. Then the script initiates the Scaphandre process, which collects energy data as functions are executing, and aggregates the collected data into a structured JSON ifle. Once an Erlang function completes execution, the same Python script takes charge of measurement termination that stops the energy measurement process. The last step is data analysis, where the script calculates the energy consumption metrics from the JSON files and stores the results in CSV files. This streamlined approach ensures precise energy profiling and optimization across platforms.
In our previous investigation [17], we conducted an in-depth analysis of Erlang’s energy usage, focusing on its performance in scenarios that align with its design principles and compared it with other programming languages. We also created a framework for the energy measurements.
In this new investigation, we delve into the impact of the JIT compiler in the latest OTP 26 on Erlang’s energy eficiency. We compare the performance of OTP 26 with that of OTP 23, using a few samples from our previous work such as a subset of the Benchmark Game and the server implementation. We also measured the n-queens problem implemented in two diferent ways [ 18]. These problems all together serve as our evaluation metrics23. To ensure a comprehensive comparison, we measured the same functions on identical hardware setups, with the only diference being the operating systems, Linux and Windows. This approach provides a clearer view of how the operating system influences the performance and energy eficiency of Erlang programs. However, we must state that some results show zeros, which could be attributed to the nature of the problem or the sampling tool. Futhermore, when the input size is small, the resulting energy may be so minimal that Scaphandre registers it as zero.
The Benchmark game and Server These implementations provide a comprehensive comparison of the performance and energy consumption of several compute-intensive programs from the Benchmark Game across diferent OTP versions.
The benchmark games present a set of problems that test the performance of the OTP versions under diferent computational tasks. In our new research, we used the fannkuch-redux, mandelbrot, and n-body problems. These problems provide a comprehensive evaluation of a language’s computational eficiency and performance.
The server implementation simulates a typical use case for Erlang, allowing us to measure its energy eficiency in a practical context. It provides a real-world scenario where Erlang’s strengths in handling 2https://github.com/joegharbi/SQAMIA_GREEN_JIT/tree/Linux 3https://github.com/joegharbi/SQAMIA_GREEN_JIT/tree/Windows concurrency and distribution can be efectively measured.
N-Queens The queens_list and n_queens modules both solve the N-Queens problem, which involves placing N number of queens on an NxN chessboard such that no two queens threaten each other. The queens_list module, uses a list comprehension to generate the configurations to check if a queen can be placed in a certain position. The n_queens module uses a diferent approach. It uses backtracking to ifnd a solution to the N-Queens problem.
We will start by sharing the measurement results and findings from our Windows analysis.
The Just-In-Time compiler in OTP 26 significantly optimises runtime and energy consumption in all problems measured. This is evident from the data shown in Table 1, which shows lower values for these parameters in OTP 26 compared to OTP 23. As an illustration, let us consider the n-body problem. In OTP 23, the energy consumption was 692.41J and the runtime was 193.98s. However, with the introduction of JIT in OTP 26, there was a significant improvement. The energy consumption was reduced to 312.29J and the runtime decreased to 84.03s. This represents a reduction in energy consumption by approximately 55% and a reduction in runtime by approximately 57%. This example demonstrates the eficiency improvements brought by the introduction of JIT in OTP 26.
Energy Consumption The energy consumption for both implementations seems to increase with the problem size on both OTP versions. However, the queens_list implementation appears to be more energy-eficient than the n_queens implementation, especially for larger problem sizes (i.e., when the number of queens is greater than 13). This could be due to the diferent algorithms used in the two implementations, with the queens_list implementation having a more eficient algorithm for handling larger problem sizes.
Runtime Performance The runtime also increases with the problem size for both implementations on both OTP versions. However, the n_queens implementation seems to have a slightly worse runtime performance for smaller problem sizes. During our experimental process, we encountered an issue where the execution of the function extended to a duration of 12 hours for an input size of N=14. This necessitated a manual intervention to halt the measurement, as further waiting was considered
Module
Arg unproductive. The observation suggests that for input sizes of 14 and beyond, the function n_queens exhibits an exponential increase in execution time. This highlights the non-linear complexity of the implementation under investigation due to the backtracking nature of the implementation. Power Usage The power usage seems to be fairly consistent across diferent problem sizes for both implementations on both OTP versions. This suggests that the energy eficiency of the implementations does not significantly change with the problem size.
OTP 23 vs OTP 26 For both implementations, the energy consumption and runtime seem slightly lower on OTP 26 compared to OTP 23. This could be due to the JIT compiler’s optimisations in OTP 26, which can lead to more eficient code execution and thus lower energy consumption and runtime.
In conclusion, while the queens_list implementation appears to be more energy-eficient, the n_queens implementation may ofer reasonable runtime performance for smaller problem sizes. However, for larger problem sizes, the n_queens implementation results in significantly longer runtime, as observed with N larger than 14. These findings highlight the trade-ofs between energy eficiency and runtime performance in the context of the N-Queens problem implemented in two distinct ways. It also shows the improvements in the OTP 26 using the JIT compiler compared to the older OTP 23 without the JIT compiler.
The investigation into the energy usage of Erlang programs on Linux follows a similar methodology as the analysis on Windows. This involves comparing diferent versions of OTP (Open Telecom Platform) to evaluate improvements brought by the Just-In-Time (JIT) compiler introduced in OTP 24 and the further enhancements in OTP 26 and 27.
The Echo Server and Benchmark Game use cases provide practical scenarios for evaluating the energy eficiency and performance of Erlang. The focus is on typical computational tasks and server operations to understand the impact of the JIT compiler on energy consumption and runtime. Benchmarks OTP 23 vs 26 on Linux The data presented in Table 3 indicates that OTP 26 ofers significant improvements in both energy consumption and runtime for most benchmarks. The fannkuchredux and mandelbrot modules show notable reductions in runtime and energy usage, illustrating the eficiency gains from the JIT compiler in OTP 26. The power usage shows a notable decrease, especially for mandelbrot, suggesting that the energy eficiency per unit of time has improved with OTP 26. Benchmarks OTP 26 vs 27 on Linux The diferences between OTP 26 and OTP 27 are more nuanced (Table 4). While OTP 27 shows marginal improvements in some areas, such as the fannkuch-redux and server benchmarks, it does not consistently outperform OTP 26 across all metrics. OTP 27 generally shows slight improvements in runtime compared to OTP 26, though not uniformly across all benchmarks. The ratios indicate minor improvements or slight declines, suggesting that while there are optimisations in OTP 27, the benefits are not as pronounced as the jump from OTP 23 to OTP 26. This suggests that while further optimisations are present in OTP 27, the benefits may be more specific to certain workloads.
The N-Queens problem provides another critical evaluation metric, examining both the n_queens and queens_list implementations for their energy usage and runtime performance across diferent OTP versions.
N Queens OTP 23 vs 26 on Linux The data presented in Table 5 shows that OTP 26 ofers better performance and energy eficiency than OTP 23 across both implementations. For the n_queens implementation, the reductions in energy usage and runtime are particularly significant for larger problem sizes. The queens_list implementation also benefits from these improvements, though the diferences are more pronounced in the larger problem sizes. The power usage generally improves, indicating better energy eficiency per unit of time with OTP 26.
N Queens OTP 26 vs 27 on Linux The transition from OTP 26 to OTP 27 showed better overall results (Table 6). However, while most scenarios demonstrate energy savings and reduced runtimes, some do not show a consistent improvement such as the n_queens with input 10 and 12, reflecting the
Module
Arg nuanced impact of optimisations specific to diferent types of workloads. The ratios are also variable, reflecting mixed improvements in energy eficiency.
The analysis of energy usage and runtime performance of Erlang programs on Linux mirrors the patterns observed on Windows. The JIT compiler introduced in OTP 24 as well as the test versions 26 and 27 ofers substantial improvements over OTP 23. Further enhancements in OTP 27 provide additional benefits. These findings underscore the importance of considering specific workload characteristics when evaluating the impact of compiler and runtime optimisations.
The detailed benchmarks across diferent Erlang modules and problem sizes illustrate the potential for significant energy savings and performance gains with newer OTP versions. However, the degree of improvement varies, indicating the complex interplay between program characteristics and runtime optimisations.
In conclusion, the move to OTP 26 and 27 on Linux demonstrates clear advantages in energy eficiency and runtime performance for many Erlang programs, aligning with the trends observed on Windows. This is not the first time we discover that Windows is generally more optimised as Sayed Najmuddin et al [12] found that Windows generally exhibits more eficient power and battery management than Linux, largely due to better driver support and optimisation. In addition, we conclude that choice of OTP version should consider the specific workload to maximise the benefits of these advancements.
In our previous research, we developed a framework for measuring the energy consumption of Erlang applications and investigated the energy behaviour of diferent Erlang implementations. In this paper, we demonstrated a cross-platform solution to carry out energy-related measurements of Erlang application. Our solution is based on Scaphandre and RAPL.
Using this framework we have investigated the energy behaviour of the Erlang BEAM virtual machine and compared OTP 23 (non-JIT) implementations with OTP 26 implementations (JIT). In addition, we have analysed the recent improvement to the BEAM JIT by comparing OTP 26 and OTP 27. we have addressed our first research question.
Furthermore, our analysis has demonstrated that irrespective of whether a JIT or non-JIT compiler is used, the implementation approach of the software profoundly influences its energy consumption. This ifnding addresses our second research question and underscores the critical role of software design in energy eficiency.
This research has made significant progress in understanding the performance of various algorithms in the Erlang runtime system. The introduction of JIT compilation in OTP 26 has shown to significantly improve the runtime and energy eficiency of these algorithms.
The research also highlighted the implicit eficiency of certain implementations, such as the queens_list module. These findings underscore the importance of both eficient algorithm design and runtime optimisations in reducing energy consumption and improving performance.
Looking ahead, there are several promising directions for future research. One potential path is to extend this analysis to other algorithms in Erlang, to gain a more comprehensive understanding of the performance characteristics of the Erlang runtime system. The findings of this research provide a solid foundation for future investigations in this dynamic and rapidly progressing field.
Project no. TKP2021-NVA-29 has been implemented with the support provided by the Ministry of Culture and Innovation of Hungary from the National Research, Development and Innovation Fund, ifnanced under the TKP2021-NVA funding scheme. [7] Z. Ournani, M. C. Belgaid, R. Rouvoy, P. Rust, J. Penhoat, Evaluating the Impact of Java Virtual Machines on Energy Consumption, in: Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), ESEM ’21, Association for Computing Machinery, New York, NY, USA, 2021. doi:10.1145/3475716.3475774. [8] Z. Ournani, R. Rouvoy, P. Rust, J. Penhoat, Comparing the Energy Consumption of Java I/O Libraries and Methods, in: ICSME 2021 - 37th International Conference on Software Maintenance and Evolution, Proceedings of the 37th International Conference on Software Maintenance and Evolution (ICSME), Luxembourg / Virtual, Luxembourg, 2021. URL: https://inria.hal.science/ hal-03269129.
[9] S. Muchnick, Advanced compiler design and implementation, Morgan Kaufmann, 1997. [10] S. Wong, Super optimization, 2020. URL: https://www.cs.cornell.edu/courses/cs6120/2020fa/blog/ super-optimization/. [11] GCC, Optimize options (using the gnu compiler collection (gcc)), 2014. URL: https://gcc.gnu.org/ onlinedocs/gcc/Optimize-Options.html. [12] S. Najmuddin, Z. Atal, R. A. Ziar, Comparative analysis of power consumption of the linux and its distribution operating systems vs windows and mac operating systems., Kardan journal of engineering and technology (2021). doi:10.31841/KJET.2021.21. [13] G. Nagy, A. A. Mészáros, I. Bozó, M. Tóth, Tools supporting green computing in Erlang, in: Proceedings of the 18th ACM SIGPLAN International Workshop on Erlang, Erlang 2019, Association for Computing Machinery, New York, NY, USA, 2019, p. 30–35. doi:10.1145/3331542.3342570. [14] V. M. Weaver, Reading RAPL energy measurements from Linux, http://web.eece.maine.edu/~vweaver/projects/rapl/, 2015. [15] Hubblo, Windows driver to get RAPL metrics, 2023. URL: https://github.com/hubblo-org/ windows-rapl-driver. [16] R. Pereira, M. Couto, F. Ribeiro, R. Rua, J. Cunha, J. P. Fernandes, J. Saraiva, Energy eficiency across programming languages: how do energy, time, and memory relate?, Proceedings of the 10th ACM SIGPLAN International Conference on Software Language Engineering - SLE 2017 (2017). doi:10.1145/3136014.3136031. [17] Y. Gharbi, M. Tóth, I. Bozó, Measuring and Analysing Erlang’s Energy Usage, in: 2024 47th MIPRO ICT and Electronics Convention (MIPRO), 2024, pp. 1993–1998. doi:10.1109/MIPRO60963.2024. 10569977. [18] N-queens problem - rosetta code, 2024. URL: https://rosettacode.org/wiki/N-queens_problem# Erlang.