Developing serverless computing platforms requires new approaches to scale computing resources optimally. Existing scaling mechanisms often cause performance issues, including cold starts, throttling, and increased costs due to the over-allocation of resources. This paper proposes an approach based on ML models that use Amazon SQS queue metrics to predict load and pre-scale Lambda functions in a distributed computing system. The research aims to develop a ready-to-use solution within the serverless computing model. The model estimates the queue depth and message arrival rate using time series forecasting using the DeepAR algorithm from AWS SageMaker. The MAE (mean absolute error) and RMSE (root mean square error) metrics assess the model's accuracy. The DeepAR model shows an MAE of 10019.21 and RMSE of 13140.91, indicating good prediction quality. The proposed solution demonstrates the capabilities of serverless platforms to solve the problem of predictive autoscaling being an effective tool for serverless applications. Empirical verification demonstrates improved performance indicators, including a decrease in the number of cold starts by (approximately) 27% and the number of unprocessed requests by (approximately) 14%.
The emergence and development of serverless computing platforms have changed the general approach to deploy and scale applications in cloud environments. With this approach, managing computing resources is mainly the cloud provider's responsibility, this allows developers to focus on writing code.
One key benefit of serverless platforms is the automatic scaling of applications in accordance with changes in load. Autoscaling implementation on such platforms requires that services should dynamically scale their resources, considering the change in the number of requests per unit of time. Thus, computing resources for user applications can be released entirely without traffic and reallocated when new requests appear, ensuring high resource efficiency.
The disadvantage of such active resource allocation is the notable initialization time. This
disadvantage is known as the “cold start” problem when time is spent on deploying and configuring
the runtime environment [
This problem becomes especially critical for services with short processing times for incoming requests, which are the majority in serverless environments, and is complicated by the fact that the created infrastructure is shared by several applications, each of which requires an individual approach to scaling solutions.
One approach to address this problem is to apply machine learning techniques to predict
workloads and automatically scale compute resources in AWS serverless environments [
The research goal is to develop a ready-to-use solution that allows us to maintain optimal performance with balanced resource utilization in cloud environments while remaining within the serverless computing model.
Many researchers are actively studying methods how to automatically scale computing resources in cloud environments. The goal of such studies is to increase application performance and ensure efficient use of cloud provider resources, which is especially critical for serverless architectures.
The traditional method for automatic scaling relies on specific resource usage or requests per
second thresholds that trigger the scaling process. This approach is easy to implement but does not
solve the problem of load spikes. For example, a cloud provider can maintain a number of prepared
containers and delay deleting function instances after launches to efficiently handle new requests.
However, this is economically inefficient since the user of serverless solutions pays only for the
resources used regardless of when these resources were allocated. “Cold start” delays will still be
present in the case of load spikes. The disadvantages and limitations of this approach are discussed
in [
One approach considered in studies on automatic scaling for serverless applications is to reduce
the number of “cold starts”. The central aspect of such an approach is the ability to predict the load
level to create the necessary resources proactively [
This becomes especially critical for serverless applications that use queues and event-driven
architectures due to sudden changes in load and data sparsity [
Solving the problem of “cold start” delays requires methods that allow predicting future
workloads based on historical data and real-time indicators. Thus, in [
Time series analysis is another popular technique for workload prediction. This technique
examines historical data and uses statistical approaches to identify trends [
A major challenge remains deploying machine learning models in the cloud and integrating them into existing serverless applications. The study [12] proposes a template for automated deployment of AWS services for training and evaluating models, which confirms the possibility of using predictive autoscaling in various serverless applications.
Despite many studies on automatic scaling and load forecasting of cloud resources, relatively little attention is paid to serverless computing models. Our research aims to create a solution for a serverless distributed application that uses message queues to automatically scale resources based on metrics and machine learning algorithms.
We developed and launched a distributed application designed for batch data processing in AWS, which is used for load forecasting and automatic resource scaling. The application's main feature is its use of serverless technologies (Fig. 1).
The main task of the developed application is to ensure distributed processing of large amounts of data with a minimum number of errors. In these systems, a queue, typically AWS SQS, is the main component, receiving events and distributing data to a set maximum number of processors, like AWS Lambda. Depending on the current load, only the number of processors required at a particular moment in time within the quota is started. Input data and processing results are stored on S3.
Another component of our application is AWS CloudWatch [13], which monitors AWS SQS, namely resource usage metrics. From the perspective of the autoscaling problem, we will be interested in two metrics: • •
ApproximateNumberOfMessagesVisible – queue length.
NumberOfMessagesSent – number of messages sent [14].
To automatically scale the developed application, we proposed a framework based on AWS SageMaker [15]. This serverless service allows us to quickly build and deploy machine learning models with minimal developer intervention.
The proposed framework consists of several components for interaction with AWS SQS (Fig. 2). First of all, AWS CloudWatch collects the necessary metrics from AWS SQS, AWS EventBridge [16], and AWS Lambda [17]. AWS EventBridge launches AWS Lambda at appropriate intervals according to the schedule, and AWS Lambda receives predicted values from the trained model and adjusts the number and main parameters of data processors, thus ensuring horizontal scaling.
The diagram of framework and distributed application interaction is presented in Figure3.
The interaction between the framework and the distributed application is as follows (Fig. 3):
AWS CloudWatch collects the necessary metrics from AWS SQS and stores them in S3. A machine learning model is created and deployed in AWS SageMaker using stored metrics.
AWS EventBridge launches AWS Lambda according to a predefined schedule. 4. 5.
AWS Lambda accesses the deployed model and obtains predicted load indicators. Based on the obtained indicators, AWS Lambda changes the settings of the data processors accordingly.
The model is built using DeepAR, a learning algorithm for time series forecasting using recurrent neural networks (RNNs). This algorithm can provide better accuracy than classical forecasting methods such as autoregressive integrated moving averages (ARIMA) or exponential smoothing (ES) traditional methods that rely on a single series. DeepAR produces point and probabilistic forecasts, which is especially critical for capacity planning applications [19].
The accuracy of the forecast distribution is estimated using weighted quantile losses. The quantile loss for the quantile q∈(0,1) is calculated as follows (1): ( , , ) = ∙ ( − ), (1 − ) ∙ ( − ), > ≤ q – the selected quantile (for example, 0.1, 0.5, or 0.9), ŷ – forecast for the corresponding quantile.
For a set of quantiles = { 1, 2, … , } the total weighted quantile loss is calculated as (2): 1 =1 ∈ ℎ = ∙ , , ( ) where [20]: – number of predicted points; – weight for quantile ; ( ) – the prediction for the quantile for the point .
Also used [21, 22] were the mean absolute error (3) and the root mean square error (4): =1
1 1
=1 =
| − | , =
( − )2.
Building a DeepAR model consists of the following steps:
Collecting and formatting AWS SQS metrics.
Defining and training hyperparameters.
Training the model.
Testing the model.
Evaluating the results.
Deploying the model for further use. (1) (2) (3) (4)
The obtained prediction results are used to automatically scale data processors according to the algorithm below using threshold values.
Algorithm 1. AWS Lambda provisioned concurrency changing 1: function AutoScaleLambda( _ 2: // Main loop that runs every minutes loop every
minutes ← GetPrediction(target) if i _ ← GetProvisionedConcurrency( _ ) // If predicted value exceeds upper threshold 6: 7: 8: 9: _ = Minimum provisioned concurrency, _ = Maximum provisioned concurrency, = Increment step for scaling, > _
then _ ← if _ >
_ + _ then _
← _ end if // Update provisioned concurrency
SetProvisionedConcurrency( _ else if < _ _
← if _ < _ ←
_ - _ then _
then end if // Update provisioned concurrency SetProvisionedConcurrency( _
minutes before next iteration minutes) , , , _ , _ , _ , _ , ,target ) target = Historical time series values.
The framework built in this way will automatically scale data processors in distributed applications using queues.
Our experimental setup consists of a distributed application and a component for predicting and automatically scaling data processors in the distributed application. The core of the application is a queue – a component designed to store and send events for processing. Each event contains the information necessary for successful processing.
To simulate the workload, Locust was used. This tool is effective for load-testing distributed systems due to its scalability, ease of scripting, and monitoring and analysis capabilities [23].
During the application testing, the queue received 1082178 events within 90 minutes. At the same time, we collected the approximate number of messages visible and number of messages sent metrics for AWS SQS. Figure 4 and Figure 5 illustrate the collected data distribution.
The collected data was used to train the DeepAR model. It is worth noting that DeepAR uses a recurrent LSTM neural network (RNN). For training, we chose a minute-by-minute time series granularity. The model required at least 30 steps of historical data for forecasting, with an initial forecast length of 50 minutes.
The training results are presented in Figure 6 and Figure 7.
To evaluate the resulting models, we used the mean absolute error and the root mean square error for different numbers of layers. Figure 8 and Figure 9 represent a differentiation of the obtained error values.
By varying the number of layers, we found that the models demonstrate the best performance at a value of 6 and can be used to predict the load of a distributed application.
The trained models were deployed in the cloud environment using AWS SageMaker, which allowed us to make queries to predict the values of selected AWS SQS metrics and reduce the number of cold starts (Fig. 10).
Our study presents a framework for predictive automatic scaling of computing resources in serverless environments. The framework enables a transition from reactive to proactive scaling, reducing “cold starts” and overload errors.
The DeepAR model shows a mean absolute error (MAE) of 10019.21 and a root mean square error (RMSE) of 13140.91, indicating good prediction quality. Combining the proposed framework with serverless applications contributes to the optimal use of computing resources without going beyond the concept of serverless computing.
It should be noted that using prediction to scale data processors automatically reduced the number of “cold starts” by (approximately) 27% and the number of unprocessed requests by (approximately) 14%.
One of the limitations of the proposed framework is the lack of historical data, which DeepAR needs to learn patterns. In scenarios where a time series has very few past observations (e.g., newly created services), DeepAR may fail to model underlying trends, reducing forecast reliability accurately [24].
Further research can focus on improving the quality and use of forecasting models. For example, DeepAR allows building models based on multiple time series, which helps improve forecast accuracy. A separate issue remains in determining the optimal values of hyperparameters for training the DeepAR model. In addition, data sparsity and model drift require constant automatic retraining.
Using additional SQS metrics enables essential horizontal and vertical scaling for queues with varying event processing times.
Another important direction is the study of more efficient implementation of computing infrastructure costs through automatic scaling [25]. Also, we can create the SageMaker pipeline to update the DeepAR model with the latest data automatically.
The proposed framework is a practical automatic scaling tool for serverless applications that improves performance and reliability in distributed systems.
During the preparation of this work, the authors used ChatGPT, Grammarly in order to: Grammar and spelling check, Paraphrase and reword. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.