COVID-19 is a life-threatening novel respiratory virus-borne disease, which was discovered in December 2019 in Wuhan and subsequently spread globally. Monitoring and predicting COVID-19 epidemic data is crucial to control pandemic outbreaks. Machine learning-based methods, including deep learning, are promising approaches to predict COVID-19 data such as new cases, infected patients, and deaths. Our study focused on short-term COVID-19 hospitalizations forecasting using two machine learning approaches- ensemble time-series method and multilayer perceptron (MLP) feedforward network method. Both methods make predictions based on hospitalization, polymerase chain reaction (PCR), and antigen (Ag) test data, which were collected between October 2020 and June 2021 in Slovakia for our study. The ensemble time-series method was more sensitive in the beginning of experimental period but failed when the number of hospitalizations began to drop. The MLP method was ineffective in the beginning because of lack of training data but improved when more robust data was available; this method is promising for monitoring the third wave of pandemic in Slovakia.
The first patients with the novel coronavirus SARSCoV-2, were hospitalized in Wuhan, China in December 2019 [1]. In January 2020 more cases were reported throughout China and abroad [2]. The most sensitive diagnostic method currently available for COVID-19 testing is the polymerase chain reaction (PCR) test [3]. However, for effective screening, frequent repetition and fast reporting is more important than sensitivity [4], which makes rapid antigen (Ag) tests or loop-mediated isothermal amplification tests (LAMP) tests advantageous in COVID19 diagnostics. There are several studies on the relationship between symptom onset, positive PCR testing, and hospitalization. The fifth day post infection is a typically when symptom onset occurs, and most infected people test false negative before this day. The decrease in probability of false negativity is noted four days before symptoms onset, from 100% four days before onset to 67% one day before onset. On the day of onset, the probability of getting a false negative result is less than 40% [5]. From zero to four days after symptom onset, PCR tests from nasopharyngeal swabs are positive [6] in most infected individuals, with a peak in the first week after onset [7]. The median time between symptom onset and hospitalization is 5 days [6], and the median number of days from symptom onset to death was 14 and less in patients aged over 70 [8]. Despite these observations, which demonstrate the importance of time in COVID-19 infections, predicting the number of hospitalized patients from positive tests and average hospitalization period is not straightforward and depends on personal and regional factors. A nationwide cohort study reported that 20% of all PCR-positive cases result in hospitalizations, and the proportion increases with age and multimorbidity [9]. In another study [10], stronger hospitalization risk is associated with men aged ≥ 75 years with comorbidities, particularly cardiovascular disease, diabetes chronic kidney disease, hyperlipidemia and obesity than in other groups.
Predicting COVID-19 epidemic data and monitoring epidemiological changes of the virus spread are crucial for controlling pandemic outbreaks [1]. Machine learning methods, including deep learning, show promise in predicting COVID-19 epidemic data such as new cases, infected patients, and mortality. A multilayer perceptron (MLP) artificial neural network was used in [11] to create a worldwide model for predicting the maximum number of infected patients in a location from available data in time. The MLP was shown to have slightly better performance for analyzing contributing factors for COVID-19 spread and deaths than the radial basis function in [12]. The authors of [13] analyzed continuous variable quantum neural networks and quantum backpropagating MLP models for predicting COVID-19 cases in India and the USA. Both methods showed better performance than classical artificial neural networks. MLP and adaptive network-based fuzzy inference systems showed promising results in outbreak predictions [14]. Hybrid machine learning prediction models of adaptive network-based fuzzy inference system and MLP-imperialist competitive algorithms were used in [15] to predict cases and mortality rate. Machine learning models, such as the linear regression, linear regressor polynomial, support vector regressor, random forest regressor, decision tree, and autoregressive moving average were used to predict outbreaks [16] [17] [18] with the highest accuracy being shown by the auto regressive moving-average [16] and random-forest approaches [18]. A random forest model was also proposed to predict mortality in the first 20–84 hours following hospitalization [19].
Deep learning forecasting methods for new or, new and recovered cases using recurrent neural networks were proposed in [20] and [21]. The authors tested the long short-term memory (LSTM), bidirectional LSTM, gated recurrent units, variational autoencoder, and convolutional LSTM; the variational autoencoder showed the best performance among all of these [20]. The convolutional LSTM outperformed other models in predicting new cases for a one-month period [21].
However, the development and subsequent practical application of models based on deep learning requires high computing power [22], which aggravates the financial disparities between different universities [23]. At least a partial solution towards the democratization of research in the field of deep learning is to use commercial cloud computing platforms, which allow the direct purchase of necessary computing power [24]. This approach has proven to be effective and is beginning to be applied in fields such as information retrieval [25], flexible maintenance [26], and time-series prediction [27], [28].
Prediction of COVID-19 hospitalizations in Slovakia based on linear regression was firstly conducted by [29]. The problem of preventing the spread of the disease is complex, and multidisciplinary approaches, including artificial intelligence methods, are required.
Our paper focuses on short-term COVID-19 hospitalizations forecasting in Slovakia with a machine learning ensemble model implemented on the MS Azure [30] cloud computing platform and an MLP feedforward network implemented locally using in MATLAB [31]. The automated machine learning (AutoML) [32] approach enables acceleration of the development and deployment of machine learning models without extensive programming knowledge, making it suitable and user-friendly for epidemiologists and data analytics professionals.
The goal of hospitalization predictions is to aid the preparedness of hospitals and health-care professionals to admit all patients required hospitalizations and provide them proper healthcare without rescheduling planned elective care; it also aids the redistribution of hospitalized patients among different regions and districts as needed. 2
We propose two different machine learning methods: the first based on ensemble learning and the second based on the MLP method. The first was developed and tested in real time during the second wave of pandemic in Slovakia. The second was tested retrospectively. 2.1
We used the dataset provided by The Institute for Healthcare Analysis [33], publicly available on Github. The dataset includes COVID-19 statistics in Slovakia, i.e., the daily number of positive and total PCR and antigen (Ag) tests; number of hospitalized patients including daily hospital admissions and discharges; vaccination statistics, etc. Available hospitalizations data are divided by districts and regions because of reporting from every hospital every day.
The model training time-series data were the daily numbers of PCR and Ag positive tests; daily percentage of positive PCR; and Ag tests from total PCR and Ag tests. All training data were filtered by a seven-day simple moving average (SMA) filter. We assumed that testing PCR or Ag positive leads to a hospitalization time of four and seven days on average, respectively. These time shifts were inspired by [29], where the average time for positive PCR and Ag test hospitalization was taken as three and seven days, respectively. We performed these time shifts with time-series training data; the schematic of a sample data preparation is shown on Fig. 1. The distributions of the variables in the dataset and correlations between all these variables with hospitalizations are provided in the Appendix. The last PCR and Ag test data from the dataset were used as inputs for predicting hospitalizations from time-series data. The ensemble method was trained on the entire time-series dataset with 1/14-day shifts until the first day of prediction without splitting into time periods. This means that a separate training dataset was used for every 14 days, ending exactly before prediction, allowing the continuation of this time series using the data from the previous days for forecast computations. The method predicts for the next 14 days using the rest of time shifted data (Fig. 1). Because of the time shifting, we can use the last four days from PCR testing and last seven days from the Ag testing as future values and consider these as prediction inputs. In this way, we created input data, allocating the last sample representing 14 days, as a test set. We used 5-fold rolling origin cross validation (ROCV) with a fixed starting point on the remaining training data. Shifting the data by 1/5 thus created cross-validation folds, which ensured that there was no data leakage.
The MLP trains with the input and output data. It needs to have a robust dataset to be well trained. Prediction was performed for 14 days using input data from 14 days before first prediction day. Training input data were moved by one day to the right until the end of the dataset. As in ensemble method, the inputs were created from daily PCR and Ag test data. In addition, hospitalizations in last 14 days were used. To enlarge the dataset, data from all regions of Slovakia (8 regions) and their summaries were used for Fig. 1. Preparing the training dataset for machine learning time-series ensemble method using time shifts. The example dataset (left) is divided into the training part (upper right) and prediction input part (bottom right) using time shifts. Missing days in the prediction input (labeled as “AUTO”) are automatically filled by MS Azure. The entire process results in 14 days of hospitalization forecasting (labeled as “14-DAYS PREDICTION”). The example is shown without normalization for better visualization. Following data are mentioned in the figure: number and percent of positive polymerase chase reaction tests (PCR and PCR%), number and percent of positive antigen tests (Ag and Ag%), number of hospitalized patients (Hosp). training. This dataset was nine times larger than one with only summary data.
For example, in one of our models, trained from 11/10/2020 till 02/02/2020 (i.e., 85 days) we obtained 58 inputs and their outputs (14 × 58 inputs together with 14 × 58 outputs moved by one day) once from every region and once from the whole of Slovakia, giving 58 × 9 = 522 input data and their outputs for training. We applied a seven-day moving average filter to the input data for preprocessing and normalization to simplify the function fit. All data were normalized to a scale of 0 to 1, with 1 corresponding to 1.5 times the maximum value in positive or hospitalized inputs and 100% as the maximum if inputs are in percent. The dataset was divided randomly and 80% assigned as training data and 20% as validation data. The last 14 days before prediction period were used as the test data to get the final 14 days prediction. The dataset was prepared in MATLAB 2020b. 2.4
The time-series machine learning ensemble method was trained in the MS Azure cloud machine learning module, which is a cloud-based machine learning service with a user interface. This allows users without programming knowledge to train; all that needs to be done is to upload the training dataset, choose the built-in method and desired features, and click to start. Azure trains huge number of models by default and compares them. After considering several models (such as decision tree, random forest, AutoArima, ProphetModel, ElasticNet, GradientBoosting, and LassoLars) we chose the voting ensemble model because of smallest normalized root mean squared error (RMSE) on prediction gained by this model compared to other available machine learning models. Best performance of this model among all available models was confirmed in most of our experiments. The architecture of Azure Voting ensemble model is shown in Fig. 2. The model consists of six soft voting base regressors: three gradient boosting regressors, one random forest regressor and two decision tree regressors, each with different parameters. The voting ensemble model considers the predictions of every regressor, which are weighted and averaged, the final prediction being the weighted average from all regressors in the ensemble model. The hyperparameters used are listed in Table 1.
After training the model, the online endpoint must be created. An online endpoint is an HTTPS endpoint which is called by the user to obtain output of trained model. It contains deployments to receive data and send responses in real time. Access to the endpoint is deployed through a Python script in combination with the prediction input data. 2.5
For the 14-day prediction, we used a standard MLP with two hidden layers. We used 15 neurons with the hyperbolic tangent activation function in each hidden layers. The output layer with 14 neurons represents a multi-step prediction of hospitalization with a linear activation function at the output.
The architecture of our MLP network is shown in Fig. 3. Training and validation data are randomly divided using early-stopping with six validation checks. The Levenberg– Marquardt training algorithm was used, and performance measured using the mean squared error. 2.6
The results of both methods were evaluated with RMSE and mean absolute percentage error (MAPE) using the standard formulas , (1) (2) where n is number of fitted points, At is the actual value, and Ft is forecast value.
The results of the time-series machine learning ensemble method are shown in Table 2. The ensemble method performed well in first three predictions in February when hospitalizations had risen and in March when it predicted the peak of the second wave in Slovakia. When comparing with real hospitalizations cases, the RMSE and MAPE values were in the range of 61.62–91.49 and 1.43–2.19, respectively. The method failed to predict the drop in hospitalizations from April (RMSE: 440.19 and MAPE: 16.21) and later (Fig. 4). The importance of the variables in the ensemble model is listed in Fig. 5. Highest importance showed percent of positive antigen test, following with Time period 3.2.–16.2. 17.2.–2.3. 5.3.–18.3. 3.–16.4. positive antigen tests. Correlations of these variables in this model are shown in Appendix section.
In contrast, the MLP prediction method was quite inaccurate during the first months of this year (RMSE in range of 114.81–328.64, MAPE in range 2.76–12.94). The prediction accuracy improved with time; more training samples led to better results. Table 3 gives the average RSME and MAPE values for all regions and for the whole of Slovakia. Values for all of Slovakia were computed as a summary of all regions. The best regional and summary results were obtained in May.
Comparisons of the performances of both the proposed methods in three random time periods are shown in Table 2 and Fig. 4.
Predictions of MLP method in all regions and the whole of Slovakia in random time periods in May improved compared to those predicted earlier (Fig. 6, Table 3marked bold).
We proposed two machine learning approaches for shortterm hospitalization forecasting in Slovakia. The first approach is time-series ensemble method and the second an MLP neural network.
The ensemble method performed well at the beginning of the experimental period, with the best RMSE being 61.62, but failed when hospitalizations decreased. This could be due to lack of training data— the method was trained with data from the whole of Slovakia only from November 2020. In that time hospitalizations had risen, and the data from the period when cases were decreasing could not be learned. Surprisingly, the peak of the second wave, which followed the decrease in hospitalizations was predicted successfully with this approach. After that, no successful predictions with decreasing of hospitalizations were made. In addition, predicting hospitalizations with cloud-based user-friendly built-in services could make this solution accessible to non-programmers and easier to implement.
Using the MLP method, the initial predictions were inaccurate. Its performance improved with time with accurate results being obtained from the time period in May. This success was also observed in regional predictions— with an average RMSE of 17.81. The RMSE for all regional summaries was 66.95, which we consider as best result for the MLP method for whole Slovakia. We assume that The improvement of the MLP results with time are due to the increase in training dataset size.
We propose that regional predictions with RMSE lower than 20 and for all of Slovakia with RMSE lower than 100 can be valuable in practice.
We took 14 days as our forecasting periods; however, shorter prediction periods are expected to give better results. As prediction period increases, the discrepancy between predicted and real numbers rises. In addition, predicting in shorter time periods in the MLP method leads to more robust dataset, which may lead to even better results. This can be a promising direction for further investigation.
Using only positive tests and previous hospitalizations as inputs may not be sufficient in the future. This experiment was done during the second wave of pandemic in Slovakia, when the vaccination status was not an important factor, and therefore, we did not notice any sudden change in the age distribution of positive tests. As vaccination begins, new input variables would be necessary, such as the percentage of vaccinated individuals in the population or in the elderly and the daily mean age of tested positive. This would be especially relevant during the third wave, when due to vaccination, the age distribution among positive tested and hospitalized may differ. We hope that with new input variables and more robust data, these methods can adapt to such changes.
Forecasting COVID-19 hospitalizations is critical for monitoring pandemic outbreaks and provide healthcare without compromising on elective care. Redistribution of patients among district and regions can be considered based on such predictions if there is a shortage of hospital beds. Our machine learning forecasting approaches are promising when sufficient training data is available. Augmenting the training dataset using data from all regions, (as in our MLP method) increases the accuracy of predictions, which gives hope for forecasting hospitalizations in the coming third wave of COVID-19.
V.K. developed the experimental premise, design, and procedures. V.K., and M.H. conducted the research, trained the ensemble method and networks, and analyzed the data.
V.K. and J.G. processed the figures and analyzed the data.
V. K. and J.G. prepared the manuscript. All authors interpreted the results, contributed to manuscript revision, and approved the submitted version.
Fig. 12, 13 Importance of Ag perc. and Ag in Ensemble model. Graphs were created in MS Azure: