Microbusinesses are a vital part of a country's economy, and forecasting their density is crucial for both governments and hosting providers. Accurate predictions enable the government to plan future benefits for business owners, and hosting providers can efficiently allocate resources. In this study, we use data provided by Forward Venturer by GoDaddy, along with data collected from the U.S. Census Bureau to develop a model for microbusiness density forecasting. During the study we performed experiments using various machine learning techniques, including linear regression (LR), Ridge, Lasso, ElasticNet regression, decision tree (DT), random forest (RF), multilayer perceptron (MLP), gradient boosting, Ada boosting, support vector machine (SVM), XGBoost, LGBM, and TensorFlow decision forest (TFDF) regressors, as well as several neural network architectures such as multilayer perceptron (MLP), recurrent neural network (RNN), long short-term memory (LSTM), N-BEATS, and autoencoder. The performance of each model was evaluated using MAE and SMAPE metrics. This study highlights the potential of various machine learning and neural network algorithms for forecasting microbusiness density, which can aid in better resource planning for hosting providers and the government. Microbusiness, density forecasting, feature selection, machine learning, regression.
2023 Copyright for this paper by its authors. CEUR
ceur-ws.org
Additionally, to the initial data, we were encouraged to use any other useful data. We have found many useful variables that could make the prediction more accurate and help to realize more advanced approaches to improve predictions, and they are described in Section II.
There is a lack of comprehensive studies analysing microbusiness density in the United States. While
there has been some research on small businesses [
The rest of the paper is organized as follows. Literature review of the microbusiness density is presented in Section II. Section III provides description of the data that will be used for forecasting. Section IV presents data preprocessing with subsections of data combination, data preparation and features selection. Forecasting results are provided in Section V. Section VI describes the selected techniques. Finally, concluding remarks regarding forecasts are discussed in Section VII.
We believe that the analysis of microbusiness is a relative new topic that has not received much
attention so far. Microbusiness density has been the subject of interest only in [
In this study, various models were utilized to predict the target variable. Models were selected based on their usage in previous competitions and included popular regression models such as Linear Regression, Decision Tree Regressor, XGB Regressor, and others. The performance of these models was evaluated using the symmetric mean absolute percentage error (SMAPE) as an accuracy metric, which was required by the competition evaluation rules and mean absolute error (MAE). These chosen metrics are defined as follows where is an actual value and is forecasted value. = 1 ∑ =1 =1
| − | (| | + | |)/2 ∑| − | (1) (2)
To examine the relationships between variables, a correlation analysis was conducted using the Pearson correlation coefficient. This statistical method was selected for its ability to quantify the strength and direction of linear associations between variables. Pearson correlation coefficient of two variables X and Y is formally defined as , = , deviation of Y. where cov( , )is covariance of the two variables, is standard deviation of X and is standard
We applied data normalization to ensure that all attributes have an equal effect on the resulting variable. Min-max normalization was used to scale the variables between zeros to one: (3) (5) (6) (7) (8) 1. Randomly select initial objects as centroids. 2. Calculate the distance between each object and centroids. 3. Assign each object to the nearest cluster. 4. Calculate the mean of each cluster as new centroid. 5. Repeat steps 2 – 4 until convergence.
The objective of K-means algorithm is to minimize the squared error function: =1 ∈ = ∑ ∑ | − |
2 where – is a point in space representing a given object, is the mean value of cluster .
To define optimal number of clusters we used Davies-Buildin Score and Elbow Method.
DaviesBuildin Score [
To better understand the differences between counties we applied K-means clustering algorithm. The idea behind K-means is to partition the data = { 1, 2, . . . , } into ( < )clusters so that so the objects within a cluster are more similar to each other than the objects in different clusters. In this case, objects similarity was measured based on Euclidean distance. Stepwise K-means clustering algorithm can be defined as follows.
where is a measure of scatter within the ith cluster defined as and , is a measure separation between ith and jth clusters: Lower value of Davies-Buildin Score indicates the better clustering. ∑ max
≠ =1 +
, 1 ∈ =
∑ ( , ) , = ( , )
The Elbow Method [
After the calculations, the results are plotted according to the number of clusters, and the location of a bend in a plot is usually considered an indicator of the appropriate number of clusters.
The initial dataset used in this study was provided by GoDaddy [
The target variable represented the number of microbusinesses per 100 people age over 18 in the given county. Due to the ACS update window, the population figures used to calculate the microbusiness density are on a two-year lag. This means that the microbusiness density for 2022 was calculated using population figures from 2020.
ID code consisting of cfip and first day of the month columns A unique county identifier using Federal Information Processing System, where first two digits corresponds to the state FIPS code, while following three numbers represents the county
The date of the observation. Consists of year, month, and day Number of microbusinesses per 100 people age over 18 in the given county. This is a target variable.
The microbusiness in the county (not provided in future forecasting) features as Table 1, but with missing target feature microbusiness_density and active column, which is yet unknown. The forecasts should cover the period from January to June 2023 for all 3135 counties.
The organizers strongly encouraged the use of external data sources that might help to improve
prediction performance. Therefore, we enriched the initial data using publicly available sources such
as the ACS website. Every externally collected dataset had the same structure as Table 2, where each
column contained information for the given year. The following information was gathered:
Business turnover [
Geographical location of counties [
To get a better understanding of the initial data, we performed the following steps: we checked the distribution of target values, performed correlation analysis, and checked for possible seasonality.
The microbusiness density distribution revealed that most observations are centered around zero, see Figure 1(a). Moreover, data distribution is skewed to the right. For this reason, we applied the log transformation, as shown in Figure 1(b). Based on these results, we can conclude that the microbusiness density follows a close-to-log-normal distribution.
(a) (b) Figure 1: (a) original microbusiness density distribution, (b) logarithmically transformed microbusiness density.
Analyzing microbusiness data on a state level, we observed that several states had higher density of microbusinesses, i.e., California, Colorado, Delaware, Wyoming, Utah, Nevada, and Florida. The average microbusiness density per county in these states was 8.38, while the rest of the U.S. states had an average of 3.41, with the standard deviations of 13.21 and 3.06 respectively. The two most outstanding states were Delaware with an average of 18.74 and Nevada with 12.42 microbusinesses density per county, as shown in Table 3. To account for these fluctuations, we decided to perform a clustering analysis. To identify the most appropriate number of clusters, we performed experiments with a different number of clusters = 2,3, . . . 30. The maximum number of clusters was based on the rule of thumb, i.e.
~√ /2, (10) where n is a number of observations.
Table 3 Highest density states information
State
Avg. microbusiness density
of microbusiness density 7.5628 8.7500 18.7419 6.9452 12.4261 8.3480 9.3745 3.8278
To compare clustering results obtained using different k values, we used Davies-Bouldin Score (DB Score) and the Elbow Method. The results are illustrated in Figure 2. In this case, the Elbow Method did not show a clear indication of an elbow point (see Figure 2(a)), while the DB Score indicated that the optimal number of clusters for this dataset is four (see Figure 2(b)). A more detailed analysis revealed that extracted clusters quite well represent the distribution of microbusiness density among counties. For instance, cluster number two represents counties having the largest density, i.e., the average value of microbusiness density in this cluster is 62.13 with a standard deviation of 51.19. In contrast, Cluster 3 has the lowest results, with an average of 2.20 and a standard deviation of 1.02. For more details, please refer to Table 4.
Based on these results, we included an additional variable representing the counties' membership among clusters.
To analyze relations between microbusinesses density and other factors we performed correlation analysis. Due to the nature of microbusiness density, we focused on two possible variations. The first approach sought to understand how a 2-year lag impacts the correlation, and second, how features correlate with each other. As can be seen in Figure 3, a 2-year lag has a positive impact on the correlation between features. A weak-to-medium linear dependency can be observed between features, with the correlation coefficient ranging between 0.2 and 0.6. The target value correlates the most with the college education feature (value 0.5), followed by broadband usage and median income features (values 0.4), as shown in Figure 3(a) (a)
Lastly, the seasonality analysis was conducted. It was noticed that the majority of the counties has a noticeable increase in microbusinesses during festive periods and decrease during inter-holiday period. The rest of the counties had either no seasonality or very small seasonality. 4.2.
The process of data preparation is essential for machine learning model performance. The following
preprocessing steps were performed to prepare the data for the modelling stage:
1. Combination of training and testing datasets. Because a one-year lag will be introduced into
the dataset, it is important to have a temporary full dataset. Before the combination two
additional features were introduced: is_test column that marked the original sets and dcount that
marked the sequence of observations in each county. This step ensures that the first row of each
county will have a full set of features.
2. Introduction of lag terms. The dataset was enriched using series’ own past values, so-called
lags. In this case, we included 1-12 month lags representing the microbusiness density of the
previous year.
3. Imputation of missing values. The merging external dataset introduced some missing values
that were imputed using the mean value of the corresponding county feature.
4. Splitting the combined dataset back into training and testing sets. To prevent data leakage
full dataset was split back into training and testing datasets using is_test feature introduced in
the first step of data preparation.
5. Removal of features with incomplete lag values in the training dataset. The first eleven
entries of each county in the training dataset were dropped due to the missing lag values.
6. Numerical feature scaling. All numerical features, except the target and lag values, were scaled
to be in the range of [
The final dataset had a total of 3162 features. Seeking to avoid unnecessary complexity caused by the dataset’s dimensionality we performed feature selection. 4.3.
The data preparation step introduced many new features that negatively impacted the model.
Therefore, statistical feature significance tests, specifically ordinary least squares regression tests [
Both original and logarithmically transformed datasets were split into training and validation sets. Training data contained observations from August 2019 to July 2022, and validation data from August 2022 to December 2022. Experiments were performed using various regression techniques, i.e., linear, Ridge, Lasso and ElasticNet regression; decision tree, random forest, multilayer perceptron, gradient boosting, Ada boosting, support vector machine, XGBoost, LGBM and TensorFlow decision forest regressors. Additionally, the following neural network architectures were trained: recurrent, multilayer perceptron, LSTM, N-BEATS and autoencoder. Finally, a validation test was used to estimate the model's performance. The obtained results are presented in Table 6.
The best performing model with original target values was a multilayer perceptron, achieving MAE of 0.055 and SMAPE of 1.696. On the other hand, the least accurate model was the AdaBoost Regressor, with MAE of 3.560 and SMAPE of 80.690. However, when using logarithmically transformed target values, the best performing models were the linear regression and Ridge regression, both with an MAE of 0.057 and SMAPE of 1.710. The worst models were ElasticNet and Lasso regressor, both models achieved an MAE of 2.290 and SMAPE of 56.297. It is important to note that the best performing models for the final submission may differ from the best performing models on the validation dataset.
Models trained on logarithmically transformed data showed superior results compared to those trained on original target values, as seen in Table 6. Consequently, the logarithmic transformation was selected for the final submissions. It is worth noticing, however, that the best performing models for the Kaggle competition differed from the models in experimentation phase. The subset of model’s forecasting, that showed the highest results, can be seen in Table 7.
The results were evaluated on data that contained only January 2023 density values. The highest performing model is the XGBoost regressor with a SMAPE of 3.3159, followed closely by the Random Forest regressor with an SMAPE of 3.3189. The least accurate model is the recurrent neural network with SMAPE of 3.8251. The final and full results will be known on June 14th, 2023.
In this paper, we presented the results of our investigation on microbusiness density forecasting using various machine learning techniques. Experiments were performed using the dataset from the GoDaddy Kaggle competition, which was enriched using external data sources. Explanatory data analysis revealed a significant positive relationship between microbusiness density, college education, broadband usage, and median income. Moreover, we observed that the distribution of microbusiness density is not uniform across counties, i.e., California, Colorado, Delaware, Wyoming, Utah, Nevada, and Florida had a higher density of microbusinesses. To account for these fluctuations, we performed a clustering analysis based on the K-means algorithm. As a result, four clusters were extracted and included in the analysis as an additional feature.
The final dataset consisted of 3162 features. To reduce the dimensionality of the data, we conducted feature selection using a statistical significance test. As a result, we identified 21 statistically significant features that were later used for modelling experiments. A detailed analysis of various regression techniques revealed that the XGBoost method performed the best, with a SMAPE of 3.3159. We also discovered that logarithmically transforming microbusiness density values produced better validation results. Consequently, we chose the logarithmically transformed dataset to forecast unseen data. However, forecasting in an open system is unpredictable due to many factors that can positively or negatively impact the results. Therefore, such forecasting requires continuous model retraining to achieve the best possible results for the upcoming months. 7. References