<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A Hybrid Pork Price Forecast Model Based on LightGBM and Segment-wise Attention 1</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Yan Wang</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Fang Deng</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Xudong Yang</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Beijing University of Posts and Telecommunications</institution>
          ,
          <addr-line>Beijing</addr-line>
          ,
          <country country="CN">China</country>
        </aff>
      </contrib-group>
      <fpage>226</fpage>
      <lpage>233</lpage>
      <abstract>
        <p>Long-term forecasting of pork prices is important for production investment and price regulation. However, the pork price series exhibits the characteristics of non-stationary, nonlinear, and pseudo-periodic, which makes the forecasting task challenging. To deal with the intricate temporal patterns, in this paper, we propose a novel hybrid model based on the decomposition-ensemble framework and provide tailored prediction methods for different components. We begin by decomposing the original series into trend and cyclical components using the HP filter. To address the problem of pseudo-cycle, we design a segment-wise neural network, which introduces the attention mechanism to model correlations at the segment level. Then LightGBM is used to implement a dynamic regression model for the trend component. Finally, the predicted series are aggregated. Predictions are carried out in a rolling forward manner to avoid the problem of information leakage. Experiments show that our model outperforms other single or hybrid models.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Time series</kwd>
        <kwd>decomposition-ensemble framework</kwd>
        <kwd>dynamic regression model，segment-wise correlation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Pork production has always been topping the list of meat production in China. According to the
National Bureau of Statistics, the average domestic pork production in the past five years reached 49.30
million tons, accounting for 59.52% of the total domestic meat production. However, pork prices have
been shrouded in frequent fluctuations. Especially in recent years, against the background of
transformation and upgrading of the hog industry chain, pork prices have shown great volatility,
impacting the nation's daily life, the operation of farmers' enterprises and the smooth operation of the
economy. Therefore, studying the pattern of pork price fluctuations and establishing an effective
longterm forecasting model are of great significance for the pig industry.</p>
      <p>
        Decomposition-ensemble model is a standard method in time series analysis, and it has been widely
used in dealing with the intricate temporal patterns[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Choosing a proper method to separate the series
and designing a suitable algorithm for each component are two key problems when constructing a
forecasting model under the decomposition-ensemble framework[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Inspired by the value theory, we
use the HP filter to decompose the original price series into the trend and cyclical components. Dynamic
regression models combine the features of explanatory and time series models and are able to model
the causes of price changes and the dependent information in historical prices. Given the good
performance of the tree model in generalization, we choose the LightGBM model to implement the
dynamic regression model and to forecast the trend series. The final prediction is the combination of
the predicted series.
      </p>
      <p>This paper is organized as follows: Section 2 introduces the data and the model we proposed. Section
3 describes the experimental settings, results and analysis. Conclusions are drawn in Section 4.</p>
    </sec>
    <sec id="sec-2">
      <title>Materials and Methods</title>
    </sec>
    <sec id="sec-3">
      <title>2.1. Data Source and Pre-processing</title>
      <p>Weekly pork and hog pricing as well as monthly breeding sow stock are collected from January
2006 to May 2022 from the National Development and Reform Commission, the China Pig website,
and the Ministry of Agriculture. For these datasets, a few pertinent statistics indices are computed and
displayed in Table 1.
(1)
(2)</p>
    </sec>
    <sec id="sec-4">
      <title>2.2. Overall Process of the Proposed Hybrid Model</title>
      <p>using the HP filter.</p>
    </sec>
    <sec id="sec-5">
      <title>2.3. HP Filter</title>
      <p>It is generally agreed that separating the temporal patterns of entanglement by time series
decomposition can highlight the intrinsic properties of the sequence of components. Inspired by the law
of market value, we choose HP filter to fit the original sequence to obtain a trend sequence that is easier
to predict. The trend series is the solution to the following optimization problem:
min
( −  ) + 
(  − 
) − (
− 
)
using the method of least squares.
shown in the following formula:
where yt is the original series and xt is the trend series. This optimization problem can be solved
The periodic series is obtained by the difference between the original series and the trend series, as</p>
      <p>=  − 

= 
, 
= 
(3)
(4)
(5)</p>
    </sec>
    <sec id="sec-6">
      <title>2.4. Prediction of Trend Component Based on LightGBM</title>
    </sec>
    <sec id="sec-7">
      <title>2.4.1. Constructing the Inputs</title>
      <p>
        Our experiments show that autoregressive time series models relying only on historical information
cannot effectively deal with the huge fluctuations in pork and hog prices, which will be discussed in
Section 3.3, so we do extensive feature engineering to extract effective features from price series and
covariate series. According to the research of relevant literature[
        <xref ref-type="bibr" rid="ref3 ref4">3,4</xref>
        ] and the exploratory analysis of the
data, the following features are selected under the framework of the dynamic regression model. On the
one hand, timestamps, holidays and breeding sow stocks are chosen as three main covariates because
capacity, seasonality and holiday effects play an important role in the formation of pork and hog prices.
A set of dummy variables are used to represent the month of the year. It should be noted that the number
of dummy variables needs to be less than the number of categories to avoid the "dummy variable trap".
For holiday features, we customize holiday variables and consider the window effect of holidays. In
addition, considering that there is a time lag in the correlation between the number of reproductive sows
and the number of live pigs, and the number of live pigs is an important determinant of pork and pig
prices, we use the following formula to calculate the number of lag periods. Then the shifted sow stock
series is included as a covariate.
( −  ̅) ( −  )
      </p>
      <p>On the other hand, the price series within the lookback window and its statistical characteristics
(variation and variance) are used as the input of the time series model to reflect the short-term
dependence of the price series.
(6)</p>
    </sec>
    <sec id="sec-8">
      <title>2.4.2. LightGBM Model</title>
      <p>
        Gradient Boosting Decision Tree (GBDT) is a member of the Boosting family in the field of
ensemble learning. Once it was proposed, it attracted widespread attention for its outstanding effects.
Although the application of GBDT to time series prediction is rarely found, it does not mean that the
algorithm is uncompetitive in this problem. Shereen2021's research shows that a properly configured
Recent research shows that a properly configured GBDT model can outperform the current SOTA DNN
model in the time series prediction field[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. LightGBM is a variant of GBDT which proposes a
gradientbased one-side sampling algorithm and an exclusive feature bundling algorithm, which solves the poor
computational efficiency problem of the XGBoost model.
      </p>
    </sec>
    <sec id="sec-9">
      <title>2.5. Prediction of Cyclical Component Based on Encoder-Decoder Model</title>
    </sec>
    <sec id="sec-10">
      <title>2.5.1. Encoder-decoder Model with Attention Mechanism</title>
      <p>
        The encoder-decoder model is widely used in the field of NLP, and attention mechanisms are
frequently applied to improve it[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. The general structure of the encoder-decoder model with attention
is shown in Figure 3. In this model, the hidden state of the final RNN layer at all time steps is the key
and value of the attention layer. At each time step of the decoding process, the final RNN layer hidden
state of the decoder at the previous time step is used as a query. With the attention score we can amplify
the attention-worthy parts of the sequence and reduce the influence of irrelevant parts. In this model,
we choose 1D-CNN as the encoder and LSTM as the decoder.
      </p>
    </sec>
    <sec id="sec-11">
      <title>2.5.2. Segment-wise Attention Model</title>
      <p>The similar sub-series search algorithm has a better effect on the time series prediction problem,
which shows that the correlation between time segments can reflect the correlation of time series better
than point-level sequence correlation. Therefore, we design a segment-wise attention model. We use
the sliding window to construct the segment list in an overlapping way as the input. Considering that
the similarity of the clips is too small if the time interval is too small, we take the step size of the window
movement as a hyperparameter. The input sequence is constructed as follows.:
where  : represents  ,  , … ,  , s represents the segment length, and step represents the
window moving step. Figure 4 shows the difference between our attention model and the traditional
attention mechanism. Our motivation is to distinguish the difference in reasoning logic between
machine translation and time series prediction. In the machine translation scenario, attention is used to
align the vocabulary of the source and target languages, but in the time series prediction scenario, the
historical segment similar to the current segment is not our target, but the subsequent segments. During
the training phase, the input to the decoder is the ground-truth value at the previous time step, and during
the prediction phase, it is the predicted value. The prediction ends when the total segment length reaches
the prediction step size.</p>
    </sec>
    <sec id="sec-12">
      <title>3. Materials and Methods</title>
    </sec>
    <sec id="sec-13">
      <title>3.1. Experimental Settings</title>
      <p>
        The disparity between the supply and demand for pig products is largely due to information lag. We
chose the prediction step as 25 weeks, which is the breeding cycle from piglets to killed pigs, in order
to make the prediction results support farmers' predictions of future income before investing. To
eliminate the possibility of experimental findings, we ran 10 experiments on each dataset. The test set
and training set were often dissected simultaneously in earlier studies, although this practice can result
in data leaks and unnaturally high accuracy[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. We carry out experiments using rolling decomposition
to get around this issue. Only the sequences that fall inside the training set window are broken down
for training in each trial. Figure 5 shows the schematic diagram.
      </p>
      <p>All experiments are implemented in Python 3.6 and 3.8. We use a few popular Python packages.
The Statsmodels package provides support for statistical correlation methods, including HP filters, STL
decomposition, ARMA, and SARIMA models. SVR modelling is carried out using the LinearSVR class
that sklearn package provides. We use darts to implement LightGBM because the library naturally
supports historically known covariates and future known covariates. The PyTorch package is used to
implement neural network models like the 1D-CNN and LSTM models.</p>
    </sec>
    <sec id="sec-14">
      <title>3.2. Performance Criteria</title>
      <p>Three statistical indicators, root mean square error (RMSE), mean absolute error (MAE) and mean
absolute percentage error (MAPE), are used to evaluate the performance of the model, which are the
most commonly used evaluation metrics for regression models. The formulas are as follows:
100%
(9)
(10)
where  and  represent the ground truth and the predicted sequence respectively, 
represent the
average of  , and  represents the length of the predicted sequence, which is 25 in this experiment.</p>
    </sec>
    <sec id="sec-15">
      <title>3.3. Result and Discussion</title>
      <p>
        To verify the effect of our model on the predictive ability and generalization ability, we conducted
two sets of experiments. The time range of the data used in the first set of experiments is 2006.1-2018.4,
which is the same as the data time range used in researches of Zhu et al. and Liu et al.[
        <xref ref-type="bibr" rid="ref2 ref8">2,8</xref>
        ]. The pig cycle
in this time range is more significant and relatively stable. The time range of the data used in the second
set of experiments is extended to 2022.5, including situations of extreme price volatility brought on by
market imbalances, in order to test the generalization ability and stability of the model. Our method is
compared with other methods for pork price series forecast, including single model (ARIMA, Prophet[
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]
and LSTM[
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]) and combined model[
        <xref ref-type="bibr" rid="ref2 ref8">2,8</xref>
        ]. Out of the 20 trials conducted in the two groups, we only chose
the results of 4 for presentation due to space restrictions (Figure 6 and 7, Table 2 and 3).
can be seen from the table that the prediction error for hog prices is typically higher than the forecast
error for pork prices. Compared with the single model, the combined model under the decomposition
ensemble framework generally performs better. On both datasets, our approach produces results that
are competitive. The results of the predictions made during times of high price volatility are displayed
in Table 3 and Figures 7. In the case of a market imbalance, all models' accuracy falls, but our model
performs the best. The mean values of MAE, RMSE, and MAPE of our model are lowered in the
prediction of pork prices when compared to the suboptimal model by 29.49%, 29.16%, and 32.23%,
respectively, while the mean errors of pig price predictions are reduced by 32.80%, 33.48%, and 33.02%.
Through the above analysis, we can draw the following conclusions: firstly, the price prediction model
proposed in this paper is effective in predicting the pork price and hog price, and secondly, the
prediction model considering the causes and historical price information of live pig and pork prices has greater
generalization ability.
      </p>
    </sec>
    <sec id="sec-16">
      <title>4. Conclusion</title>
      <p>This paper proposes a hybrid model based on a decomposition ensemble framework for predicting
hog prices and pork prices in the next hog breeding cycle. Considering the historical dependence
information of the time series and the impact of covariates that affect the price of live pigs, the dynamic
regression model based on LightGBM is chosen to forecast the trend component of the price. This is
done in order to ensure the stability and generalization ability of the model. In order to deal with
pseudoperiodic components, we extend the recognition ability of similar subsequence search algorithm with
the potent embedding capability of neural networks and propose an encoder-decoder model based on
segmental attention for periodic components. Finally, we combine the trend and cyclical components
to provide the forecast outcomes. The advantages of the proposed method are reinforced by experiments
on real datasets.</p>
      <p>Considering the distinct characteristics of Chinese provinces in the pork industry and the
transmission of pork price fluctuations between provinces, we will consider introducing geospatial information
into the pig price forecast model to achieve more fine-grained price forecasting in the future.</p>
    </sec>
    <sec id="sec-17">
      <title>5. Reference</title>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>HYNDMAN</surname>
            <given-names>R J</given-names>
          </string-name>
          , ATHANASOPOULOS G.
          <source>Forecasting: principles and practice[M]. 2nd Edition</source>
          . Melbourne, Australia,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>ZHU</surname>
            <given-names>H</given-names>
          </string-name>
          ,
          <string-name>
            <surname>XU</surname>
            <given-names>R</given-names>
          </string-name>
          ,
          <string-name>
            <surname>DENG H</surname>
          </string-name>
          .
          <article-title>A novel STL-based hybrid model for forecasting hog price in China</article-title>
          [J/OL].
          <source>Computers and Electronics in Agriculture</source>
          ,
          <year>2022</year>
          ,
          <volume>198</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>LUO</surname>
            <given-names>Q</given-names>
          </string-name>
          , ZHANG L.
          <article-title>Fluctuation Characteristics of Hog Price in China: Evidence Based on the Method of B-N Decomposition[J/OL]</article-title>
          .
          <source>Journal of Agrotechnical Economics</source>
          ,
          <year>2018</year>
          ,
          <volume>7</volume>
          :
          <fpage>93</fpage>
          -
          <lpage>106</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>DU</given-names>
            <surname>YANG</surname>
          </string-name>
          .
          <article-title>Analysis, prospect and countermeasures of periodic fluctuation of pig price in China</article-title>
          [J/OL].
          <source>Prices Monthly</source>
          ,
          <year>2020</year>
          ,
          <volume>515</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>5</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>ELSAYED</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>THYSSENS</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>RASHED</surname>
            <given-names>A</given-names>
          </string-name>
          , et al.
          <source>Do We Really Need Deep Learning Models for Time Series Forecasting?[J/OL]. arXiv preprint arXiv</source>
          ,
          <volume>2101</volume>
          .02118,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>KONDO</surname>
            <given-names>K</given-names>
          </string-name>
          ,
          <string-name>
            <surname>ISHIKAWA</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>KIMURA M.</surname>
          </string-name>
          <article-title>Sequence to sequence with attention for influenza prevalence prediction using Google Trends[J/OL]</article-title>
          .
          <source>ACM International Conference Proceeding Series</source>
          ,
          <year>2019</year>
          :
          <fpage>1</fpage>
          -
          <lpage>7</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>YU</surname>
            <given-names>L</given-names>
          </string-name>
          ,
          <string-name>
            <surname>MA</surname>
            <given-names>Y</given-names>
          </string-name>
          , MA M.
          <article-title>An effective rolling decomposition-ensemble model for gasoline consumption forecasting</article-title>
          [J/OL].
          <source>Energy</source>
          ,
          <year>2021</year>
          ,
          <volume>222</volume>
          :
          <fpage>119869</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>LIU</surname>
            <given-names>Y</given-names>
          </string-name>
          ,
          <string-name>
            <surname>DUAN</surname>
            <given-names>Q</given-names>
          </string-name>
          ,
          <string-name>
            <surname>WANG</surname>
            <given-names>D</given-names>
          </string-name>
          , et al.
          <article-title>Prediction for hog prices based on similar sub-series search and support vector regression</article-title>
          [J/OL].
          <source>Computers and Electronics in Agriculture</source>
          ,
          <year>2019</year>
          ,
          <volume>157</volume>
          :
          <fpage>581</fpage>
          -
          <lpage>588</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>TAYLOR S J</surname>
            ,
            <given-names>LETHAM B. Forecasting</given-names>
          </string-name>
          at Scale[J/OL].
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>SABU K M</surname>
            ,
            <given-names>KUMAR T K M.</given-names>
          </string-name>
          <article-title>Predictive analytics in Agriculture: Forecasting prices of Arecanuts in Kerala</article-title>
          [J/OL].
          <source>Procedia Computer Science</source>
          ,
          <year>2020</year>
          ,
          <volume>171</volume>
          :
          <fpage>699</fpage>
          -
          <lpage>708</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>