<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Swedish AI Society Workshop</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Navigating demand forecasting in make-to-order manufacturing: the role of global models and intermittent time-series</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Jonatan Flyckt</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Niklas Lavesson</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Software Engineering, Blekinge Institute of Technology</institution>
          ,
          <addr-line>Valhallavägen 10, 371 79, Karlskrona</addr-line>
          ,
          <country country="SE">Sweden</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Herenco AB</institution>
          ,
          <addr-line>Skolgatan 24, 553 16, Jönköping</addr-line>
          ,
          <country country="SE">Sweden</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2025</year>
      </pub-date>
      <volume>16</volume>
      <fpage>16</fpage>
      <lpage>17</lpage>
      <abstract>
        <p>Demand forecasting can optimise production and supply chain practices in manufacturing organisations. However, demand forecasting is not widely adopted among make-to-order (MTO) manufacturers with mass customisation ofers. Building efective demand forecasting systems is challenging in such organisations due to the numerous unique manufactured articles and sparse demand patterns. This position paper argues that make-to-order manufacturers should employ demand forecasting to a larger extent, and that the forecasting community should address challenges related to the domain. Key challenges include creating models capable of predicting both demand size and timing of intermittent forecasts, as well as a deeper insight into the efects of global deep learning time-series models. We perform a pilot experiment using demand forecasting in a purchasing decision support system to validate the usefulness of demand forecasting for MTO manufacturing organisations with mass customisation ofers. A research roadmap is proposed to address the identified challenges.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Demand Forecasting</kwd>
        <kwd>Intermittent Time-Series</kwd>
        <kwd>Global Models</kwd>
        <kwd>Machine Learning</kwd>
        <kwd>Make-to-Order Manufacturing</kwd>
        <kwd>Mass Customization</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Demand forecasting is an important tool in many domains, e.g., retailing and manufacturing, to help
ensure that goods are available when customers need them. Demand forecasting involves training
machine learning models on historical demand patterns for products or product groups. During
inference, the trained models are supplied with recent demand data to predict the demand for a future
period of interest [1]. The goal is to estimate when customers will purchase items, and how many items
they will purchase.</p>
      <sec id="sec-1-1">
        <title>1.1. Demand forecasting challenges</title>
        <p>
          Accurate demand forecasts allow manufacturing organisations to plan production proactively to have
goods ready when the customers need them and become more eficient at doing so. Retail and
make-tostock (MTS) manufacturing organisations adopt demand forecasting into their supply chain planning
to a larger extent than make-to-order (MTO) manufacturing organisations [
          <xref ref-type="bibr" rid="ref3 ref4">2, 3</xref>
          ]. MTO manufacturing
organisations generally wait until the customer places an order until they produce the final goods and
therefore cannot use demand forecasts to the same extent that MTS manufacturing organisations do.
        </p>
        <p>
          Demand forecasts can help MTO organisations decide when to produce extra finished goods to stock,
making production more eficient. However, due to the challenges involved, MTO organisations often
avoid using demand forecasting in their workflows [
          <xref ref-type="bibr" rid="ref3 ref4">2, 3</xref>
          ]. MTO manufacturing organisations often
produce many articles and need to know when demand will occur for each article and customer [
          <xref ref-type="bibr" rid="ref1">4</xref>
          ].
However, it is not feasible to train separate machine learning models for thousands of unique articles
(local models). Instead, focus shifts to global models that train on many time-series and forecast for all
of them with one model [
          <xref ref-type="bibr" rid="ref6">5</xref>
          ].
        </p>
      </sec>
      <sec id="sec-1-2">
        <title>1.2. Aim and scope</title>
        <p>This position paper highlights knowledge gaps in the system requirements for demand forecasting for
MTO manufacturing organisations and specific challenges that need to be addressed to close them. We
believe more attention should be given to training and assessing intermittent time-series that predict
both demand size and timing, especially from a global perspective when dealing with many time-series.
To support this position, we conduct a pilot experiment where demand forecasting is incorporated into
a smart purchasing system used by an MTO manufacturer. The aim of the experiment is to evaluate the
usefulness of demand forecasting for improving purchasing decisions in an MTO context.</p>
      </sec>
      <sec id="sec-1-3">
        <title>1.3. Outline</title>
        <p>In the sections that follow, we first present the background and relevant terminology for the paper. This
is followed by a description of the use of demand forecasting both in general and from a manufacturing
perspective, highlighting specific demand forecasting challenges that arise in MTO manufacturing
organisations. The argumentation section of the paper argues three main points:
• The need for a better understanding of the requirements of demand forecasting systems in MTO
manufacturing organisations.
• The need for models capable of forecasting intermittent time-series with both accurate demand
timings and sizes.
• The need for a deeper insight into the efects of using global deep learning time-series models
compared to local models, and how to improve global models.</p>
        <p>Following this, we present results from a pilot experiment where demand forecasting is integrated
into a purchasing system in a mass customisation purchasing task, demonstrating the usefulness and
potential of demand forecasting in an MTO context. Lastly, we present a research roadmap of open,
tangible demand forecasting problems which we believe should be addressed.</p>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>2. Background</title>
      <p>
        Demand forecasting has been used for decades in MTS manufacturing to manage inventory of
semiifnished and finished goods [
        <xref ref-type="bibr" rid="ref9">6, 7</xref>
        ]. However, demand forecasting continues to see limited use in the
MTO and mass customisation manufacturing domain [
        <xref ref-type="bibr" rid="ref3">2</xref>
        ]. MTO organisations want to keep the stock
levels low to avoid tying up capital or producing goods which become obsolete. In our experience, MTO
organisations often use qualitative methods, relying on expert knowledge to forecast contract orders
without historical data or macro trends. However, these approaches are time-consuming and dificult
when there are many articles [
        <xref ref-type="bibr" rid="ref10">8</xref>
        ].
      </p>
      <sec id="sec-2-1">
        <title>2.1. Terminology</title>
        <p>
          A make-to-stock manufacturing strategy aims to keep adequate stock of manufactured goods in
anticipation of future demand [
          <xref ref-type="bibr" rid="ref3">2</xref>
          ]. A make-to-order manufacturing strategy means that goods are
only manufactured after an order has been placed, and is a common strategy among manufacturers who
have mass customisation ofers [
          <xref ref-type="bibr" rid="ref11 ref3">9, 2</xref>
          ]. The purpose of mass customisation is to provide customers
with customised products tailored to their specific needs [
          <xref ref-type="bibr" rid="ref11">9</xref>
          ], e.g., unique cosmetic designs integrated
into base products, or tools unique to certain customers. We use the term article to describe goods that
have been customised for a specific customer. Organisations with repeat customers can use a hybrid
manufacturing strategy and manufacture customised goods ahead of demand with a calculated risk to
improve production eficiency.
        </p>
        <p>
          The purpose of demand forecasting is to estimate when customers are likely to have a demand for
goods which an organisation supplies. Customers can sometimes provide their own demand forecasts,
but they are often not accurate enough to use for decision support. Time-series analysis can be used
to estimate demand of goods by treating historical sales orders as time-series. These time-series, either
univariate or multivariate with covariates, can inform the training of machine learning models to
predict future demand for a specified time-period ( forecasting horizon) [
          <xref ref-type="bibr" rid="ref12 ref8">1, 10</xref>
          ].
        </p>
        <p>
          Mass customisation often leads to sparse demand due to many unique articles, making the time-series
intermittent or lumpy. Intermittent and lumpy time-series consist of many zero values followed by a
single demand or burst of demands [
          <xref ref-type="bibr" rid="ref13">11</xref>
          ]. Such time-series are dificult to forecast because most methods
handle continuous values [1].
        </p>
        <p>
          Deep learning time-series models can be local or global [
          <xref ref-type="bibr" rid="ref6">5</xref>
          ]. Local time-series models train on
historical samples of a single time-series for forecasting that series only, whereas global time-series
models train on multiple time-series and forecast future values for any series in the dataset. We use
semi-global model to describe multiple models trained on subsets of data to leverage global model
strengths while having a more homogeneous training data. Both global and local models handle one
time-series (and covariates) at a time during inference.
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Related work</title>
        <sec id="sec-2-2-1">
          <title>2.2.1. Demand forecasting and applications</title>
          <p>
            Demand forecasting is widely employed in the retail and MTS manufacturing domains [
            <xref ref-type="bibr" rid="ref1 ref15 ref20">4, 12, 13</xref>
            ]. The
most common methods for forecasting univariate time-series with machine learning are statistical
models like ARIMA [14] for local models aimed at forecasting one time-series, and deep learning models
like LSTMs and transformers, which can be both local or global, i.e., forecasting one or multiple
timeseries using the same model [
            <xref ref-type="bibr" rid="ref12 ref8">1, 10</xref>
            ]. Intermittent demands are common in both retail and manufacturing
domains [
            <xref ref-type="bibr" rid="ref1">4</xref>
            ]. A hybrid MTS/MTO approach: make-to-forecast (MTF) has been proposed which uses
forecasts to manufacture or purchase partially completed products based on forecasts, modifying them
when the orders are placed [
            <xref ref-type="bibr" rid="ref3 ref4">2, 3</xref>
            ]. However, there are few practical real-world applications of such
approaches to date [
            <xref ref-type="bibr" rid="ref10">8</xref>
            ]. Recent work on time-series have attempted to create foundation models which
would be capable of predicting on time-series from completely new domains [
            <xref ref-type="bibr" rid="ref18">15, 16</xref>
            ].
          </p>
        </sec>
        <sec id="sec-2-2-2">
          <title>2.2.2. Intermittent forecasting</title>
          <p>
            The most widely adopted method for forecasting intermittent demand is the Croston method [
            <xref ref-type="bibr" rid="ref19">17</xref>
            ] and
its many derivatives [
            <xref ref-type="bibr" rid="ref24 ref25">18, 19, 20, 21</xref>
            ]. The Croston method divides intermittent time-series into period
length and demand size, then smooths them separately before combining. This estimates the average
demand rate for a future time-period [
            <xref ref-type="bibr" rid="ref19">17</xref>
            ]. Another statistical approach from the retail domain is to use
a hierarchical forecasting structure with a greedy aggregation-decomposition to forecast stock-keeping
units for intermittent demand products [
            <xref ref-type="bibr" rid="ref20">13</xref>
            ].
          </p>
          <p>
            Deep learning models like LSTMs [
            <xref ref-type="bibr" rid="ref26">22</xref>
            ] or transformers [
            <xref ref-type="bibr" rid="ref12 ref27 ref8">10, 23</xref>
            ] often predict zeros when faced with
intermittent demand. Several specialised approaches have been proposed to forecast intermittent
demand with deep learning [
            <xref ref-type="bibr" rid="ref1">1, 4</xref>
            ]. One such approach, DeepAR, uses non-linear transformations
combined with appropriate likelihoods to overcome the issue of predicting only zero values, creating
a model architecture that works for both smooth and intermittent time-series [1]. Non-sequential
deep learning models have also been shown to perform well on intermittent time-series by using the
demands and intervals as separate inputs into a multi-layer perceptron with a single hidden layer [
            <xref ref-type="bibr" rid="ref1">4</xref>
            ].
Most intermittent forecasting methods predict an average demand rate, which is useful for domains
which focus on stock-keeping eficiency. However, few studies focus on predicting both the timing and
size of demands
          </p>
        </sec>
        <sec id="sec-2-2-3">
          <title>2.2.3. Global models</title>
          <p>
            Deep learning forecasting has mostly used local models historically [
            <xref ref-type="bibr" rid="ref6">1, 5</xref>
            ]. Many deep learning
architectures train both local and global models [
            <xref ref-type="bibr" rid="ref12 ref8">1, 10</xref>
            ], but papers often don’t specify which is used. Rožanec
et al. [
            <xref ref-type="bibr" rid="ref28">24</xref>
            ] used anomaly detection and explainable AI technologies to enable users to determine when
the output of forecasting models could be trusted. They noted that global models can perform better, but
there’s a gap in explaining why they sometimes produce bad forecasts. Montero-Manso and Hyndman
[
            <xref ref-type="bibr" rid="ref6">5</xref>
            ] found that global models could be both more complex and better at generalisation than local models.
          </p>
          <p>
            Semi-global models trained on groups of homogeneous time-series have been shown to perform better
than fully global models trained on more heterogeneous data [
            <xref ref-type="bibr" rid="ref29">25</xref>
            ]. Strong models can also be produced
by training global base models and using transfer learning for local or semi-global contexts [
            <xref ref-type="bibr" rid="ref30">26</xref>
            ]. Bandara
et al. used clustering on a set of 18 time-series features proposed by Hyndman [
            <xref ref-type="bibr" rid="ref31">27</xref>
            ] to determine which
time-series should be grouped together for semi-global training [
            <xref ref-type="bibr" rid="ref29">25</xref>
            ]. [
            <xref ref-type="bibr" rid="ref32">28</xref>
            ] used an iterative refinement
approach to the clustering where they let forecasting accuracy guide cluster assignment. These studies
show good results in clustering time-series for semi-global models, but it is unclear which features
are most relevant. Is there a diference in feature importance for smooth, intermittent, and lumpy
time-series? It has been noted that disparate time-series in the same training cluster are detrimental to
model performance [
            <xref ref-type="bibr" rid="ref29">25</xref>
            ]. Further exploration is required into which characteristics determine good
semi-global training clusters, and whether there is a "similarity cutof" to aim for.
2.2.4. Metrics
Forecast success can be assessed with diferent metrics, the most common method being some variation
of either the Mean Absolut Deviation (MAD), percentage error, or Mean Squared Error (MSE) calculated
from the prediction error of each time-step [
            <xref ref-type="bibr" rid="ref33">29</xref>
            ]. MSE, MAD, and a multitude of other metrics, e.g.,
MASE [30] and RMASE [
            <xref ref-type="bibr" rid="ref20">13</xref>
            ] have been used to assess intermittent forecasts with a per-time-step error
in various ways [
            <xref ref-type="bibr" rid="ref33">29</xref>
            ]. Although current metrics are efective in some contexts, more specialised metrics
are called for, as the large number of zero values make absolute errors a blunt assessment of the actual
forecasting performance [30].
          </p>
          <p>
            Some metrics account for the timing of demands to address the issue of over-penalising near-misses:
MAD and MSE calculate the error when a demand occurs by summing the forecasted values leading
up to the demand point and measuring its agreement with the specific demand for that time-step
[
            <xref ref-type="bibr" rid="ref33">29</xref>
            ]. Cumulative Forecast Error (CFE) measures positive and negative errors at each time-step for
the forecasting horizon before adding the errors together, essentially forming a sum aggregate error
[
            <xref ref-type="bibr" rid="ref33">29</xref>
            ]. This bypasses some of the issues with misjudging near-misses, but these metrics only help for
near-misses which occur before the demand, not after. Additionally, they penalise forecasts across the
forecasting horizon equally, meaning that the timing of the forecasts is not assessed.
          </p>
          <p>
            There are also domain-specific metrics such as Periods In Stock (PIS) [
            <xref ref-type="bibr" rid="ref33">29</xref>
            ] or Stock-keeping-oriented
Prediction Error Cost (SPEC) [31] which simulate the performance of a forecasting model in a
stockkeeping scenario at retail organisations. Such metrics only work for a stock-keeping oriented approach,
however, and are not suitable for use in an MTO manufacturing organisation or in domain-agnostic
approaches.
          </p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Forecasting intermittent demand: global and local model approaches for an MTO manufacturing context</title>
      <p>This section will explore the challenges and needs in demand forecasting for MTO manufacturing
organisations. We will focus on understanding system requirements, forecasting intermittent time-series,
and comparing global and local models.</p>
      <sec id="sec-3-1">
        <title>3.1. MTO manufacturing</title>
        <p>
          MTO manufacturers should forecast the demand of their goods more, but first, it is essential to
understand the specific requirements of demand forecasting systems in MTO manufacturing. Efective
forecasts would allow many mass customisation manufacturers to take calculated risks to occasionally
produce extra finished goods to stock, or to purchase semi-finished goods before demand arrives. By
doing so they could improve the eficiency of their inventory management, production planning, staf
scheduling, and material purchasing [
          <xref ref-type="bibr" rid="ref36">32</xref>
          ]. They could also serve customers better by contacting them
with "win-win" opportunities when a demand is forecasted. This could both help the manufacturing
organisation be more eficient and give the customer a more stable delivery and potentially better prices.
        </p>
        <p>In our experience from working in the domain, customer-supplied forecasts are the primary decision
support used to gauge future demand. Unfortunately, these forecasts are often either inaccurate
or not detailed enough to be useful beyond high-level budgeting. Manual forecasts are sometimes
made by observing past sales and using expertise to estimate future demand. These manual estimates
inform production planning and purchasing decisions. Unfortunately, this process is time-consuming
and dificult to scale accurately. This highlights the need for automated forecasting, which, besides
increasing forecasting performance, could also free up a lot of time for other tasks. It is dificult
to know beforehand which level of forecasting performance is needed to support decisions in MTO
manufacturing organisations, as there have not been many studies on this. To fully support
decisionmaking in MTO organisations where demands are often intermittent, forecasts should predict both
demand timing and size. These forecasts could help reduce setup times and increase machine up-times
by adjusting production order timing.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Intermittent time-series forecasting</title>
        <p>Forecasting intermittent time-series is particularly challenging due to the sporadic nature of demand.
Intermittent time-series forecasts usually predict an average demand rate per time-step, rather than the
size and timing of individual demand occurrences. It should be a priority to develop models capable of
predicting both demand size and timing, as they would allow many organisations to use the forecasts
in ways which are not possible with average demand rates. To accomplish this, the focus should be on
modelling the time-series in a way which deep learning models can understand. Due to their non-linear
nature, deep learning models can capture underlying patterns and detect subtle trends ranging across
time-series, whereas statistical models often oversimplify time-series problems.</p>
        <p>Of course, the efort to cater to every situation with a black box model may be too large, and the
stability and consistency of the standard statistical models is one of the reasons that they have been
used for a long time. The black box nature of deep learning models also makes them less interpretable
than statistical models, which can be detrimental to user trust in the models.</p>
        <p>
          Model performance metrics should relate to the real-world goal one is trying to accomplish. There
are good domain-specific metrics that simulate stock-keeping in e.g., the retail domain [
          <xref ref-type="bibr" rid="ref1 ref33">29, 4, 31</xref>
          ], but
such metrics are unfeasible for an MTO manufacturing setting. When training a deep learning model,
the loss function should ideally correlate strongly to the metric, and simulation metrics may be too slow
to use eficiently. The non-domain specific metrics often judge a sum aggregate error or the error per
time-step in the forecasting horizon [
          <xref ref-type="bibr" rid="ref33">29</xref>
          ]. These metrics are insuficient with models that forecast both
the timing and size of demands. Therefore, it is essential to develop domain-agnostic metrics capable of
correctly judging how well both the timing and size of demands are predicted.
        </p>
      </sec>
      <sec id="sec-3-3">
        <title>3.3. Global vs. local time-series models</title>
        <p>
          Global models, which train on multiple time-series, ofer the advantage of simplicity: There is no need
to tune the hyper-parameters and to spend time and compute resources on training tens of thousands of
diferent models. Previous work indicates that global models can outperform local models for time-series
forecasting [
          <xref ref-type="bibr" rid="ref6">5</xref>
          ]. We argue that the forecasting community should focus more on understanding the
strengths and drawbacks of global deep learning models compared to local models.
        </p>
        <p>
          A large disparity among time-series in the training data can negatively afect global model
performance. Several studies suggest that semi-global models could help address this issue [
          <xref ref-type="bibr" rid="ref29 ref32">25, 28</xref>
          ]. This
way we would get the best of both worlds: Robust models which generalise well and are not too
resource intensive to train, but that are still specific enough to yield performance on a par with, or
better than local models. To accomplish this, however, more knowledge is needed about how to best
group time-series together for training. It is not understood today which characteristics of time-series
are relevant for semi-global training clusters, and which characteristics are detrimental to forecasting
performance. It would also be interesting to examine whether there are cut-ofs in the diversity of
time-series or in the number of time-series needed in a training cluster to produce high-performing
models.
        </p>
        <p>A counterargument to using clustering for semi-global models is that forecasting systems are already
complex. Adding a clustering step to the forecasting process can make model maintenance more
challenging. This could lead to slower development of such systems, specifically in general upkeep
of models, and the designing of automatic re-training regimen. Additionally, it may be challenging to
leverage conditional models with exogenous variables in a global or semi-global context, because the
efects of the exogenous variables could difer too much across the time-series. Seasonal components
would also need to be accounted for, which adds additional complexities. However, the performance
increase could very well outweigh these issues, but more work is needed to fully understand this.</p>
        <p>In real-world situations, it is common that some time-series are more important to forecast well than
others. For this reason, a purely global forecasting approach may not produce the best results. A hybrid
mix of local and global models could work well to ensure satisfactory performance on the important
timeseries. However, this requires a clearer understanding of which time-series characteristics determine
whether global or local models are more efective. Perhaps transfer learning from a global model to a
ifne-tuning on a single time-series can yield stronger performance than either a global or local model
can? Our experience from working with forecasting models is that some time-series are more dificult
to forecast than others, and that training local models on such time-series often does not yield any
valuable results whatsoever. However, when a global model trained on other time-series is applied to the
same time-series the forecasting performance is improved. Although this is an anecdotal observation, it
highlights that it could be worthwhile to study whether certain inherent characteristics make some
time-series more dificult to forecast using local models.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Pilot experiment: demand forecasting in smart purchasing system</title>
      <p>To validate the usefulness of demand forecasting in an MTO context, we perform a pilot experiment
integrating demand forecasting into an existing purchasing decision support system. We simulate the
system for one year and compare the savings with and without demand forecasting. We first provide a
high-level description of the system’s purpose and implementation, before detailing the experiment
and results.</p>
      <sec id="sec-4-1">
        <title>4.1. Purchasing task description and system implementation</title>
        <p>An AI-assisted purchasing decision support system has been built for a company with an MTO and
mass customisation strategy. The system helps purchase material sub-components that are unique to
each product and customer, and need to be bought frequently and in large quantities. The material
sub-component is one of several components used to produce the articles sold to customers. There are
2000 - 3000 unique articles active at any time, and anything between 50 and 100 diferent components
are purchased each week at varying quantities.</p>
        <p>The supplier uses bulk pricing: Purchases can be combined to reach diferent price tiers, afecting the
ifnal price per component. The system helps taking calculated risks outside of the known requirements
to get a lower price per component. Components become obsolete if customers change their product
design, so the company risks any component bought above the order-based quantities (purchasing
requirements). Additionally, components have a shelf-life of roughly 9 months, after which they degrade
too much in quality and cannot be used to produce the finished articles.</p>
        <p>The decision support system uses a machine learning model which can estimate the probability of
obsolescence  for a component purchase at a specific quantity . The target variable is based on
historical purchases and component usage before obsolescence: For training instances with historical
known data, if we are assessing a purchasing quantity of  = 10,000 components, and the historical
data shows that 100,000 components were used up before the customer discontinued the product version,

then the regression target becomes 100,000 = 0.1. The input features include factors like customer
consistency in purchase quantities, purchase frequency, and component lifetime before design changes.
Other features include time since design swap, volume since design swap, and price list specific features.</p>
        <p>The system takes a list of known purchase requirements and returns purchasing suggestions to the
purchaser, who makes the final decision. Because the price lists are available during use, we know the
following for a potential quantity increase : (1) The monetary risk of increasing the quantity; (2)
The best-case monetary reward of increasing the quantity. With these components together with the
probability of obsolescence , we can compute the expected value E() of increasing the purchase
quantity:</p>
        <p>E() = (1 − ) · reward() −  · risk()
(1)</p>
        <p>The system considers several quantity increases per component and chooses the one with the highest
expected value. After this the system must calculate the expected values for the other quantity increases
again, because the conditions have changed due to the previous quantity increase. Quantity increases
are made until there are no more positive expected values, or other stopping criteria such as a time
limit are reached.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Experiment setup</title>
        <p>The experiment compares three diferent purchasing approaches: (1) Human-based purchasing; (2)
System-based purchasing; (3) System-based purchasing enhanced with demand forecasting. We simulate
one year of system use and compare the purchasing outcomes to the actual human purchasing done for
the same year. The simulation tracks excess stock from purchase increases and subtracts these quantities
from future purchase requirements to simulate the correct requirements. Additionally, expired and
obsolete stock are tracked to further correct the excess stock. Expired stock occurs when a component
has lied unused in stock for more than 9 months, and obsolete stock occurs when the customer updates
their component to a new version, rendering the old version useless.</p>
        <p>Summary statistics are computed after the simulation: How much did we spend compared to the
human-based purchasing, and how many extra components did we buy? What component quantities
went unused either due to expiration, or due to becoming obsolete because customers changed versions?
How much stock remained at the end of the simulation (which would not become obsolete), and how
much would that have cost to purchase with human purchasing? We estimate the cost of excess
components using human purchasing and add it to the baseline cost to compute an estimated ground
truth. The system’s total spending is compared to this ground truth to estimate the percentage of
monetary savings.</p>
        <p>
          Two diferent simulations are run: The standard system-based purchasing uses the implementation
described in subsection 4.1. The second simulation builds on the system by including demand forecasting
to improve the decision making. We use TSB [
          <xref ref-type="bibr" rid="ref25">21</xref>
          ] for intermittent and lumpy demands and Prophet
[
          <xref ref-type="bibr" rid="ref37">33</xref>
          ] for smooth and erratic demands. These models are chosen because they are readily available and
can produce decent forecasts without much efort. TSB applies separate single exponential smoothing
to estimate the demand probability and average demand size, which are then multiplied to produce
a demand rate forecast [
          <xref ref-type="bibr" rid="ref25">21</xref>
          ]. Prophet performs an automatic decomposition into trend, holiday, and
seasonal components, which are forecasted separately and then combined additively to produce the
ifnal forecast [
          <xref ref-type="bibr" rid="ref37">33</xref>
          ]. For each purchasing occasion, demand forecasts are computed for all included
components monthly. The demand forecasts are based on the sales orders of the finished article which
contain the material sub-component. The forecasted demand for the coming 9 months is set as an
upper limit of the purchasing quantity for that article’s sub-component, with the hope that this will
reduce expired components. In addition to this, we set a max quantity cap of five times the required
quantity. For articles with fewer than 3 historical sales, the forecast is skipped and a much lower max
cap (2· the required quantity) is set to reflect the relatively high uncertainty of these articles. These
limit parameters were found through trial and error by observing forecasts and purchasing suggestions
in diferent scenarios.
        </p>
      </sec>
      <sec id="sec-4-3">
        <title>4.3. Results and analysis</title>
        <p>Adding forecasts to the system improves on all metrics (Table 1): The system achieves a significantly
higher savings across one year of purchasing, and manages to do so while taking smaller risks and
over-purchasing a smaller quantity in total. Additionally, the system is more successful at taking its
risks (Table 2), mainly due to components expiring less frequently. This shows that adding demand
forecasts to the system achieves the desired results: Fewer components expire due to not being used
within 9 months of being purchased. The system without demand forecasts still outperforms human
purchasing, but does so by taking less-calculated risks and relying on a more brute-force approach.
The non-demand forecasting approach takes significantly larger risks (5.19 % across one year) than the
demand forecasting approach which spends roughly the same amount of money as a human purchaser
for a much higher quantity, most of which ends up being used. This occurs because of the bulk pricing,
which causes many small purchases to have a very high component price, whereas bundling several
purchases together in one larger purchase can lower the cost despite increasing the quantity. Overall,
the results show that including demand forecasts in the system increases the profit margin and also has
the potential to reduce the environmental impact from unnecessary transports.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Summary and roadmap</title>
      <p>Demand forecasting is an important challenge for many organisations. MTO manufacturers face added
complexities, causing them to avoid demand forecasting. The large number of articles warrants using
global models that can forecast many time-series without training each one explicitly. Additionally,
the per-article demand in MTO organisations is often sparse, causing an intermittent nature in the
time-series.</p>
      <p>Global and semi-global models could be trained better if we had more insights into how performance
is afected by diferent time-series characteristics. It is essential to develop intermittent forecasting
models which do not only produce an average demand rate across time, but that predict when demand
is likely to occur and how large it will be. To assess such forecasts, new metrics need to be developed
that assess how well each actual demand is predicted both in timing and size, and not just on a macro
level.</p>
      <p>The pilot experiment demonstrates that it is possible to use demand forecasting to significantly
improve processes in an MTO context, and that even simple forecasting models can yield valuable
results when embedded into operational decision support systems. The system with forecasts achieved
higher savings and took smaller risks compared to the system without forecasts, resulting in a higher
profit margin and a lower environmental impact. These results motivate putting focus on addressing
demand forecasting challenges from an MTO manufacturing perspective, both in developing forecasts
better suited to the problems for such organisations, and in performing experiments to validate the
approaches.</p>
      <p>A research roadmap is proposed to address the knowledge gaps discussed in this position paper:
• MTO forecasting challenges
– What are the challenges faced by MTO organisations when it comes to demand forecasting?
What type of information and systems would be needed to make full use of demand forecasts
in operations? What are the data and software requirements for such forecasting systems?
Do explanatory methods need to be developed for the forecasts to be useful as decision
support to humans? We are currently investigating this though a real-world case study.
– In what areas of operation can demand forecasts be used in MTO organisations, and what
performance is needed? An important next step is to perform empirical studies which
examine actual uses of forecast in diferent scenarios to support production planning and
purchasing. The pilot experiment in this paper is a promising first step, but future
experiments should also study additional aspects such as the efects of using global models, or on
incorporating intermittent demand forecasts which predict both demand timing and size.
• Global time-series models
• Intermittent time-series models
– How can the correct "likeness" or "diversity" be determined for data used to train semi-global
models for producing better forecasts than local or global models? How do we construct
hyperparameter searches on specific datasets to find the optimal training data diversity?
Which time-series features are the most relevant to represent diversity in this regard?
– What time-series characteristics determine whether time-series benefit from using global,
semi-global, or local models?
– How can intermittent time-series data be processed, and how do we construct deep learning
models that train on the data in order to forecast both demand size and demand timings
accurately? Can these models be global instead of local, and how are the methods for
building global intermittent models similar or diferent to global models aimed at smooth
time-series?
∗ Are some intermittent time-series too dificult for deep learning models to learn? Should
these time-series instead be left to proven statistical methods such as the Croston
method? What are the characteristics of such time-series, and is it possible to identify
them automatically?
– What is the best way to assess the strength intermittent time-series forecasts both in terms of
demand size and timing? Which metrics best represent the usefulness in MTO organisations
or other organisations where sparse demand is common?</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgments</title>
      <p>This research was sponsored via the KKS project SERT (Software Engineering ReThought),
www.rethought.se.</p>
    </sec>
    <sec id="sec-7">
      <title>Declaration on Generative AI</title>
      <p>During the preparation of this work, the authors used ChatGPT-4o in order to: Paraphrase and reword.
After using this tool/service, the authors reviewed and edited the content as needed and take full
responsibility for the publication’s content.
[14]
[15]
[18]
[19]</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [4] [6] [1]
          <string-name>
            <given-names>David</given-names>
            <surname>Salinas</surname>
          </string-name>
          et al. “
          <article-title>DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks”</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          en.
          <source>In: International Journal of Forecasting 36.3 (July</source>
          <year>2020</year>
          ), pp.
          <fpage>1181</fpage>
          -
          <lpage>1191</lpage>
          . issn:
          <volume>01692070</volume>
          . doi:
          <volume>10</volume>
          .1016/j.ijforecast.
          <year>2019</year>
          .
          <volume>07</volume>
          .001.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Jack</given-names>
            <surname>Meredith</surname>
          </string-name>
          and
          <string-name>
            <given-names>Umit</given-names>
            <surname>Akinc</surname>
          </string-name>
          . “
          <article-title>Characterizing and Structuring a New Make-to-Forecast Production Strategy”</article-title>
          .
          <source>In: Journal of Operations Management 25.3</source>
          (
          <issue>2007</issue>
          ), pp.
          <fpage>623</fpage>
          -
          <lpage>642</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.jom.
          <year>2006</year>
          .
          <volume>04</volume>
          .006.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Umit</given-names>
            <surname>Akinc and Jack R. Meredith</surname>
          </string-name>
          . “
          <article-title>Make-to-Forecast: Customization with Fast Delivery”</article-title>
          . en.
          <source>In: International Journal of Operations &amp; Production Management 35.5 (May</source>
          <year>2015</year>
          ), pp.
          <fpage>728</fpage>
          -
          <lpage>750</lpage>
          . issn:
          <fpage>0144</fpage>
          -
          <lpage>3577</lpage>
          . doi:
          <volume>10</volume>
          .1108/IJOPM-12-2012-0567.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <given-names>Nikolaos</given-names>
            <surname>Kourentzes</surname>
          </string-name>
          . “
          <article-title>Intermittent Demand Forecasts with Neural Networks”. en</article-title>
          .
          <source>In: International Journal of Production Economics</source>
          <volume>143</volume>
          .1 (May
          <year>2013</year>
          ), pp.
          <fpage>198</fpage>
          -
          <lpage>206</lpage>
          . issn:
          <volume>09255273</volume>
          . doi:
          <volume>10</volume>
          .1016/j.ijpe.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Pablo</given-names>
            <surname>Montero-Manso</surname>
          </string-name>
          and
          <string-name>
            <given-names>Rob J.</given-names>
            <surname>Hyndman</surname>
          </string-name>
          . “
          <article-title>Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality”. en</article-title>
          .
          <source>In: International Journal of Forecasting 37</source>
          .4 (
          <issue>Oct</issue>
          .
          <year>2021</year>
          ), pp.
          <fpage>1632</fpage>
          -
          <lpage>1653</lpage>
          . issn:
          <volume>01692070</volume>
          . doi:
          <volume>10</volume>
          .1016/j.ijforecast.
          <year>2021</year>
          .
          <volume>03</volume>
          .004.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <given-names>N.S.</given-names>
            <surname>Adshead</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.H.R.</given-names>
            <surname>Price</surname>
          </string-name>
          . “
          <article-title>Demand forecasting and cost performance in a model of a real manufacturing unit”</article-title>
          .
          <source>In: International Journal of Production Research 25.9</source>
          (
          <issue>1987</issue>
          ), pp.
          <fpage>1251</fpage>
          -
          <lpage>1265</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <source>doi: 10</source>
          .1080/00207548708919913.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Shaw</surname>
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Chen</surname>
            and
            <given-names>Maling</given-names>
          </string-name>
          <string-name>
            <surname>Ebrahimpour</surname>
          </string-name>
          . “
          <article-title>Demand forecasting for a jewelry packaging company: pattern identification and assessment”</article-title>
          .
          <source>In: International Journal of Production Economics 22.3</source>
          (
          <issue>1991</issue>
          ), pp.
          <fpage>203</fpage>
          -
          <lpage>209</lpage>
          . doi:
          <volume>10</volume>
          .1016/
          <fpage>0925</fpage>
          -
          <lpage>5273</lpage>
          (
          <issue>91</issue>
          )
          <fpage>90096</fpage>
          -C.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>Marius</given-names>
            <surname>Syberg</surname>
          </string-name>
          et al. “
          <article-title>Framework for Predictive Sales and Demand Planning in Customer-Oriented Manufacturing Systems Using Data Enrichment and Machine Learning”</article-title>
          . en.
          <source>In: Procedia CIRP 120</source>
          (
          <year>2023</year>
          ), pp.
          <fpage>1107</fpage>
          -
          <lpage>1112</lpage>
          . issn:
          <volume>22128271</volume>
          . doi:
          <volume>10</volume>
          .1016/j.procir.
          <year>2023</year>
          .
          <volume>09</volume>
          .133.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>B.</given-names>
            <surname>Joseph Pine</surname>
          </string-name>
          <string-name>
            <given-names>II</given-names>
            ,
            <surname>Bart</surname>
          </string-name>
          <string-name>
            <surname>Victor</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Andrew C.</given-names>
            <surname>Boynton</surname>
          </string-name>
          . “Making Mass Customization Work”.
          <source>In: Harvard Business Review</source>
          <volume>71</volume>
          (
          <year>1993</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>Bryan</given-names>
            <surname>Lim</surname>
          </string-name>
          et al. “
          <article-title>Temporal Fusion Transformers for Interpretable Multi-Horizon Time Series Forecasting”</article-title>
          . en.
          <source>In: International Journal of Forecasting 37</source>
          .4 (
          <issue>Oct</issue>
          .
          <year>2021</year>
          ), pp.
          <fpage>1748</fpage>
          -
          <lpage>1764</lpage>
          . issn:
          <volume>01692070</volume>
          . doi:
          <volume>10</volume>
          .1016/j.ijforecast.
          <year>2021</year>
          .
          <volume>03</volume>
          .012.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Aris</surname>
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Syntetos</surname>
            , John E. Boylan, and
            <given-names>J. D.</given-names>
          </string-name>
          <string-name>
            <surname>Croston</surname>
          </string-name>
          . “
          <article-title>On the categorization of demand patterns”.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          en.
          <source>In: Journal of the Operational Research Society</source>
          <volume>56</volume>
          .5 (May
          <year>2005</year>
          ), pp.
          <fpage>495</fpage>
          -
          <lpage>503</lpage>
          . doi:
          <volume>10</volume>
          .1057/ palgrave.jors.
          <volume>2601841</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>Fatima</given-names>
            <surname>Zohra</surname>
          </string-name>
          Benhamida et al. “
          <article-title>Demand Forecasting Tool for Inventory Control Smart Systems”</article-title>
          . en.
          <source>In: Journal of Communications Software and Systems 17.2</source>
          (
          <issue>2021</issue>
          ), pp.
          <fpage>185</fpage>
          -
          <lpage>196</lpage>
          . issn:
          <volume>18456421</volume>
          , 18466079. doi:
          <volume>10</volume>
          .24138/jcomss-2021-0068.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <given-names>George E. P.</given-names>
            <surname>Box and Gwilym M. Jenkins</surname>
          </string-name>
          .
          <source>Time Series Analysis: Forecasting and Control</source>
          . San Francisco: Holden-Day,
          <year>1970</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <given-names>Mononito</given-names>
            <surname>Goswami</surname>
          </string-name>
          et al. “
          <article-title>MOMENT: A Family of Open Time-series Foundation Models”</article-title>
          . In: vol.
          <volume>235</volume>
          .
          <year>2024</year>
          , pp.
          <fpage>16115</fpage>
          -
          <lpage>16152</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>Yuxuan</given-names>
            <surname>Liang</surname>
          </string-name>
          et al. “
          <article-title>Foundation Models for Time Series Analysis: A Tutorial and Survey”</article-title>
          .
          <source>In: 2024</source>
          , pp.
          <fpage>6555</fpage>
          -
          <lpage>6565</lpage>
          . doi:
          <volume>10</volume>
          .1145/3637528.3671451.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>J.D.</given-names>
            <surname>Croston</surname>
          </string-name>
          . “
          <article-title>Forecasting and Stock Control for Intermittent Demands”</article-title>
          .
          <source>In: Operational Research Quarterly 23.3</source>
          (
          <issue>1972</issue>
          ), pp.
          <fpage>289</fpage>
          -
          <lpage>303</lpage>
          . doi:
          <volume>10</volume>
          .1057/jors.
          <year>1972</year>
          .
          <volume>50</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>Chongshou</given-names>
            <surname>Li</surname>
          </string-name>
          and
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Lim</surname>
          </string-name>
          .
          <article-title>“A Greedy Aggregation-Decomposition Method for Intermittent Demand Forecasting in Fashion Retailing”</article-title>
          . en.
          <source>In: European Journal of Operational Research</source>
          <volume>269</volume>
          .3 (
          <issue>Sept</issue>
          .
          <year>2018</year>
          ), pp.
          <fpage>860</fpage>
          -
          <lpage>869</lpage>
          . issn:
          <volume>03772217</volume>
          . doi:
          <volume>10</volume>
          .1016/j.ejor.
          <year>2018</year>
          .
          <volume>02</volume>
          .029.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <given-names>Aris A.</given-names>
            <surname>Syntetos</surname>
          </string-name>
          and
          <string-name>
            <surname>John E. Boylan.</surname>
          </string-name>
          “
          <article-title>On the Bias of Intermittent Demand Estimates”</article-title>
          . en.
          <source>In: International Journal of Production Economics</source>
          <volume>71</volume>
          .
          <fpage>1</fpage>
          -
          <lpage>3</lpage>
          (May
          <year>2001</year>
          ), pp.
          <fpage>457</fpage>
          -
          <lpage>466</lpage>
          . issn:
          <volume>09255273</volume>
          . doi:
          <volume>10</volume>
          .1016/S0925-
          <volume>5273</volume>
          (
          <issue>00</issue>
          )
          <fpage>00143</fpage>
          -
          <lpage>2</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <given-names>Aris A.</given-names>
            <surname>Syntetos</surname>
          </string-name>
          and
          <string-name>
            <surname>John E. Boylan.</surname>
          </string-name>
          “
          <article-title>The Accuracy of Intermittent Demand Estimates”</article-title>
          . en.
          <source>In: International Journal of Forecasting 21.2</source>
          (
          <issue>Apr</issue>
          .
          <year>2005</year>
          ), pp.
          <fpage>303</fpage>
          -
          <lpage>314</lpage>
          . issn:
          <volume>01692070</volume>
          . doi:
          <volume>10</volume>
          .1016/j.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>ijforecast.</surname>
          </string-name>
          <year>2004</year>
          .
          <volume>10</volume>
          .001.
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Estelle</surname>
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Shale</surname>
            , John E. Boylan, and
            <given-names>F. R.</given-names>
          </string-name>
          <string-name>
            <surname>Johnston</surname>
          </string-name>
          . “
          <article-title>Forecasting for Intermittent Demand: The Estimation of an Unbiased Average”</article-title>
          .
          <source>In: Journal of the Operational Research Society 57.5</source>
          (
          <issue>2006</issue>
          ), pp.
          <fpage>588</fpage>
          -
          <lpage>592</lpage>
          . doi:
          <volume>10</volume>
          .1057/palgrave.jors.
          <volume>2602031</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Ruud</surname>
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Teunter</surname>
          </string-name>
          , Aris A.
          <string-name>
            <surname>Syntetos</surname>
          </string-name>
          , and Mohamed Zied Babai. “Intermittent Demand:
          <article-title>Linking Forecasting to Inventory Obsolescence”</article-title>
          .
          <source>In: European Journal of Operational Research 214.3</source>
          (
          <issue>2011</issue>
          ), pp.
          <fpage>606</fpage>
          -
          <lpage>615</lpage>
          . issn:
          <fpage>0377</fpage>
          -
          <lpage>2217</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.ejor.
          <year>2011</year>
          .
          <volume>05</volume>
          .018.
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>Sepp</given-names>
            <surname>Hochreiter</surname>
          </string-name>
          and
          <string-name>
            <given-names>Jürgen</given-names>
            <surname>Schmidhuber</surname>
          </string-name>
          .
          <article-title>“Long Short-Term Memory”</article-title>
          .
          <source>In: Neural Computation 9.8</source>
          (
          <issue>1997</issue>
          ), pp.
          <fpage>1735</fpage>
          -
          <lpage>1780</lpage>
          . doi:
          <volume>10</volume>
          .1162/neco.
          <year>1997</year>
          .
          <volume>9</volume>
          .8.1735.
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>Ashish</given-names>
            <surname>Vaswani</surname>
          </string-name>
          et al. “
          <article-title>Attention is All You Need”</article-title>
          .
          <source>In: Advances in Neural Information Processing Systems</source>
          . Vol.
          <volume>2017</volume>
          -December.
          <year>2017</year>
          , pp.
          <fpage>5999</fpage>
          -
          <lpage>6009</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>Jože</given-names>
            <surname>Rožanec</surname>
          </string-name>
          et al. “
          <source>Explaining Bad Forecasts in Global Time Series Models”. en. In: Applied Sciences 11.19 (Oct</source>
          .
          <year>2021</year>
          ), p.
          <fpage>9243</fpage>
          . issn:
          <fpage>2076</fpage>
          -
          <lpage>3417</lpage>
          . doi:
          <volume>10</volume>
          .3390/app11199243.
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Kasun</surname>
            <given-names>Bandara</given-names>
          </string-name>
          , Christoph Bergmeir, and Slawek Smyl. “
          <source>Forecasting Across Time Series Databases Using Recurrent Neural Networks on Groups of Similar Series: A Clustering Approach”. en. In: Expert Systems with Applications</source>
          <volume>140</volume>
          (Feb.
          <year>2020</year>
          ), p.
          <fpage>112896</fpage>
          . issn:
          <volume>09574174</volume>
          . doi:
          <volume>10</volume>
          .1016/j.eswa.
          <year>2019</year>
          .
          <volume>112896</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>Rui</given-names>
            <surname>Ye</surname>
          </string-name>
          and
          <string-name>
            <given-names>Qun</given-names>
            <surname>Dai</surname>
          </string-name>
          . “
          <article-title>Implementing Transfer Learning Across Diferent Datasets for Time Series Forecasting”</article-title>
          . en.
          <source>In: Pattern Recognition</source>
          <volume>109</volume>
          (
          <issue>Jan</issue>
          .
          <year>2021</year>
          ), p.
          <fpage>107617</fpage>
          . issn:
          <volume>00313203</volume>
          . doi:
          <volume>10</volume>
          .1016/j.patcog.
          <year>2020</year>
          .
          <volume>107617</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Rob</surname>
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Hyndman</surname>
            ,
            <given-names>Earo</given-names>
          </string-name>
          <string-name>
            <surname>Wang</surname>
            , and
            <given-names>Nikolay</given-names>
          </string-name>
          <string-name>
            <surname>Laptev</surname>
          </string-name>
          . “
          <string-name>
            <surname>Large-Scale Unusual</surname>
          </string-name>
          Time Series Detection”. en.
          <source>In: 2015 IEEE International Conference on Data Mining Workshop (ICDMW)</source>
          .
          <source>Atlantic City</source>
          , NJ, USA: IEEE, Nov.
          <year>2015</year>
          , pp.
          <fpage>1616</fpage>
          -
          <lpage>1619</lpage>
          . isbn:
          <fpage>978</fpage>
          -1-
          <fpage>4673</fpage>
          -8493-3. doi:
          <volume>10</volume>
          .1109/ICDMW.
          <year>2015</year>
          .
          <volume>104</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>Angel</given-names>
            <surname>López</surname>
          </string-name>
          <string-name>
            <surname>Oriona</surname>
          </string-name>
          , Pablo Montero Manso, and José Antonio Vilar Fernández.
          <source>Time Series Clustering Based on Prediction Accuracy of Global Forecasting Models. en. arXiv:2305</source>
          .00473 [cs, stat].
          <source>Apr</source>
          .
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>Peter</given-names>
            <surname>Wallström</surname>
          </string-name>
          .
          <article-title>“Evaluation of Forecasting Techniques and Forecast Errors”</article-title>
          . Available at https: //www.diva- portal.org/smash/get/diva2:990519/FULLTEXT01.pdf.
          <source>Licentiate thesis</source>
          . Luleå, Sweden: Luleå University of Technology, May
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          <string-name>
            <given-names>Nikolaos</given-names>
            <surname>Kourentzes</surname>
          </string-name>
          . “
          <article-title>On Intermittent Demand Model Optimisation and Selection”. en</article-title>
          .
          <source>In: International Journal of Production Economics</source>
          <volume>156</volume>
          (Oct.
          <year>2014</year>
          ), pp.
          <fpage>180</fpage>
          -
          <lpage>190</lpage>
          . issn:
          <volume>09255273</volume>
          . doi:
          <volume>10</volume>
          .1016/j.ijpe.
          <year>2014</year>
          .
          <volume>06</volume>
          .007.
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          <string-name>
            <given-names>Dominik</given-names>
            <surname>Martin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Philipp</given-names>
            <surname>Spitzer</surname>
          </string-name>
          , and Niklas Kühl.
          <article-title>“A new metric for lumpy and intermittent demand forecasts: Stock-keeping-oriented Prediction Error Costs”</article-title>
          .
          <source>In: Proceedings of the Annual Hawaii International Conference on System Sciences. Vol. 2020-January</source>
          .
          <year>2020</year>
          , pp.
          <fpage>974</fpage>
          -
          <lpage>983</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [32]
          <string-name>
            <given-names>Jamal</given-names>
            <surname>Fattah</surname>
          </string-name>
          et al. “
          <article-title>Forecasting of demand using ARIMA model”</article-title>
          .
          <source>In: International Journal of Engineering Business Management</source>
          <volume>10</volume>
          (
          <year>2018</year>
          ). doi:
          <volume>10</volume>
          .1177/1847979018808673.
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Sean J Taylor and Benjamin Letham</surname>
          </string-name>
          . “Forecasting at scale”.
          <source>In: PeerJ Preprints</source>
          <volume>5</volume>
          (
          <issue>Sept</issue>
          .
          <year>2017</year>
          ),
          <year>e3190v2</year>
          . doi:
          <volume>10</volume>
          .7287/peerj.preprints.3190v2.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>