Organizational Information improves Forecast Efficiency of Correction Techniques

Abstract

Financial services within corporations have an essential need for accurate forecasts. In corporations, experts typically generate judgmental cash flow forecasts in a decentralized fashion and provide data that is important in corporate risk management. But the accuracy of these forecasts is most likely reduced by biases of the organizational structure. As for the importance of cash flow forecasts, usually correction techniques are applied with statistical methods based on historical data. In most cases the organizational biases are not included into the correction techniques. This paper argues that disregarding the organizational information actually decreases forecast efficiency. Forecast efficiency provides statistical information for the amount of structure within forecasts and errors. In case of aggregated cash flows in accounting, the forecasts highly depend on return margins. The empirical results in this paper show that debiasing with forecasts correction based on organizational information can improve forecast efficiency by 56 % to a statistical approach. The reduction of inefficient pattern show statistics arguing for forecast correction that rely on organizational biases (standard deviation of error 0.20) instead of basic statistical approaches that harm forecast efficiency (standard deviation of error 0.28).

1 Introduction

Corporations with global operations typically generate forecasts for cash flow items on a regular basis (e.g., monthly or quarterly), at different organizational levels, business divisions, and countries. These forecasts are often generated in a decentralized fashion by the subsidiaries, where the subsidiaries send thousands of item-level forecasts and revisions to corporate headquarters. These forecasts are then consolidated and used in crucial tasks of the corporate finance department (such as in [ 14 ] or even to access with cash flow forecasts the company’s stock market value [ 13 ]).

The tasks in corporate departments strongly depend on the quality of the forecasts, as they provide the data base for the financial planning operations and subsequent management activities. For instance, due to forecast inaccuracies, the corporate hedging to reduce foreign exchange risks will result in increased costs or uncovered currency risks. 1.1

The Problem of Judgmental Forecasts

Usually, cash flow forecasts result from the judgment of human experts [ 24 ] and are revised several months or quarters after the initial forecast until the date of the actual realization finalizes the sequence of forecasts. The initial forecast and the sequence of adjusted forecasts is referred to as forecasting process, while the sequence of adjustments in revisions is usually coined as revisioning process or simply revisioning. When judgmental forecasting takes place, the forecasts can be prone to individual biases and latent human factors that entail forecasting processes in many ways [ 16, 18 ]. Additionally, the organizational structures and dependencies of the environment can change the forecaster’s expectation, resulting in organizational biases that result in forecast inaccuracies [ 6 ]. 1.2

Correction Techniques and Organizational Biases

Improving biased forecasts is possible with forecast correction techniques that analyze and change the human prediction with statistical models [ 12 ]. For instance, [ 15 ] found dependencies of timing and magnitude of cash flow revisions. Their results state that cash flow forecast processes are more accurate when they show a high revision at a late state of the process compared to a high revision at the early stage.

However, current forecast correction techniques often employ solely statistical methods – leaving out the organizational biases for approaches of forecast improvement. In corporate finance, several important key performance indicators (KPI) exist that aggregate many figures. An example of such key figure is Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA) margin, which can be used as one of the primary proxies for a company’s current operating profitability [ 19 ]. When humans try to achieve personal objectives (e.g., bonus payments by financial incentives) predefined targets that rely on these figures, for instance percentage return margins, these organizational biases can alter forecasts and their adjustments in a revisioning process [ 11 ].

In addition, in the realm of cash flows, several business functions might influence the realization volume of cash flows. The looming failure to meet earnings targets (which might reduce manager’s bonus payments) is an incentive to hold-back invoices received within term of credit. Alternatively, managers can trigger invoices issued earlier or might change payment terms in order to align annual cash results with targets. Conversely, if earning targets have been met already, there might be an incentive to delay the issuing of invoices until the next year to increase the probability of meeting next year’s targets. In particular, the papers of [ 4 ], [ 7 ], and [ 5 ] show that realizations are often shifted according to earnings management policies. When the volumes are shifted, the forecast errors can be expected to exhibit a systematic bias. 1.3

Efficiency Theory

Biases often translate to observable patterns in forecasting processes and one measurement to analyze the systematic behavior of revisioning is the efficiency theory . The theory in market finance [ 10 ] and forecasting [ 22 ] suggest that processes are efficient if they describe a random walk. The theory states that non-random walks promote inefficient forecasting since correlations among revisions with revisions or errors are expected to show statistical insufficiency that has the potential to anticipate future adjustments or errors. The application of this theory provides evidence that correlations exist in many cases [ 2, 17, 1, 9, 8 ]. 1.4

Our Contribution

This paper argues that efficiency provides a statistical tool to evaluate different correction approaches. The analysis of efficiency figures can provide insights for the differences of model predictions. The analyses for accounting cash flows contribute to the current research as they show that including organizational information into correction models is key for further improvements in correction techniques. When the empirical outcomes of these organizational models are compared to purely statistical model approaches they show that both models reduce the error, but the disregard of organizational information in the purely statistical approach does crucially harm the forecast efficiency. Moreover, this insight is also applicable to other domains, where exploratory data analysis and forecast correction play an important role in time series forecasting. 1.5

Structure of the Paper

The remainder of the paper is structured as follows. The data description in Section 3 is followed by the notation that is introduced in Section 4. Section 5 describes the design for the empirical analysis and the concept of forecast efficiency in detail. Section 6 presents the results and interpretation of the analysis. In Section 7 discusses the implications of this work for future improvements in forecast correction. 2

Related Work

Organizational biases can result in forecast inaccuracies as pointed out by Daniel et al. [ 6 ] but does not correct them in any way. He identified "dividend thresholds" as a organizational bias, which alters the forecasts.

In the paper [ 21 ], the authors analyzed short time series within the year and used a Bayesian method to account for sub-seasonal information for the seasonal based correction. In contrast to their setting, our forecast series are even shorter (5 reference points instead of 12), the application of linear regression models (instead of Bayesian models), and we account for one single information in our paper focuses a margin target at the end of year (instead of the whole sub-annual pattern).

Regarding seasonality, Yelland [ 27 ] concludes that a simple stable seasonal pattern model can perform surprisingly well, if it uses “theory-free” descriptions of booking processes. His findings are in resonance to the theme that simple empirically-based models do frequently better than complex ones.

The authors of [ 3 ] promote that in marketing and finance simple models sometimes predict more accurately than complex models. The authors argue that “the benefits of simplicity are often overlooked because the importance of the bias component of prediction error is inflated, and the variance component of prediction error (based on oversensitivity to different samples) is neglected.” Reasoned by their study, we correct the forecasts with a simple linear regression model. 3

Empirical Cash Flow Data

The data stems from a record of cash flow forecasts and realizations provided by a multinational sample corporation. With over 100,000 employees, the company generates annual revenues in the billion Euro range. The corporation is headquartered in Germany, but has worldwide more than 300 separate legal entities. The subsidiaries are grouped into four distinct divisions (D1 – D4), based on their business portfolios.

Each subsidiary operates officially independently of the corporation, while there are some organizational dependencies. First, based on the set of local plans, the corporation re-adjusts the planning to an overall view, and sets the target requirements for local operations for being rated as a “successful” subsidiary. Second, in the corporation the fiscal year ends in December and the subsidiaries that meet targets is assumed to be most pronounced at the end of the year. Third, as the subsidiaries operate independently, they have their own financial information system, a heterogeneous payment structure (e.g., incentivization bonuses) and have to ensure liquidity for their operations (e.g., with earnings management processes). Fourth, each subsidiary that is participating in the forecasting process – mostly large-volume entities – enters its expectations on future cash flow in a digital, corporate-based forecasting system.

Financial risk management is centralized, with the local subsidiaries reporting cash flows to the corporation’s central finance department, where these serve as the basis for further actions in corporate finance. Therefore, the corporate finance department receives cash flow forecasts (forecasts) generated by the subsidiaries worldwide, denominated in foreign currencies. After the realization date, the corporation receives in every month the cash flow figures for realizations (actuals). The data available cover itemtypes of invoices issued (II) and invoices received (IR) from the corporate IT system. In order to evaluate possible strategies and provide further information for KPI figures such as percentage return ratio the forecasts and actuals are aggregated for the corporate risk management. As a proxy for the percentage return margin within a fiscal year, the entity’s ratio of aggregated revenues (II) and expenses (IR) is calculated.

The aggregated data set used in the analysis for this paper covers forecasts and actual for the entity’s ratios. Delivered by the subsidiaries on a quarterly basis, the forecasts cover intervals with horizons of up to 15 months (five quarters). The dataset for actual invoices ranges from January 2008 to December 2013 with the corresponding forecasts covering the actuals’ period.

In total, actuals and forecasts are available for the 67 largest subsidiaries resulting in 25 different currencies for the dataset. Actuals grouped by division, subsidiary, currency and item-type result in 72 actual time series. Overall, the dataset consists of 3,087 monthly invoice actuals, with five associated forecasts each. The underlying raw dataset of non-aggregated forecasts cover 102.360 items. Table 1 gives a brief summary of the dataset. 4

Notation and Forecasting Process

The notation presented in this section is commonly used in current literature on [ 22 ].

Denoting the actual of cash flow margin ratio as 0R, the lead time t > 0 of a forecast t R for 0R refers to a quarter of the year until the actual date (t = 0). Figure 1 visualizes the temporal structure of an example forecasting process in five steps for an actual0R. The initial forecast ratio 5R is delivered with a lead time of five periods and is revised four times until the last one–period–ahead forecast 1R is generated.

Since ratios are specific for an entity, for reasons of comparability, this work focuses on normalized ratios (Def. 1). Therefore, the notation t R refers to the normalized ratio instead of the entity specific ratio (t R := t R(E)). Definition 1 (Normalized ratio). Normalized ratio is obtained by subtracting the minimum ratio within an entity from R and dividing by the difference of its maximum and minimum ratio. The values are always between zero and one per entity. t R(yE=)Y,m=M = t Ryen=tYit,ym==EM − min(S R) max(S R) − min(S R) while:

[ R = {t Rednattiety : entity = E ∧ date < (Y, M)} Definition 2 (Target ratio). The suggested annual return target (target ratio) that an entity has to reach at the end of the year y = Y is defined as:

T (0Ry=Y )

As targets are unknown (to us), but business development measured with EBITDA figures seem rather stable over the years, the target ratio in y = Y is estimated by averaging the December actual ratios of the three preceding years (0Ry=Y − j,m=12, for j ∈ {1, 2, 3}).

Definition 3 (Revision). The revision for ratios describes the adjustment from the second to last forecast before the actual. It is formally defined as;

12R = 1R − 2R

This paper uses the last revision because generally the latest judgmental forecast incorporates the most information and is the most accurate [ 20 ].

Definition 4 (Difference from target). The difference from target is defined as: Definition 5 (Error). Finally, the error is defined as: TargetDiff = T (0R) − 1R

t E = 0R − t R Improving forecast accuracy is an important goal, where usually correction techniques such as linear regressions are applied in the literature for analysis and correction of biases. These statistical forecast correction techniques build models that usually employ information of basic features based on historical data. An example of such a basic statistic model can be found in Def 6. Here, the forecast error 1E is regressed using basic variables such as regression intercept, ratio 1R, and revision 12R. Theoretically valid, this model optimizes the error based on the human forecaster’s prediction and revisioning behavior. But, this paper argues that correction approaches should incorporate important organizational information too. As noted before, reaching predefined target KPIs is an important strategic goal. The difference to the percentage return margin target is symbolized with TargetDiff and measures the distance to the organizational prerequisites. To overcome this organizational bias, the information of TargetDiff is integrated into the regression model as shown in Def 7.

Typically, correction techniques evaluate their results with some error metric, such as error (deviation), absolute error, percentage error, absolute percentage error, and so on. Slightly different use cases can favor a specific error measure as most of them have known flaws that suit one case but not the other ones. The research presented in this paper tries to be independent of those restrictions that make comparison of scientific results difficult and hinders reproducibility. Therefore, the comparison of both models is evaluated in an error-metric-independent way.

Based on the efficiency theory [ 22 ], proposed tests for the structure in terms of correlations amongst revisions and between revisions and errors. Forecast processes that show no correlation structures (with significant p-values) are considered as weak-form efficient . Otherwise, existing structures hint to information that could be incorporated into revisions because revisions are predictable. With t ∈ R0+ denoting the lead-time to the realization of an actual (at t = 0), Nordhaus suggests testing for weak-form efficiency using the Propositions (P1) and (P2). Proposition 1 (P1). Forecast error at t is independent of all revisions up to (t + 1).

Proposition 2 (P2). Forecast revision at t is independent of all revisions up to (t + 1).

Combining the argumentation for organizational debiasing and efficiency, the authors propose the following hypotheses: Hypothesis 1. Does forecast correction that incorporates organizational information (that organizationally biases forecasts) improve forecast efficiency? Hypothesis 2. How does efficiency for organizational correction differ from basic statistical approaches?

These hypotheses are evaluated based on the two regression models. Both models are trained for each month of the year independently to consider the seasonality in the business data. Therefore, the data is split into 12 subsets that are accessed to train one specific model for each month (resulting in 24 models). To show the benefit of the organizational information empirically, the model prediction needs to add the original forecast 1R to derive a new model prediction. These model predictions will then be compared to the original forecasts (M∅ symbolizes the expert forecast) and with each other in terms of forecast efficiency. The baseline for comparison is the original forecast based on M∅, which will be evaluated first. For reasons of clarity, the model forecast substitutes the original forecast, which leads to three possible forecast processes “ 5R, 4R, 3R, 2R, 1R(M{∅,Orga,Basic}), 0R” with changed revision and error measures for 12R and 1E depending on the selected model. Logically, the evaluation focuses on these changed measurements only. Additionally, the indication for error quantiles and statistics for efficiency are provided. 6

Empirical Analysis

This section presents the empirical results. These consist of correlation analysis for efficiency, with a revision and error analysis, followed by the analysis of the underlying statistics. For the correlation analysis the experiments use the R programming language [ 23 ] and the libraries corrplot [ 25 ] and knitr [ 26 ].

As noted before, the forecast efficiency is an important goal of forecasting processes. The forecast efficiency of the resulting prediction of the models MOrga and MBasic are compared to each other and the baseline M∅. The baseline of forecast efficiency forM∅ is shown in Figure 2. It should be noted that in the figures, we hide irrelevant cells (marked using "x" sign) and we show all and only the cells relevant for the efficiency analysis as proposed in [ 22 ].

4E 3E 2E 1E 45R 34R 23R 12R x x x x x x

E 4 x x x x

E 3 x x

E 2

E 1

R 5 4

R 4 3

R 3 2

R 2 1

The comparison of models M∅ − MBasic and MBasic − MOrga (difference in correlation) are depicted in Figure 3 and Figure 4 respectively.

The Figure 3 shows that the basic statistical model increases efficiency (marked in blue) compared to the baseline by (12R,1 E) = 92% and (23R,1 E) = 70%. But, all the other dependencies have decreased efficiency (marked in red). Comparison between the basic statistical model and the organizational model in Figure 4 shows an additional increase of efficiency relative to MBasic by 56% for the final forecast. More remakable, the whole forecasting process is more efficient (see(12R,23 R), (12R,34 R) and (12R,45 R)) stating that the organizational debiasing approach is superior to the basic statistical approaches.

The Figure 5 shows important information for the error quantiles of the forecasts. This figure also provides additional support for the performance of MOrga through the 1E measure. The organizational model outperforms the statistical model especially for the 1. quartile (Δ = 0.072), median (Δ = 0.017), and 3. quartile (Δ = 0.120). Only for minimum, maximum, and for mean error (Δ = 0.002) the statistical model seems beneficial.

The results for Cor(12R,1 E) are not significant after cor

M0

MBasic

MOrga

rection due to the high efficiency, but the details are shown in Table 3. The Spearman covariance for the approaches states that revisions and error have a lower joint variability. The organizational model has a positive covariance, while the statistical model has a negative covariance with a higher magnitude. Also, the table shows that organizational model increases standard deviation for the revision, but it reduces for the error. It is arguable with these numbers that the organizational model’s revision focuses with meaningful revisions on the reduction of the error, while the statistical model’s revision focuses on changing the error with minor corrections. This enables future approaches to detect other, currently unknown biases to be identified and removed.

Overall, the results state several advantages of the organizational model in comparison to the statistical model. First, in the sense of Nordhaus the organizational debiasing model improves forecast efficiency forCor(12R,1 E), supporting Hypothesis 1. Second, the error distribution is narrowed, especially for the 1st and 3rd Quartile. Third, the advantage of bias reduction instead of error optimization. The second and third finding support Hypothesis 2. 7

Conclusions and Outlook

Empirical analyses on forecast efficiency or on cash flow biases might be a very interesting paper topic for the specific research communities and therefore easy to find. However, linking these settings to forecast correction techniques that account for organizational biases in a predictive model have not been explored in the forecast community so far.

This research addresses two research gaps: (1) Linking organizational information to forecast correction techniques and evaluating the result independently from a specific error metric. The results show that organizational information is beneficial to forecast efficiency. (2) Analyses of correction models that compare basic statistical approaches to organizational approaches have been left unattended.

This study contributes with the conclusion that the different results for corrective models may be inherent to each approach.

Relevance for the Data Mining Community

For the data mining community the paper might change the understanding of the link between exploratory data analysis and forecast correction. Exploring data can actually show the way how to correct forecasts in a modelindependent way. We would like to stress that the results of this paper were not achieved with a neural network, a random forest, or a complex machine learning algorithm. Instead, the results are achieved with a simple linear regression models.

The importance of exploratory data analysis is strengthened as data understanding additionally allows a differentiation between biases with pattern and errors.

The most important result of this study is probably the statement that a basic statistical model “just” tries to optimize the selected component (e.g, the error), while an organizational model tries to reduce the bias itself. As a result of the organizational model enables the possibility to identify further unknown biases and correct these biases with a second model. Understanding the error components is important. When a forecaster distinguishes the signal from the noise, the error should decrease by the way or making predictions more confident. Therefore, even if no error decrease is achieved with one organizational debiasing model, a patch of models for the most important organizational biases will definitely increase the accuracy.

Managerial Implications

From the perspective of a manager and forecast researcher it is important to understand in which way business-related factors may affect forecasts and indirectly correction models. In the case of cash flow forecasts in a corporate setting one important factors is the percentage margin target, as these might provide incentivization to alter forecasts and actuals of cash flows. The underlying value of this information is stated in terms of forecast efficiency. The analysis showed that efficiency increases.

Based on this research, application of the presented approach would be interesting also for forecasting in other domains. The efficiency theory could provide an alternative approach to understand the value of specific information within forecast correction (compared to other measures such as entropy or information gain).

Outlook

It might be reasonable to recommend in the forecasting community that future approaches shall not minimize the error component, by changing forecasts and revisions marginally. Instead, maximization or at least the change of forecasts and revisions in an acceptable big magnitude that Approach M∅ (Baseline) MOrga (Organizational) MBasic (Statistical)

Covariance(12R,1E)

Std.Dev.(12R)

Std.Dev.(1E) -246092.58 result in marginally errors is recommended. A high revision will determine how long the forecast result is aligned to the bias pattern. Based on the results, the understanding of forecasts and best applied correction techniques is obtained on the way.

References

[1] Masahiro Ashiya . Testing the Rationality of Forecast Revisions Made by the IMF and the OECD . Journal of Forecasting , 25 ( 1 ): 25 - 36 , 2006 .

[2] D. A. Bessler and J. A. Brandt . An Analysis of Forecasts of Livestock Prices . Journal of Economic Behavior & Organization , 18 ( 2 ): 249 - 263 , 1992 .

[3] Henry Brighton and Gerd Gigerenzer . The Bias Bias. Journal of Business Research , 68 ( 8 ): 1772 - 1784 , 2015 .

[4] David Burgstahler and Ilia Dichev . Earnings Management to Avoid Earnings Decreases and Losses . Journal of Accounting and Economics , 24 ( 1 ): 99 - 126 , 1997 .

[5] David Burgstahler and Michael Eames . Management of Earnings and Analysts' Forecasts to Achieve Zero and Small Positive Earnings Surprises . Journal of Business Finance & Accounting , 33 ( 5-6 ): 633 - 652 , 2006 .

[6] N. D. Daniel , D. J. Denis , and L. Naveen . Do Firms Manage Earnings to Meet Dividend Thresholds? Journal of Accounting and Economics , 45 ( 1 ): 2 - 26 , 2008 .

[7] Francois Degeorge , Jayendu Patel , and Richard Zeckhauser. Earnings Management to Exceed Thresholds. Journal of Business , 72 ( 1 ): 1 - 33 , 1999 .

[8] Bruno Deschamps and Christos Ioannidis . Can Rational Stubbornness Explain Forecast Biases? Journal of Economic Behavior & Organization , 92 : 141 - 151 , 2013 .

[9] Jonas Dovern and Johannes Weisser . Accuracy, Unbiasedness and Efficiency of Professional Macroeconomic Forecasts: An Empirical Comparison for the G7 . International Journal of Forecasting , 27 ( 2 ): 452 - 465 , 2011 .

[10] E. F. Fama . Efficient Capital Markets: A Review of Theory and Empirical Work . The Journal of Finance , 25 ( 2 ): 383 - 417 , 1970 .

[11] F. Guidry , A. J. Leone , and S. Rock . Earnings-Based Bonus Plans and Earnings Management by Business-Unit Managers . Journal of Accounting and Economics , 26 ( 1 ): 113 - 142 , 1999 .

[12] Jiawei Han , Jian Pei , and Micheline Kamber . Data Mining: Concepts and Techniques . Elsevier, 2011 .

[13] S. N. Kaplan and R. S. Ruback . The Valuation of Cash Flow Forecasts: An Empirical Analysis . The Journal of Finance , 50 ( 4 ): 1059 - 1093 , 1995 .

[14] C. S. Kim , D. C. Mauer , and A. E. Sherman. The Determinants of Corporate Liquidity: Theory and Evidence . Jour. of Financial and Quant. Analysis , 33 ( 3 ): 335 - 359 , 1998 .

[15] Florian Knöll , Verena Dorner, and Thomas Setzer. Relating Cash Flow Forecast Errors to Revision Patterns . In Proc. of MKWI , pages 1217 - 1228 . MKWI - Prescriptive Analytics in IS , Universitätsverlag Ilmenau, 2016 .

[16] Michael Lawrence , Paul Goodwin, Marcus O'Connor , and Dilek Önkal . Judgmental Forecasting: A Review of Progress Over the Last 25 Years . International Journal of Forecasting , 22 : 493 - 618 , 2006 .

[17] Michael Lawrence and Marcus O'Connor. Sales Forecasting Updates: How Good are They in Practice? International Journal of Forecasting , 16 ( 3 ): 369 - 382 , 2000 .

[18] J. Leitner and U. Leopold-Wildburger . Experiments on Forecasting Behavior with Several Sources of InformationA Review of the Literature . European Journal of Operational Research , 213 ( 3 ): 459 - 469 , 2011 .

[19] B. Marr. Key Performance Indicators (KPI): The 75 Measures Every Manager Needs to Know . Pearson UK , 2012 .

[20] S. K. McNees . The Role of Judgment in Macroeconomic Forecasting Accuracy . International Journal of Forecasting , 6 ( 3 ): 287 - 299 , 1990 .

[21] M. Mendoza and E. de Alba. Forecasting an accumulated series based on partial accumulation ii: A new bayesian method for short series with stable seasonal patterns . International Journal of Forecasting , 22 ( 4 ): 781 - 798 , 2006 .

[22] W. D. Nordhaus . Forecasting Efficiency: Concepts and Applications . The Review of Economics and Statistics , 69 ( 4 ): 667 - 674 , 1987 .

[23] R Core Team. R: A Language and Environment for Statistical Computing . Vienna, Austria: R Foundation for Statistical Computing., 2013 .

[24] N. R. Sanders and K. B. Manrodt . The Efficacy of Using Judgmental Versus Quantitative Forecasting Methods in Practice. Omega , 31 ( 6 ): 511 - 522 , 2003 .

[25] Taiyun Wei and Viliam Simko . corrplot: Visualization of a Correlation Matrix , 2016 . R package version 0 .77.

[26] Yihui Xie . knitr: A Comprehensive Tool for Reproducible Research in R. Chapman and Hall/CRC, 2014 . ISBN 978- 1466561595.

[27] P. M. Yelland . Stable Seasonal Pattern Models for Forecast Revision: A Comparative Study . International Journal of Forecasting , 22 ( 4 ): 799 - 818 , 2006 .