We propose a security price data cleaning technique based on Reynold's decomposition that uses T0 (the time period of integration) to determine the de-noise level of the price data. The goal of this study is to find the optimal T0 that reveals an underlying price trend, possibly indicating the intrinsic value of the security. The DJIA (Dow Jones Industrial Average) Index and the thirty companies comprising the index are our fundamental interest. Preliminary results suggest that the graphs of ↵ (a key percentage measure) versus T0 of the thirty companies and the DJIA Index exhibit at least two properties: (1) ↵ drops exponentially as T0 increases when T0 / order of magnitude of 100 days, and (2) ↵ drops linearly as T0 increases when T0 ' order of magnitude of 100 days. For the DJIA Index itself, T0 is less than order of magnitude of 100 days. The result of applying our technique to each component stock of the DJIA parallels the result of the technique applied to the DJIA Index itself.
Understanding and analyzing financial data in
order to forecast and make cost-effective decisions
is challenging because of the complex and volatile
nature of security prices. The most recent
financial market meltdown in 2008-09 casted doubts
on financial data analysis and forecasting.
Inability to recognize or acknowledge financial
distress signaled by pertinent financial data was a
significant factor leading to these catastrophic
economic results
Several financial computation models that
deal with cleaning financial data employ
similar methodologies, such as candlestick
strategies
To this end, we propose a data cleaning
technique based on Reynold’s decomposition in order
to decompose the price data into a mean part and
a fluctuating part. Fluctuations in stock prices are
perpetual and irrational in time because the weak
form of market efficiency and different types of
market participants create a complex dynamic of
behavioral finance
In the study of turbulence in fluid dynamics, each
component of the velocity is characterized by
fluctuations over time. One method to study the
dynamics in this regime is to perform a Reynold’s
decomposition such that the mean part of the
velocity is separated from the fluctuations. We
propose that this technique could also be used to study
financial data. In other words, we propose that the
price as a function of time, p(t), can be
decomposed into the following:
p(t) = p¯(t) + p0(t)
(1)
where p¯(t) is the mean portion and p0(t) is the
fluctuating portion of the price. We define p¯(t) to be
a moving time-average that can be found by
performing the following integral
p¯(t) =
1 Z t+T0/2
T0 t T0/2
p(t0)dt0
(2)
where T0 is the time period of integration. T0 must
be a time period that is greater than the time
period of the fluctuations, ⌧ , and less than the time
period of interest, T . T is dependent on each
particular analysis; for example, T could be weeks,
months, or years. Thus, ⌧ < T 0 < T .
Furthermore, the time-averaged value of the fluctuating
portion over the entire time period of interest is
zero
In this study, we focus on the thirty
companies comprising the Dow Jones Industrial
Average (DJIA) as of May 13, 2015, and the DJIA
Index because, being the second oldest financial
index, the DJIA is the benchmark that tracks
financial market performance as a whole. Thus,
it represents a broad market, and its validity is
intensely scrutinized and followed by at least 10
Wall Street analysts
Daily adjusted close stock price data for the thirty Dow Jones companies listed above from the time of inception of the company to May 13, 2015, are obtained from Yahoo! Finance. For the DJIA Index, the daily adjusted close stock price data from Jan. 29, 1985, to May 13, 2015, are also obtained from Yahoo! Finance. The adjusted close stock price is used because it accounts for stock dividends and splits. Only days in which the stock price is provided, i.e., business days, are considered in this study. Thus, the time from Friday to Monday is taken as only one (business) day.
We estimate the time period of fluctuations to be a day, ⌧ ⇠ 1 business day, and the time period of interest to be the total number of business days since the inception of the stock, T ⇠ 260 ⇥ n business days, where n represents the number of years since the inception of the stock. Further, we chose the following time periods of integration, T0, for this study: 4 days, 10 days, 20 days, 30 days, 64 days, 130 days, 194 days, 260 days, 390 days, and 520 days, which roughly represent the following time periods: one week, two weeks, one month, one-and-a-half months, a quarter of a year, half of a year, three-quarters of a year, one year, one-and-a-half years, and two years, respectively.
p¯(t) is calculated by only considering the analysis time period from T0/2 after the day of inception to T0/2 before May 13, 2015, such that for each day p¯(t) is calculated, the full time period of integration is used. To exemplify, consider the case where T = 1000 days and T0 = 100 days. Then the first 50 days (day 1 to day 50) are not included in the analysis, and neither are the last 50 days (day 951 to day 1000). For each day in the analysis time period, the integration stated in Eq. (2) is performed numerically to find p¯(t) for that day. p0(t) is found by subtracting p¯(t) from p(t), the actual price, for that day.
For each specific T0, the statistics of p0(t) are analyzed. Specifically, a histogram with 25 bins of p0(t) is created for each T0. As an example, Fig 1 shows a histogram for GS (The Goldman Sachs Group Inc). Note that like Fig 1, most of the histograms are centered around 0, which suggests that most of the fluctuations for the stocks are nearly zero. Therefore, the actual stock price is near or nearly equal to the local time-average for most of the time period analyzed. For most stocks, as T0 increases, the maximum height achieved by the histogram decreases and the histogram tails become heavier. Thus, as T0 increases, there are more observations away from the center of the distribution. This is observed because as the time period of integration increases, more points are considered in the average. Therefore, there is a greater likelihood that p¯(t) is different from the actual price.
To measure the fidelity of p¯(t) to p(t), the number of data points of p0(t) that are within 1 dollar from zero are counted and divided by the total number of data points in the analysis period. We will call this percentage measure ↵ , and this measure should be as close as possible to 100% to reflect that p¯(t) is a good approximation of p(t). As stated previously, if p0(t) is near zero, that means that p¯(t) is close to p(t) because p(t) = p¯(t) + p0(t). As T0 increases, ↵ decreases because the mean is farther away from the actual price when the integration period is larger. Using the MATLABr curve fitting tool, it is found that for all of the thirty stocks the relationship between ↵ and T0 is best represented by the following equation ↵ (T0) = a1 eb1 T0 + c1 ed1 T0 (3) where a1, b1, c1, and d1 are curve fitting parameters. In fact, the lowest goodness-of-fit measure R2 among all thirty stocks is 0.9909. As an example, the curve fitting parameters for GS are a1 = 0.3613, b1 = 0.09018, c1 = 0.1034, and d1 = 0.003687. The first derivative of this equation is d↵ dT0
= a1 b1 eb1 T0 + c1 d1 ed1 T0 and the second derivative is d2↵ dT02 = a1 (b1)2 eb1 T0 + c1 (d1)2 ed1 T0 For most of the stocks, it was discovered that when T0 is fewer than 100 days, the measure ↵ drops exponentially as T0 increases. However, the second derivative (Eq. (5)) becomes near zero in a range from 96 days to 387 days for the thirty stocks analyzed, with the most common being approximately 125 days. Thus, when T0 is at least (4) (5) an order of magnitude of 100 days, ↵ starts to decrease linearly for nearly all of the stocks analyzed. Fig 2 plots the curve fitted ↵ (T0) for all thirty analyzed stocks. As we can see, the general trend among the thirty stocks is that ↵ drops exponentially when T0 is fewer than 100 days, but ↵ drops linearly when T0 is greater than 100 days. An appealing fact is that the graph of ↵ against T0 for the DJIA Index, Fig 3, also exhibits similar trends in ↵ , as shown in Fig 2. Note the different scales of the vertical axes of Fig 2 and Fig 3, which means that Fig 3 is much flatter than Fig 2.
Mathematically, we will define the point where the slope is constant by the following
d2↵ T0! tc dT02 = 0 lim (6) where tc is the time period of integration at which the second derivative of ↵ approaches zero. Thus, for the thirty stocks analyzed, tc is in the following range 96 days < tc < 387 days. Therefore, for time periods of integration larger than tc, the change in ↵ will be relatively small.
The standard deviation of the fluctuations p0(t), defined as v u = tu N 1 1
N X(p0(ti) i=1 p¯0(t))2, (7) is also analyzed where N is the total number of data points and p¯0(t) is the total time average of the fluctuations. A large of the fluctuations reflects that p¯(t) is not equal to p(t). To remove the miscellaneous fluctuations of the price, should be as large as possible. As indicated by the straight lines in the log-log plot in Fig 4, and T0 are related by a power law where the slope of the line indicates the exponent.
Using the MATLABr curve fitting tool, we fit the following equation for each of the thirty stocks: = a2 T0b2 + c2 (8) where a2, b2, and c2 are curve fitting parameters. As an example, a2 = 2.194, b2 = 0.4317, and c2 = 1.425 for GS. The lowest goodness-of-fit measure R2 among the thirty stocks is 0.9849. b2, the exponent, varies from 0.35 to 0.69 for all thirty stocks. The average exponent is 0.5. 4
This paper demonstrates preliminary results of an ongoing security price data cleaning research. We found that the graphs of ↵ versus T0 of the thirty companies and the DJIA Index exhibit at least two properties: (1) ↵ drops exponentially as T0 increases when T0 / order of magnitude of 100 days, and (2) ↵ drops linearly as T0 increases when T0 ' order of magnitude of 100 days. Thus, the optimal T0 for the thirty companies studied is approximately 100 days. For the DJIA Index itself, the optimal T0 appears to be less than 100 days. One of the possible explanations is that the DJIA Index might show the counter measure effect of fluctuation points among the thirty companies since the DJIA is a composite of the thirty companies that collectively provide a balance view of the market. As a result, T0 might be even smaller for the second derivative to approach zero. We also found that and T0 are related by a power law. As for future research, we plan to define mathematical metrics in our study of security price valuations and trading strategies.