Intraday volatility prediction and estimation

GARCH has been shown to be a reasonable estimator of variance for daily or longer period returns. Some have adapted GARCH to use intraday returns to improve daily returns. GARCH does very poorly in estimating intra-day variance, however.

The GARCH model is based on the empirical observation that there is strong autocorrelation in the square of returns for lower frequencies (such as daily). This can be easily seen by observing clustering and "smooth-ish" decay of squared returns on main daily series.

Intra-day squared returns, however, have many jumps, with little in the way of autocorrelated decay pattern. Here is a sample of squared returns for EUR/USD. There are many spikes followed by immediate drops in return (as opposed to smoother decay). There does appear to be a longer-term pattern, though, allowing for a model.

With expanded processing power and general access to tick data, research has begun to focus on intra-day variance estimation. In particular, expressing variance in terms of price duration has become an emergent theme. Andersen, Dobrev, and Schaumburg are among a growing community developing this in a new direction.

At this point have disqualified GARCH as a useful measure for my work, but am investigating a formulation of a duration based measure.

Hawkes Process & Strategies

Call me unread, but I had not encountered the Hawkes process before today. The Hawkes process is a "point process" modeling event intensity incorporating empirical event occurrence.

The discrete form of the process is:

where t_i is the i^th occurrence at time t_i < t for some t. The form of the function is typically an exponential, but can be any function that models decay as a counting process:

Ok, that's great but what are the applications in strategies research?

Intra-day Stochastic Volatility Prediction
The recent theme in the literature has been to replace the quadratic-variance approach with a time-based approach. The degree of movement within an interval of time is equivalent in measure to the amount of time required for a given movement, and can be interchanged easily as Andersen, Dobrev, and Schaumburg have shown in "Duration-Based Volatility Estimation".

Cai, Kim, and Leduc in "A model for intraday volatility" approached the problem by combining an Autoregressive Conditional Duration process and a Hawkes process to model decay, showing that:

and then equivalently expressed in terms of intensity (where N represents the number of events of size dY):


relating back to volatility measure as:

The intensity process is comprised of an ACD part and a Hawkes part:




They claim to model the intra-day volatility closely and propose a long/short straddle strategy to take advantage of the predictive ability.

High Frequency Order Prediction Strategy
The literature suggests the use of Hawkes processes to model the buying and selling processes of market participants.

John Carlsson in "Modeling Stock Orders Using Hawkes's Self-Exciting Process", suggests a strategy where if the Hawkes predicted ratio of buy/sell intensity exceeds a threshold (say 5) buy (sell) and exit position within N seconds (he used 10).

This plays on the significant autocorrelation (ie non-zero decay time) of the intensity back to the mean. A skewed ratio of buy vs sell orders will surely influence the market in the direction of order skew.

The strategy can be enhanced to include information about volume, trade size, etc. We can also look at the buy/sell intensity of highly correlated assets and use to enhance the signal.

Price Path Probability (Again)

So I completed a model and calibration which determines the probability of a price going through a level within a given time period. If one can arrive at a high confidence level, this is incredibly useful for multi-leg execution and as a prop strategy in its own right.

The model uses a SDE with mean reversion, a trend component, and an evolving distribution to determine the price across time. The SDE is evaluated as a monte carlo simulation on a grid. We determine the conditional probability of going from one price level to the next for a given (small) time interval. The sum of the product of the probabilities along the price path represents the posterior probability of being at the given price node at a given time.

With the grid in hand, one can query the grid to determine the probability of a price being above or below a level within a given time, etc. For some markets, we are seeing a 75% confidence level, meaning we are right 3/4 of the time. There were some markets where there was no distinct edge in the approach, I have ideas on how to adjust this, but have not had the time to revisit.

The evolution of the distribution was the most complex to model. Unlike idealized option models, where the distribution is stationary and generally gaussian, the observed intra-day distribution over short periods is neither gaussian nor stationary. We noted that the first 3 or 4 moments have dynamics which can be modelled and fitted on top of an empirical distribution.

The trending and mean reversion functions were fitted using a maximum likelihood estimate, which was easily obtained from the distribution for each time step under given assumptions. The parameters were evolved with a GA to maximize the likelihood.

Evolution of Distribution

For a number of problems, understanding the evolution of the distribution over time is important. The distribution tends to be stable over longer periods and unstable over shorter periods.

The distribution is going to be measured from a sample over some time period. One may want to take a blend of distributions measured over different time periods, combined as basis functions with weights summing to 1.

The interesting bit is predicting the distribution forward with some statistical accuracy. The order book and momentum indicators should tell us something about how the distribution is going to transform over the next period or based on when a certain price level is achieved.

We are going to use a GA to calibrate the transformation function against historical data. There are many different functions we could use, so we use a GP approach to play with the permutations.

GP for option pricing

As you probably know GP (Genetic Programming) is an extension of GA which rearranges algebraic or functional instruction trees to fit to a solution.


I had not thought of it previously, but could use such an approach with the right set of functional constructors to converge on an option pricing GP. Now if all we were trying to do was to replicate the Black / Scholes, CEV, or other gaussian distribution based model, would not be very interesting.

We know that the actual distribution are often non-gaussian. Could we produce a more accurate approximation of the hedging cost against a non-gaussian distribution (implying the true risk free price of the option) with GP?

Interestingly, Neural Networks are just special cases of a GP tree, so in the end GP is the most general approach to non-linear regression.

Price Path Probability

What is the probable path of a security over the next 1 second, 5 seconds, 30 seconds?

I attended a quantitative algorithmic trading seminar 3 weeks ago where one presenter was discussing fill probability (in general terms). The presenter claimed that their model predicts the price path over the next few minutes to determine how best to read a VWAP strategy.

While I don't believe it is possible to predict a specific price path, it is possible to determine the probability of any given price path. If we can determine the probability of any given path through time from the current price to some final price in N seconds or minutes, we can compute the expected probability of going through a price level within some period of time.

The expected probability through a node at time Tn at price level Pa on a multinomial tree will simply be the sum of the probability of all sub-paths from Ts to Tn going through Pa. That part may be simple, but accurately determining the likely paths / probabilities is a hard research problem.

Given that the number of paths is exponential with time, the farther out we look the more time it takes to compute a precise expectation. We must use a monte carlo analysis, sampling a calibrated timeseries equation, to approximate the expectation function.

Determining the timeseries function that accurately reflects the market is a very hard research problem. Alas, if I told you my approach would have to kill you ;)