One of the things I originally intended to blog on, as you can see from my early posts, was some of the issues related to complex systems and the unpredictability associated with non-normal distributions. Possibly because my ideas were too badly formulated I’ve made few comments on this as I wanted more time to consider rather than posting uninformed crap.
Anyway, I have been reading some of the work of Nassim Nicholas Taleb an ex-derivatives trader who runs a hedge fund, is a fan of Karl Popper and has published a book on the subject of errors people make in the face of randomness some years ago called Fooled By Randomness with another one on the way apparently. He is now an academic. I have the book on order and will let people know what I think in due course. Anyhow it seems that a major theme of Taleb’s work is the idea that the widespread use of the normal distribution in finance (and other areas but he’s a finance guy so this is his focus) leads to people persistently underestimating the possibility of rare events. Which leads to his attacks on the whole idea of calculating risk measures such as VAR not to mention option pricing.
His belief in this idea is strong enough that according to this New Yorker article, he runs a hedge fund who’s main strategy is to systematically buy options (never sell) on the basis that the market persistently undervalues the chance of big moves. Rather than try to make money in “normal” market conditions, and then get occasionally take a hit when Russia defaults on its debt or a major terrorist attack occurs etc, the strategy is such that you usually make a loss, but every so often you make a very large profit.
An illustration of the difference that the big moves make it this graph of the S&P with and without the 10 largest moves. If you fit a normal distribution to the time series you should never get these, and particularly not ten over this timescale. [ref]
He goes as far as to say that distributions with only normal variances are not even really random. The tail events become so unlikely that you really can disregard them in the way that in physics can ignore quantum fluctuations with the correspondence principle for macroscopic objects. A summary of his opinions on this subject appear in this summary.
In my opinion this is good stuff and well worth remembering, but he seems to be verging on the situation where someone trying to break a dogmatic orthodoxy, rejects all valuable parts of the existing orthodoxy along with the rubbish. The normal distribution is and will continue to remain an incredibly valuable part of much of statistics, because it is useful and easy to work with. No doubt there are many areas where its being used inappropriately, but what we are looking for is better ways of detecting what those situations are.
Risk management is fumbling its way towards a better understanding of longer tailed distributions and how we can deal with them, and most places now make some attempt to incorporate this into VAR. While this is hardly the end of the problems with VAR it’s a start. As is pointed out in this response to Taleb’s opinion on VAR, while there may be things that it can’t quantify it does show up a great many other risks as I was pointing out in the article on the NAB options debacle.