In 1963 Mandelbrot published research into the distribution of cotton prices based on a very long time series which found that, contrary to the general assumption that these price movements were normally distributed, they instead followed a pareto-levy distribution. While on the surface these two distributions don’t appear to be terribly different, (many small movements, and a few large ones), the implications are significantly different, most notably the pareto-levy distribution has an infinite variance.

This implies that rather than extreme market moves being so unlikely that they make little contribution to the overall evolution, they instead come to have a very significant contribution. In a normally distributed market, crashes and booms are vanishingly rare, in a pareto-levy one crashes occur and are a significant component of the final outcome.

It has taken years for this to be taken seriously, and in the mean time financial theory has gone on using the assumption of normally distributed returns to derive such results as the Black-Scholes option pricing equation, ultimately winning an Nobel Prize in Economics for the discoverers Scholes and Merton (Black having already died), not to mention Modern Portfolio theory (also winning Nobels). That modern finance ignored Mandelbrot’s discovery and went onto honor those working under assumptions shown to be false has clearly annoyed Mandelbrot immensely and as mentioned previously he spends much of the book telling us of his prior discoveries and how he was ignored.

That Black-Scholes has significant short comings due to unrealistic assumptions is very well recognised. In the market there exists what is known as a volatility smile, options with strikes not near the current market (ie. out-of, or in-the-money options) are priced with different volatilities to at-the-money options. The very existence of such a thing is a contradiction of the basic assumptions of the B-S model and one of a number of ways the market in practice tries to compromise between using equations that roughly works in some circumstances and getting a fair price. For all known flaws of the Black-Scholes framework, no one has been able to figure out anything else that uses improved assumptions and enables calculation of real prices. GARCH models are an attempt to fix this but embed many of the same assumptions.

While the book shows some nice fractal schemes for generating much more realistic market models than are generated by a straight Brownian motion scheme and this is all very interesting. The discussion of how varying time scales may explain some of the observed behaviour is also good, the speed that trading occurs may be a better measure of market time than the clock is.

However in the end we are left at the end with a lot of criticism, a few good ideas but little to show for it. Attempts to try to tinker with calculations such as VAR to include estimates of fat tails are dismissed as being like the Ptolemaic system. I don’t disagree with this analogy, but essentially we are in the situation even worse than astronomy was after Copernicus. Until Kepler calculated the orbits to be elliptical, the predictions of the Copernican system were no better than the Ptolemaic system. Similarly while it is widely recognised that many of the assumptions of modern finance are wrong it does give us a framework to make calculations and will continue to be used until something better comes along.

In the end he calls for research to be done into better developing a theory to understand market behaviour, which is a good thing. In the interim though work in the old paradigm will continue with some recognition that there are flaws which will be dealt with in an *ad hoc* way. Continued railing against Efficient Markets, the normal distribution and the independence of returns this will not change this without some solid results.

I thought I should comment, as I was thinking today that I hadn’t discovered anyone else mathematically minded blogging (outside the IT community). Obviously I hadn’t looked very hard! I’ll be back with something more relevant.

But I would say that even knowing that VAR is flawed, it’s better than not having a risk measure at all. Clearly if some distribution are less normal than others, then your comparisons might not be right, but at least it lets you compare risks to some extent.

Thanks for the comments, That’s very much the point it gives you some measure of risk. If you believe that its the only measure of risk then you are fooling yourself, but at the same time it gives you some feel for what is going on.

Steve,

Has this research been replicated for commodities other than cotton? I might have to buy a copy of the book. I have been doing assignments at commodities dealers and this would be a significant piece to add to VaR analysis.

Fat tailed problems are known to occur with VaR models, but I was not aware that an appreciation of the problem goes that far back. Looks like I will have to do something to sort through this.

Andrew,

His early work was done on Cotton because it had one of the best long term daily price records – over a century of daily records in the early sixties when he did the work. I believe he did it for other commodoties, and he cites Eugene Fama, who was his doctoral student, as having done similar analysis for the top 30 Dow Stocks in the mid 60s. His point is that it occurs in pretty much any asset class.

In my experience most VAR systems if they take this into account they try to do it in an Ad Hoc basis by using mixed normals, a low probability wide distribution and a high probability narrower one (this also lets you model skewness) . Mandelbrot would probably take a dim view on this as it ultimately still is bound by normal distributions, and doesn’t ever reflect the real extreme movements you get if you really do have a powerlaw tail. eg. something like the Cauchy y=1/(1+x^2)

There is also the question of what you do with correlations in these extreme situations, correlation is a measure of the linear variation between variables where as the normal correlations may break down entirely under extreme moves eg. the LTCM experience.

I first read this book during the winter of 2005. Most of my career has been in the arena of financial analysis, and in that field, I am not really concerned so much with the purity of mathematical theory as I am about getting results. But as soon as I picked up the book, I sensed that it was saying something extremely profound that cut to the core of what I did for a living. In modern business schools, we are indoctrinated with concepts of the random walk and the normal distribution – so much so – that we take them for gospel. But when we transition from the world of academia to the world of enterprise, we are consistently confronted by the reality that the models don’t work as well as they should. We usually chalk this up to being a problem of implementation rather something fundamentally wrong with our most basic assumptions. But when I read “Misbehavior of Markets”, I knew that Mandelbrot was articulating a great truth. Suddenly I could see dozens of situations from my past, where assumptions about probability had fallen short of reality – not only in the financial markets, but also in the metrics of corporate planning. The critic above it correct in saying that Mandelbrot raises questions without providing the answers. This book is not a cookbook. It will remain for others to come up with recipes. But just knowing the short-comings of traditional models is an important first step. My financial consulting firm has already begun that process in our own work. This book was the catalyst for that process.

I agree with the fact that people need to realise the random walk model is not accurate. I guess from where I’m sitting I feel its pretty well recognised, which is why I tend to find attacks on it without something better to put on in its place a little tiresome. Anyway perhaps I have misread the “popular conception” of the theory and more people do need to have the flaws in brownian motion model spelt out.

[…] visar att kriser tenderar att förstärka sig själva i en sorts komplexitetsrundgång och att de standardmodeller som utvecklats inom ekonomin konsistent underskattar denna tendens. I en sådan rundgång kan ett […]

Thanks for the insightful review! One smells something wrong with the carping against Gaussian, ergodic, stationary, co-integrable, whatever processes. Yes, the mathematics is wrong, but … maybe what smells wrong is that I’m not smelling a computable solution.

The standard approach of the efficient markets concept dangerously underestimates the risks in the financial system. The long-term capital management crash of 1998 is one of many examples of the disastrous effects of market risk underestimation. In the 1970s and 1980s, this approach became “…the guiding principle for many of the standard tools of modern finance . . . taught in business schools and shrink-wrapped into financial software packages… [It] is a house built on sand.” (Mandelbrot 2008). The efficient market theory promised “expected” returns through clear, simple risk assessment and profile models based on risk aversion. In this model, option markets flourished and structured finance boomed (CDOs…). This model was adopted by financial markets, economics department and business schools who trained students on the basis of the modern portfolio theory (MPT), which attempts to maximise returns for a given amount of risk, or, alternatively, minimises risk for a given amount of return.

Professor B. Mandelbrot (1924-2010) changed the way we view the world by introducing the fractal geometry of nature. He gave us a tool to describe different systems with common characteristics. Applied to finance it allows us to understand market instability as described by Professor H. Minsky (1919-1996). Wild prices, fat tails (that is, heavy-tailed market distributions that exhibit extreme skewness or kurtosis), and the long-term memory effect, led Mandelbrot to view the financial series as fractal series.

Fractals are everywhere from cauliflowers to our blood vessels. Fractals have helped model the weather, measure online traffic, compress computer files, analyse seismic tremors and the distributions of galaxies. Mandelbrot hope was to build a stronger financial industry and an enhanced system of regulation. He developed a multifractal model with variable market time, exponential price distributions, and fractal generators. Some say that Mandelbrot described the financial system without explaining it, but no one can elucidate it. He was a wise observer and had a special way of scrutinizing objects and nature… thinking in a fractal way in finance is about observing, analysing and trading opportunities out off the markets anomalies! His hope was that people will accept the reality of risky markets and stop pretending otherwise.

By identifying markets structures, what we essentially learn from B. Mandelbrot is that markets risks can be modelled so that people can avoid big losses – the question he left unanswered is how to do this in practice. A measure of risk should take into account long term price dependence or the tendency of bad news to come in flocks. Our focus shouldn’t be on how to predict prices; but on how to foresee risks. Understanding the nature of the market allows us to use the clustering property of volatility as a tool to measure and forecast risk, “opportunities are in small packages of time, large price changes tend to cluster and follow one another. If there was a large price change yesterday, then today is a risky day.” (Mandelbrot 2008,p.233). We may not be able to forecast direction, but with this understanding we can get out of the market at its clustery periods and reduce the chances of loss.

Extreme Weather…[…]Misbehavior of Markets: Mandelbrot « Criticality[…]…

[…] […]