The Black Swan

29 October, 2007

My original intention was to write a longish post discussing the main ideas. Rather I will do an overall impression here, and then do a few follow ups to talk about what I think are the interesting points.

All up I found The Black Swan an very interesting read. If you’ve read fooled by randomness then some of the arguments will have already appeared but certainly not all. The Black Swan is concerned with wild randomness, the randomness that according to Taleb dominates modern society. Fool by randomness largely concerns the randomness of games, what Taleb would describe as mild randomness. So mild that it is hardly random at all.

As we are reminded in several places in the book Taleb made his fuck off money (i.e. enough money to be comfortably independent even if not mega wealthy) in the 87 stockmarket crash betting on the fact that the market under-appreciated Black Swans. He doesn’t need to fawn to the establishment be it economics, philosophy or publishing and we get this tone all through the book.

While the book touches on finance applications its hardly a finance book. Rather Taleb considers it to be as much a work of Epistomology. His chief concern is with what he calls epistemic arrogance. He hates those who profess to know more than they really do know, be they economic modellers, historians or political scientists, but his chief enemy is the normal distribution which is described as the Great Intellectual Fraud. To use normal distributions in fields where it is easy to show are not normal (such as finance), is both fraudulent and causes Black Swans.

Its strong stuff which he backs with examples, logic and the findings of behavioural finance and similar studies. Its also fairly convincing for the most part. Its true we don’t need sophisticated statistical studies to find whether market moves are normally distributed. We only need the fact that we get 1 in 10,000 year events (as modelled by a normal distribution) occurring every few years. Risk managers who run such models (such as me) are either ignorant fools, who actually believe wrongly in what they do, or deceptive frauds who know better but persist in fooling others for the money. I’d contend I’m neither but I’ll discuss later on.

Anyway I leave the rest to further discussion. I particularly want to mention the Narrative Fallacy, which I have actually mentioned before, the problem of prediction as well as whether I really am a fraud, idiot, or neither.

Hypocrisy, relativism and the Diamond Age

8 November, 2006

Hypocrisy is a poorly regarded activity as Ted Haggard, a US pastor who has admitted to having gay sex could tell you. Criticism and focus of most stories about this case has concentrated mostly on his anti-gay views and the inherent hypocrisy.

Hypocrisy is defined as “a feigning to be what one is not or to believe what one does not; especially : the false assumption of an appearance of virtue or religion”. While it’s clear that Haggard hasn’t met the virtuous standards he has espoused it’s also not clear that he is feigning his belief in those standards. People following their instincts rather than doing what they believe is right is not uncommon.

A passage in Neal Stephenson’s SF book The Diamond Age, has an interesting take on hypocrisy espoused by one of characters, one of the leaders of a group known as “Neo-Victorians”.

You know, when I was a young man, hypocrisy was deemed the worst of vices,” Finkle-McGraw said. “It was all because of moral relativism. You see, in that sort of a climate, you are not allowed to criticise others-after all, if there is no absolute right and wrong, then what grounds is there for criticism? … Now, this led to a good deal of general frustration, for people are naturally censorious and love nothing better than to criticise others’ shortcomings. And so it was that they seized on hypocrisy and elevated it from a ubiquitous peccadillo into the monarch of all vices. For, you see, even if there is no right and wrong, you can find grounds to criticise another person by contrasting what he has espoused with what he has actually done. In this case, you are not making any judgment whatsoever as to the correctness of his views or the morality of his behaviour-you are merely pointing out that he has said one thing and done another. Virtually all political discourse in the days of my youth was devoted to the ferreting out of hypocrisy. Read the rest of this entry »

Misbehavior of Markets: Mandelbrot

26 September, 2006

In 1963 Mandelbrot published research into the distribution of cotton prices based on a very long time series which found that, contrary to the general assumption that these price movements were normally distributed, they instead followed a pareto-levy distribution. While on the surface these two distributions don’t appear to be terribly different, (many small movements, and a few large ones), the implications are significantly different, most notably the pareto-levy distribution has an infinite variance.

This implies that rather than extreme market moves being so unlikely that they make little contribution to the overall evolution, they instead come to have a very significant contribution. In a normally distributed market, crashes and booms are vanishingly rare, in a pareto-levy one crashes occur and are a significant component of the final outcome.
Read the rest of this entry »

The ethics of what we eat – Part III

2 September, 2006

In the earlier parts of this I discussed what I see as the problems with the book from an Australian perspective and my discussion of the issues they have with factory farming.

The later sections of the book concentrate on “conscientious omnivores”, essentially people who still eat meat but make a conscious effort to buy such things as “free range”, “organic” and “locally grown” produce. Then finally, veganism.

Firstly they discuss the fact that many of the labels that indicate humanely grown or similar produce are not probably what they expect. In some cases only very token efforts have been made to gain these labels. I didn’t find this unexpected. Whatever the system is people will be trying to game it for their own advantage.

On the other hand they visit a number of farms that do practice legitimate free range production, and as expected, the animals have happier lives. We’ll never really know how much happier but pretty clearly happier. In the low density flocks many of the practices that are routine for factory farms such as debeaking chickens and tail docking pigs are not necessary.
Read the rest of this entry »

Spinning a narrative

1 September, 2006

One of the topics discussed in Fooled by Randomness was the tendency of people in general, but journalists and business economists in particular, to attempt to spin any random series of events into some sort of coherent story. Rather than being content to report an event in its own right there is a compulsion to explain even if the link is tenuous.

He gives some typical “Bloomberg” examples (from the widely used news/trading service). Comments such as “Dow is up 1.03 on lower interest rates”, when there is little evidence that the two events are related, particularly when a 1 point move on the Dow is essentially it remaining unchanged. Essentially they are trying to explain noise as something real and this goes wider than the reporting on markets.

Other classic examples are journalistic explanation of polling data. A drop of a percent or two in the figures of one poll is read as having been the voter’s perceptions. This article in the SMH from earlier this year is a classic example:

The downturn in support for the Government over the past month indicates the tax cuts have failed to assuage public dissatisfaction over the rise in interest rates and petrol prices, and the fiasco following the death in Iraq of Private Jacob Kovco.

How much of a swing? 3 percentage points, which as Andrew Leigh explained at the time, once the margin of errors are taken into account, is a result not significant enough for any credible social scientist to draw a conclusion from.
Read the rest of this entry »

Fooled by Randomness

22 August, 2006

I’ve just finished reading Fooled by Randomness by Nassim Nicholas Taleb (2nd edition). Its a good book which makes some great points, but also has plenty of stuff that is annoying in it. The style as he describes it is not mathematical but literary-philosophical, which I don’t have issue with but I did find the author’s sneering and arrogance sometimes a little overwhelming.

The book is structured as a series of essays about the nature of probability and randomness, and how people can deal with it. Taleb is a derivatives trader who worked on Wall Street for many years and now runs a hedge fund of his own. As the heading quote on his website states:

My major hobby is teasing people who take themselves & the quality of their knowledge too seriously & those who don’t have the guts to sometimes say: I don’t know….

and much of the book is just this, mocking all those who make predictions, well beyond their knowledge. Journalists and others who ascribe causal relationships to random outcomes also get considerable attention. He makes much of the way that survivor bias distorts our opinions on things. We look at someone who had been extraordinarily successful in some activity and assume it relates to skill without asking how many people who did something similar failed. Are they good and we have something to learn from them, or are they just the lucky 1-in-32 who got five heads in a row? Without knowing how many people started in the endeavour and whether the successes have survived longer than we would expect in a random environment is the only way of having some confidence.

More than anything else Taleb’s focus is on what he calls “Black Swans”, the rare occurrences that inductive reasoning will never tell you about. His points are coloured particularly by two experiences. Growing up in Lebanon in the early 80’s war and by his experience of highly successful traders who, after years of success, lost everything they had made and much more in less than one month including their jobs by the Russian default in 1998. In the aftermath many would claim that such an event was completely unusual and unexpected, Taleb argues that our past experience will never be a good guide to such things and we will always run into these outlier events.
Read the rest of this entry »

The Ethics of What We Eat – part II

20 August, 2006

In the first part of this I dicussed what I saw as shortcomings from an Australian perspective, here I’ll actually discuss the book.

The book itself it is divided into roughly three sections, the first being a typical American family, who on analysis eat food that mostly comes from “factory farms”. The second family are “conscientious omnivores”, they are more discriminating the father is vegetarian and the meat that is bought is mostly free range. They also purchase on the basis on such concepts as buying locally, fair trade and organic food. The third family are vegan family, their main ethical concerns are about avoiding GM eating organic food and the question of whether it is ethical to raise children as vegans.

The attempt is made to trace where the food they purchase in each situation comes from, how that food is produced and what are the effects of producing it.
Read the rest of this entry »