The Black Swan

29 October, 2007

My original intention was to write a longish post discussing the main ideas. Rather I will do an overall impression here, and then do a few follow ups to talk about what I think are the interesting points.

All up I found The Black Swan an very interesting read. If you’ve read fooled by randomness then some of the arguments will have already appeared but certainly not all. The Black Swan is concerned with wild randomness, the randomness that according to Taleb dominates modern society. Fool by randomness largely concerns the randomness of games, what Taleb would describe as mild randomness. So mild that it is hardly random at all.

As we are reminded in several places in the book Taleb made his fuck off money (i.e. enough money to be comfortably independent even if not mega wealthy) in the 87 stockmarket crash betting on the fact that the market under-appreciated Black Swans. He doesn’t need to fawn to the establishment be it economics, philosophy or publishing and we get this tone all through the book.

While the book touches on finance applications its hardly a finance book. Rather Taleb considers it to be as much a work of Epistomology. His chief concern is with what he calls epistemic arrogance. He hates those who profess to know more than they really do know, be they economic modellers, historians or political scientists, but his chief enemy is the normal distribution which is described as the Great Intellectual Fraud. To use normal distributions in fields where it is easy to show are not normal (such as finance), is both fraudulent and causes Black Swans.

Its strong stuff which he backs with examples, logic and the findings of behavioural finance and similar studies. Its also fairly convincing for the most part. Its true we don’t need sophisticated statistical studies to find whether market moves are normally distributed. We only need the fact that we get 1 in 10,000 year events (as modelled by a normal distribution) occurring every few years. Risk managers who run such models (such as me) are either ignorant fools, who actually believe wrongly in what they do, or deceptive frauds who know better but persist in fooling others for the money. I’d contend I’m neither but I’ll discuss later on.

Anyway I leave the rest to further discussion. I particularly want to mention the Narrative Fallacy, which I have actually mentioned before, the problem of prediction as well as whether I really am a fraud, idiot, or neither.


Card Counting

19 December, 2006

This will probably be a real spam magnet but anyhow….

Some time ago, while I was still at uni, with a few friend I tried card counting blackjack. Like all casino games blackjack is in the house’s favour. However over the course of a deck (or several decks) this is not always the case for every hand. By keeping track of the cards that have come out of the deck, you can determine when the game is in your favour and increase your bet accordingly. Contrary to popular opinion you don’t need to track the actual cards that have been removed but rather keep a running tally on their effect on the game.

Certain cards are good for the player. A deck rich in aces gives more chance of blackjack with its higher pay out and a deck rich in tens gives the dealer who must hit automatically until he gets 17 or more, a much higher chance of going bust. Conversely a deck rich in 4, 5 and 6’s is bad for the player. The dealer will more rarely go bust as these cards will save him from going bust when he gets to totals of 14-16. The basics of such a system of counting and betting were first shown by Edward O. Thorpe and rely on both inferring a proportional measure of the expectation for a single bet and the use of the Kelly Criteria, in essence betting amounts proportional to your expected chance of winning.

In a small deck this is particularly effective as you get towards the end of the cards, but it’s also effective in large decks as well. Particularly if they place the cut card, which determines the point when the decks are reshuffled, near the end. Casinos of course take a dim view of the idea that someone else other than the house could be playing a positive expectation game and so try to stop it typically by banning the players in question. Counting is not illegal, but the casino is free to exclude anyone they don’t like.

The reality of the situation though is that unless you have a large bank roll you are better off working at McDonald’s, and if you are playing alone its pretty easy to detect someone with wild swings in their betting patterns, particularly if they are winning money. Still it was possible for us on many trips to the casino to sit off a table not playing until the deck was “hot” and then jump in and lay some bets. This makes you pretty obvious but if you are serious low rollers like we were when we were uni students then I doubt they are worried particularly.

Successful counters these days work in teams. A reasonably interesting book on the subject is Bringing Down the House, which although it concentrates too much on the glamour of the high-roller lifestyle and not enough on the actual scheme.

As for our little project, it slowly disappeared into nothing more than an occasional drunken trip to the casino where we would attempt to count through the haze. I note though now that the Star City Casino has put in continuous shuffle machines and the whole hope of counting is gone. I wonder whether it is actually a positive revenue deal for them. After all people like me will only play if they have the knowledge that they might just be able to have an edge even though in practice they rarely will. For the casino I guess giving this money away is worth it if it means avoiding serious counters.

What probability your vote decides?

10 October, 2006

Recently Andrew Leigh posted some comments about the number of elections in Australia where a single vote has decided the result and commented that this implied there was a probability that there was a 1 in 4500 chance that your vote would be the decisive one.

I have a few objections to this number, the first being that in a preferential system there are a number of points where a single vote can have an impact on the outcome as the preferences are redistributed up to two parties, so even though the final two party outcome is decisive you may have been decisive in deciding which party won. Whether this has ever occurred, I don’t know but it would seem possible.

A second objection is that a victory by two votes is also decisive, as otherwise you could have driven the result to a tie. I don’t know whether this has ever occurred.

Main objection though is with the calculating probabilities based on the historical outcome rather than the expected outcome for an electorate. Voters know when they go to the booth whether they are in a safe seat or not and whether their vote is likely to count and I would presume that this effects the way they view their vote.
Read the rest of this entry »

Misbehavior of Markets: Mandelbrot

26 September, 2006

In 1963 Mandelbrot published research into the distribution of cotton prices based on a very long time series which found that, contrary to the general assumption that these price movements were normally distributed, they instead followed a pareto-levy distribution. While on the surface these two distributions don’t appear to be terribly different, (many small movements, and a few large ones), the implications are significantly different, most notably the pareto-levy distribution has an infinite variance.

This implies that rather than extreme market moves being so unlikely that they make little contribution to the overall evolution, they instead come to have a very significant contribution. In a normally distributed market, crashes and booms are vanishingly rare, in a pareto-levy one crashes occur and are a significant component of the final outcome.
Read the rest of this entry »

Benoit Mandelbrot: Annoyingly full of himself.

4 September, 2006

I’m currently reading Mandebrot’s “The misbehavior of markets” which in time I will write a review of, but about 100 pages in I have to say I’m pretty tired of him stating every couple of pages how he discovered fat tails in market prices forty years ago, and has known the whole time that Modern Portfolio Theory, Black-Scholes etc was based on the wrong assumption of a normal distribution all that time. There appears to be some interesting stuff in the book, and there is no doubt some of the basic assumptions of finance theory need a look at and possibly re-grounding, but its not like no one in the past forty years hasn’t been dealing with this.

Even though it’s very much ad hoc, markets already attempt to account for the fact that Black-Scholes is flawed, by pricing in an implied volatility smile. Similarly fat tails have been used in risk models pretty widely by now. Neither are completely satisfactory but are both clear signs that market participants at least realised the flaws in the models being used, and have tried to adjust accordingly while retaining some framework that actually lets you get on with things. Certainly this is not the impression you get from the early parts of this book where it seems keen to show that Mandelbrot is virtually alone in appreciating the problems.

There is a maxim in creative writing that an author should “show not tell” as in you reveal the abstract qualities of personality etc by showing the details not telling the abstractions directly. It would be well for non-fiction authors to realise that they will come off much better by showing us how smart they are by revealing the genius of their arguments and discoveries rather than by telling us how smart they are every couple of pages.

Australian wins Fields Medal

23 August, 2006

It deserves mention that an Australian, Terence Tao has won the Fields Medal for mathematics becoming the first Australian to win the prize. Up to four Fields Medals are handed out once every four years, and it is the mathematics equivalent of the Nobel Prize.

AT THE age of two, Terence Tao could already add up and subtract using the magnetic numbers his parents stuck on the fridge.

At eight, he scored better than 99 per cent of 17-year-old prospective university students on an international aptitude test for mathematics.

The Adelaide-born prodigy was appointed a professor at 24, and now, at 31, has become the first Australian to win a Fields Medal, the mathematics equivalent of a Nobel prize.

The award was presented in Madrid yesterday by Spain’s King Juan Carlos I at a congress attended by 4000 international mathematicians.

Its a bit difficult to work out what exactly he won it for as the Medal is not awarded for specific pieces of work. His website states a fairly broad area of mathematical interest, although it is suggested here that:

It is awarded for a body of work rather than a single achievement but Professor Tao is most recently celebrated for showing, with Ben Green of Cambridge, that there are long strings of prime numbers a constant distance apart, work that is important for the coding of information such as banking details.

Well done Terence.
Read the rest of this entry »

Fooled by Randomness

22 August, 2006

I’ve just finished reading Fooled by Randomness by Nassim Nicholas Taleb (2nd edition). Its a good book which makes some great points, but also has plenty of stuff that is annoying in it. The style as he describes it is not mathematical but literary-philosophical, which I don’t have issue with but I did find the author’s sneering and arrogance sometimes a little overwhelming.

The book is structured as a series of essays about the nature of probability and randomness, and how people can deal with it. Taleb is a derivatives trader who worked on Wall Street for many years and now runs a hedge fund of his own. As the heading quote on his website states:

My major hobby is teasing people who take themselves & the quality of their knowledge too seriously & those who don’t have the guts to sometimes say: I don’t know….

and much of the book is just this, mocking all those who make predictions, well beyond their knowledge. Journalists and others who ascribe causal relationships to random outcomes also get considerable attention. He makes much of the way that survivor bias distorts our opinions on things. We look at someone who had been extraordinarily successful in some activity and assume it relates to skill without asking how many people who did something similar failed. Are they good and we have something to learn from them, or are they just the lucky 1-in-32 who got five heads in a row? Without knowing how many people started in the endeavour and whether the successes have survived longer than we would expect in a random environment is the only way of having some confidence.

More than anything else Taleb’s focus is on what he calls “Black Swans”, the rare occurrences that inductive reasoning will never tell you about. His points are coloured particularly by two experiences. Growing up in Lebanon in the early 80’s war and by his experience of highly successful traders who, after years of success, lost everything they had made and much more in less than one month including their jobs by the Russian default in 1998. In the aftermath many would claim that such an event was completely unusual and unexpected, Taleb argues that our past experience will never be a good guide to such things and we will always run into these outlier events.
Read the rest of this entry »