The Folly of Stock Market Forecasts and Predictions

One of the themes of my book Invest Like a Fox…Not Like a Hedgehog is that people often misuse or misunderstand statistics. This is particularly true in investing, but it’s also true in other areas. Nate Silver made a big name for himself in 2012 by bolding predicting a big victory for President Obama. Previously, he worked with baseball statistics. Silver wrote a book, The Signal and the Noise, that explains many of the problems in the ways people use and interpret statistics. You can learn a lot from it. But if you aren’t ready to read the book, consider this post describing 12 principles from Silver that you can use to evaluate information, numbers, and news.

The ratings agencies got the housing bubble wrong because they were behaving like first-time drunk drivers

It’s called an out of sample error, and here’s how he describes it. Someone who has never driven drunk needs to decide whether to call a cab home.

Out of a sample of 20,000 car trips, you’d gotten into just two minor accidents, and gotten to your destination safely the other 19,998 times. Those seem like pretty favorable odds. […] The problem of course, is that of those 20,000 car trips, none occurred when you were anywhere near this drunk. Your sample for drunk driving is not 20,000 trips but zero, and you have no way to use your past experience to forecast your accident risk.

Silver then points out that this is the exact type of forecast error made by ratings agencies with regards to housing, and the fact that they based their mortgage default correlation models on a set of data where housing never decreased in value.