From January 2008 through April 2011, we publicly posted 124 predictions related to stock prices in the S&P 500, GDP, new unemployment claims and other economic or business-related topics. We've established the outcome for 116 of those predictions and determined that 79 were on target while 37 were not.

Overall, that gives us an overall prediction accuracy percentage of 68.1% and a plus-minus statistic of +42 for our prediction project. But we still have a burning question:

How does that compare to others?

Is that 68.1% prediction accuracy good? Average? Lousy? We don't know, so today we're going to find out. And the way we're going to do that is to compare our performance with the gurus!

Who are the gurus? Well, they're the (mostly) guys who put out their own economic and investing predictions via newsletters or magazine articles. Several even have radio shows and even turn up with some regularity on business or investing-related television programs. And luckily for us, guys whose prediction track records have been sampled and analyzed by CXO Advisory who, really luckily for us, utilizes a system that's very similar to the plus-minus based approach we used to measure our own performance!

We've taken CXO Advisory's recorded sample data for each the gurus for whom they've tabulated data and put it in the dynamic table below - if you're accessing the table directly on Political Calculations, you can rank the data from low-to-high or from high-to-low according to the category you select by clicking the column headings.

In looking over the rankings, although we appear to have done well, we should note that there's some apples-to-oranges comparisons going on between how we tracked our performance and how CXO measured the prediction accuracy of the gurus.

Our standard procedure in making predictions is to both state the prediction and the specific time period for which it would apply - if we were off on either count, we would score a miss. By contrast, CXO's method consider the gurus' predictions, but provides a full year of time following when they were made to determine the outcome, which is a much looser standard.

To be fair, that's because many of the gurus' predictions were open-ended in not specifying the period of time for which their predictions would apply. CXO's prediction accuracy measurement approach then is a reasonable way to consider that element, but one that is perhaps overly generous to the gurus. We also found that many of the guru's predictions were somewhat vague, in that a lot of predictions were essentially "up or down" calls, lacking precision in quantifying "how much?"

Overall, the gurus (including our results) were right 2,593 times and wrong 2,833 times, for an overall prediction accuracy score of 47.8% and a plus-minus statistic of -240. Omitting our results from the gurus total drops their overall prediction accuracy score to 47.3% and the plus-minus statistic to -282.

The bottom line however is that there are a few names in the list above that are genuinely worth listening to for economic and investing matters - particularly those with more than 100 recorded predictions and above average accuracy. We were surprised to see some very well respected names rank low in prediction accuracy, such as Richard Russell and John Mauldin, who performed less well than, say, Jim Cramer, perhaps the best known so-called guru in the list.

That's not necessarily a strike against these three gurus, as each (and particularly John Mauldin through his weekly newsletters) offers insights into economic and market matters that we find highly valuable. (Yes, believe it or not, Jim Cramer in serious mode can be very insightful!)

Finally, while we came out pretty well overall, our sense is that there are others who aren't ranked in the table above who would likely do as well or better. We'd love to see where Barry Ritholtz would rank for one (he would definitely qualify as a guru), as well as Ivan Kitov, who hunts bigger game in developing predictions that span multi-year periods of time, rather than predictions that apply for the next week or the next month.

Previously on Political Calculations

The following links will take you to our previous prediction outcome reports, which we've presented below in the order they've appeared here approximately every three months beginning with April 2009. You can get the most recent status updates by clicking the "track record" tag at the bottom of the post.

• Plus-Minus for Predictions - 16 April 2009
• Our Plus-Minus Is Now Seventeen! - 16 July 2009
• Predictions Plus-Minus Update - 15 October 2009
• Predictions Plus-Minus Update - 21 January 2010
• Predictions Plus-Minus Update - 26 April 2010
• Predictions Plus-Minus Update - 30 July 2010
• Predictions Plus-Minus Update - 29 October 2010
• Predictions Plus-Minus Update - 31 January 2011
• Predictions Plus-Minus Update: The Final Regular Tally - 2 May 2011

Image Credits: Joe Paduda and Yes Yes Way

Labels: forecasting, track record

- posted by Ironman at 2:05 PM | Permalink | Home |

In April 2009, we began keeping track of how accurate all of our predictions have been since January 2008, providing updates for our readers once a quarter ever since. Today, that comes to an end, as we've now completed our two year long experiment in publicly forecasting future economic and market events and reporting the results.

During that time, we've measured the accuracy of our predictions using one of the most unforgiving metric ever developed for measuring performance: the plus-minus statistic from hockey and basketball!

Using this system, we gain a point if we're right, lose a point when we're wrong, and score a zero for when the outcome of a prediction either cannot yet be determined or, in the case where we make multiple predictions, when we have contrary results that cancel each other out. And like a game of horseshoes, close doesn't count where near misses are involved!

Ultimately, our plus-minus score will reveal the number of times we scored hits rather than misses in making predictions of the future. If we're no better at predicting the future than a coin toss, our plus-minus score will gravitate toward zero. If we're better at predicting the future than that, our score will rise over time, and if we're really bad, our score will surely fall.

In January 2011, our recorded plus-minus score stood at +38. Here's how we stand today for all the predictions we've made through our final public prediction at the beginning of April 2011 or for which we are still waiting to discover the outcomes:

By our final tally, we've made a total of 124 predictions since January 2008 and we're still waiting to find out the outcomes of eight. As for being off or on target, we've been off target on 37 occasions and on target for 79, giving us a plus-minus score of +42 overall. Our overall prediction accuracy rate is 68.1%, which is what you get when you divide our 79 hits by the 116 predictions where we've been able to establish the outcome and convert the result to a percentage.

But now, there's the little matter of how we compare to others, which we'll post soon to complete the grand finale for our forecasting series!

Previously on Political Calculations

The following links will take you to our previous prediction outcome reports, which we've presented below in the order they've appeared here approximately every three months beginning with April 2009. You can get the most recent status updates by clicking the "track record" tag at the bottom of the post.

• Plus-Minus for Predictions - 16 April 2009
• Our Plus-Minus Is Now Seventeen! - 16 July 2009
• Predictions Plus-Minus Update - 15 October 2009
• Predictions Plus-Minus Update - 21 January 2010
• Predictions Plus-Minus Update - 26 April 2010
• Predictions Plus-Minus Update - 30 July 2010
• Predictions Plus-Minus Update - 29 October 2010
• Predictions Plus-Minus Update - 31 January 2011

Image Credit: Random Glenings

Labels: forecasting, track record

- posted by Ironman at 12:23 PM | Permalink | Home |

Assuming typical levels of noise in the markets, and in the 1 in 5 chance that a short-term correction that may be waiting in the wings will stay there, we can expect the average of the daily closing values for the S&P 500 to fall into a range between 1289 and 1321 during February 2011.

Our first chart shows why we think so:

Our second chart shows our track record for predicting where the S&P 500 will go a month ahead of time:

We'll leave it as an exercise to our investing-oriented readers to work out what our up-or-down call accuracy might be.

Labels: forecasting, investing, SP 500, track record

- posted by Ironman at 8:55 AM | Permalink | Home |

In April 2009, we began keeping track of how accurate all of our predictions have been since January 2008, providing updates for our readers once a quarter ever since.

Today, it's time once again to update our prediction accuracy score, in which we make use the most unforgiving metric ever developed for measuring performance: the plus-minus statistic from hockey and basketball!

Using this system, we gain a point if we're right, lose a point when we're wrong, and score a zero for when the outcome of a prediction either cannot yet be determined or, in the case where we make multiple predictions, when we have contrary results that cancel each other out. Like a game of horseshoes, close doesn't count where near misses are involved!

Ultimately, our plus-minus score will reveal the number of times we scored hits rather than misses in making predictions of the future. If we're no better at predicting the future than a coin toss, our plus-minus score will gravitate toward zero. If we're better at predicting the future than that, our score will rise over time, and if we're really bad, our score will surely fall.

Three months ago, our plus-minus score was +35. Here's how we stand today:

• Number of Predictions Made Since January 2008: 113
• Number of Correct Predictions: 72
• Number of Incorrect Predictions: 34
• Number of Outcomes Not Yet Determined or No Decisions: 7

Overall, for the predictions where we've been able to establish the outcomes to date, our plus-minus score has risen to +38. Measured as a percentage, our prediction accuracy rate is 67.9%.

The table below updates the status of all the predictions that we've successfully determined the outcomes during the last three months, as well as those for which we are still waiting for the outcome. The blow-by-blow commentary is just a bonus that you'll hopefully find to be entertaining!

We're coming up on the end of this project, which we'll terminate with our two-year predictions plus-minus score accounting anniversary in April 2011. As part of the grand finale, we'll unveil just how we rank among the community of financial and economic analysts and gurus!

Previously on Political Calculations

The following links will take you to our previous prediction outcome reports, which we've presented below in the order they've appeared here approximately every three months beginning with April 2009. You can get the most recent status updates by clicking the "track record" tag at the bottom of the post.

• Plus-Minus for Predictions - 16 April 2009
• Our Plus-Minus Is Now Seventeen! - 16 July 2009
• Predictions Plus-Minus Update - 15 October 2009
• Predictions Plus-Minus Update - 21 January 2010
• Predictions Plus-Minus Update - 26 April 2010
• Predictions Plus-Minus Update - 30 July 2010
• Predictions Plus-Minus Update - 29 October 2010
• Predictions Plus-Minus Update - 31 January 2011

Labels: forecasting, track record

- posted by Ironman at 11:37 AM | Permalink | Home |