Is there a better way to weight stocks within a market index?

Modern portfolio theory suggests that the optimal way to weight stocks within an index is according to their market capitalization, where the percentage representation of each company within a particular index is based on the product of its share prices and number of shares outstanding with respect to the total sum of all the market capitalizations of the companies whose stocks make up the index. Examples of market cap-weighted indices includes the S&P 500 and the Nasdaq 100 (Ticker: QQQQ).

There are, of course, other ways to weight the holdings of the stocks of individual companies within a market index. For example, the Dow Jones Industrial Average (Ticker: DJI) is a price-weighted index. There are also equal weighted indices, in which the number of shares of each component stock within the index is periodically adjusted so that each represents the same market capitalization.

A little over 10 years ago however, Rob Arnott of Research Affiliates introduced a new method for setting the weight of each stock within a market index, called fundamental indexing. Here, instead of using either market cap or price, stocks within the index would be weighted according to fundamental measures of their companies' business performance, such as their revenue, dividend rates, or book values.

Here is the promise of fundamental indexing that Arnott noted back in 2006, based on his analysis of the 1,000 stocks that made up his proposed index:

How consistent is this approach? It's awfully consistent. During economic expansions, you add almost two percent a year. During recessions -- when you most need those returns -- you add three and a half percent. During bull markets you add 40 basis points. You don't really add anything in bull markets, because they are driven more by psychology than by the underlying fundamental realities of the companies. And so during bull markets you keep pace. Which is good; it's important. During bear markets you find yourself adding 600 to 700 basis points per annum. Bear markets are when reality sets in and people say, "Show me the numbers." Bear markets are when this really comes on strong. Also, during periods of rising rates, two and a half percent added. During periods of falling rates, one and a half percent added.

Soon, it became possible to invest in Arnott's new kind of index, where the first one was an Exchange Traded Fund based on his original 1,000 stocks, Powershares RAFI US 1000 ETF (NYSE: PRF). The follwwing chart compares the relative performance of the fundamentally weighted PRF against the price weighted DJI and the market cap-weighted S&P 500 over its entire history.

In this chart, we see that through the close of trading last Friday, 14 October 2016, the relative value of the fundamentally weighted PRF was up by 84.15% over its initial 30 December 2005 value, while the Dow Jones Industrial Average was 66.78% higher and the S&P 500 was 68.15% higher.

But we also see that there were periods where DJI either outperformed or performed as well as PRF. We also noticed that most of the PRF's gains over the market cap-weighted S&P 500 came in 2009, so we wondered how the relative performance of the PRF has fared since that time. So, we reset the chart to show the relative performance of each major type of index to begin in January 2010. The next chart shows what we found.

We see that PRF has outperformed both the S&P 500 and DJI over the period covered in this chart, having risen to be 96.32% above its 8 January 2010 level, compared to the S&P 500's 93.48% and the DJI's comparatively lackluster 73.23%. Meanwhile, we also confirm that much of PRF's relative outperformance over the longer period of time with respect to the S&P 500 occurred as an almost singular event in mid-2009, when whatever factors boosted it also boosted the DJI.

What this outcome suggests is that the considerable outperformance of PRF during 2009 has not been replicated in the years since, where through 14 October 2016, it has largely found itself within spitting distance of the performance of the S&P 500.

And yet, the fundamentally weighted PRF has generally performed either equivalent to or slightly better than the market capitalization weighted S&P 500 over that time.

As an investment then, it has some very attractive qualities that suggest a place for it in one's long term holdings. However, the near-performance of PRF with either the DJI or the S&P 500 over shorter durations means that it will be important to consider other factors, such as transaction costs, when choosing between investing in these different kinds of indices.

On a final note, we've been wanting to get back to this particular investing topic for some time, where the last time we discussed it here was back in August 2006!

What can we say? Some of our analytical projects have very long development periods!