Thursday 21 August 2014

A Stock Ranking Backtest

The Mechanical Bull method is a strategy based on theory. The logic is appealing, and it has performed reasonably well over the last 15 months. However, I have had no real way of testing how well it is going to perform over the long term. I would love to be able to backtest the approach. Unfortunately, this is not possible because Stockopedia doesn't provide a backtesting service (at least not yet).

Recently, I made a rather intriguing discovery. I used to be a frequent user of Sharelock Holmes, until Stockopedia launched its subscription service. Over the weekend, I dipped back into the site and stumbled over some new metrics that looked remarkably like Stockopedia's ranking measures. Sharelock now has composite Quality, Momentum and Value scores, as well as the various blended composites, including the "Market Score" which is their version of the StockRank. It uses a similar ranking score from 1 to 100 (although 1 is the best through to 100 for the worst).

Now, I am certainly not advising anyone to cancel their Stockopedia subscriptions and switch to Sherlock Holmes. It is a rather dull and drab website and the screening functionality is fairly limited. More importantly, I suspect the data is not always up to scratch. I've come across errors in the past relation to handling of share splits. However, the one area where it exceeds the Stockpedia offer is in terms of backtesting. So I feel this provides a useful opportunity to see how these composite strategies play out over the longer term.

It is important to note that these Sharelock Holmes rankings do not produce the same set of stocks as the Stockopedia ranking system. Of the current StockRank top decile stocks, about 60 per cent are in the top decile "Market Score" list. Most of the remainder rank are in the second decile although a few got very poor scores.

So these rankings seem to be based on very similar principles, but the implementation differs. There are also clearly differences with some of the data. Just one example I picked out was the Piotroski score for Staffline, which is 8 according to Stockopedia but just 4 according to Sharelock. Not with standing these differences some back testing of these ranking scores seemed too me to be a worthwhile project.

This first chart shows "Market Score" performance by deciles (rebalanced each quarter) going back to Jun 2003:

This gives the familiar and reassuring fanning out pattern seen on the Stockopedia discussion forums, except going all the way back to mid-2003. The top decile stocks blitz the field with an especially impressive performance over the past 5 years. The chart also nicely illustrates the compounding effect over the longer term. This just goes to show how tilting the odds in your favour, even just slightly, can deliver very impressive returns over the longer term.

However, it is also interesting to note that that all deciles fall away during the bear market that started in mid-2007 through to early 2009. I think this raises some interesting questions about market psychology. It seems that the market as a whole acts fairly rationally during bull markets, identifying quality, undervalued and winning stocks and dumping junk, expensive and losing stocks. However, during a bear market, fear takes over and investors seem to be willing to irrationally dump stocks irrespective of fundamentals. So, really, we should be grateful for bear markets since they generate the fertile ground for rational investors to exploit in the next bull run.

My next piece of analysis involved selecting the top decile stocks for Quality, Value and Momentum, along with all the composites (QV, QM, VM, "Market Score"). For each strategy I generated a set of relative performance data for every quarter back to June 2003 for a range of time periods from 3 months to 5 years. For each of these time periods I have then averaged the performance and then annualised the data. For example, for performance data over two years, I halved the average performance to give a figure that could be compared with all the other time periods.

It is only possible to compare the performance across all these various time scales on a fully like for like basis for the first five years (i.e. Jun 2003 to Jun 2009). In other words, to get the 5 year performance results for June 2014, you would have to wait until June 2019 to see how those stocks will eventually perform. So it is important to bear in mind that this data does not include the bull market of the past five years (and also remember this is relative not absolute performance). The results of this comparison is shown below:


This shows that momentum over the short term is is the single most effective strategy. However, performance drops of quickly after 3-6 months so in terms of translating this into an effective strategy you would need to factor in trading costs and spreads etc. Also, momentum tends to do badly in falling markets.

A strategy based on a pure value rank performed worse of all. This result seems pretty surprising considering that value investing is so widely respected. A likely explanation is that a rules based approach to value investing may be susceptible to picking value traps, that is, stocks that are a cheap, but for very good reasons that doesn't appear in the financial data. The only time that a value based rules strategy seems to really pays-off is after the market has completely tanked.

A strategy based purely on quality does consistently well over all time periods. Quality also holds up  best when the market is under pressure. This suggests that quality stocks are not just for buy and hold type investors, but should probably be key consideration for any investment strategy.

The combined QVM (Market) strategy does fairly well but it doesn't particularly stand out as the best approach. I suspect the under performance of value acts as a slight drag its overall performance. On this basis, the greater safety offered by quality would seem to be a price worth paying for only slightly greater short-term returns.

I also produced another version for the full ten years. As explained above, this shows only partial data for the longer time periods, but it is worth showing as it incorporates the bull market since early 2009.


This shows that the returns for both the Market and QM strategies have picked up relative to pure quality. If one considers that the typical holding period for a stock in the MB portfolio is about a year, then either of these strategies would offer at least a 10 percent annual return above the market.

There is a lot more I would like to look at here, but something I am starting to think about is a strategy based on different phases of the stockmarket cycle. This could consist of quality, value and momentum "pots" where the size of the pots would vary according to the state of the market. For example, given where we are at the moment I might have about 70% quality and 30% momentum stocks. If the market drifts down I might go 70% quality 30% value. If another bull market I might go 50% quality and 50% momentum.

Anyway, lots of ideas here and lots more thinking to do.

Friday 8 August 2014

A Riddle Solved

I regularly check out the very slick dashboard on Stockopedia's home page. I've been a particular fan of the Guru Screens section. This includes a Guru index, which is a composite of all 60 or so long guru screens maintained by Stockopedia.

Over the past few months I have been increasingly puzzled by the droopy looking shape of the Guru composite index versus the FTSE 100 index. I've replicated the chart below to show you what I mean:


This shows the composite Guru index steadily pulling away from the FTSE 100, accelerating away during the second half of 2013 but then falling back over the last six months or so. In contrast,the FTSE 100 has been holding its ground pretty well during 2014.

Stockopedia handily group Guru screens by 'style performance' (e.g. Value, Growth, Momentum, Income etc.) so you can dive a bit more into the detail (although this is subscriber only service). Unfortunately, this doesn't really shed much light on proceedings. The FTSE 100 outperforms every one of these style indices over the past six months. In other words, whatever you investment style, you have probably lost money over the past six months. Unless of course your guru is a monkey with a pin and a list of FTSE 100 companies. In that case, you are probably slightly ahead.

This seemed to me to be a very strange state of affairs and so I decided to really put in some effort to work out what is going on. I spent a lot of time stratifying performance data and looking for patterns, After a number of false leads, I stumbled over something very interesting. It turns out I was looking in the wrong place. I should have been looking into the benchmark rather than performance factors of the guru screens. I will just cut to the chase and show you what happens when you overlay the FTSE 250 index over the composite Guru screens:


This clearly shows that the Guru composite index has been basically tracking the FTSE 250 over this period. Of course, a lot of FTSE 250 stocks appear in these Guru screens. I have done some rough calculations (rough as there isn't a straightforward way of screening out FTSE 250 stocks on Stockopedia) and I reckon about 30 per cent of all Guru screen appearances are FTSE 250 stocks. In contrast, less than 10 per cent of appearances are for FTSE 100 stocks. Even so, the strength of the correlation between the Guru index and the FTSE 250 seems remarkable.

What this shows is that if you really want to test whether a strategy is outperforming, you need an appropriate benchmark. Ideally you want a benchmark that broadly represents the investment universe your picking from so you can determine whether you can identify stocks that systematically outperform. The FTSE 100 Index isn't ideal in this case because it represents such a small part of the investment universe from where the Guru strategies are drawn.

There is the FTSE All share index, but that just seems to follow the FTSE 100 pretty closely, so I assume that the index is weighted by market cap. What you really want is an unweighted index of all FTSE and AIM stocks, but I am not sure if there is such a thing. If anyone knows something about these things please let me know!

This also throws up the obvious question as to whether these guru screens actually help in picking winning stocks. The evidence above suggests that using the monkey with a pin method on FTE250 stocks might work just as well.

I have said before that stock screening has its limitations because it is binary blunt instrument. Stocks are either in or out and there is no differentiation between a stock that clears a hurdle by a country mile and ones that just scrape in. The same principle applies for those that just miss out. I also have a pet theory that stock screening tools have become so widely available that perhaps they no longer give the average investor much of a edge.

These things aside, I am pleased to have solved this riddle. When I see different markets moving in certain ways, I am always trying to work out where the money in the system is going from and going to. The pattern over the last six months clearly points to a net move of capital from smaller to large caps.

Portfolio update:

The MB portfolio was down 1.3 per cent in July. This pretty much followed the major indices, with the FTSE 100 falling about 0.2 per cent and the FTSE 250 down 1.4 per cent. The was one change to the portfolio with Sweett (CSG) making me reach for on 8 July to be replaced by Castings (CGS). So that was CGS replacing CSG, which has a nice symmetery about it.