Did The Eastwood Index Beat The Bookmakers?
Introduction
It’s the end of the season so it’s time to review how the Eastwood Index performed over the year and how it compared with the bookmakers.
Ranked Probability Scores
One of the most important aspects to me is how accurate the forecasts were and I’ve assessed this using Ranked Probability Scores, as recommended by Constantinou and Fenton. I’ve discussed Ranked Probability Scores on the blog before but for people new to them they measure the difference between the forecasts and what really happened. Scores range between 0–1 and represent the amount of error in the predictions so lower Ranked Probability Scores are better and signify greater accuracy.
Comparison With Bookmakers
Looking at Figure 1 you can see that the Eastwood Index has consistently outperformed the bookmakers all season – and this isn’t just one bookmaker that the Eastwood Index has beaten but the combined knowledge of the industry as I’ve aggregated multiple bookmakers’ odds together and stripped out the overround to make the comparison as tough as possible.
Figure 1: Eastwood Index Vs Aggregated Bookmakers
Interestingly, the difference in accuracy seems to be greatest as both ends of the season. I expected the start of the season to be difficult to forecast as new teams have been promoted, players have been bought and sold, and managers may have changed clubs but the Eastwood Index seems to have coped with these variables better than the bookmakers’ odds have.
Over the course of the season the bookmakers’ forecasts improved until there was very little difference between them and the Eastwood Index but I was somewhat surprised to see how far out their accuracy drifted over the final few weeks of the season.
In theory these should be the easiest matches to forecast as we have the most information but in reality they can be tricky as team’s motivations change. For example, Manchester United have been playing their reserve goalkeeper so he gets enough appearances to earn his winners medal while Swansea’s players may as well have been on holiday since they won the league cup.
These changes seem to have thrown the bookmakers’ odds out quite noticeably while the Eastwood Index’s accuracy has remained constant. In fact, it suggests that bookmakers may be over-compensating for these apparent end-of-season effects as the Eastwood Index does not currently take them into account and has not struggled because of it.
Conclusions
Overall, I am pleased with the Eastwood Index’s debut season. I was slightly reticent to publish the forecasts at first in case the model did not hold up but it has remained accurate throughout the year. The next stage of its development is to identify any patterns as to where its forecasts differ from the bookmakers and how that could be combined with various staking strategies as well as looking at expanding to cover other leagues too.
Comments
George - May 21, 2013
Cool outcome. From my cursory looks into this, it is definitely possible, its just a case of trying to maximise any returns by only taking picks of a certain ratio etc. I’ve looked at other sports as well and sometimes find you can get a good number on a particular team for a period of weeks etc. Out of interest how are you translating %’s into home, away or draw (or are you assuming where the EI percentage is greater than the bookmakers % you are taking that result)? Things to look at going forward could be things like home field advantage (e.g. like the Clarke, Norman paper from 1995) or the effect of travel or fatigue. Good luck.
Martin Eastwood - May 21, 2013
Thanks George, that’s the tricky bit – how to convert from my %’s to winning staking strategy. My plan for the summer is to work on that further as I’ve got some ideas ready to test out for next season :)
Alex - May 21, 2013
Okay – perhaps this is naive, but if you’ve demonstrated that your index has outperformed the bookies all season, why can’t you invest, say, $100/matchday and split the bets according to the outcomes predicted by the EI over the course of the season? You should win the majority of the bets and show some return for your money, no?
Martin Eastwood - May 22, 2013
Yes and I would make a profit over the season doing that. What I want to work out though is what would be the best way of splitting that $100/matchday between the matches to maximize the profit made, e.g. something like the Kelly Criterion.
Alex - May 23, 2013
Ahh, I see. This is extremely interesting stuff. I’ve been tinkering around with a Monte Carlo approach in Matlab (using data from Football Manager 2013) for predicting game results, but I haven’t had great – or even remotely convincing – success. Your stuff here is brilliant.
Martin Eastwood - May 23, 2013 Thanks Alex :)
Is the data easy to extract from Football Manager 13? I’ve never touched it since the all Championship Manager days because it is too addictive but it could be an interesting data source…
Alex - June 6, 2013
Yep! There’s a few editors available (just google for ‘em), it becomes pretty straightforward to pull data.
Man, if you can show that you can outperform the bookies by even 4%, I think this would make a great alternative to savings accounts. Are you going to be publishing predictions for matchdays over the next season? I think putting in, say, $20 a matchday might be fun.
George - May 23, 2013
Don’t know which source you use for data but I use football-data.co.uk, which is great for any of the European Leagues (including the Premiership). What’s really nice for doing this kind of thing, is they usually put stuff in an xls or csv, so you can just do your thing straight away. They also usually have a range of bookmaker odds (as I am expecting that certain bookmakers deal with “sharp”er customers than others (if you know what I mean) so I expect that you could probably find a bookmaker that the EI is more prone to exploiting (or perhaps that the EI for a particular team is more prone to exploring).
Martin Eastwood - May 23, 2013
Something else to add to my to-do list, which is the best bookmaker for using the EI with!
amir - May 23, 2013
Please read this article for optimal allocation of bets: http://www.academia.edu/1027427/Algorithms_for_optimal_allocation_of_bets_on_many_simultaneous_events
amir - May 23, 2013
I see you already wrote another article stating Kelly Criterion…
Martin Eastwood - May 23, 2013
Thanks Amir that link looks really interesting!
Lars - October 18, 2013
If I see this correctly, you introduced the EI in February 2013.
Now you are comparing its predictions with the whole of the 2012/13 season.
That is not really “beating” the bookmakers. To beat them, you need to make your predictions BEFORE the matches not after them.
If you beat them again next year without changing your methods I’ll give you all the credit you deserve.
Martin Eastwood - October 18, 2013
Lars – I actually went back and recreated predictions for every match of the season based on just the data that would have been available at the point in time when the match was originally played.
Nic - November 24, 2013
Thanks for the good read. It’s really interesting.
When you compare against the bookmakers are you taking their opening odds? Because after the odds are published the fluctuations are due to general populace betting patterns.
This could account for your sudden strong finish vis-a-vis the bookmakers as the public was betting more against academic results etc which then the bookmakers have to tweak the odds for.
Another question. What’s the average spread a bookmaker keeps. To beat them do you mean you overcome the spread as well?
Martin Eastwood - November 25, 2013
Thanks Nic,
The odds were taken from the football-data website. There were then normalised to account for the overround and aggregated IIRC. Not sure how much the average bookmaker keeps as the overound, charges etc vary so much between traditional highstreet bookmakers and online betting.
Cheers,
Martin
Patrick - December 10, 2013
You state that you stripped out the bookmakers over-round in your comparison. Should you not be comparing your predictions against the bookmakers prices WITH over-round included?
Otherwise is it not a little bit pointless (in betting terms) as you won’t be able to take prices with the over-round removed? ie the edge you’ve found only exists when betting against 100% books…
Cheers
Martin Eastwood - December 13, 2013
Yes, I agree. I was undecided as to the best way of looking at it so ended up removing the over round as gave me the worst results so I went with the worst case scenario. If I leave the over round in which is probably more realistic then I actually got even better results