Introduction
It’s the end of the season so it’s time to review how the Eastwood Index performed over the year and how it compared with the bookmakers.
Ranked Probability Scores
One of the most important aspects to me is how accurate the forecasts were and I’ve assessed this using Ranked Probability Scores, as recommended by Constantinou and Fenton. I’ve discussed Ranked Probability Scores on the blog before but for people new to them they measure the difference between the forecasts and what really happened. Scores range between 0–1 and represent the amount of error in the predictions so lower Ranked Probability Scores are better and signify greater accuracy.
Comparison With Bookmakers
Looking at Figure 1 you can see that the Eastwood Index has consistently outperformed the bookmakers all season – and this isn’t just one bookmaker that the Eastwood Index has beaten but the combined knowledge of the industry as I’ve aggregated multiple bookmakers’ odds together and stripped out the overround to make the comparison as tough as possible.
Figure 1: Eastwood Index Vs Aggregated Bookmakers
Interestingly, the difference in accuracy seems to be greatest as both ends of the season. I expected the start of the season to be difficult to forecast as new teams have been promoted, players have been bought and sold, and managers may have changed clubs but the Eastwood Index seems to have coped with these variables better than the bookmakers’ odds have.
Over the course of the season the bookmakers’ forecasts improved until there was very little difference between them and the Eastwood Index but I was somewhat surprised to see how far out their accuracy drifted over the final few weeks of the season.
In theory these should be the easiest matches to forecast as we have the most information but in reality they can be tricky as team’s motivations change. For example, Manchester United have been playing their reserve goalkeeper so he gets enough appearances to earn his winners medal while Swansea’s players may as well have been on holiday since they won the league cup.
These changes seem to have thrown the bookmakers’ odds out quite noticeably while the Eastwood Index’s accuracy has remained constant. In fact, it suggests that bookmakers may be over-compensating for these apparent end-of-season effects as the Eastwood Index does not currently take them into account and has not struggled because of it.
Conclusions
Overall, I am pleased with the Eastwood Index’s debut season. I was slightly reticent to publish the forecasts at first in case the model did not hold up but it has remained accurate throughout the year. The next stage of its development is to identify any patterns as to where its forecasts differ from the bookmakers and how that could be combined with various staking strategies as well as looking at expanding to cover other leagues too.