Did The Eastwood Index Beat The Bookmakers?

Posted by Martin Eastwood May 21, 2013 19 Comments 3832 views

It’s the end of the season so it’s time to review how the Eastwood Index performed over the year and how it compared with the bookmakers.

Ranked Probability Scores

One of the most important aspects to me is how accurate the forecasts were and I’ve assessed this using Ranked Probability Scores, as recommended by Constantinou and Fenton.  I’ve discussed Ranked Probability Scores on the blog before but for people new to them they measure the difference between the forecasts and what really happened. Scores range between 0–1 and represent the amount of error in the predictions so lower Ranked Probability Scores are better and signify greater accuracy.

Comparison With Bookmakers

Looking at Figure 1 you can see that the Eastwood Index has consistently outperformed the bookmakers all season – and this isn’t just one bookmaker that the Eastwood Index has beaten but the combined knowledge of the industry as I’ve aggregated multiple bookmakers’ odds together and stripped out the overround to make the comparison as tough as possible.

Figure 1: Eastwood Index Vs Aggregated Bookmakers

Figure 1: Eastwood Index Vs Aggregated Bookmakers

Interestingly, the difference in accuracy seems to be greatest as both ends of the season. I expected the start of the season to be difficult to forecast as new teams have been promoted, players have been bought and sold, and managers may have changed clubs but the Eastwood Index seems to have coped with these variables better than the bookmakers’ odds have.

Over the course of the season the bookmakers’ forecasts improved until there was very little difference between them and the Eastwood Index but I was somewhat surprised to see how far out their accuracy drifted over the final few weeks of the season.

In theory these should be the easiest matches to forecast as we have the most information but in reality they can be tricky as team’s motivations change. For example, Manchester United have been playing their reserve goalkeeper so he gets enough appearances to earn his winners medal while Swansea’s players may as well have been on holiday since they won the league cup.

These changes seem to have thrown the bookmakers’ odds out quite noticeably while the Eastwood Index’s accuracy has remained constant. In fact, it suggests that bookmakers may be over-compensating for these apparent end-of-season effects as the Eastwood Index does not currently take them into account and has not struggled because of it.

Conclusions

Overall, I am pleased with the Eastwood Index’s debut season. I was slightly reticent to publish the forecasts at first in case the model did not hold up but it has remained accurate throughout the year. The next stage of its development is to identify any patterns as to where its forecasts differ from the bookmakers and how that could be combined with various staking strategies as well as looking at expanding to cover other leagues too.

About Martin Eastwood

Martin is football fan and data scienist. In his spare time he likes to combine the two and write about the mathematical analysis of football.

View all post by Martin Eastwood

There are 19 Comments

  1. George
    - May 21, 2013
      -   Reply

    Cool outcome. From my cursory looks into this, it is definitely possible, its just a case of trying to maximise any returns by only taking picks of a certain ratio etc. I’ve looked at other sports as well and sometimes find you can get a good number on a particular team for a period of weeks etc. Out of interest how are you translating %’s into home, away or draw (or are you assuming where the EI percentage is greater than the bookmakers % you are taking that result)? Things to look at going forward could be things like home field advantage (e.g. like the Clarke, Norman paper from 1995) or the effect of travel or fatigue. Good luck.

    • Martin Eastwood
      - May 21, 2013
        -   Reply

      Thanks George, that’s the tricky bit – how to convert from my %’s to winning staking strategy. My plan for the summer is to work on that further as I’ve got some ideas ready to test out for next season :)

  2. Alex
    - May 21, 2013
      -   Reply

    Okay – perhaps this is naive, but if you’ve demonstrated that your index has outperformed the bookies all season, why can’t you invest, say, $100/matchday and split the bets according to the outcomes predicted by the EI over the course of the season? You should win the majority of the bets and show some return for your money, no?

    • Martin Eastwood
      - May 22, 2013
        -   Reply

      Yes and I would make a profit over the season doing that. What I want to work out though is what would be the best way of splitting that $100/matchday between the matches to maximize the profit made, e.g. something like the Kelly Criterion.

      • Alex
        - May 23, 2013
          -   Reply

        Ahh, I see. This is *extremely* interesting stuff. I’ve been tinkering around with a Monte Carlo approach in Matlab (using data from Football Manager 2013) for predicting game results, but I haven’t had great – or even remotely convincing – success. Your stuff here is brilliant.

        • Martin Eastwood
          - May 23, 2013
            -   Reply

          Thanks Alex :)

          Is the data easy to extract from Football Manager 13? I’ve never touched it since the all Championship Manager days because it is too addictive but it could be an interesting data source…

          • Alex
            - June 6, 2013

            Yep! There’s a few editors available (just google for ‘em), it becomes pretty straightforward to pull data.

            Man, if you can show that you can outperform the bookies by even 4%, I think this would make a great alternative to savings accounts. Are you going to be publishing predictions for matchdays over the next season? I think putting in, say, $20 a matchday might be fun.

  3. George
    - May 23, 2013
      -   Reply

    Don’t know which source you use for data but I use football-data.co.uk, which is great for any of the European Leagues (including the Premiership). What’s really nice for doing this kind of thing, is they usually put stuff in an xls or csv, so you can just do your thing straight away. They also usually have a range of bookmaker odds (as I am expecting that certain bookmakers deal with “sharp”er customers than others (if you know what I mean) so I expect that you could probably find a bookmaker that the EI is more prone to exploiting (or perhaps that the EI for a particular team is more prone to exploring).

    • Martin Eastwood
      - May 23, 2013
        -   Reply

      Something else to add to my to-do list, which is the best bookmaker for using the EI with!

  4. amir
    - May 23, 2013
      -   Reply
    • amir
      - May 23, 2013
        -   Reply

      I see you already wrote another article stating Kelly Criterion…

    • Martin Eastwood
      - May 23, 2013
        -   Reply

      Thanks Amir that link looks really interesting!

  5. Pingback MLS: You Go Sit Down The Table… | The Shin Guardian

  6. - October 18, 2013
      -   Reply

    If I see this correctly, you introduced the EI in February 2013.

    Now you are comparing its predictions with the whole of the 2012/13 season.

    That is not really “beating” the bookmakers. To beat them, you need to make your predictions BEFORE the matches not after them.

    If you beat them again next year without changing your methods I’ll give you all the credit you deserve.

    • Martin Eastwood
      - October 18, 2013
        -   Reply

      Lars – I actually went back and recreated predictions for every match of the season based on just the data that would have been available at the point in time when the match was originally played.

  7. Nic
    - November 24, 2013
      -   Reply

    Thanks for the good read. It’s really interesting.

    When you compare against the bookmakers are you taking their opening odds? Because after the odds are published the fluctuations are due to general populace betting patterns.

    This could account for your sudden strong finish vis-a-vis the bookmakers as the public was betting more against academic results etc which then the bookmakers have to tweak the odds for.

    Another question. What’s the average spread a bookmaker keeps. To beat them do you mean you overcome the spread as well?

    • Martin Eastwood
      - November 25, 2013
        -   Reply

      Thanks Nic,

      The odds were taken from the football-data website. There were then normalised to account for the overround and aggregated IIRC. Not sure how much the average bookmaker keeps as the overound, charges etc vary so much between traditional highstreet bookmakers and online betting.

      Cheers,

      Martin

  8. - December 10, 2013
      -   Reply

    You state that you stripped out the bookmakers over-round in your comparison. Should you not be comparing your predictions against the bookmakers prices WITH over-round included?

    Otherwise is it not a little bit pointless (in betting terms) as you won’t be able to take prices with the over-round removed? ie the edge you’ve found only exists when betting against 100% books…

    Cheers

    • Martin Eastwood
      - December 13, 2013
        -   Reply

      Yes, I agree. I was undecided as to the best way of looking at it so ended up removing the over round as gave me the worst results so I went with the worst case scenario. If I leave the over round in which is probably more realistic then I actually got even better results

Write Your Comment

  • OI

    Two Questions: -Did you use two different samples …

  • Anonymous

    Thanks a lot! :) I need to brush up on my stats …

  • Martin Eastwood

    I split the shots into different bins by distance …

  • Anonymous

    If you could run me through the process or maybe s …

  • Anonymous

    Poisson distribution of course. My bad! …

Latest Tweets

  • Loading...