Blog List.

Backtesting PlayerRatings

Introduction

Following my last article discussing my PlayerRating model for quantifying footballers, JackIO challenged me to test the model by looking at what young players it recommended. Rather than looking at what players score highly now though, I thought it would be more interesting to go back in time and look at what players it rated ten years ago. This way we can see whether the PlayerRating model was accurate and whether these players actually went on to have successful careers or not.

A Decade Ago

Okay, let's go back a decade to April 2005. Tony Christie and Peter Kay were at number one in the music charts, Tony Blair and George Bush were at war with Iraq and Bolton Wanderers were five points away from the Champions League with just seven matches to play.

So who were the top players to sign back then? The table below shows the top twenty rated players aged under 21 on the 1st April 2005. I've also added in the player's values at that point according to transfermarkt, the peak value they reached, how many full international caps they've had to date and the team they were playing for at the time.

Player Nationality Age Team Value (£m) Peak (£m) Caps
Alexis Ruano Delgado Spain 19 Málaga 1.75 8.8 0
Wesley Sneijder Netherlands 20 AFC Ajax 10 39.6 113
Bastian Schweinsteiger Germany 20 Bayern Munich 6 35.2 109
Llorente Spain 20 Atlético Madrid 0.5 26.4 23
David Clarkson Scotland 19 Motherwell FC 0.15 0.8 2
Nigel de Jong Netherlands 20 AFC Ajax 4 18.5 80
Michael Rensing Germany 20 Bayern Munich 0.75 4 0
Gonzalo Rodríguez Argentina 20 Villarreal CF 3 12.3 6
Cristiano Ronaldo Portugal 20 Manchester United 20.1 105.6 119
Glen Johnson England 20 Chelsea FC 5 15 54
Christian Lell Germany 20 Bayern Munich 0.6 3.5 0
Javier Mascherano Argentina 20 River Plate 2 26.4 111
Iniesta Spain 20 Barcelona 11 61.6 102
Leighton Baines England 20 Wigan 1 20 30
Wayne Rooney England 19 Manchester United 27 57 103
Fernando Torres Spain 20 Atlético Madrid 22 44 110
Obafemi Martins Nigeria 20 Inter Milan 8.8 14 39
Andreas Ottl Germany 20 Bayern Munich 0.4 2.2 0
Piotr Trochowski Germany 20 Hamburger SV 0.75 8.8 35
Per Mertesacker Germany 20 Hannover 96 4.4 16.7 104

Reassuringly, of these 20 players 16 went on to receive full international caps for their countries and the overwhelming majority (if not all depending on your football geekiness) are recognizable names such as Ronaldo, Iniesta, Schweinsteiger, Sneijder etc. There's also a nice mixture of positions suggesting the model isn't necessarily biased to certain positions.

A somewhat surprise inclusion on the list is David Clarkson who's had somewhat of a journeyman career to date. Looking back at his playing history though he broke into Motherwell's first team shortly after his 17th birthday and went on to score 14 goals the following season, so his inclusion back in 2005 seems valid. I don't know enough about Scottish football though to know whether that was a fluke season but his time afterwards in English football was far from successful, culminating with him being part of the Bristol Rovers squad that was relegated to non-league football at the end of the 2013/2014 season. If anyone knows more about him I'd be intrigued to hear!

Another point worth highlighting is that all the players on the list are aged 19 or 20 and one reason for this is that minutes-played is an important factor in how the model determines a player's true rating - the more game time a player has then the more confidence the model has in its estimation of the player's true ability. Also, the majority of players in the top 20 are seemingly already at big teams by this age. It's alright if you're Chelsea and you have the finances to Hoover up all the young talent from other teams but for smaller teams looking to find promising players you've potentially missed the boat by that age.

Because of this, I also took a quick look at what players aged 18 or under the model recommended too. I fully expected these results to look worse because these players have likely had less game time meaning there is less data for the PlayerRating model to work with but even so the results still look promising.

Player Nationality Age Team Value (£m) Peak (£m) Caps
Vincent Kompany Belgium 18 RSC Anderlecht 10 30 60
Yoan Gouffran France 18 Caen 1 2.5 0
Aiden McGeady Ireland 18 Celtic 1 8 75
Tony McMahon England 18 Middlesbrough 0.04 1.3 0
Sergio Ramos Spain 18 Sevilla FC 6 40 126
Cesc Fàbregas Spain 17 Arsenal FC 6.6 48.4 94
Igor Akinfeev Russia 18 CSKA Moscow 1 17.6 78
Tom Huddlestone England 18 Derby County 0.3 10.6 4
Yoann Gourcuff France 18 Rennes 1 21.1 31
Fernando Gago Argentina 18 Boca Juniors 0 20 57
Quincy Owusu-Abeyie Ghana 18 Arsenal FC 0.5 2.5 28
Manuel Neuer Germany 18 Schalke 04 0.06 40 58
Christian Fuchs Austria 18 SV Mattersburg 0.18 6.6 65
Edin Džeko Bosnia-Herzegovina 18 Zeljeznicar 0 28.2 71
James Morrison Scotland 18 Middlesbrough 0.7 6.6 36
Ibrahim Afellay Netherlands 18 PSV Eindhoven 0.1 12.3 49
Ryan Babel Netherlands 18 AFC Ajax 1 15.2 43
Marco Motta Italy 18 Atalanta 2 6 1
Charles NZogbia France 18 Newcastle Utd 0.5 10.6 2
Stéphane Mbia Cameroon 18 Rennes 0.2 13.2 52

Anyone fancy signing an 18-year-old Manuel Neuer for £60k? How about Christian Fuchs for £180k? Tony McMahon for £40k? Who? To be fair to Tony McMahon, back in 2005 he was aged 18, had represented England at under-16, under-17 and under-19 level, captained Middlesbrough to the FA Youth championship, was playing regular first team football and had appeared in the UEFA Cup. It's an impressive start to anyone's career, and who knows what would have happened had he not knackered his knees and broken his leg from pretty much that point onwards.

I know the results of all this are certainly not perfect (or even scientific) and there are most likely some really good players who were ranked outside the top 20 and so haven't made this list (e.g a very young Lionel Messi at position 28) but even so the results still so far look feasible, especially considering the young age of the players. Had anyone taken a punt on this group of players then they could have put together a really exciting squad for a tiny price that would have made a hefty long term profit. Sounds good to me!

Conclusions

Okay, let's get one thing straight before my inbox is filled with people moaning - I'm not advocating teams should sign players based just on numbers let alone my PlayerRating model - I firmly believe football scouting should be a mixture of the qualitative and quantitative. Also, I know this article is somewhat subjective about what classes as successful / not successful and just because Transfermarkt says Manuel Neuer was worth £60k back in 2005 doesn't mean he was actually available for that price. I'm pretty sure Shalke would have twigged he was rather good at football by then and not wanted to sell him off quite so cheaply!!!

However, considering the paucity of data available for young players and the high attrition rate of young players making the grade I think the results of the PlayerRating model look really promising - I haven't tweaked anything here, these are the actual players the model would have rated back then based purely on the data available up until April 2005. I've still got lots more ideas for improving the PlayerRating system further but even so I'm really excited by the results so far.

Comments

Get In Touch!

Submit your comments below, and feel free to format them using MarkDown if you want. Comments typically take upto 24 hours to appear on the site and be answered so please be patient.

Thanks!

About

Pena.lt/y is a site dedicated to football analytics. You'll find lots of research, tutorials and examples on the blog and on GitHub.

Social Links

Get In Touch

You can contact pena.lt/y through the website here