penaltyblog

Why I Wrote My Own MCMC Sampler for Penaltyblog

2026-01-08T10:42:00+00:00

Introduction

A little while back, I made the difficult decision to remove the Bayesian goal models from penaltyblog. It wasn't a choice I took lightly, but many users were hitting a wall. The models relied on powerful libraries like Stan and PyMC for which the installation process was proving to be overly difficult. What should have been a simple modelling task often turned into wrestling with complex dependencies.

That frustration is precisely why I'm so excited to announce the latest release of penaltyblog. It now includes its very own native, dependency-free Bayesian engine, built from the ground up to make my Bayesian goal models accessible to users, without the installation headache.

Moving Beyond Point Estimates

The existing models in penaltyblog (like Dixon-Coles and Poisson) rely on Maximum Likelihood Estimation (MLE). MLE is great as it finds the single best set of parameters (Attack, Defense, Home Advantage) that explains the history of goals.

But football is gloriously unpredictable. Is Liverpool's attack strength exactly 1.45? Or is it somewhere between 1.30 and 1.60 depending on who's injured, who's in form, and whether the team bus got stuck in traffic?

MLE gives you a specific number. Bayesian inference gives you a distribution of possibilities. By understanding the range of likely strengths, we can better quantify the risk in our odds and predictions. It's the difference between saying "Liverpool will win 2-0" and "Liverpool will probably win, but here's the full range of possible outcomes and how likely each one is."

The Solution: Cythonized MCMC

To bring Bayesian modelling to penaltyblog without the previous dependency hell, I decided to write a bespoke MCMC (Markov Chain Monte Carlo) sampler from scratch using Cython.

Under the hood, it uses an Affine Invariant Ensemble Sampler with Differential Evolution moves. If that sounds scary, here's what it means in plain English:

It's lightning fast: The likelihood functions and the sampler are compiled to C. It leaves pure Python implementations in the dust.
It's robust: Football data is messy. This specific type of sampler excels at navigating the correlated parameters often found in sports models (like the relationship between a team's attack and their opponent's defense).
No bloat: It's built directly into the package and is designed for these specific use cases. pip install penaltyblog is all you need to get started.

How to use it

I've designed the API to mimic the existing MLE models, so you'll feel right at home. Here's how you fit a Bayesian Dixon-Coles model.

First, let's grab the data using the built-in scrapers:

import penaltyblog as pb

# Get Premier League data
df = pb.scrapers.FootballData("ENG Premier League", "2023-2024").get_fixtures()

# Calculate weights (optional, but recommended to weigh recent games higher)
weights = pb.models.dixon_coles_weights(df["date"], xi=0.001)

Next, we fit the model. Instead of DixonColesGoalModel, we use BayesianGoalModel:

from penaltyblog.models import BayesianGoalModel

# Initialize the model
model = BayesianGoalModel(
    df["fthg"],
    df["ftag"],
    df["team_home"],
    df["team_away"],
    weights=weights
)

# Fit using MCMC
# n_samples: How many steps to take
# burn: How many initial steps to throw away (warm-up)
model.fit(n_samples=2000, burn=1000)

A Quick Note on the Arguments

You might notice a few extra arguments in the .fit() function. Since MCMC works by "walking" around the probability landscape, we need to give it a little guidance:

burn: Think of this as the warm-up lap. The walkers start at random positions and need time to find the "sensible" parameter values. We discard these initial steps so they don't skew our results.
n_samples: This is how long we record the walkers after the warm-up. More samples mean a smoother, more accurate distribution, but it will take longer to run.
n_chains: Why rely on just one explorer? This tells the model to send out multiple independent walkers starting from different random locations. If they all end up describing the same probability landscape, we can trust the results. If they disagree (get stuck in different places), it’s a warning sign that our model hasn't converged.
thin: (Optional) If the walkers are moving very slowly, consecutive samples might look too similar. "Thinning" tells the model to only keep every 5th or 10th sample etc to ensure they remain independent.

Unlike the MLE models which finish instantly, this will take a moment as the "walkers" explore the parameter space. Grab a cup of tea, check the latest transfer rumors, and let the mathematics work its magic.

Diagnostics

In Bayesian analysis, we can't just trust the result blindly. We need to ensure our sampler converged properly. I've added helper functions to visualize the "trace" (the path the sampler took). Ideally, we want the traces to look like "fuzzy caterpillars," which indicates the model has explored the space well.

# Plot trace for specific parameters
model.plot_trace(params=["attack_Arsenal", "home_advantage"])

When analyzing these trace plots, we are looking for the classic 'fuzzy caterpillar' shape. The dense, rectangular block of noise is exactly what you want to see - it confirms that the model has converged and is confidently exploring the possible values for parameters like Arsenal's Attack Strength or Home Advantage.

The fact that the plot remains horizontal (not drifting up or down) and the lines are perfectly blended together proves that our independent sampling chains all agree on the answer, giving us a statistically sound result.

Prediction

Once we're happy with the convergence, prediction works exactly the same way as before. The difference is that penaltyblog is now calculating the probabilities across thousands of possible parameter combinations, giving you a marginalized probability that accounts for uncertainty.

probs = model.predict("Man City", "Arsenal")

print(f"Home Win: {probs.home_win}")
print(f"Draw: {probs.draw}")
print(f"Away Win: {probs.away_win}")

The Hierarchical Model

I've also added a HierarchicalBayesianGoalModel for those who want to take it a step further.

In the standard model, we usually clamp the team parameters so they average to zero. In a Hierarchical model, we treat the league's variance (how wide the gap is between the best and worst teams) as an unknown parameter to be learned.

Think of it this way: instead of just looking at how teams perform against each other, the model also learns about the "character" of the league itself. Is this a season with a dominant team running away with the title? Or is it a tightly contested battle where anyone can beat anyone?

The model learns two extra parameters: sigma_attack and sigma_defense.

High Sigma: The league is very unequal (think Man City vs. Sheffield Utd).
Low Sigma: The league is very competitive and tightly packed.

This approach allows the model to "shrink" estimates for teams with fewer games toward the league average, which is especially useful early in the season to prevent wild predictions based on tiny sample sizes.

from penaltyblog.models import HierarchicalBayesianGoalModel

h_model = HierarchicalBayesianGoalModel(
    df["fthg"], df["ftag"], df["team_home"], df["team_away"], weights=weights
)

h_model.fit()

Summary

This update was born out of my own frustration with setting up complex environments just to run simple sports models. Life's too short to spend it debugging C++ compilers when you could be analyzing football!

The new Bayesian capabilities allow you to:

Quantify Uncertainty: See the full distribution of team strengths, not just point estimates.
Avoid Installation Hell: No external dependencies required - just one simple pip install.
Fast: Powered by custom Cython MCMC sampler.

Ready to dive into Bayesian football modelling without the headaches? The update is available now.

pip install penaltyblog --upgrade

Happy modeling!

Introducing the Opta API Connector for matchflow

2025-11-30T19:30:00+00:00

Introduction

I'm excited to announce a new feature for matchflow: direct integration with the Stats Perform (Opta) Soccer API!

For a while now, matchflow (part of my penaltyblog package) has been a powerful tool for building data pipelines from local files. But what if you could start those pipelines directly from Stats Perform's Opta API?

That's exactly what this new addition enables. If you have access to the Stats Perform Opta API, you can now build lazy, powerful matchflow pipelines straight from their feeds. This update is all about letting you focus on your analysis, not on the data engineering "grunt work" that comes with using a complex API.

The "Why?": Handling the Hard Parts for You

Let's be honest: while the Opta API is an incredibly rich data source, it's not always the easiest to work with. You have to manage:

Authentication tokens
Building the correct URL for each of the 20+ feeds
Handling paginated endpoints (and knowing which ones are paginated)
Parsing the deeply nested, complex JSON responses
Flattening statistics and un-nesting event data

The new opta connector, available in penaltyblog.matchflow, handles that for you.

Lazy Execution: It builds a matchflow plan. No API calls are made until you run .collect(), .to_pandas(), or write to a file.
Automatic Pagination: Feeds like matches or venues are automatically paginated. You get a single, clean stream of all records, whether it takes 1 request or 100.
Smart Parsing: It automatically unnests and flattens data. For example, event data (MA3) is yielded one event at a time, and match stats (MA2) are yielded one player-stat or team-stat record at a time. It turns complex JSON into analysis-ready rows.

Getting Started: A Quick Example

First, let's set our Stats Perform credentials as environment variables:

export OPTA_AUTH_KEY="your_auth_key_here"
export OPTA_RT_MODE="your_rt_mode_here" # (e.g., "b")

Now, we can import the opta object and use it to build a Flow. Let's get a list of all active tournament calendars (Feed OT2).

from penaltyblog.matchflow import Flow

flow = (
    Flow.opta.tournament_calendars(status="active")
    .select("id", "competitionCode", "name")
)

flow.show()

competitionCode	id	name
ACO	ax1yf4nlzqpcji4j8epdgx3zl	Africa Cup of Nations Qualification
17Q	64fygrchlfuz3q4lc7k2ffj84	Africa U17 Cup of Nations Qualification
ACO	7dauoeun2gnkofl7f4y510s4f	Africa U20 Cup of Nations
20Q	4fht4nyqpp5dzzv6ucm057dp0	Africa U20 Cup of Nations Qualification
23Q	27zeqzs85uxv3eej1mhot6gic	Africa U23 Cup of Nations Qualification

The Real Power: Building a Pipeline

The true magic happens when you combine this source with matchflow's other methods. Let's build a common pipeline: getting all shots from a single match.

We'll use Feed MA3 (Match Events) and filter it down.

from penaltyblog.matchflow import Flow, where_opta_event

# The match we want to analyze
FIXTURE_UUID = "zhs8gg1hvcuqvhkk2itb54pg" 

# 1. Start the flow from the Opta MA3 (events) feed
# This is lazy! No data is downloaded yet.
event_flow = Flow.opta.events(fixture_uuid=FIXTURE_UUID)

# 2. Chain matchflow methods to build our pipeline
# We'll use the new 'where_opta_event' helper
shot_flow = event_flow.filter(
    where_opta_event(["Goal", "Attempt Saved", "Miss", "Post"])
)

# 3. Now, execute the full pipeline to 
# download and filter the data
shot_df = shot_flow.to_pandas()

# You now have a clean DataFrame of *only* the shot events
shot_df[['playerName', 'timeMin', 'timeSec', 'x', 'y', 'typeId']].head()

	playerName	timeMin	timeSec	x	y	typeId
0	H. Ekitiké	2	4	76.8	68.7	15
1	Mohamed Salah	3	39	85.6	31.3	15
2	V. van Dijk	4	27	91.4	50.5	13
3	A. Semenyo	5	55	91.1	45	13
4	Evanilson	9	44	94.9	41.1	13

This is what it's all about. You defined your source (opta.events) and your transformation (.filter), and matchflow handled the API call, JSON parsing, and filtering, delivering a clean DataFrame of shots.

Say Goodbye to "Magic Numbers"

You might have noticed in the example above that I filtered for event types like "Goal" and "Miss" directly. If you’ve worked with Opta data before, you know usually you have to memorize specific IDs to do this. You stare at your code wondering, "Is Event Type 15 a Goal or an Attempt Saved? Is Qualifier 1 a Long Ball or a Cross?"

If you are not careful, you end up with code full of "magic numbers" that is hard to read and hard to debug:

# The Old Way: Hard to read, hard to debug
shot_flow = flow.filter(
    lambda x: x['typeId'] in [13, 14, 15, 16]  # Wait, which ID is which?
)

I wanted to fix this developer experience. That's why I’ve included a comprehensive mapping of standard Opta definitions directly into matchflow.

In addition to where_opta_event, I've also added where_opta_qualifier. These helpers let you filter using human-readable names, handling the ID lookups behind the scenes.

Here is how you might filter for specific pass types without needing to look up a documentation PDF:

from penaltyblog.matchflow import where_opta_event, where_opta_qualifier

# The New Way: Readable and explicit
# Get all Passes that are specifically 'Long balls'
long_ball_flow = flow.filter(
    where_opta_event("Pass"),
    where_opta_qualifier("Long ball")
)

These helpers support list inputs for multiple events and value checks for qualifiers, making your analysis code self-documenting and much easier to share with colleagues.

Why matchflow? Cloud Storage & More

While the API integration is the headline feature of this update, it's worth remembering why matchflow exists in the first place. It isn't just a script to download data, it's a complete processing engine for football data.

Because matchflow is built on top of the excellent fsspec library, you aren't limited to saving your data locally. You can stream the API response directly into S3, Google Cloud Storage, or Azure Blob Storage.

This is huge for building data lakes or automated pipelines. You don't need to download the data to memory and then upload it as matchflow handles the data streaming for you.

# Stream Opta events directly to an S3 bucket
Flow.opta.events(fixture_uuid=FIXTURE_UUID) \
    .to_jsonl(
        "s3://my-football-data/raw/events.jsonl",
    )

And because matchflow is lazy, it optimizes your plan before execution. If you chain a .limit(10) to an API call, matchflow knows to stop processing as soon as it has 10 records, saving you time and compute resources.

The "Killer Feature": Joining Streams

If you have worked with football data before, you know the data is often fragmented. You get match events in one feed, but the player details are often in a completely different feed.

Traditionally, linking these together forces you to make a choice:

The RAM Hog: Load everything into Pandas DataFrames and merge them (risking memory crashes with large historical dumps).
The Hard Way: Write complex Python loops to manually look up player IDs as you iterate through events.

matchflow solves this with lazy joins. You can join two data streams just like you would in SQL or Pandas, but it remains a lazy iterator.

Pro Tip: Working with IP Whitelists

Stats Perform feeds are often IP-restricted, which can be difficult if you are working from a laptop or a dynamic cloud environment.

To solve this, matchflow supports standard Python proxy dictionaries. If you have a static IP proxy set up to match your whitelist, you can pass it directly to any opta method:

# Define your secure proxy
proxies = {
    "http": "http://user:pass@10.10.1.10:3128",
    "https": "http://user:pass@10.10.1.10:1080",
}

# Pass it into your flow
event_flow = Flow.opta.events(
    fixture_uuid=FIXTURE_UUID, 
    proxies=proxies
)

What Feeds Are Supported?

I've added support for a wide range of the most popular feeds, with more to come. You can now build flows from:

Tournament Feeds (OT2, MA0): tournament_calendars(), tournament_schedule()
Venue & Area Feeds (OT3, OT4): venues(), areas()
Match Feeds (MA1, MA2, MA3): matches(), match(), match_stats_player(), match_stats_team(), events()
Advanced Match Feeds (MA4, MA5): pass_matrix(), possession()
Player/Team Feeds (PE2, TM1, TM2, TM3, TM4): player_career(), teams(), team_standings(), squads(), player_season_stats(), team_season_stats()
And more! Including referees(), rankings(), injuries(), and transfers().

Conclusion

This latest update transforms matchflow from a file processing tool into an end-to-end solution for working with Opta feeds. By handling authentication, pagination, and complex JSON parsing for you, it removes the friction between you and the insights you're looking for.

Whether you are building a simple shot map or a complex, cloud-based data lake, the new Opta connector is designed to make your life easier.

This feature is available now in the latest version of penaltyblog. I’m excited to see what you build with it - if you have any feedback or run into issues, please let me know!

Shrinkage, Uncertainty, and Son Heung-min: Using Bayesian Methods to Identify Finishing Ability

2025-10-01T19:30:00+00:00

Introduction

In football analytics, we often turn to a simple metric: Goals minus Expected Goals (G-xG) to assess a player's finishing ability. It’s a decent first look, often used to argue that a player is "running hot" or getting lucky.

However, it has a fundamental flaw: it struggles to separate temporary overperformance from genuine, repeatable ability.

Imagine two players:

Player A scores 4 goals from 2.0 xG. His G-xG is +2.0.
Player B scores 20 goals from 18.0 xG. His G-xG is also +2.0.

Are they equally good finishers?

Our intuition says no. Player A might just be on a lucky hot streak over a handful of shots, while Player B has a more proven track record. The core problem is that simple metrics, such as G-xG, can't distinguish between repeatable skill and random luck. To do that, we need a better approach.

The Trouble with a Single Number

The issue with a simple metric like Goals - xG is that it hides the two most important factors in any analysis: context and confidence.

It Ignores Sample Size: A player who scores one goal from a 0.1 xG chance (+0.9 G-xG) looks like a world-class finisher. But with only one shot, we have almost no information. Simple metrics treat this the same as a player who consistently overperforms over hundreds of shots.
It Gives a False Sense of Precision: By providing a single number, G-xG suggests a definitive value for a player's skill. But in reality, there's a range of plausible skill levels for every player. A striker's +5.0 G-xG might be a true reflection of their talent, or it might be the high end of a lucky season where their "true" skill is closer to +2.0. We have no way to know how certain we are.

To truly measure finishing, we need a model that can think more like a human scout: it should be skeptical of small samples and express its confidence in its own conclusions.

A Better Approach: Thinking in Probabilities

Instead of relying on simple subtraction, we can get a much truer picture of finishing skill using a Bayesian hierarchical model. While that sounds complex, the idea behind it is intuitive.

The model assumes that most players are average finishers and requires significant evidence to believe a player is truly exceptional. It achieves this in two ways:

"Borrowing Strength" and Shrinkage: The model analyzes all players simultaneously and learns the skill distribution of the entire league. For players with little data, their skill estimate is "shrunk" towards this league average. A player with 2 goals from 3 shots won't be crowned a superstar as the model remains skeptical and keeps their estimate stable. This prevents outliers from topping the leaderboards by luck alone.
Quantifying Uncertainty: This is the biggest advantage. Instead of a single number, the model produces a full range of plausible skill levels for each player, known as a credible interval. We can finally move from "Player X is a +2.0 finisher" to "We are 94% certain that Player X is between a +1.5 and +2.5 finisher." This allows us to see not just what the model thinks a player's skill is, but how confident it is in that assessment.

To achieve this, the model intelligently isolates a player's individual talent. For every shot, it accounts for three key factors: the quality of the chance (the xG), the baseline difficulty of the competition (it's harder to score in the Premier League than in some other leagues), and finally, the unique skill of the player taking the shot. This final value - the player's individual contribution after all other factors are considered - is what we define as true finishing skill. We'll call this metric Finishing Skill Above Average (FSAA).

The Results: Ranking Europe's Elite Finishers

After running the model on hundreds of thousands of non-penalty shots from Europe’s top leagues, we get a stable, uncertainty-aware ranking of finishing skill.

The table below shows the top 10, a list dominated by players renowned for their clinical ability in front of goal. The players are ranked by the lower bound of their skill estimate (HDI 3%), which shows our minimum level of confidence in their ability. This means we are ranking them by their estimated 'worst-case' skill, not just their average estimate.

Player	FSAA	HDI 3%	HDI 97%	Prob > Average
Son Heung-Min	0.354	0.192	0.543	1.00000
Lionel Messi	0.292	0.151	0.425	1.00000
Antoine Griezmann	0.305	0.138	0.471	0.99950
James Rodríguez	0.324	0.108	0.555	0.99600
Harry Kane	0.247	0.101	0.381	0.99875
Kylian Mbappe-Lottin	0.246	0.097	0.392	0.99925
Kevin De Bruyne	0.256	0.076	0.456	0.99250
Iago Aspas	0.245	0.069	0.413	0.99725
Manolo Gabbiadini	0.266	0.061	0.476	0.98975
Dries Mertens	0.232	0.051	0.424	0.98800

Understanding the Rankings: What Do the Numbers Mean?

The table provides a rich picture of each player's finishing ability, far beyond a simple rank. Here’s a quick guide to interpreting the columns:

FSAA: This is the model's best estimate of a player's finishing skill. A positive number means they are an above-average finisher, while a negative number indicates a below-average one. The higher the value, the more goals a player adds compared to an average player taking the same shots.

HDI 3% / HDI 97%: This is our "confidence" range. We can be 94% certain that the player's true finishing skill lies between these two values. If the entire range is positive (above zero), as it is for all the players in the top 10, we can be highly confident that their performance is due to genuine skill, not just a lucky streak.

Prob > Average: This is the most direct measure of confidence. It tells us the probability that a player is a better-than-average finisher. For the truly elite, this number is very close to 100%. For example, the model is 100% certain that both Son Heung-Min and Lionel Messi are above-average finishers.

Sense Check: The Elite (Lionel Messi)

The model ranks Son Heung-Min and Lionel Messi as the top two finishers, a result that makes perfect sense. Both are famous for consistently outperforming their expected goals. For Messi, the model is 100% certain he is an above-average finisher, which I don't think many people would disagree with.

Sense Check: The Not-so Elite (Jesús Navas)

To show the model works at both ends of the spectrum, consider Jesús Navas. Famous for his hard work but not his finishing, the model ranks him in the bottom 10 of the 7,500+ players analysed by the model. It is 88% certain he is a below-average finisher, confirming that the model's rankings align with reality. Here's the rest of the bottom 10 finishers.

Player	FSAA	HDI 3%	HDI 97%	Prob > Average
Rodrigo Palacio	-0.178	-0.405	0.045	0.06750
Jesús Navas	-0.158	-0.406	0.105	0.12225
Dominic Calvert-Lewin	-0.199	-0.409	0.006	0.03575
Giampaolo Pazzini	-0.159	-0.412	0.101	0.12350
Florian Sotoca	-0.189	-0.413	0.046	0.06050
Keane Lewis-Potter	-0.138	-0.413	0.121	0.17050
Emmanuel Rivière	-0.152	-0.414	0.093	0.13200
Hans Hateboer	-0.160	-0.416	0.092	0.11475
Alexander Djiku	-0.152	-0.419	0.100	0.13075
Mijat Gacinovic	-0.167	-0.426	0.088	0.10725

Putting It Into Practice

So, what does a FSAA of +0.292 for Lionel Messi actually mean?

It's an estimate of Messi's finishing skill compared to an "average" finisher on a log-odds scale, but we can translate it into real-world impact. For a standard shot with a 10% chance of being a goal (0.10 xG), Messi's skill increases that probability to nearly 13%.

While an absolute increase of 3% may not sound like a lot, it's a nearly 30% relative increase in the likelihood of scoring from a given shot compared with an "average" player.

Over the course of 100 of these shots, an average player would score 10 goals. Messi would be expected to score 13. His elite finishing ability creates 3 extra goals from the exact same set of chances.

Practical Applications for Clubs

This finishing skill model offers several advantages for professional clubs over traditional metrics.

In recruitment, scouts can identify undervalued strikers who consistently outperform their Expected Goals but haven't yet caught the market's attention, while avoiding overpaying for players on unsustainable hot streaks.

The uncertainty intervals help clubs assess risk - a striker with a wide credible interval might represent a bigger gamble, while one with tighter intervals represents a safer investment.

The model also supports contract negotiations by providing objective evidence of a player's true finishing ability, separate from temporary form or lucky seasons.

Perhaps most valuably, the approach helps clubs avoid the costly mistake of building their attack around a player whose apparent finishing prowess is actually just statistical noise. Looking at you Dominic Calvert-Lewin.

Methodology Details

Data for the analyses were collected for non-penalty shots for the Premier League, Ligue 1, Serie A, Budesliga 1, La Liga for seasons 2014/2015 to 2025/2026.

The rankings were produced by a multilevel Bayesian logistic regression. The model included crossed random effects for player and competition to isolate finishing skill from league-wide differences. Priors were chosen to be weakly informative, and inference was performed using a NUTS MCMC sampler with multiple chains. Convergence was confirmed by ensuring the r_hat statistic for all parameters was below 1.01.

Conclusion

For years, Goals - xG has been the default metric for finishing skill, but it's a noisy measure that can be misleading.

As any clichéd football pundit will tell you, "form is temporary, class is permanent" - but most finishing metrics struggle to separate the two. By using Bayesian approaches, we can build models that properly account for sample size and quantify uncertainty, giving us a much truer and more stable picture of player ability.

This model provides more than just a ranking; it offers objective evidence of a player's true finishing ability backed by statistical confidence. We can now quantify exactly how much better one finisher is than another - Messi's +0.292 skill creates 3 extra goals per 100 shots compared to an average player, while Son's +0.354 creates nearly 4. The credible intervals tell us precisely how confident we should be in these assessments.

The results speak for themselves, identifying a list of world-class players whose elite finishing talent is not just a lucky streak but a genuine, repeatable skill. Bayesian methods offer a robust way to evaluate one of football's most crucial talents.

From Biased Odds to Fair Probabilities: Removing the Bookmaker's Overround

2025-09-14T19:30:00+00:00

Introduction

If you've ever converted betting odds to probabilities, you've likely noticed they sum to a value greater than 100%. This surplus is the bookmaker's margin, often called the "overround" or "vig", and it's how they guarantee a profit. For any serious quantitative analysis, removing this margin to find the "true" underlying probabilities is a critical first step.

The challenge, however, is that there is no single, universally agreed-upon way to remove this margin. The process depends on our assumptions about how the bookmaker has applied it. Is it distributed evenly across all outcomes, or is it weighted towards the longshots?

To help analysts tackle this problem with greater flexibility, I've updated to the implied odds functions in the penaltyblog package. This article explores the new features and takes a deep dive into the different theoretical models you can now use.

What's New in penaltyblog?

The core of this update is a move to a more modern and robust architecture. I've have replaced standard dictionaries with type-safe dataclasses, namely OddsInput and ImpliedProbabilities.

This change provides several key benefits for users:

Clarity and Safety: The new structure makes the API self-documenting. You know exactly what data to provide and what the function will return, allowing tools like mypy to catch potential errors before you run your code.
Ease of Use: With type-safe objects, IDEs can provide better auto-completion, making the library more intuitive to work with.

All functionality is now consolidated into a single, powerful function: penaltyblog.implied.calculate_implied(). This serves as a unified entry point for all the supported methods.

A Deep Dive into Implied Probability Methods

The fundamental question when removing the overround is: how is the margin distributed? To illustrate the different approaches, we'll use a typical 1X2 football market with the following decimal odds: Home: 2.7, Draw: 2.3, Away: 4.4.

The unadjusted probabilities, calculated as $1/odds$, are:

$1/2.7=0.370$
$1/2.3=0.434$
$1/4.4=0.227$

The sum of these is $1.032$, which gives us a bookmaker's margin of $3.2\%$. Each method below attempts to reduce this sum to exactly $1.0$ based on a different set of assumptions.

The Basic Approaches

These two methods are the most common and serve as excellent baselines.

Multiplicative ("multiplicative"): This is the simplest and most widely used method. It assumes the margin is distributed proportionally across all outcomes and normalises the raw probabilities by dividing each one by their sum.

$$p_i = \frac{1/o_i}{\sum_{j=1}^{n} 1/o_j}$$

Additive ("additive"): This method assumes the margin is an equal, fixed amount subtracted from each outcome's raw probability.

$$p_i = \frac{1}{o_i} - \frac{M}{n}$$

where $M$ is the total margin and $n$ is the number of outcomes.

Iterative Approaches

These methods are more complex and often require solving for a parameter that normalises the probability distribution.

Power ("power"): This method raises the raw probabilities to a power, k, which is solved for iteratively. It provides more flexibility than the basic methods in modelling how the margin is applied.

$$p_i = \left(\frac{1}{o_i}\right)^k$$

Shin ("shin"): Developed by Hyun Song Shin, this method is derived from a model that assumes a mix of informed and uninformed bettors in the market. It solves for a parameter, z, which can be interpreted as the proportion of uninformed money. It is a popular method in academic literature for modelling the favourite-longshot bias.

Odds Ratio ("odds_ratio"): Proposed by Keith Cheung, this method models the relationship between the bookmaker's probabilities and the true probabilities using an odds ratio formulation, solving for a constant c.

Differential Margin Weighting ("differential_margin_weighting"): This approach, popularised by Joseph Buchdahl, works on the assumption that the margin is applied proportionally to the odds themselves, which results in a greater proportion of the margin being applied to longshots.

$$\text{fair_odds}_i = \frac{n \cdot o_i}{n - (M \cdot o_i)}$$

Logarithmic ("logarithmic"): This method converts the bookmaker's probabilities into "log-odds" space and assumes the margin is applied as an equal subtraction from each outcome. It solves for a single constant, c, that is removed from all log-odds values to ensure the final probabilities sum to 1.0.

Putting It All Together: A Practical Example

The new calculate_implied function makes it easy to apply and compare these methods.

import penaltyblog as pb

# Our example odds for a Home, Draw, Away market
odds = [2.7, 2.3, 4.4]
market_names = ["Home", "Draw", "Away"]

# Calculate using the default Multiplicative method
result_mult = pb.implied.calculate_implied(
    odds,
    market_names=market_names
)
print(f"Multiplicative: {result_mult.probabilities}")
# >> Multiplicative: {'Home': 0.3587, 'Draw': 0.4211, 'Away': 0.2201}

# Now try Shin's method
result_shin = pb.implied.calculate_implied(
    odds,
    method="shin",
    market_names=market_names
)
print(f"Shin: {result_shin.probabilities}")
# >> Shin: {'Home': 0.3593, 'Draw': 0.4232, 'Away': 0.2174}

An Empirical Test: Which Method is Most Accurate?

While the theoretical differences are interesting, the crucial question is whether one method produces more accurate probabilities than another. To test this, we can take a historical dataset of odds and outcomes and evaluate the accuracy of each method using a proper scoring rule.

For this analysis, I used the closing odds for all 380 matches from the 2024/25 English Premier League season. For each match, I calculated the implied 1X2 probabilities for Bet365's odds using every method. I then scored these probabilities against the actual match outcome (Home Win, Draw, or Away Win) using the Ranked Probability Score (RPS).

The RPS is a measure of the distance between a probabilistic forecast and the observed outcome, where a lower score indicates a more accurate forecast.

import penaltyblog as pb
import pandas as pd

# Get the fixture data for the 2024-2025 Premier League season
df = pb.scrapers.FootballData("ENG Premier League", "2024-2025").get_fixtures()

# Map the full time result to a numerical outcome (0 for Home, 1 for Draw, 2 for Away)
df["outcome"] = df["ftr"].map({"H": 0, "D": 1, "A": 2})

# Define a function to normalise the odds using a specific method
def normalise(x, method):
    """
    Calculates implied probabilities from betting odds using a specified method.

    Args:
        x (pd.Series): A row of the DataFrame containing 'b365_h', 'b365_d', 'b365_a' columns.
        method (str): The normalisation method to use.

    Returns:
        pd.Series: A pandas Series with the normalised probabilities for home, draw, and away outcomes.
    """
    res = pb.implied.calculate_implied(
        x[["b365_h", "b365_d", "b365_a"]],
        method=method
    )
    # Return the probabilities as a Series with named columns
    return pd.Series(res.probabilities, index=['prob_h', 'prob_d', 'prob_a'])

# Define the list of normalisation methods to compare
methods = [
    "additive",
    "multiplicative",
    "power",
    "differential_margin_weighting",
    "shin",
    "odds_ratio",
    "logarithmic"
]

results = {}
# Iterate through each normalisation method
for method in methods:
    # Apply the normalise function to each row of the DataFrame to get the probabilities
    norm_probs = df.apply(normalise, axis=1, args=(method,))

    # Calculate the average Ranked Probability Score (RPS) for the current method
    rps = pb.metrics.rps_average(norm_probs[["prob_h", "prob_d", "prob_a"]], df["outcome"])

    # Store the rounded RPS value in the results dictionary
    results[method] = round(rps, 4)

# Print the results
print(results)

Results

The results show a tight race with a compelling conclusion. While the simple multiplicative method was technically the most accurate with the lowest RPS of 0.19724, the differences among the top methods are negligible.

The more theoretically grounded Odds Ratio and Logarithmic methods were virtually indistinguishable in performance, tying for a close second place. This suggests that for a highly efficient market like the English Premier League, several different models for how bookmakers apply their margin can produce similarly accurate probabilities.

Method	RPS
multiplicative	0.19724
logarithmic	0.19730
odds_ratio	0.19730
shin	0.19731
additive	0.19736
differential_margin_weighting	0.19736
power	0.19739

Conclusion

Removing the bookmaker's margin is a foundational step in quantitative sports analysis. With the latest update to penaltyblog, analysts now have a comprehensive and easy-to-use toolkit to perform this task with greater precision.

Penaltyblog v1.5.0: Faster Models, Smarter Queries, and a Sharper Edge

2025-08-14T19:30:00+00:00

Introduction

The latest release of the penaltyblog Python package brings a major upgrade to your football analytics toolkit.

Highlights include:

Interactive plotting for rich, dynamic pitch visualisations.
Faster goal-model fitting with improved stability.
More accurate goal-expectancy estimation.
Powerful Flow query tools for streamlined data access.

Pitch - The Interactive Football Plotting Library

The standout feature in this release is the brand-new Pitch plotting API - a fully interactive, Plotly-powered framework for building rich football pitch visualisations in Python.

Key capabilities:

Multiple built-in pitch dimensions and themes.
Horizontal or vertical layouts.
Flexible view modes for zooming into specific areas.
Layering of scatter points, heatmaps, kernel-density surfaces, comets, arrows, and more.
Custom hover tooltips, colour schemes, opacity, and orientation controls.

You can:

Display charts inline in Jupyter notebooks.
Save as static images.
Export as standalone HTML for embedding anywhere.

In short, it’s a single, versatile tool for producing publication-ready, explorable football data visuals.

Example: Shot Map

The example below uses the new Pitch API to visualise every shot taken by Liverpool in match ID 22912 from StatsBomb.

Process:

Data is pulled directly from StatsBomb via the Flow API.
We filter for Liverpool’s shots.
We render them as an interactive Plotly chart.

Each marker shows a shot’s location, with hover tooltips displaying the player’s name and exact coordinates.

from penaltyblog.viz import Pitch
from penaltyblog.matchflow import Flow, where_equals, get_field


# 1) Pull and prep the data with Flow
def fmt(v):  # stringifier (avoids None issues)
    return "" if v is None else str(v)

flow = (
    Flow
    .statsbomb
    .events(22912, optimize=True)
    .filter(where_equals("type.name", "Shot"))
    .filter(where_equals("team.name", "Liverpool"))
    .assign(
        hover_text=lambda x: f"{fmt(get_field(x,'player.name'))}: "
                             f"({fmt(get_field(x,'location.0'))}, "
                             f"{fmt(get_field(x,'location.1'))})"
    )
)

# 2) Create the pitch
pitch = Pitch(
    provider="statsbomb",
    orientation="horizontal",
    view="full",
    theme="night",
    show_axis=False,
    show_legend=False,
    title="Liverpool shots (StatsBomb 22912)",
    subtitle="Hover for player + shot location",
)

# 3) Plot the shot map
pitch.plot_scatter(flow, "location.0", "location.1", hover="hover_text")

pitch.show()

Example: Layering Plots

One of the most powerful features of the new Pitch API is layering - the ability to stack multiple plot types in a single interactive view.

You can:

Start with a clean base pitch.
Add scatter plots for individual events.
Overlay heatmaps or kernel density estimates for spatial intensity.
Draw arrows to indicate passes or runs.
Use “comet” trails to visualise player or ball movement.

Each layer:

Is fully customisable in colour, opacity, and tooltip content.
Can be shown, hidden, or reordered at any time.
Works seamlessly together to create rich, multi-dimensional visualisations.

import numpy as np
import pandas as pd
from penaltyblog.viz import Pitch

# 1) Pull and prep the data with Flow
flow = (
    Flow
    .statsbomb
    .events(22912, optimize=True)
    .filter(where_equals("type.name", "Pass"))
    .filter(where_equals("team.name", "Liverpool"))
    .assign(hover_text=lambda x: get_field(x, "player.name") + ": ")
)

# 2) Create the pitch
pitch = Pitch(
    provider="statsbomb",
    orientation="horizontal",
    view="full",
    theme="night",
    show_axis=False,
    show_legend=False,
    title="Liverpool Passes (StatsBomb 22912)",
    subtitle="Hover for player + location",    
)

# 3) Plot a smooth KDE (continuous) with fine grid
pitch.plot_heatmap(
    flow,
    x="location.0",
    y="location.1",
    show_colorbar=True,
    colorscale="Viridis",
    opacity=0.75,
)

# 4) Overlay the raw points 
pitch.plot_scatter(
    flow,
    x="location.0",
    y="location.1",    
    size=4,
    color="white",
    hover="hover_text"
)

# 5) Show the result
pitch.show()

Pitch Documentation

You can explore the full range of Pitch features in the penaltyblog documentation.

This is the first release of Pitch, so there may be a few edge cases I haven’t encountered yet. If you run into any unexpected behavior, please:

Open an issue on Github
Or send me a message directly via the blog

Your feedback will help refine Pitch and shape future updates.

Blazingly Fast Goal Models

Goal model fitting in penaltyblog is now 5–10x faster thanks to two major changes:

Analytical Jacobian gradients are now used during optimisation, replacing slower numerical approximations.
More core routines are implemented in Cython for low-level performance.

With exact gradient information, the optimiser converges more quickly and with greater numerical stability. In practice, you can fit models on large datasets in a fraction of the time, without sacrificing the accuracy or robustness of the original implementation.

More Control Over Optimisation

The updated fitting API also exposes a minimizer_options parameter, allowing you to pass arguments directly to scipy.optimize.minimize.

This means you can:

Increase maxiter for tricky fits.
Tighten gtol or ftol for higher precision.
Fine-tune convergence settings without modifying library code.

Combined with the new Cython-powered gradients, this flexibility makes it easier to adapt training to different datasets, convergence requirements, and performance constraints.

Example: Dixon–Coles Model with Custom Optimisation

import penaltyblog as pb

model = pb.models.DixonColesGoalModel(gh, ga, th, ta, weights=weights)

model.fit(
    use_gradient=True,                # optional; can be False for back-compat
    minimizer_options={               # optional; passes to `scipy.optimize.minimize`
        "maxiter": 3000,              # more iterations if needed
        "gtol": 1e-8,                 # gradient tolerance
        "ftol": 1e-9,                 # function tolerance
    }
)

Improved `FootballProbabilityGrid`

Every penaltyblog goals model returns a FootballProbabilityGrid from .predict(). It contains the full scoreline probability matrix for a match and makes it easy to calculate market-ready probabilities.

The latest update adds normalization, expanded markets, better Asian Handicap handling, and richer representation, all while maintaining backwards compatibility.

The new implementation includes:

1. Optional Normalization

You can now pass normalize=True (default) to .predict() to ensure the probability grid sums to exactly 1.0.

Why it matters:

Without normalization: Preserves the true Poisson/Dixon–Coles mass beyond max_goals (purist approach), but markets like 1X2 + totals may not sum exactly to 1.0.
With normalization: Rescales probabilities so all markets are internally consistent - important for pricing, trading, and backtesting.

For a more detailed discussion, see the documentation.

2. Expanded Market Calculations

The grid now computes more markets out of the box:

1X2: home_win, draw, away_win
Convenience: home_draw_away → [home_win, draw, away_win]
Both Teams to Score (BTTS): btts_yes, btts_no
Totals: total_goals(over_under, strike) for any goal line (e.g. 2.5, 3.0)
Asian Handicap: asian_handicap and asian_handicap_probs with push handling

3. Push-Aware Asian Handicap Calculations

The updated asian_handicap method correctly handles:

Full win
Half win / Half loss
Push (stake returned)

It supports quarter, half, and full goal handicaps, making it possible to price complex AH markets directly from the grid — no manual matrix work required.

4. Richer Representation

Calling repr() now shows:

Model name
Home/Away goal expectations
Core 1X2 probabilities

Great for quick inspection in interactive sessions.

5. Backwards Compatibility

All previous methods still work.
normalize defaults to True for consistency, but you can disable it for raw probabilities.

Example: Predicting & Using the Grid

# Fit your goals model
model.fit()
pred = model.predict("Arsenal", "Chelsea", max_goals=15, normalize=True)

# Core markets (1x2)
print("P(Home win), P(Draw), P(Away win):", pred.home_draw_away)
print("P(Home win):", pred.home_win)
print("P(Draw):", pred.draw)
print("P(Away win):", pred.away_win)

# Goal expectancy
print("Home xG:", pred.home_goal_expectation)
print("Away xG:", pred.away_goal_expectation)

# Both teams to score
print("BTTS (Yes):", pred.btts_yes)
print("BTTS (No):", pred.btts_no)

# Totals
u, p, o = pred.totals(2.0)
print("Totals 2.0  -> Under, Push, Over:", (u, p, o))

u, p, o = pred.totals(2.5)
print("Totals 2.5  -> Under, Push, Over:", (u, p, o))

print("P(Over 2.5):", pred.total_goals("over", 2.5))

# Asian Handicaps
print("AH Home -0.5  (win prob only):", pred.asian_handicap("home", -0.5))
print("AH Home -0.25 (Win/Push/Lose):", pred.asian_handicap_probs("home", -0.25))
print("AH Away +1.0  (Win/Push/Lose):", pred.asian_handicap_probs("away", +1.0))

# Double chance and DNB
print("Double chance 1X:", pred.double_chance_1x)
print("Double chance X2:", pred.double_chance_x2)
print("Double chance 12:", pred.double_chance_12)
print("DNB Home (conditional win prob):", pred.draw_no_bet_home)
print("DNB Away (conditional win prob):", pred.draw_no_bet_away)

# Exact scores and distributions
print("P(Exact score 2-1):", pred.exact_score(2, 1))
print("Home goal distribution (P(H=k)):", pred.home_goal_distribution())
print("Away goal distribution (P(A=k)):", pred.away_goal_distribution())
print("Total goals distribution (P(T=k)):", pred.total_goals_distribution())

With the improved FootballProbabilityGrid, you can go from a fitted model to fully priced betting markets in one step - no direct NumPy work required.

Flow Query DSL

Flow.query(expr) lets you filter records with a compact, readable string expression. It’s safe (parsed via Python’s AST, not eval), validated, and compiled into an efficient predicate.

Use it to:

Prototype filters quickly.
Keep pipelines readable.
Let end users define filters without writing Python.

How It Works

The query string is parsed into an AST.
The AST is validated for safety.
It’s compiled into a fast predicate function.
Variables from your caller’s local scope can be injected with @var.

Supported syntax

Comparisons

==, !=, >, >=, <, <=

Chained comparisons are supported and expanded internally (e.g., 0 <= goals <= 5).

flow.query("goals_home >= 2")
flow.query("0 <= goals_home <= 5")

Logical operators

and, or, not (use parentheses to control precedence)

flow.query("home_team == 'Liverpool' and goals_home > goals_away")
flow.query("not (home_team == 'Arsenal' or away_team == 'Arsenal')")

Field access (dot notation)

Access nested fields with . (resolved via get_field):

flow.query("venue.city == 'London'")
flow.query("player.stats.minutes >= 60")

Membership

in, not in - note that field must be on the left-hand side

flow.query("home_team in ['Chelsea', 'Tottenham']")
flow.query("league not in ['Premier League', 'La Liga']")
# "Man City" in home_team  ← not currently supported

NULL / missing checks

Identity comparisons with None:

flow.query("player.injury_status is None")
flow.query("player.injury_status is not None")

String transforms (inside comparisons)

len(x)
.lower()
.upper()

flow.query("home_team.lower() == 'manchester united'")
flow.query("len(home_team) < 8")

These must be used in a comparison (e.g., field.lower() == 'x'). Standalone field.lower() as a predicate will raise an error.

Predicate-style string methods (standalone)

.contains(substring)
.startswith(prefix)
.endswith(suffix)
.regex(pattern, flags)

flow.query("home_team.contains('united')")
flow.query("away_team.startswith('West')")
flow.query("player.name.regex('^Mo')")

Regex flags

Pass flags from the re module via a local variable:

import re

pattern = r"liverpool"
flags = re.IGNORECASE
flow.query("home_team.regex(@pattern, @flags)")

Variables from Python (`@var`)

Inject values from your local scope with @var:

team = "Liverpool"
cutoff = date(2023, 1, 1)

flow.query("home_team == @team and match_date >= @cutoff")

Variables must exist in the local namespace where you call .query(...).

Date & datetime literals

Create date/datetime values inside the expression:

flow.query("match_date > date(2024, 6, 30)")
flow.query("kickoff >= datetime(2025, 8, 11, 19, 45)")

Supported literal constructors: date(Y, M, D), datetime(Y, M, D, h, m, s).

Common Patterns

flow.query("""
    league == 'NLD Eredivisie'
    and season == '2024-2025'
    and total_goals >= 3
""")  # NOTE: arithmetic isn't supported directly; see tip below

# Use an assigned field for arithmetic first:
flow.assign(total_goals=lambda r: r["goals_home"] + r["goals_away"]) \
    .query("total_goals >= 3")

# Team in a rolling window (with local variable)
from datetime import date

start = date(2025, 6, 1)
flow.query("(team_home == 'PSV' or team_away == 'PSV') and match_date >= @start")

# Regex with flags
import re

pattern = r"^[A-Z][a-z]+$"
flags = re.IGNORECASE
flow.query("player.name.regex(@pattern, @flags)")

Performance notes

Each call compiles the expression once, then applies the predicate efficiently.
Filtering is fastest when you reduce nested lookups early (e.g., call .flatten() if appropriate).
Use .assign(...) to precompute values you’ll filter on (e.g., total_goals), then query on that key.

Gotchas & limitations

Arithmetic inside queries is not currently supported.
Do the math in .assign(...) or .map(...), then filter on the derived field.
For in / not in, the field must be on the left.
.lower() / .upper() / len() must be used within a comparison.
@var substitution uses the caller’s local variables only.
Only the following literal functions are currently supported inside queries: date(...), datetime(...).

Quick reference

Comparisons: == != > >= < <= (chained comparisons allowed)
Logic: and · or · not
Membership: in · not in (field on LHS)
Nulls: is None · is not None
Transforms (in comparisons): len(x) · .lower()· .upper()
Predicates: .contains() · .startswith() · .endswith()· .regex()
Literals: date(Y, M, D) · datetime(Y, M, D, h, m, s)
Variables: @var (from caller’s locals)

Conclusions

v1.5.0 is all about speed, clarity, and control.

From the new Flow.query DSL for filtering with clean, Pythonic expressions, to the upgraded FootballProbabilityGrid that’s market-ready out of the box, to the Pitch API for building rich, interactive visualisations - every change in this release helps you move from raw data to reliable, actionable insights faster than ever.

Whether you’re exploring match data, pricing markets, or building visuals for analysis, penaltyblog v1.5.0 gives you the tools to work smarter and faster.

How Accurate Are Soccer Odds? A Data Dive into 250 Million Betting Lines

2025-07-16T19:30:00+00:00

Introduction

The global soccer betting market is a high-volume predictive system, where bookmakers generate and update millions of probability estimates in real time. But how accurate are these predictions and how do different bookmakers stack up against each other?

To explore these questions, I’ve collected a dataset of over 250 million betting lines from 1.25 million soccer fixtures worldwide, spanning multiple years, leagues, and market types. This gives us a unique dataset to evaluate bookmaker performance on a large scale.

For this article, I’ll be narrowing the focus to just the major European leagues and analyzing the moneyline (1X2) markets - the outcome of the match: home win, draw, or away win.

We'll be digging into:

Overrounds and how bookmakers build in margin
Bookmaker accuracy using Ranked Probability Scores (RPS)
Calibration curves to test how well odds translate to real-world outcomes

What Is the Overround?

In any betting market, the overround is the bookmaker’s built-in profit margin. This is the amount by which the implied probabilities of all outcomes exceed 100%.

For example, in a simple 3-way soccer market (Home Win / Draw / Away Win), the odds might be:

Home Win: 2.00
Draw: 3.60
Away Win: 3.60

To convert these odds into implied probabilities, we take the inverse:

Home Win: 1 / 2.00 = 0.500
Draw: 1 / 3.60 ≈ 0.278
Away Win: 1 / 3.60 ≈ 0.278
Total: 0.500 + 0.278 + 0.278 = 1.056, or 105.6%

That extra 5.6% is the overround, or the bookmaker’s edge. In a perfectly fair market with no margin, the total implied probabilities would sum to 100%.

Overrounds by Competition and Season

The table below shows average overrounds for moneyline (1X2) markets across the major European leagues, broken down by season. Several patterns stand out immediately:

Top-tier leagues like the Premier League, La Liga, Serie A, and the Bundesliga consistently have the lowest overrounds - typically between 4.1% and 5.1%. These markets are the most competitive, liquid, and efficient, so bookmakers tend to apply tighter margins.

Second divisions and lower leagues, such as League One, Serie B, or La Liga 2, often feature significantly higher overrounds, often pushing 7% or more. These markets may be less efficient and attract less volume, giving bookmakers more pricing power.

Also, there’s been a noticeable upward trend in overrounds over time across most competitions. For example:

The Premier League has crept from 4.12% to 4.65% over five seasons.
Serie B rose from 6.51% to 7.18%.
La Liga 2 shows a particularly sharp increase: 6.77% to 7.59%.

Season	2020/2021	2021/2022	2022/2023	2023/2024	2024/2025
Belgium: Jupiler League	6.04	6.15	6.14	6.34	6.43
England: Championship	5.06	5.15	5.43	5.32	5.63
England: League One	6.27	6.25	6.45	6.51	7.00
England: League Two	6.32	6.35	6.65	6.61	6.94
England: Premier League	4.12	4.21	4.31	4.36	4.65
France: Ligue 1	5.12	5.06	5.03	5.11	5.19
France: Ligue 2	6.83	6.87	6.90	7.08	7.10
Germany: 2. Bundesliga	5.98	5.95	6.28	6.39	6.57
Germany: Bundesliga	4.72	4.63	4.71	4.76	5.02
Italy: Serie A	4.74	4.72	4.94	5.02	5.12
Italy: Serie B	6.51	6.50	6.55	6.76	7.18
Netherlands: Eredivisie	5.58	5.64	5.81	5.64	5.90
Portugal: Liga Portugal	6.05	6.08	5.96	5.72	6.19
Spain: La Liga	4.53	4.56	4.72	4.76	4.95
Spain: La Liga 2	6.77	6.86	7.07	7.11	7.59

Table 1: Overrounds by Competition and Season

Premier League Overrounds by Bookmaker

Zooming in on the English Premier League, we can see clear differences in how various bookmakers price their moneyline markets. Table two below shows average overrounds for each bookmaker over the last five seasons.

Several key themes emerge:

Pinnacle consistently has the lowest overrounds, averaging between 2.39% and 3.00%. This aligns with Pinnacle’s reputation as a low-margin, high-volume bookmaker that attracts sharper bettors. Their pricing reflects high market efficiency.
1xBet and Unibet also offer relatively tight margins, typically under 3%, though Unibet shows a notable spike in 2024/2025, possibly indicating a change in pricing strategy.
Mainstream bookmakers like bet365, bwin, and William Hill tend to operate with higher overrounds, often between 5% and 6%, with William Hill even reaching 7.32% in 2023/2024. These firms serve a broader customer base and tend to target recreational bettors.
888sport, Betfair, and Betway show a trend of steadily increasing margins over the seasons, suggesting a shift toward more conservative pricing.
10bet jumps dramatically to 6.18% in 2024/2025 from a relatively sharp 3.78%, suggesting a drastic repositioning of itself in the market.

When we break down overrounds in the Premier League by bookmaker, a clear divide emerges between two groups:

Low-margin operators like Pinnacle and 1xBet, which consistently offer tighter pricing.
Mainstream bookmakers such as William Hill, bet365, and bwin, which price with significantly higher margins.

This split reflects two fundamentally different business models:

Mainstream bookmakers like William Hill, bet365, and bwin typically operate with overrounds in the 5–7% range. These firms primarily cater to recreational bettors - users who are betting more for entertainment than profit. Many of these customers are less likely to shop around for better odds or understand how bookmaker margin erodes long-term returns. As such, these bookmakers can afford to be less competitive on price, and may even benefit from it since tighter pricing can attract sharp bettors they’d rather avoid.

In contrast, low-margin operators such as Pinnacle or 1xBet attract price-sensitive or professional bettors. Their overrounds often sit below 3%, even in top-tier markets. These firms rely on a volume-based model, welcoming sharp action and managing risk through efficient pricing and faster line movement, rather than blocking or limiting users.

Season	2020/2021	2021/2022	2022/2023	2023/2024	2024/2025
10bet	3.78	4.49	4.01	4.23	6.18
1xBet	3.10	3.08	2.81	2.95	3.03
888sport	4.19	4.69	5.56	6.03	5.81
Betfair	4.03	4.47	5.38	5.68	5.87
Betway	5.43	5.00	5.08	5.83	6.07
Pinnacle	2.39	2.43	2.46	2.93	3.00
Unibet	2.56	2.59	2.56	2.27	5.18
William Hill	5.11	6.05	6.25	7.32	5.57
bet365	5.47	5.54	5.41	5.39	5.52
bwin	5.10	5.63	5.78	5.64	5.77

Table 2: Premier League Overrounds by Bookmaker

Bookmaker Accuracy by League: Ranked Probability Scores

To evaluate how well bookmaker odds align with actual match outcomes, we use the Ranked Probability Score (RPS) - a strictly proper scoring rule that measures the accuracy of probabilistic forecasts across ordered outcomes (like home/draw/away in football). Lower values indicate better predictive performance, with perfect accuracy scoring 0.

Table 3 below shows average RPS values for moneyline markets across major European leagues, broken down by season:

Several key observations stand out:

Top-tier leagues tend to have lower RPS values, indicating more accurate and better-calibrated odds. For example:
- The Netherlands' Eredivisie, Portugal’s Liga Portugal, and Italy’s Serie A consistently sit at the top of the accuracy table, with RPS scores as low as 0.175–0.185.
- The Premier League, La Liga, and Bundesliga also show strong performance, generally ranging between 0.18 and 0.20.
In contrast, lower-tier leagues such as England’s League One/Two, Serie B, and La Liga 2, tend to produce higher RPS scores, often above 0.21, suggesting greater uncertainty and less precise odds.
Accuracy varies season-to-season within leagues, but not that wildly, suggesting relative stability in how bookmakers price different competitions. Some leagues (like France's Ligue 1) show very consistent RPS values year over year.

Season	2020/2021	2021/2022	2022/2023	2023/2024	2024/2025
Belgium: Jupiler League	0.223	0.196	0.201	0.195	0.196
England: Championship	0.219	0.217	0.221	0.212	0.207
England: League One	0.220	0.200	0.207	0.216	0.209
England: League Two	0.223	0.214	0.214	0.224	0.219
England: Premier League	0.211	0.191	0.198	0.180	0.193
France: Ligue 1	0.204	0.201	0.198	0.206	0.203
France: Ligue 2	0.204	0.209	0.208	0.221	0.211
Germany: 2. Bundesliga	0.220	0.217	0.213	0.220	0.219
Germany: Bundesliga	0.198	0.202	0.202	0.186	0.202
Italy: Serie A	0.185	0.193	0.197	0.184	0.184
Italy: Serie B	0.206	0.204	0.211	0.209	0.204
Netherlands: Eredivisie	0.183	0.186	0.184	0.176	0.185
Portugal: Liga Portugal	0.188	0.175	0.177	0.179	0.182
Spain: La Liga	0.190	0.195	0.202	0.182	0.190
Spain: La Liga 2	0.214	0.206	0.201	0.210	0.202

Table 3: Bookmaker Accuracy by League: Ranked Probability Scores

Accuracy by Bookmaker: RPS in the Premier League

Having seen how bookmaker margins differ, it’s natural to ask: Do tighter margins translate to more accurate odds? One way to answer that is by comparing Ranked Probability Scores (RPS) across bookmakers in the Premier League.

At first glance, the differences in RPS are remarkably narrow, typically falling between 0.18 and 0.20 across seasons. That’s perhaps expected in a highly liquid, competitive market like the Premier League, where bookmakers converge on efficient prices.

Still, some patterns emerge:

No clear link between margin and accuracy: Despite having the lowest overrounds, Pinnacle does not clearly outperform higher-margin books like William Hill or bet365 in terms of RPS. In fact, William Hill occasionally posts the lowest (i.e., best) RPS, such as 0.188 in 2021/2022. This supports the idea that bookmaker margin and forecast accuracy are not tightly correlated. Higher-margin books may simply be padding prices, not offering worse forecasts.
1xBet, Betway, and Pinnacle show near-identical performance, suggesting they may be operating with similar models or reacting similarly to market movement.

So while Pinnacle leads on pricing (low overround), and books like William Hill apply large margins, there’s no strong evidence here that sharp-friendly books are forecasting significantly better, at least not when measured by RPS alone in the Premier League.

season	2020/2021	2021/2022	2022/2023	2023/2024	2024/2025
1xBet	0.211	0.191	0.197	0.180	0.193
888sport	0.212	0.199	0.197	0.180	0.187
Betway	0.211	0.191	0.197	0.181	0.193
Pinnacle	0.211	0.192	0.197	0.180	0.192
Unibet	0.212	0.191	0.198	0.179	0.194
William Hill	0.211	0.188	0.198	0.181	0.193
bet365	0.212	0.189	0.196	0.180	0.193
bwin	0.212	0.194	0.197	0.181	0.193

Table 4: Accuracy by Bookmaker: RPS in the Premier League

Are Bookmakers Well-Calibrated?

A final test of accuracy is calibration: how well do predicted probabilities match observed outcomes? If a bookmaker assigns a 70% probability to an outcome, that outcome should occur roughly 70% of the time for the model to be considered well-calibrated.

The plot below shows calibration curves for bookmaker probabilities across home wins, draws, and away wins. The dashed black line represents perfect calibration where predicted probability equals observed frequency.

The results are impressive:

All three outcome types - Home Win, Draw, and Away Win - track the diagonal line closely, indicating that bookmakers’ odds are well-calibrated overall.
There's no major systematic bias: bookmakers aren't consistently under- or overestimating the likelihood of any specific outcome across probability ranges.
Deviations are small and mostly in the extremes, where data can be noisier due to fewer data points.

Taken together with the low RPS scores in previous sections, this reinforces a key conclusion:

Bookmakers aren’t just padding prices, their forecasts are statistically sound and well-aligned with real-world outcomes, especially in major markets like the Premier League.

Plot 1: Calibration Curves for Bookmaker Probabilities

Beating the Bookmaker in Obscure Leagues?

A common belief among many bettors is that focusing on lower-profile leagues, where information is scarce, offers an edge. The logic is simple: if you know more than the bookmaker, you can find value. However, the data tells a more cautionary story.

Let's take the Seychelles and Mongolian Premier Leagues as examples:

Seychelles Premier League: Overround = 9.76%
Mongolia Premier League: Overround = 12.03%

These are massive compared to the 4-5% typical in top European leagues. But high margins alone don’t mean sloppy forecasts. In fact, the RPS scores tell a different story:

Seychelles RPS = 0.230
Mongolia RPS = 0.192

Mongolia, in particular, is priced as accurately as many major European leagues despite the bookmaker charging a hefty margin. Even Seychelles, with a worse RPS, still falls within the range seen in many second-tier European competitions.

So while obscure leagues may feel like an edge, the data suggests bookmakers are doing a surprisingly good job and charging extra for the uncertainty.

If anything, the takeaway is this: the more obscure the league, the higher the margin - not necessarily the worse the forecast. Bookmakers are aware of their informational disadvantage and price accordingly to protect themselves. You’re not beating the market by knowing more about Tuv Azarganuud's starting XI but you are paying a premium for the privilege of trying.

Conclusion

Across millions of betting lines, one thing is clear: bookmakers, especially in major markets, are very good at what they do. Their odds are well-calibrated, their forecasts consistently accurate, and their margins finely tuned to the level of competition and market liquidity. Even in obscure leagues, they’re cautious but not careless.

But that doesn’t mean the market is unbeatable. If anything, it shows that finding an edge requires more than intuition - it demands scale, rigor, and nuance. That’s exactly what a data-driven approach is designed to uncover.

Whether you're looking to evaluate specific bookmakers, spot outliers, or understand where pricing deviates from probability, I’ll be continuing to dig into this dataset and sharing anything interesting I find.

For those looking for signal in a noisy market, I hope this is an interesting start.

MatchFlow 1.4.0: Optimizing, Visualizing, and Validating your Data Pipelines

2025-06-10T19:30:00+00:00

Introduction

Over the past few weeks, I've been diving into the world of query plan optimization. It's a fascinating (but complex) rabbit hole that has led to several new features in the upcoming MatchFlow release. Here's what's new, why I built it, and how it can help you.

Query Plan Optimization (optional, but awesome!)

As data pipelines become more complex, I realized MatchFlow could benefit from smarter optimization. Enter the internal FlowOptimizer class. This is an optional feature you can enable to automatically rewrite your pipeline plans, pushing filters and limits closer to data sources, fusing steps, and generally improving efficiency.

You can opt-in to using it by:

from penaltyblog.matchflow import Flow

flow = Flow.from_jsonl("events.jsonl", optimize=True)

Why make it optional? Correctness and safety are my top priorities. Since this is the initial release of the optimizer, I'm making it opt-in while it gets thoroughly battle-tested.

Once I'm confident it handles edge cases correctly, I'll make it the default. For now though, enabling it is straightforward but the choice remains yours.

Importantly, the optimizer is also transparent and explainable. Use .explain(compare=True) to see exactly what optimizations have taken place.

Improved Plan Visualization with `.plot_plan`

Understanding exactly what your pipeline does is crucial. With Flow.plot_plan, you can now generate visual representations of your pipelines. Quickly visualize steps, identify bottlenecks, and simplify debugging. You can also set compare=True to compare the plan before and after the optimizer has potentially rewritten it.

from penaltyblog.matchflow import Flow


data = [
    {"team": "A", "score": 1},
    {"team": "B", "score": 2},
    {"team": "A", "score": 3},
    {"team": "B", "score": 4},
]

flow = (
    Flow.from_records(data, optimize=True)
    .select("team", "score")
    .assign(x=lambda r: r["score"] * 2)
    .limit(2)
)

flow.plot_plan(compare=True)

Figure 1: Flow plan before and after optimizer

You can see from the example above how the optimizer has pushed the limit step closer to the data source, reducing the amount of data that needs to be processed. This can lead to significant performance improvements, especially for larger datasets.

Let's look at another example:

from penaltyblog.matchflow import Flow


data = [
    {"team": "A", "score": 1},
    {"team": "B", "score": 2},
    {"team": "A", "score": 3},
    {"team": "B", "score": 4},
]

flow = (
    Flow.from_records(data, optimize=True)
    .filter(lambda r: r["team"] == "A")
    .filter(lambda r: r["score"] > 2)
    .limit(2)
)
flow.plot_plan(compare=True)

Figure 2: Flow plan before and after optimizer

This time the optimizer has recognized that the two filters are redundant and has fused them into a single filter step. This can lead to significant performance improvements by reducing the number of steps that need to be executed.

Rolling Summaries for Groups

Sports data often needs rolling summaries, think moving averages or cumulative sums. MatchFlow now supports this natively using a sliding window:

from penaltyblog.matchflow import Flow, where_equals
import datetime

flow = (
    Flow.statsbomb.events(16023)
      .filter(where_equals("type.name", "Shot"))
      .assign(
        timestamp=lambda r: datetime.timedelta(
            minutes=r["minute"], 
            seconds=r["second"]
        )
      )
      .group_by("team.name")
      .sort_by("timestamp")
      .rolling_summary(
         window="5m",                                  
         aggregators={
            "xg": ("sum", "shot.statsbomb_xg"),
            "shot_count": ("count", "shot.statsbomb_xg")
         },
         time_field="timestamp",                                                          
      )
      .select("team.name", "timestamp", "xg", "shot_count")
)

flow.show()

What it does

Groups your stream by one or more keys (e.g. team).
Sorts each group by a timestamp (must be pre-sorted).
Slides a fixed-duration window (5m here but can also be the number of records) over each group.
Computes aggregations (moving average, running sum, count, etc.) within each window.

Why pre-sorting matters

To guarantee correct sliding-window results, you must call .sort_by() before .rolling_summary(). If records arrive at the window out of order, you risk skewed or invalid metrics.

Automatic safety checks

I debated with myself a lot about auto-sorting inside .rolling_summary(), but eventually decided against it because it would:

Add overhead when data is already sorted
Hide errors when upstream sources emit out-of-order events

Instead, I settled on MatchFlow’s new optimizer emitting a warning whenever it detects that .rolling_summary() follows a non-sorted stream. This reminds you to verify order without penalizing optimized pipelines.

Figure 3: Flow plan before and after optimizer

Time Buckets

If you’d rather have one summary per uniform time interval, say total xG every 5 minutes, you can use .time_bucket():

from penaltyblog.matchflow import Flow, where_equals
import datetime


flow = (
    Flow.statsbomb.events(16023)
      .filter(where_equals("type.name", "Shot"))
      .assign(
        timestamp=lambda r: datetime.timedelta(
            minutes=r["minute"], 
            seconds=r["second"]
        )
      )
      .group_by("team.name")
      .time_bucket(
         freq="5m",                                 
         aggregators={
            "xg": ("sum", "shot.statsbomb_xg"),
            "shot_count": ("count", "shot.statsbomb_xg") 
         },
         time_field="timestamp",                       
         label="left"                                
      )
)

flow.show()

What it does

Non-overlapping bins: divides time into [0–5), [5–10), … minute windows.
Single row per bucket: each group emits exactly one record per interval, with your chosen aggregations.
Flexible labeling: choose label="left" (interval start) or label="right" (interval end), and name that column with bucket_name.

Quick Tips

Always .assign() your timestamp field as a proper datetime or timedelta when using time-based windows.
Don't forget to use .sort_by() before .rolling_summary() to ensure correct results.
Use .time_bucket() when you need uniform reporting intervals; use .rolling_summary() for sliding-window analytics.

Schema Validation

Messy data can be frustrating so I've also introduced .with_schema(), to help catch data issues early:

from datetime import datetime

def parse_datetime(dt_str):
    return datetime.strptime(dt_str, "%Y-%m-%d %H:%M:%S")

flow.with_schema({"team.name": str, "score": int, "timestamp": parse_datetime}, strict=True, drop_extra=True)

You can set strict=True to raise an error if any data doesn't match the schema, and drop_extra=True to drop any fields that are not defined in the schema.

Quick Data Previews

Because I'm often working in notebooks or terminals, I've added a simple .show() method to quickly preview data in a nice, readable format:

flow.limit(10).show()

Quality-of-Life Improvements

I've also added some quality-of-life improvements to existing methods, including:

Optional progress bars on .collect()
Improved .explain() for grouped pipelines
Documentation updates

Conclusions

These improvements reflect my ongoing plan to make MatchFlow as powerful while remaining simple as possible.

MatchFlow is still new and I'm sure there's edge cases I haven't discovered yet so if you find any issues or or have any feedback, please do let me know. I'm always open to collaboration and excited about new possibilities.

What's Next?

As a quick teaser, the new optimizer opens the door to exciting future developments like predicate pushdown and parallel processing. I'm also currently experimenting with a custom binary data format designed for faster loading and even more efficient filtering. Lots of great stuff on the roadmap so stay tuned!

Introducing MatchFlow: a JSON-native query engine for football data.

2025-05-25T19:30:00+00:00

MatchFlow: A JSON-Native Query Engine for Football Data

Over the years, I’ve spent a lot of time wrangling football data - from open data projects to internal club work. A lot of it comes as JSON these days: events, lineups, matches, tracking, metadata. And while that format is powerful and expressive, the tools we use to work with it often aren’t.

What’s the first step in most workflows?

Flatten everything. Hope the structure is consistent. Write some glue code to handle edge cases. Cross fingers that json_normalize() doesn’t choke. And repeat, every time.

I got tired of that. So I started building MatchFlow, a Python library that lets me work with football data more naturally.

Why I Built MatchFlow

I don’t want to spend hours figuring out how to flatten the data just to get started on a new project. I want to load the data, explore it, and see what’s there. And that’s where MatchFlow really helps.

Show me all the shots in a match
Group events by player
Compute some summary stats
Build a dataset for modelling

The problem is, doing that with nested JSON style data often means jumping through hoops: flattening things, renaming fields, re-shaping arrays, cleaning as you go. It gets in the way of thinking clearly.

MatchFlow is my attempt to fix that, or at least make it less annoying.

It’s built around a simple idea: don’t fight the structure. Just let it flow through, and shape it as you go.

What MatchFlow Does

MatchFlow builds a directed acyclic graph (DAG) of operations - a chain of lazy, composable steps that can be optimized and executed efficiently. This architecture makes MatchFlow fast, composable, and ready for upcoming features like pushdown filtering, transform fusion, and automatic caching.

You can load data from JSON files, folders, or even directly from APIs like StatsBomb, and then apply transformations step by step:

from penaltyblog.matchflow import Flow, where_equals

flow = (
    Flow.from_folder("data/events/")
    .filter(where_equals("type", "Shot"))
    .assign(xT=lambda r: model.predict(r))
    .select("player.name", "xT", "location")
    .to_json("shots.json")
)

Nothing is loaded into memory until you ask for it. You can stream, group, filter, join, and export. And most importantly, you can do all of that without flattening everything first.

It Works with the Data We Actually Get

Football data is full of nested structures:

Events have players, tags, locations
Matches contain teams, lineups, substitutions
Freeze-frames as arrays of objects inside a single field

Flattening all that up front means you lose some of the meaning, or have to write extra code to glue it back together. MatchFlow lets you work with it in-place, using dot notation (player.name, player.location.0, etc.) and stream-like transformations.

MatchFlow isn’t trying to replace dataframes - it’s built for a different job: helping you explore, transform, and build pipelines over nested, unstructured data without flattening everything first.

Whether that means pulling down a bunch of JSON files and exploring them interactively, or turning them into a repeatable data engineering pipeline, MatchFlow makes that easy without needing to reshape everything upfront.

A Quick Example (with StatsBomb)

Here’s how you might pull shots from a StatsBomb match using the built-in API client:

from penaltyblog.matchflow import Flow, where_equals

shots = (
    Flow.statsbomb.events(match_id=22912)
    .filter(where_equals("type.name", "Shot"))
    .select("player.name", "location", "shot.statsbomb_xg")
    .collect()
)

No flattening. No munging. Just a stream of records that you can work with directly.

What's Included in v1

Read from JSON files, folders, generators, or the StatsBomb API
Chainable methods like .filter(), .select(), .assign(), .group_by(), .join()
Export to .to_json(), .to_jsonl(), .to_pandas()
Nested field access using dot notation
Lazy evaluation - everything streams until you call .collect() or .to_pandas()

What’s Coming Next

🚀 Performance

A custom file format with partitioning and fast reading
Predicate pushdown using file-level indexes and Bloom filters
Compilation of query steps into native Python functions for speed
Cython/JIT-optimized inner loops for group-by and sorting

🔧 Features

Support for Wyscout, Opta, and remote data (S3, GCS)
Built-in operations like extract_shots() and extract_passes()
DSL for natural query expressions like flow.query("player.name == 'Messi'")

🛠 Ergonomics

Plotting helpers for quick exploration
Command-line tools for scripting pipelines
Caching for faster re-runs

Try It Out

If you want to give it a spin:

pip install penaltyblog

You can also check out the docs - there’s a guides, an API reference, and examples of some common tasks.

Final Thoughts

I don’t think MatchFlow is for everyone. If you're happy working in SQL or pandas and it's doing what you need then great, stick with it. But if you’ve ever found yourself spending too much time writing flatteners, normalizers, or trying to keep your data in shape just to ask a basic question, MatchFlow might help.

It's early days, and I'm sure there are still edge cases I haven't uncovered. But this approach has saved me a lot of time over the years, and hopefully it can do the same for you.

I'd love to hear how you use it, or where it breaks. That’s how it gets better 😁

Better Metrics for Football Forecasts: Moving Beyond the Ranked Probability Score

2025-05-01T19:30:00+00:00

Introduction

What if the metric you've been using to evaluate football forecasts is quietly rewarding worse predictions?

Probabilistic forecasting is a crucial tool in sports analytics - whether it's to inform betting strategies, guide player and tactical decisions, or simply enhance the way we follow the sport. As these models have grown in popularity, however, one important question has often been overlooked: how do we evaluate the quality of our forecasts?

Despite the increase in use of football (soccer) prediction models, the discussion around how best to score and compare them has been relatively limited. In many cases, evaluation methods are taken for granted without much scrutiny. This is a significant gap because without the right tools for measuring forecast quality, it's easy to draw misleading conclusions about model performance.

On my blog, I've often used the Ranked Probability Score (RPS) to assess the accuracy of predictive models. RPS has an intuitive appeal as it rewards forecasts not just for getting the result right, but also for placing higher probability on outcomes close to the actual result. A forecast that leaned towards a draw when the match ended as a narrow home win was treated better than one that backed an away win.

For a long time, that logic seemed sound. However, I've recently been questioning whether RPS is really the best tool for evaluating football forecasts - especially when our ultimate goal is to identify the most informative models as efficiently and fairly as possible.

In this article, I’ll explain why RPS might not be the optimal choice, introduce alternative scoring metrics like Log Loss (also known as Ignorance Score) and the multiclass Brier score, and share some experiments / ideas I've tested to explore which metrics are best suited for evaluating football predictive models.

Background: How We Score Probabilistic Football Forecasts

When evaluating a model that outputs probabilities, such as predicting a 45% chance of a home win, 30% draw, and 25% away win, we need a way to judge how good those probabilities actually were once the match result is known. That's where scoring rules come in.

A scoring rule is simply a mathematical method that assigns a numerical score to a forecast based on the eventual outcome. A good scoring rule rewards forecasts that were well-calibrated and honest, and penalizes forecasts that placed high confidence on the wrong results.

The Ranked Probability Score (RPS)

The Ranked Probability Score (RPS) has become a popular choice for evaluating football forecasts. Its appeal lies in how it measures how close a forecast was to the actual outcome, taking the ordering of results into account.

In football, match outcomes - home win, draw, away win - can be thought of as ordered, with a draw being closer to a home win than an away win is. The RPS rewards forecasts that spread probability sensibly across outcomes near the correct result, not just those that hit the exact outcome.

Technically, RPS calculates the squared differences between the cumulative forecast probabilities and the cumulative actual outcomes across all possible results. It sums these differences to produce a final score, with lower scores indicating better forecasts.

In summary:

Correct predictions get rewarded.
Predictions that almost got it right (e.g., favouring a draw when it was a home win) are penalized less harshly than predictions that were way off (e.g., favouring an away win).

This sensitivity to distance has often been cited as a key advantage of RPS but, as we will explore, it also introduces potential problems.

Alternative Scoring Rules: Brier Score and Log Loss

While RPS rewards proximity to the correct outcome, there are other scoring rules that take a different approach:

The Brier Score (Multiclass Version)

The Brier score measures the squared difference between the predicted probability and the actual outcome, across all possible results but without considering the ordering between outcomes. A wrong prediction for an away win is penalized the same as a wrong prediction for a draw if the match ends in a home win.

In formula terms, it’s simply the mean squared error between the forecast probabilities and the actual result (coded as 1 for the outcome that occurred, 0 for those that did not).

Like RPS, lower Brier scores indicate better forecasts. However, the Brier score is insensitive to distance as it treats all incorrect outcomes equally wrong.

Log Loss (Ignorance Score)

The log loss, also known as the ignorance score, takes an even sharper approach. It focuses only on the probability assigned to the outcome that actually occurred, ignoring how probability was distributed across other outcomes.

Log loss measures how surprised the forecast was by the actual result using concepts from information theory.

If you assigned a high probability to the correct result, you get a low (good) score.
If you assigned a low probability to the correct result, you're heavily penalized.

Smaller log loss values (closer to zero) indicate better forecasts. Importantly, log loss is a local scoring rule meaning it doesn't reward or penalize probability placed on outcomes that didn’t happen.

Key Properties of a Good Scoring Rule

When choosing how to evaluate football forecasts, a few key properties matter:

Propriety: A scoring rule is proper if it encourages honest forecasting - assigning probabilities that truly reflect beliefs.
Locality: A scoring rule is local if it only depends on the probability of the outcome that occurred (log loss is local; RPS and Brier are not).
Sensitivity to Distance: Some rules (like RPS) reward probability placed on outcomes "near" the correct one; others (like Brier and log loss) do not.

These properties have deep implications for how fairly and effectively different metrics evaluate forecasts, and, as we'll see, they raise important questions about whether RPS is really the best choice.

What’s Wrong with the Ranked Probability Score?

At first glance, the RPS seems like an ideal tool for evaluating football forecasts. It rewards accurate predictions and gives partial credit to forecasts that are almost right, reflecting the natural ordering of outcomes like home win, draw, and away win.

However, when we dig deeper into the properties of RPS, issues emerge especially when we think carefully about what we actually want a scoring rule to do.

Sensitivity to Distance Isn't Always Useful

RPS assumes that outcomes have a natural order, and that being close to the right result is better than being completely wrong. But in practice, this sensitivity to distance may not make sense for evaluating probabilistic forecasts.

Consider a match where a team wins 3-0 at home. From a probability standpoint, a forecast that heavily backed a draw is no closer to reality than one that backed an away win. The match was decisively a home win, and both alternative forecasts were wrong. Yet RPS would treat the forecast favoring a draw more kindly even though both forecasts misrepresented what actually happened.

In probabilistic forecasting, what matters most is the probability placed on the outcome that actually occurred, not how close other outcomes seemed. Sensitivity to distance risks rewarding forecasts that were still wrong, simply because they were less wrong in some arbitrary way.

Non-Locality Dilutes the Signal

A second issue is that RPS is non-local: it takes into account the entire probability distribution, not just the probability assigned to the true outcome.

This means a forecast can achieve a better RPS score by adjusting probabilities on outcomes that did not happen. In theory, a forecast could place relatively little probability on the actual outcome but still score well by distributing probabilities nicely across nearby results.

This dilutes the signal we care about most: how much belief did the forecast put on the correct outcome? A good evaluation metric should focus sharply on that and not get distracted by how probabilities were assigned to events that never occurred.

Inefficiency in Identifying Better Forecasts

From a practical standpoint, another downside of RPS is that it can require more data - as in more matches and outcomes - to reliably identify which forecasting model is actually better.

In simulation experiments (including those I’ll describe later), scoring rules like log loss tend to distinguish better models more quickly and more reliably than RPS. This matters because in the real world, we often have limited sample sizes, especially when evaluating new models, seasonal predictions, or niche competitions.

A scoring rule that uses the available information more efficiently gives us a better chance of recognizing model quality earlier.

RPS Summary

While the Ranked Probability Score has attractive theoretical features, it also carries significant practical and conceptual drawbacks.

It can reward forecasts that are still substantially wrong.
It spreads focus across outcomes that didn’t happen.
It may be slower and less reliable at identifying genuinely better models.

Given these issues, it's worth asking whether we should move beyond RPS when evaluating football forecasts and whether alternatives like log loss (Ignorance Score) and the multiclass Brier score might offer a better foundation.

In the next sections, I'll show some ideas that explore exactly that question.

Experiment One: RPS Can be Slower to Identify Better Forecasts

To compare how well different scoring rules distinguish better forecasts in football, I ran a simple simulation.

For each match:

Forecast A represented a "better" forecast - more accurate and better calibrated.
Forecast B represented a "worse" forecast - slightly less accurate and less confident.

Specifically:

Forecast A assigned probabilities of 60% home win, 25% draw, and 15% away win.
Forecast B assigned 50% home win, 30% draw, and 20% away win - a slightly more spread-out, less confident forecast.

In each simulation:

I simulated match outcomes by randomly drawing results according to the better forecast (Forecast A).
For each match, I scored both forecasts using three different metrics:
- Log Loss (Ignorance Score)
- Multiclass Brier Score
- Ranked Probability Score (RPS)
I then compared the total scores for Forecast A and Forecast B.
- If a metric gave a lower (better) score to Forecast A, it was counted as a "correct" selection for that metric.

I repeated this process 1,000,000 times for each sample size tested - ranging from just 10 matches up to 1000 matches - and calculated the proportion of simulations where each scoring rule correctly identified Forecast A as better.

This setup mimics a realistic football forecasting evaluation:

Limited numbers of matches.
Models that are not drastically different - a common scenario in practice.
Need for scoring rules to efficiently and reliably identify better forecasts even when differences are subtle.

The code for this is shown below. However, please note that you may get different results due to the random sampling. The general pattern should be similar though.

import numpy as np
import penaltyblog as pb


def simulate(sample_size, n_repeats):
    forecast_a = np.array([0.6, 0.25, 0.15])
    forecast_b = np.array([0.5, 0.3, 0.2])

    outcomes = np.random.choice(3, size=(n_repeats, sample_size), p=forecast_a)

    results = {"brier": 0, "logloss": 0, "rps": 0}

    for i in range(n_repeats):
        o = outcomes[i]

        # Broadcast forecasts
        probs_a = np.repeat(forecast_a[np.newaxis, :], sample_size, axis=0)
        probs_b = np.repeat(forecast_b[np.newaxis, :], sample_size, axis=0)

        score_a_brier = pb.metrics.multiclass_brier_score(probs_a, o)
        score_b_brier = pb.metrics.multiclass_brier_score(probs_b, o)

        score_a_logloss = pb.metrics.ignorance_score(probs_a, o)
        score_b_logloss = pb.metrics.ignorance_score(probs_b, o)

        score_a_rps = pb.metrics.rps_average(probs_a, o)
        score_b_rps = pb.metrics.rps_average(probs_b, o)

        results["brier"] += int(score_a_brier < score_b_brier)
        results["logloss"] += int(score_a_logloss < score_b_logloss)
        results["rps"] += int(score_a_rps < score_b_rps)

    for key in results:
        results[key] /= n_repeats

    return results


def run_experiment(sample_sizes, n_repeats=1000000):
    results = {metric: [] for metric in ["brier", "logloss", "rps"]}

    for n in sample_sizes:
        res = simulate(n, n_repeats)
        for metric in res:
            results[metric].append(res[metric])

    return results

run_experiment([10, 25, 50, 100, 200, 500, 1000], 1000000)

Results

Sample Size	Log Loss	Brier Score	RPS
10	62.9%	63.0%	61.3%
25	70.4%	72.9%	67.7%
50	76.9%	77.1%	76.5%
100	84.9%	87.0%	84.1%
200	93.3%	92.6%	92.1%
500	99.0%	98.8%	99.0%
1000	100.0%	100.0%	100.0%

Table 1: Proportion of simulations where each scoring rule correctly identified the better forecast

Interpreting the Results

With one million repetitions per sample size, the results paint a stable and revealing picture. At smaller sample sizes (10–100 matches), RPS consistently underperforms relative to both Log Loss and Brier Score. For example, at just 25 matches, RPS correctly identifies the better forecast only 67.7% of the time, compared to 70.4% for Log Loss and 72.9% for Brier Score.

Throughout most sample sizes, Brier Score slightly edges out Log Loss in terms of raw accuracy, particularly in the 25 to 200 match range. However, Log Loss remains nearly as effective, and its theoretical strengths as a strictly proper local scoring rule make it a more principled choice for model evaluation, especially when distinguishing subtle differences between forecasts.

These results reinforce the idea that RPS is the least efficient of the three: it requires more data to reach the same level of confidence in model comparison, making it a less suitable option for practical use in football analytics.

Experiment Two: RPS Can Favour a Forecast That Believes Less in the Truth

One of the key problems with the Ranked Probability Score is that it's non-local, meaning it evaluates forecasts based on the entire cumulative distribution, not just the outcome that actually occurred. This can lead to unintuitive results.

Results

Forecast	Probabilities			P(Correct)	RPS	Log Loss
Forecast	Home	Draw	Away	P(Correct)	RPS	Log Loss
A	0.70	0.10	0.20	0.70	0.065	0.515
B	0.65	0.30	0.05	0.65	0.062	0.621

Table 2: RPS Can Favor a Forecast That Believes Less in the Truth

Interpreting the Results

Although Forecast A assigns a higher probability to the correct outcome (70% vs 65%), the Ranked Probability Score (RPS) gives a better (lower) score to Forecast B. This happens because RPS evaluates the entire distribution of probabilities, and Forecast B’s more even spread is treated as closer to the true outcome in cumulative terms.

In contrast, Log Loss (a local scoring rule) correctly favours Forecast A, since it focuses solely on the probability assigned to the observed result. This example highlights a key flaw of RPS: it can reward worse forecasts simply because they are smoother, even when they express less belief in what actually happened.

Experiment 3: A Real-World Test Using Bookmaker Odds

The previous experiments in this article used synthetic forecasts - carefully constructed examples designed to highlight how different scoring rules behave under controlled conditions. While these are useful for understanding the theoretical properties of metrics like RPS, Log Loss, and Brier Score, real-world forecasting is rarely so tidy.

To evaluate how these scoring rules perform in more practical settings, I next used historical bookmaker odds - widely regarded as one of the most accurate publicly available sources of football match probabilities - and intentionally distorted them to simulate worse forecasts.

The goal was to see which scoring rules could reliably detect the better forecast under more realistic, subtly challenging conditions.

Here's how the experiment was structured:

Bookmaker odds from Bet365 (sourced via football-data.co.uk) were converted into probabilities by inverting and normalizing the odds (to remove the overround).
These bookmaker-derived probabilities were treated as the "true" forecast (Forecast A).
A slightly degraded forecast (Forecast B) was created by distorting the original probabilities using temperature scaling, a method that flattens or sharpens the distribution to simulate reduced model quality.
For each match, an outcome was simulated from Forecast A, and both forecasts were scored using:
- Log Loss (Ignorance Score)
- Brier Score
- Ranked Probability Score (RPS)
The process was repeated thousands of times, and I tracked how often each scoring rule correctly favoured the higher-quality (bookmaker) forecast.

If you want to have a go at repeating this, code is shown below. However, please note that you may get slightly different results due to the random sampling. The general pattern should be similar though.

import numpy as np
import penaltyblog as pb
import pandas as pd


def distort_probs(probs, temperature=1.25):
    """Distort probability distribution using temperature scaling."""
    scaled = np.power(probs, 1 / temperature)
    return scaled / scaled.sum()

def run_betting_experiment(odds_list, n_repeats=100000, distort_fn=distort_probs):
    """Compare scoring rules on bookmaker vs distorted forecasts."""
    correct = {"logloss": 0, "brier": 0, "rps": 0}

    for _ in range(n_repeats):
        # Sample match odds from the bookmaker data
        odds = odds_list[np.random.randint(len(odds_list))]
        true_probs = pb.implied.shin(odds)["implied_probabilities"]

        # Distort to create worse forecast
        distorted_probs = distort_fn(true_probs)

        # Simulate an outcome based on true_probs
        outcome = np.random.choice(3, p=true_probs)

        # Score both forecasts
        log_a = pb.metrics.ignorance_score([true_probs], [outcome])
        log_b = pb.metrics.ignorance_score([distorted_probs], [outcome])

        brier_a = pb.metrics.multiclass_brier_score([true_probs], [outcome])
        brier_b = pb.metrics.multiclass_brier_score([distorted_probs], [outcome])

        rps_a = pb.metrics.rps_average([true_probs], [outcome])
        rps_b = pb.metrics.rps_average([distorted_probs], [outcome])

        # Compare which forecast scored better
        correct["logloss"] += int(log_a < log_b)
        correct["brier"] += int(brier_a < brier_b)
        correct["rps"] += int(rps_a < rps_b)

    # Normalize to proportion
    for k in correct:
        correct[k] /= n_repeats

    return correct

df = pd.concat([
    pb.scrapers.FootballData("ENG Premier League", "2020-2021").get_fixtures(), 
    pb.scrapers.FootballData("ENG Premier League", "2021-2022").get_fixtures(), 
    pb.scrapers.FootballData("ENG Premier League", "2022-2023").get_fixtures(),    
    pb.scrapers.FootballData("ENG Premier League", "2023-2024").get_fixtures(),
    pb.scrapers.FootballData("ENG Premier League", "2024-2025").get_fixtures(),
])

home = df["b365_h"]
draw = df["b365_d"]
away = df["b365_a"]

odds_list = np.array(list(zip(home, draw, away)))

results = run_betting_experiment(odds_list, n_repeats=1000000)
print(results)

Results

After running the simulation thousands of times, the proportion of cases in which each scoring rule correctly identified the better forecast was:

Scoring Rule	% Correct
Log Loss	58.6%
Brier Score	57.8%
RPS	56.1%

Table 3: Proportion of cases in which each scoring rule correctly identified the better forecast.

Interpreting the Results

Even in this more realistic setting, where the differences between forecasts were subtle and the "ground truth" forecasts came from actual bookmaker odds, Log Loss continued to outperform the other metrics. Although all three scoring rules perform similarly, Log Loss was more consistent in identifying the better forecast.

The gap may seem small, but in practice, these marginal improvements in sensitivity and reliability matter, especially when comparing models in leagues with fewer matches or when evaluating forecasts from similar models. RPS, once again, proved less effective at distinguishing higher-quality forecasts, consistent with the findings from the earlier synthetic experiments.

Conclusion

Across both synthetic and real-world tests, Log Loss And Briar Scores consistently proved to be a more sensitive and reliable metric for evaluating football forecasts than the Ranked Probability Score (RPS). While all three scoring rules eventually converge with enough data, RPS lags behind at smaller sample sizes and can even reward worse forecasts under certain conditions as a direct consequence of its non-locality. While Brier Score sometimes outperforms Log Loss in raw accuracy, particularly at moderate sample sizes, Log Loss’s sharper theoretical grounding and consistency make it my preferred choice.

In real football analytics and betting, we rarely have huge sample sizes to evaluate models with. We often deal with limited data, closely matched forecasts, and the need for efficient, trustworthy evaluation tools. Based on this evidence, Log Loss stands out as the most appropriate scoring rule for comparing football predictive models, offering sharper discrimination, stronger theoretical foundations, and more reliable guidance when it matters most.

Pi Ratings: The Smarter Way to Rank Football Teams

2025-04-14T19:30:00+00:00

Introduction

Football analytics has come a long way in recent years, moving from simple league tables to more sophisticated methods of quantifying team performance. If you’ve ever looked at Elo ratings or FIFA rankings, you know that rating systems attempt to provide a clearer picture of how good a team really is, beyond just the wins and losses. But are these systems as accurate as they could be?

Imagine two teams: Team A beats Team B 1-0 in a closely fought match, while Team C thrashes Team D 5-0. Should Team A and Team C gain the same rating boost? Many traditional rating systems don't differentiate much between these results, even though one clearly signals a more dominant performance. This is where Pi Ratings come in — a dynamic rating system designed to better reflect team ability by considering score discrepancies, home vs. away performances, and recent form.

Pi Ratings were first introduced by Constantinou & Fenton in their research on dynamic football team ratings. Their study showed that Pi Ratings not only provided a more accurate measure of team strength compared to traditional systems like Elo, but also demonstrated profitability against bookmaker odds over five English Premier League seasons.

In this article, we’ll explore:

Where traditional rating systems, like Elo, lack the nuance for football’s specific demands
How Pi Ratings improve upon them
Some real-world results comparing the two
How you can use Pi Ratings in your own football analytics work

If you’re interested in football (soccer) data or just want a better way to evaluate your team’s strength, this is for you. Let’s dive in.

Why Traditional Ratings Systems (Like Elo) Fall Short

Rating systems play in important role in football analytics, providing a way to compare teams and predict future performance. One of the most widely used methods is the Elo rating system, originally developed for ranking chess players and later adapted for sports, including football. While Elo has proven useful in capturing team strength over time, it has several key limitations when applied to football.

Elo Ratings: A Brief Overview

Elo ratings work by assigning a numerical value to each team, which is adjusted after every match based on the result. If a higher-rated team wins, it gains only a small increase in its rating, whereas an underdog victory leads to a more significant rating adjustment. The formula accounts for the expected probability of winning, meaning that an upset results in a greater shift than an expected victory.

The appeal of Elo lies in its simplicity: teams are ranked on a single scale, and their relative strength is updated dynamically based on match outcomes. However, despite its widespread use, Elo has several shortcomings that reduce its effectiveness in football.

Draws Are Not Handled

An significant limitation of the standard Elo system is that it only accounts for wins and losses — it doesn't natively handle draws. While this works well in sports like chess or tennis, it's a major drawback in football, where draws are a common and meaningful outcome.

Some football adaptations of Elo attempt to account for this by treating draws as "half a win," but prediction-wise, Elo doesn't model draws probabilistically — it simply doesn't output a draw probability. We'll return to this point later when we compare how Pi and Elo predictions perform head-to-head.

Score Margins Are Ignored

Elo ratings consider only whether a team wins, loses, or draws, but do not take into account the margin of victory. A 1-0 win is treated the same as a 5-0 win, even though the latter provides a much stronger indication of dominance. Since goal differences carry valuable information about team strength, ignoring them can lead to inaccurate assessments of performance.

Home and Away Performances Are Not Handled Separately

Football teams often exhibit significantly different performances at home and away due to factors such as crowd support, pitch familiarity, and travel fatigue. Traditional Elo ratings apply the same formula regardless of match location, failing to account for these home and away discrepancies. Some adaptations of Elo introduce a fixed home advantage adjustment, but this is often static and uniform across teams, whereas in reality, home advantage could vary by club and competition.

Slow Adaptation to Recent Form

Elo ratings update dynamically, but the changes are incremental and cumulative. This means that a team experiencing a sudden surge or decline in form may not have its rating adjust quickly enough to reflect its current strength. For instance, a team suffering from injuries to key players or undergoing a managerial change may take several matches before its Elo rating adequately reflects the shift in performance.

What Are Pi Ratings?

Pi Ratings (Constantinou & Fenton (2013)) are a dynamic rating system designed specifically for football, addressing several key shortcomings of traditional methods like Elo.

Understanding Pi Ratings

Pi Ratings are built on three key principles:

Score Margins Matter: A team winning 5-0 should receive a greater rating boost than a team winning 1-0, as the larger margin suggests a stronger performance. Conversely, a narrow loss may not indicate a major decline in ability, whereas a heavy defeat should trigger a more significant rating drop.
Home and Away Ratings Are Separate: Instead of using a fixed home advantage adjustment, Pi Ratings maintain distinct ratings for home and away performances. This allows the system to recognize that some teams perform significantly better at home than away, while others are more consistent across venues.
Recent Performance Is More Important: A team’s current form is more relevant than performances from months ago. Pi Ratings incorporate a learning rate that ensures recent matches influence a team’s rating more strongly than older results - you can think of this as being somewhat similar to the Dixon and Coles decay rate we discussed in my previous article on predicting football match outcomes.

Interpreting Pi Ratings

Each team begins with a rating of 0, which represents the level of an average team in the data. The system is zero-centered, meaning that when one team gains rating points, the other team loses the same amount. This ensures that all ratings are relative — a team with a rating of +1.0 is one goal better, on average, than the typical opponent. This property also makes it possible to compare teams across leagues or seasons.

How Pi Ratings Are Calculated

Pi Ratings update dynamically after each match by comparing what was expected to happen with what actually happened — specifically, in terms of the goal difference. Here's how the process works:

All teams start with a neutral rating of 0, representing the average team in the dataset. Ratings rise or fall based on performance over time
Before a match, an expected goal difference is calculated using the home team’s Pi Rating and the away team’s Pi Rating
After the match, the actual goal difference is compared to this expectation. If a team overperforms or underperforms significantly, the rating adjustment is larger.
The greater the surprise, the bigger the adjustment. For example, if a team was expected to lose 2–0 but instead wins 3–0, that’s a major signal of strength and leads to a sharp increase in rating.
Home and away performances are rated separately. A strong home result boosts the home rating, and vice versa for away games. However, each update also slightly nudges the other (home affects away, and away affects home), allowing the model to learn cross-context performance with a catch-up learning rate.

Now that we’ve covered the fundamentals of how Pi Ratings work, let’s see them in action. In the rest of the article, we’ll apply the Pi system to recent football seasons and compare its performance to an Elo model, to see whether Pi Ratings provide a more accurate representation of team strength.

Installing the penaltyblog Python Package

If you've not used the penaltyblog Python package before, you can install it using pip.

pip install penaltyblog

Calculating Pi Ratings Using `penaltyblog`

Let’s start by downloading some historical match data using penaltyblog, which provides an easy way to import football results from football-data.co.uk.

import penaltyblog as pb
import matplotlib as mpl
import matplotlib.dates as mdates
import matplotlib.pyplot as plt


df = pd.concat([
    pb.scrapers.FootballData("ENG Premier League", "2018-2019").get_fixtures(),
    pb.scrapers.FootballData("ENG Premier League", "2019-2020").get_fixtures(),
    pb.scrapers.FootballData("ENG Premier League", "2020-2021").get_fixtures(),
    pb.scrapers.FootballData("ENG Premier League", "2021-2022").get_fixtures(),
    pb.scrapers.FootballData("ENG Premier League", "2022-2023").get_fixtures(),
    pb.scrapers.FootballData("ENG Premier League", "2023-2024").get_fixtures(),
    pb.scrapers.FootballData("ENG Premier League", "2024-2025").get_fixtures(),
])

df = df[df["date"] < "2025-04-01"]

Next, we'll loop through all the historical results from oldest to latest and update each team's Pi Rating based on the score line.

pi_ratings = pb.ratings.PiRatingSystem()

for idx, row in df.iterrows():
    goal_diff = row["goals_home"] - row["goals_away"]
    pi_ratings.update_ratings(row["team_home"], row["team_away"], goal_diff, row["date"])

Manchester City's Pi Rating

Let's take a look at Manchester City's Pi Rating over time.

ratings = [x for x in pi_ratings.rating_history if x["team"] == "Man City"]
ratings = pd.DataFrame(ratings).sort_values("date")

plt.figure(figsize=(10, 5)) 
plt.plot(ratings["date"], ratings["home_rating"],  label="home", linewidth=3)
plt.plot(ratings["date"], ratings["away_rating"], label="away", linewidth=3)
plt.xlabel("Date", fontsize=14)  
plt.ylabel("Pi Rating", fontsize=14) 
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
plt.legend(fontsize=14)
plt.legend()
plt.tight_layout()
plt.show()

Figure 1: Manchester City's Pi Rating over time

We can see that it took about a year of data for Manchester City’s Pi Rating to stabilise after starting at zero, followed by a long stretch of dominance. A noticeable dip occurred during the first half of the 2023/2024 season, before a strong run of form saw them recover and eventually claim the title.

The dramatic downturn in the 2024/2025 season — at least by Manchester City’s usual standards — is clearly reflected in the ratings, with the lowest point arriving at the end of 2024. As someone who’s watched a lot of Manchester City over the years, the ratings align closely with how the team’s form felt to me watching.

Chelsea's Pi Rating

Let's take a look at Chelsea's Pi Rating next and overlay the dates they changed managers.

ratings = [x for x in pi_ratings.rating_history if x["team"] == "Chelsea"]
ratings = pd.DataFrame(ratings).sort_values("date")

plt.figure(figsize=(10, 5)) 
plt.plot(ratings["date"], ratings["home_rating"],  label="home", linewidth=3)
plt.plot(ratings["date"], ratings["away_rating"], label="away", linewidth=3)
plt.xlabel("Date", fontsize=14)  
plt.ylabel("Pi Rating", fontsize=14) 
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
plt.legend(fontsize=14)

date_num = mdates.datestr2num("2024-07-01") 
plt.vlines(x=date_num-5, ymin=-0.25, ymax=1.5, linestyles="dotted", colors="black")
plt.text(date_num, plt.ylim()[1]-0.1, "Maresca", ha="left", va="top", rotation=90) 

date_num = mdates.datestr2num("2023-07-01") 
plt.vlines(x=date_num-5, ymin=-0.25, ymax=1.5, linestyles="dotted", colors="black")
plt.text(date_num, plt.ylim()[1]-0.1, "Pochettino", ha="left", va="top", rotation=90) 

date_num = mdates.datestr2num("2023-04-06") 
plt.vlines(x=date_num-5, ymin=-0.25, ymax=1.5, linestyles="dotted", colors="black")
plt.text(date_num, plt.ylim()[1]-0.1, "Lampard", ha="left", va="top", rotation=90) 

date_num = mdates.datestr2num("2022-09-08") 
plt.vlines(x=date_num-10, ymin=-0.25, ymax=1.5, linestyles="dotted", colors="black")
plt.text(date_num, plt.ylim()[1]-0.1, "Potter", ha="left", va="top", rotation=90) 

date_num = mdates.datestr2num("2021-01-26") 
plt.vlines(x=date_num-15, ymin=-0.25, ymax=1.5, linestyles="dotted", colors="black")
plt.text(date_num, plt.ylim()[1]-0.1, "Tuchel", ha="left", va="top", rotation=90) 

date_num = mdates.datestr2num("2019-07-04") 
plt.vlines(x=date_num-15, ymin=-0.25, ymax=1.5, linestyles="dotted", colors="black")
plt.text(date_num, plt.ylim()[1]-0.1, "Lampard", ha="left", va="top", rotation=90) 

plt.legend()
plt.tight_layout()

Figure 2: Chelsea's Pi Rating over time

Figure 2 tells a familiar story for Chelsea fans: manager changes often followed periods of declining performance, as reflected in sharp dips in the Pi Ratings. The chart clearly shows how Graham Potter and Frank Lampard oversaw particularly difficult spells, with both home and away ratings dropping significantly. Mauricio Pochettino, by contrast, made a noticeable impact with the Pi Ratings climbing steadily during his tenure. Based on how quickly Pochettino turned Chelsea's rating around, it's perhaps a shame that he was only able to stay for one season.

Using Pi Ratings to Predict Matches

Beyond rating teams, Pi Ratings can also be used to estimate the probability of a home win, draw, or away win between two sides. In my previous article, we explored how well more detailed goals-based models performed on this dataset. Now, let’s put Pi Ratings to the test on the same matches and see how their predictive power compares.

Let's start off by downloading the same data we used in the previous article.

import numpy as np
import pandas as pd
import penaltyblog as pb
from tqdm import tqdm

df = pd.concat([
    pb.scrapers.FootballData("NLD Eredivisie", "2015-2016").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2016-2017").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2017-2018").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2018-2019").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2019-2020").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2020-2021").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2021-2022").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2022-2023").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2023-2024").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2024-2025").get_fixtures(),
])

df = df.sort_values('date').set_index('date', drop=False)
ftr_map = {"H": 0, "D": 1, "A": 2}
df['ftr_numeric'] = df['ftr'].map(ftr_map)

Next, we loop through the test data, using the Pi Ratings to predict each fixture's outcome. After making a prediction, we update the model with the actual result. To evaluate how well the model performs, we use the Ranked Probability Score (RPS), which measures the accuracy of probabilistic predictions.

I explained more about the code and the RPS metric in my previous article , so I won’t repeat everything here — but if you're new to these concepts, I definitely recommend giving it a read first.

start_date = df.query("season == '2023-2024'")["date"].min()
run_dates = df["date"][df["date"] >= start_date].unique()

pi_ratings = pb.ratings.PiRatingSystem()

predictions = []
observed = []

for date in tqdm(run_dates, desc="Processing dates"):
    test = df[df["date"] == date]
    for i in range(len(test)):
        try:
            homes = test["team_home"].values
            aways = test["team_away"].values
            fthg = test["fthg"].values
            ftag = test["ftag"].values
            outcomes = test["ftr_numeric"].values

            probabilities = pi_ratings.calculate_match_probabilities(homes[i], aways[i])
            predictions.append([probabilities["home_win"], probabilities["draw"], probabilities["away_win"]])
            observed.append(outcomes[i])

            goal_diff = fthg[i] - ftag[i]
            pi_ratings.update_ratings(homes[i], aways[i], goal_diff)
        except Exception as e:
            continue  

rps = pb.metrics.rps_average(predictions, observed)
print(f"RPS: {rps}")

RPS: 0.19905621086882397

The average RPS for Pi Ratings on this dataset was 0.199, which gives us a solid benchmark to compare against other models. While this number doesn’t mean much in isolation, lower values indicate more accurate probabilistic predictions — so the real test is how it stacks up next to Elo and the goals-based models from the previous article.

Comparing Elo and Pi Ratings

As mentioned earlier, standard Elo Ratings only handle wins and losses. This makes them less suitable for football, where draws are common and carry meaningful information — especially when it comes to making probabilistic predictions.

To allow for a fair comparison with Pi Ratings, I extended the standard Elo system in penaltyblog to handle draws in both rating updates and match predictions. In the rating updates, draws are treated as being worth half a win (0.5), a common adaptation in football Elo models.

To handle draws — which are common in football but not natively supported by standard Elo — I added a Gaussian-shaped draw probability model. This approach assumes draws are most likely when teams are closely matched and become less likely as the rating gap widens.

The draw probability is computed as:

$$\text{p_draw} = \text{draw_base} \cdot \exp\left(-\frac{(\text{elo_diff})^2}{2 \cdot (\text{draw_width})^2}\right)$$

Where:

$\text{draw_base}$ is the peak probability (default 0.3)
$\text{elo_diff}$ is the difference in Elo ratings between the two teams.
$\text{draw_width}$ is the width of the Gaussian distribution, controlling how quickly the probability of a draw decreases with increasing Elo rating difference.

start_date = df.query("season == '2023-2024'")["date"].min()
run_dates = df["date"][df["date"] >= start_date].unique()

elo_ratings = pb.ratings.Elo()

predictions = []
observed = []

for date in tqdm(run_dates, desc="Processing dates"):
    test = df[df["date"] == date]

    for i in range(len(test)):
        try:
            homes = test["team_home"].values
            aways = test["team_away"].values
            fthg = test["fthg"].values
            ftag = test["ftag"].values
            outcomes = test["ftr_numeric"].values

            probabilities = elo_ratings.predict_match_outcome_probs(homes[i], aways[i])
            predictions.append([probabilities["home_win"], probabilities["draw"], probabilities["away_win"]])
            observed.append(outcomes[i])
            elo_ratings.update_ratings(homes[i], aways[i], outcomes[i])
        except Exception as e:
            continue    

rps = pb.metrics.rps_average(predictions, observed)
print(f"RPS: {rps}")

RPS: 0.2041672568256303

The results show that Pi Ratings outperform Elo when it comes to predicting football matches. While both systems are simple and fast, Pi Ratings offer several advantages that make them better suited to the dynamics of football. They were designed specifically for the sport, natively handle draws, and update team ratings based on score margins — capturing not just whether a team won or lost, but how convincingly.

Pi Ratings also separate home and away performances and adapt more quickly to changes in form. These enhancements result in more accurate probability estimates, as reflected in the lower Ranked Probability Score. For anyone working with football data — especially in situations where simplicity and speed matter — Pi Ratings offer a powerful upgrade over traditional Elo.

Comparing Ratings Models to Full Statistical Models

While Pi Ratings offer a clear improvement over traditional Elo in the context of football, it's worth asking how they stack up against more advanced, purpose-built statistical models. These models — like Dixon and Coles, Bivariate Poisson, and Zero-Inflated Poisson — are designed specifically for predicting football scores, often incorporating detailed assumptions about goal distributions, time decay, and team-specific strengths.

Let’s compare the predictive accuracy of these models against both Pi Ratings and Elo, using the same dataset and evaluation metric (Ranked Probability Score).

Model	Ranked Probability Score (RPS)
Dixon and Coles	0.19137780685608083
Weibull Count	0.19141358825225932
Poisson	0.19154229559464445
Zero-inflated Poisson	0.19154043298013113
Negative Binomial	0.19155750459845977
Bivariate Poisson	0.19161764011301444
Pi Ratings	0.19905621086882397
Elo Ratings	0.2041672568256303

Table 1: Rank Probability Scores for the different model types

While the statistical models clearly lead in predictive accuracy, this comes with trade-offs. They often require more data, greater model complexity, and longer computation times — along with domain-specific assumptions that aren’t always easy to explain or adapt.

Pi Ratings, by contrast, strike an effective middle ground. They’re more accurate than Elo, require only basic match results, and are fast and interpretable — making them ideal for many real-world applications where simplicity, speed, and solid predictive power matter more than squeezing out marginal gains.

Conclusions

If you’re rating chess players, Elo is exactly what you need — it’s simple, elegant, and perfectly suited to win/loss outcomes. But football is a different game: draws are common, scorelines matter, and form fluctuates quickly. That’s where Pi Ratings shine.

They offer a smarter, football-specific alternative to Elo — capturing performance nuances, handling home and away differences, and adapting more quickly to change. While not as accurate as heavyweight statistical models, Pi Ratings strike a valuable middle ground: fast, interpretable, and amazingly powerful for something so simple.

If you're building a football model, running simulations, or just want a better way to track your team’s strength — start with Pi.

Football Prediction Models: Which Ones Work the Best?

2025-03-10T19:30:00+00:00

Introduction

I've recently released version 1.1.0 of my penaltyblog Python package, bringing significant improvements to the speed and predictive performance of football (soccer) goals models. With this update, I thought it would be a great opportunity to compare the different models available — such as Poisson, Dixon and Coles, and more — exploring how they work, how to optimize their parameters, and how they perform on real-world data.

Let's start off with a high-level look at the different models available, looking at how they work, what their strengths are and what their weaknesses are.

What is the Poisson Goals Model?

One of the most widely used models for predicting football scores is the Poisson goals model. It assumes that the number of goals scored by each team follows a Poisson distribution — essentially meaning that goals occur randomly over time, but with a certain average rate (known as λ). This rate is determined by factors like a team’s attacking strength, their opponent’s defensive ability, and whether they are playing at home or away.

Strengths of the Poisson Goals Model

Simple and efficient – It’s easy to understand and requires only a few parameters, making it quick to calculate.
Decent predictive power – Despite its simplicity, it often performs well in forecasting match outcomes.
Useful for betting and analytics – Many bookmakers and analysts use variations of the Poisson model as a baseline.

Weaknesses of the Poisson Goals Model

Overestimates extreme results – Because it assumes goal events are independent, it sometimes predicts too many high-scoring games.
Doesn’t handle low-scoring bias well – Football has more 0-0 and 1-1 draws than the model naturally expects.
Ignores game dynamics – It doesn’t account for strategic shifts (e.g., teams playing more defensively when leading).

To address some of these issues, more sophisticated models have been developed to try and combat some of the Poisson model's limitations.

What is the Dixon and Coles Goals Model?

The Dixon and Coles goals model is an improvement on the standard Poisson approach for predicting football scores. While the Poisson model assumes goals are scored independently, Dixon and Coles recognized that this assumption doesn’t always hold in real-world matches. In particular, low-scoring games, especially 0-0, 1-0, and 1-1, happen more often than a basic Poisson model would predict. To correct this, they introduced an additional adjustment factor that modifies the probabilities of low-score outcomes, making the model more accurate for practical football forecasting.

Strengths of the Dixon and Coles Goals Model

More realistic score predictions – It better accounts for the observed tendency of football matches to produce more low-score draws.
Improves predictive accuracy – By adjusting goal probabilities, it refines match outcome predictions, especially for betting and analytics.
Still relatively simple – It builds on the Poisson model without adding excessive complexity, making it practical to implement.

Weaknesses of the Dixon and Coles Goals Model

Less effective for high-scoring matches – Since it primarily corrects low-score probabilities, it may not offer much improvement for teams or leagues where high-scoring games are common.
Parameter estimation can be trickier – The extra adjustment introduces an additional parameter that needs to be optimized, making implementation more complex.
Assumes the same adjustment applies to all matches – The correction factor is applied uniformly, meaning it doesn’t adapt dynamically to different match contexts or team styles.

Overall, the Dixon and Coles model offers a small but meaningful refinement over Poisson, making it a better choice when predicting match results, particularly in low-scoring leagues or competitions where draws are common.

What is the Bivariate Poisson Goals Model?

The Bivariate Poisson goals model is a more advanced approach to predicting football match scores that builds on the standard Poisson goals model by introducing correlation between the number of goals scored by each team.

Unlike the basic Poisson goals model, which assumes that each team’s goal count is independent of the other, the Bivariate Poisson goals model acknowledges that certain match factors, such as overall game tempo, attacking intent, or defensive frailties, can influence both teams simultaneously and therefore both teams' scores are dependent on each other.

To achieve this, the model introduces a shared dependency term, which captures the extent to which goal-scoring events are linked. For example, in high-tempo matches where both teams play aggressively, a high-scoring game (like 3-2) may be more likely than two independent Poisson processes would suggest. And, in defensive matchups, both teams might struggle to score, increasing the likelihood of a low-scoring draw.

Strengths of the Bivariate Poisson Goals Model

Accounts for correlation in goal-scoring – Unlike simpler models, it recognizes that teams don’t score in isolation; match dynamics often affect both sides.
Improves predictions across all scorelines – While Dixon and Coles only corrects for low-score biases, the Bivariate Poisson goals model affects high-score predictions as well.
More flexible than Poisson and Dixon and Coles – it extends the standard Poisson goals framework without imposing arbitrary score adjustments.

Weaknesses of the Bivariate Poisson Goals Model

More complex to estimate – the additional correlation parameter makes it harder to fit the model to data, requiring more advanced statistical techniques.
Less interpretable than simpler models – while Poisson-based models are easy to explain, the dependency structure in Bivariate Poisson makes it more abstract.
Can be unnecessary for low-scoring leagues – if most matches end with 0-0, 1-0, or 1-1 scorelines, the extra complexity might not give noticeable accuracy gains over simpler models.

Overall, the Bivariate Poisson goals model is a strong choice for capturing the interaction between teams, particularly in leagues or matchups where attacking styles, game tempo, or defensive weaknesses lead to both high and low-scoring outcomes. However, its added complexity means it may not always be worth the effort compared with a simpler model like Dixon and Coles.

What is the Zero-inflated Poisson Goals Model?

The Zero-Inflated Poisson (ZIP) goals model is an extension of the standard Poisson goals model that accounts for the fact that football matches often have more goalless results (0-0, 1-0, 0-1) than a standard Poisson process predicts. In simple terms, the ZIP model assumes that some matches have a high probability of producing zero goals due to defensive tactics, lack of attacking quality, or other match-specific factors that the standard Poisson goals model does not capture.

This is done by introducing an extra parameter, which represents the probability that a match belongs to a special zero-inflated category rather than following the usual Poisson goal distribution. If a match is in this category, its goal count is forced to be zero. Otherwise, the number of goals is drawn from a standard Poisson distribution. This allows the model to explicitly account for the excess number of goalless matches while still modeling other scorelines using a Poisson process.

Strengths of the Zero-Inflated Poisson Goals Model

Better at modeling goalless matches – it accounts for the tendency of football matches to have more 0-0 results than a pure Poisson process would predict.
Improves accuracy for defensive teams – useful in competitions or teams where ultra-defensive tactics result in frequent low-scoring games.
Still relatively simple – it only adds one extra parameter to the Poisson model.

Weaknesses of the Zero-Inflated Poisson Goals Model

Only adjusts for excess zero-score games – the ZIP goals model specifically corrects for too many goalless matches but does not address other common Poisson goals model issues, such as overestimating extreme scorelines (e.g., 4-3 or 5-2 results).
Assumes zero-inflation is the same across all matches – the model applies a single probability for zero-goal inflation, meaning it doesn’t adapt dynamically to different teams, leagues, or match conditions (e.g., some teams naturally play more defensive football than others).
May not improve predictions in high-scoring leagues – in competitions where 0-0 results are not unusually frequent, the extra complexity of zero-inflation is unnecessary and may not lead to better forecasts.

Overall, the Zero-Inflated Poisson goals model is a useful model when working with leagues or teams that have a higher-than-expected number of goalless results; however, it doesn’t adjust for other scoreline distortions.

What is the Negative Binomial Goals Model?

The Negative Binomial goals model is an extension of the standard Poisson goals model that addresses the issue of overdispersion — where the variance in goal counts is greater than the mean, which the Poisson model cannot handle properly. In football data, overdispersion often occurs because real-world score lines include more variability than the simple Poisson process predicts.

The Negative Binomial goals model attempts to solve this by introducing an extra parameter, which allows the variance to be larger than the mean, making it more flexible in capturing a wider range of goal distributions. Instead of assuming a fixed rate of goal-scoring for each team, it accounts for additional variability in scoring ability between different matches. This makes it particularly useful for leagues or teams where results can fluctuate significantly from game to game.

Strengths of the Negative Binomial Goals Model

Handles overdispersion effectively – it allows for greater variance in goal counts, making it more accurate for leagues or teams with unpredictable scorelines.
More realistic high-score predictions – unlike the standard Poisson model, it doesn’t underestimate the frequency of extreme results (e.g., 4-2, 5-3), making it more reliable for goal-heavy leagues.
Still interpretable and relatively simple – it’s a natural extension of Poisson, meaning it retains much of the intuitiveness and ease of implementation while improving accuracy.

Weaknesses of the Negative Binomial Goals Model

Doesn’t model goal correlation between teams – while it improves goal variance, it still treats each team’s goal count as independent, meaning it doesn’t account for game dynamics where both teams’ performances are linked (unlike the Bivariate Poisson goals model).
Can be unnecessary for low-scoring leagues – if a competition has mostly 0-0, 1-0, or 1-1 matches, the added flexibility of the Negative Binomial goals model may not provide a significant advantage over simpler models like Poisson or Dixon and Coles goal models.
Requires an extra parameter to estimate – while not overly complex, it adds another layer of statistical estimation, which can make optimization and model fitting slightly more challenging compared to a basic Poisson model.

The Negative Binomial goals model is particularly useful in high-scoring leagues or tournaments where teams display inconsistent scoring patterns. It’s a strong alternative when Poisson-based models struggle with overdispersion, but it won’t help much in low-scoring leagues where correcting draw probabilities (as Dixon-Coles does) is more important.

What is the Weibull Count + Copula Goals Model?

Instead of assuming that goals follow a Poisson or Negative Binomial distribution, this approach uses a Weibull distribution, which allows for more flexible goal distributions. This can be useful since real-world goal distributions often don't follow the Poisson. The Weibull goals model may be able to better capture the empirical shape of goal distributions, accommodating overdispersion and other nuances in scoring patterns.

The model also incorporates a copula, which allows for goal-scoring correlation between teams. Unlike say the standard Bivariate Poisson model, which assumes a specific form of dependence, the copula framework is more flexible and can capture different types of relationships between teams' goal counts, such as how a team's attacking performance influences their opponent's defensive response.

Strengths of the Weibull Count + Copula Goals Model

More flexible than Poisson-based models – the Weibull count model does not assume a fixed mean-variance relationship, making it better at handling overdispersion and improving accuracy for extreme scorelines.
Accounts for goal correlation in a more general way – unlike the Bivariate Poisson model, which assumes a specific type of dependency, the copula approach allows for a wider range of correlations between teams' goal counts.
Better predictive power in certain contexts – Weibull Count + Copula approach can often outperform traditional Poisson-based models, particularly in leagues with more complex goal-scoring patterns.

Weaknesses of the Weibull Count + Copula Goals Model

Significantly more complex to implement – Unlike simpler Poisson-based models, this approach requires some heavy mathematical techniques to estimate the Weibull parameters and copula dependency structure, making it harder to apply in practice.
More computationally expensive – The added flexibility comes at a cost: fitting the model requires more intensive calculations, which can be impractical for large-scale applications compared to Poisson-based models.
Not always a clear improvement over simpler models – while it offers greater flexibility, in many cases simpler models like Dixon and Coles or Negative Binomial goals models still perform well enough without the added complexity, making this model potentially unnecessary for certain leagues or datasets.

This model is most useful in situations where traditional Poisson-based models struggle, for example, in leagues where goal distributions exhibit strong overdispersion or where team interactions significantly influence each other's scoring potential. However, it is significantly more time consuming to fit and may not always provide a clear improvement over simpler models.

The table below provides a quick reference to help you decide which model best suits your needs based on factors like scoring patterns, goal correlation, and computational complexity.

Model	Strengths	Weaknesses	Best Used For
Poisson	Simple, efficient, widely used	Overpredicts high scores, doesn’t handle low-score bias well	General forecasting, fast model training
Dixon & Coles	Corrects low-score biases, better real-world accuracy	Assumes fixed adjustment across all matches, extra parameter tuning needed	Low-scoring leagues, competitions with many draws
Bivariate Poisson	Models goal correlation between teams, useful for high-scoring matches	Complex, harder to interpret, computationally expensive	High-scoring leagues, capturing match dynamics
Zero-Inflated Poisson	Better at modeling 0-0 draws, improves accuracy for defensive teams	Only addresses goalless matches, assumes fixed zero-inflation probability	Ultra-defensive teams, low-scoring competitions
Negative Binomial	Handles overdispersion, more realistic high-score predictions	Still assumes independent goal counts, may be unnecessary for low-scoring leagues	High-scoring leagues, competitions with unpredictable scorelines
Weibull Count + Copula	More flexible goal distribution, accounts for complex scoring relationships	Highly complex, computationally expensive, difficult to implement	Leagues with extreme goal-scoring patterns, advanced predictive modeling

Table 1: Comparison of the different model types currently available in penaltyblog

Installing the penaltyblog Python Package

Now we've got the theory out of the way, let's look at how to use these models via the penaltyblog Python package. If you've not used it before, you can install it using pip.

pip install penaltyblog

Downloading Football Data the Easy Way

Next, we are going to need some data to fit the models to. We'll use penaltyblog's built in functionality to download data for the English Premier League from football-data.co.uk.

We start off by downloading the data, then we index the date column to speed up filtering the dataframe later on. Finally, we create a new column called ftr_numeric which maps the ftr column to a numeric value, so we can use it in the models more easily.

We're going to use the Dutch Eredivisie for this example, but you can use any league you want by changing the league and season arguments in the code.

import numpy as np
import pandas as pd
import penaltyblog as pb
import matplotlib.pyplot as plt
from tqdm import tqdm

df = pd.concat([
    pb.scrapers.FootballData("NLD Eredivisie", "2015-2016").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2016-2017").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2017-2018").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2018-2019").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2019-2020").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2020-2021").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2021-2022").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2022-2023").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2023-2024").get_fixtures(),
    pb.scrapers.FootballData("NLD Eredivisie", "2024-2025").get_fixtures(),
])

df = df.sort_values('date').set_index('date', drop=False)
ftr_map = {"H": 0, "D": 1, "A": 2}
df['ftr_numeric'] = df['ftr'].map(ftr_map)

To evaluate and compare the different models in our package, we employ a rolling time-based validation approach. Instead of training each model on the entire dataset at once, we simulate how they would perform in a real-world betting or forecasting scenario.

The process involves iterating through the dataset one date at a time, fitting the model only to the data available up to that date, and then using it to predict the outcomes of fixtures on that specific day.

This method ensures that our predictions are made without knowledge of future results, mimicking the conditions under which these models would be used in practice. By avoiding data leakage and preserving the natural temporal structure of football matches, we obtain a more fair and realistic assessment of each model’s predictive power.

Fitting the Goals Models

Let's start off by getting all the unique dates since the start of the 2023-2024 season to test the models on.

start_date = df.query("season == '2023-2024'")["date"].min()
run_dates = df["date"][df["date"] >= start_date].unique()
len(run_dates)

The next block of code implements our rolling time-based validation approach to evaluate a Poisson model by predicting match outcomes over time. The process iterates through each date in run_dates, simulating a real-world forecasting scenario where only past data is available when making predictions.

For each date:

A training set is created, including matches from the previous three years up to (but not including) the current date.
A test set is created, containing matches that occur on the current date.
The model is initialized and trained using the training set.
If the model successfully fits, predictions are made for the test fixtures.
The predicted probabilities (home_draw_away) and actual outcomes are stored for evaluation.

To ensure robustness:

The code skips dates with no fixtures.
Errors during prediction are caught and ignored to prevent the loop from breaking.
The final Ranked Probability Score (RPS) is computed to measure model performance.

We can repeat this process for each of the model types we're interested in to get a sense of how well they perform on our dataset.

predictions = []
observed = []

for date in tqdm(run_dates, desc="Processing dates"):
    lookback = pd.Timestamp(date) - pd.DateOffset(years=3)
    train = df[(df["date"] < date) & (df["date"] >= lookback)]
    test = df[df["date"] == date]

    clf = pb.models.DixonColesGoalModel(
        train["goals_home"],
        train["goals_away"],
        train["team_home"],
        train["team_away"],
    )
    try:
        clf.fit()

        if len(test) > 0:
            homes = test["team_home"].values
            aways = test["team_away"].values
            outcomes = test["ftr_numeric"].values

            for i in range(len(test)):
                try:
                    prediction = clf.predict(homes[i], aways[i])
                    predictions.append(prediction.home_draw_away)
                    observed.append(outcomes[i])
                except Exception as e:
                    continue
    except:
        continue

pb.metrics.rps_average(predictions, observed)

Measuring Performance of the Goals Models

We will use the Ranked Probability Score (RPS) to evaluate the accuracy of the models' forecasts, which is a metric that measures how well a model’s predicted probability distribution aligns with the actual outcome.

RPS is calculated as the squared difference between the cumulative predicted probabilities and the cumulative observed outcome. It ranges from 0 to 1, where lower values indicate better predictive performance.

In football modeling, a model with a lower RPS assigns higher probabilities to correct outcomes while distributing probabilities meaningfully across alternatives, making it more reliable for decision-making.

Table 2 below shows the results for all the different models against our Eredivisie dataset. We can see that:

Dixon and Coles performed the best with the lowest RPS.
Weibull Count followed closely behind.
Bivariate Poisson had the highest RPS, indicating the weakest performance on this set of data.

Model	Ranked Probability Score (RPS)
Dixon and Coles	0.19137780685608083
Weibull Count	0.19141358825225932
Poisson	0.19154229559464445
Zero-inflated Poisson	0.19154043298013113
Negative Binomial	0.19155750459845977
Bivariate Poisson	0.19161764011301444

Table 2: Rank Probability Scores for the different model types

Optimising the Goal Model's Lookback Window

In the previous section, we used a fixed three-year lookback window to train our models before making predictions. However, the optimal amount of historical data to use is not always clear — too much data might include outdated information, while too little could lead to overfitting to recent trends.

Next, we'll optimise this lookback window by systematically varying the amount of past data used for training. By looping through different window sizes (e.g., 1 year, 2 years, 3 years, etc.), we can assess how the model’s predictive performance changes based on the amount of historical data it has access to and see if we can improve the RPS further.

For simplicity, we'll just optimise the Dixon and Coles model since it performed best, but ideally you'd repeat this process for all the models you're interested in to find the best performing model on your dataset.

We'll use the same rolling time-based validation approach as before, but this time we'll vary the lookback window by just tweaking the lookback variable by changing the number of years we go back. For example, the code below gives us a lookback window of two years.

lookback = pd.Timestamp(date) - pd.DateOffset(years=2)

Figure 1 below shows the results for different lookback windows. We can see that initially adding more data improves the model's performance, but after around four seasons the RPS starts to increase again and the model's performance starts to degrade.

Figure 1: Dixon and Coles RPS using different lookback windows

Incorporating Time-Weighted Data: Enhancing Predictions with the Dixon and Coles Approach

Next, we'll optimise the weighting applied to the data based on the methodology proposed by Dixon and Coles in their seminal paper. Their approach acknowledges that more recent matches carry greater predictive value than older ones when modeling football outcomes so should be given more importance in the model's predictions.

This is done by introducing an exponential decay function so that older games contribute less to the model’s parameter estimates, allowing it to adapt more effectively to recent team performances. This weighting method helps balance the trade-off between using sufficient historical data and ensuring that the model remains responsive to current trends.

How the Dixon and Coles Weights are Calculated

The weighting function typically follows this form:

$$[ w_t = e^{-\xi (T - t)} ]$$

Explanation of Variables

$w_t$: Weight assigned to a match played at time $t$
$T$: The current date
$t$: The date the match is played
$\xi$: The decay factor (higher values cause faster decay)

How it Works

If $\xi$ = 0, all matches are weighted equally.
If $\xi$ is small (e.g., 0.01), older matches retain some influence.
If $\xi$ is large (e.g., 0.03), older matches lose influence faster.

To help visualise the effect of the weights, I've plotted a range of $\xi$ values in Figure 2 below. With a $\xi$ of zero, no weighting is applied and all historical fixtures carry the same importance. As we increase $\xi$, older matches carry less and less weight in the model's calculations and the model becomes more responsive to recent team performance trends.

Figure 2: Example Dixon and Coles weights

Optimising the Value of $\xi$

All of the models in the penaltyblog package support applying weights to the data being trained on, so we can easily optimise the value of $\xi$ by looping through a range of values and selecting the one that gives the lowest RPS.

penaltyblog also provides a function to automatically create the weights values based on Dixon and Coles approach, but you can also use your own weighting system if you prefer, as shown in the example below.

xi = 0.001
weights = pb.models.dixon_coles_weights(train["date"], xi)

clf = pb.models.ZeroInflatedPoissonGoalsModel(
    train["goals_home"],
    train["goals_away"],
    train["team_home"],
    train["team_away"],
    weights,
)

clf.fit()

Now we just need to update our rolling time-based validation from before to loop through a range of $\xi$ values and select the one that gives the lowest RPS.

results = []
for xi in np.arange(0.000, 0.005, 0.0005):
    predictions = []
    observed = []
    for date in tqdm(run_dates, desc="Processing dates"):
        lookback = pd.Timestamp(date) - pd.DateOffset(years=4)
        train = df[(df["date"] < date) & (df["date"] >= lookback)]
        test = df[df["date"] == date]

        weights = pb.models.dixon_coles_weights(train["date"], xi)
        clf = pb.models.ZeroInflatedPoissonGoalsModel(
            train["goals_home"],
            train["goals_away"],
            train["team_home"],
            train["team_away"],
            weights,
        )
        clf.fit()

        if len(test) > 0:
            homes = test["team_home"].values
            aways = test["team_away"].values
            outcomes = test["ftr_numeric"].values

            for i in range(len(test)):
                try:
                    prediction = clf.predict(homes[i], aways[i])
                    predictions.append(prediction.home_draw_away)
                    observed.append(outcomes[i])
                except Exception as e:
                    continue

    result = {
        "decay": xi,
        "rps": pb.metrics.rps_average(predictions, observed),
    }
    results.append(result)

x = [x["decay"] for x in results]
y = [x["rps"] for x in results]

plt.plot(x, y)
plt.xlabel("Decay")
plt.ylabel("RPS")

Figure 3 below shows the results for different values of $\xi$. We can see that the optimal value is around 0.001, reducing our RPS to 0.1890924613913105.

Figure 3: Dixon and Coles RPS using different values of $\xi$

Conclusions

In this article, we've explored penaltyblog's predictive models from the Poisson goal model to more advanced approaches like Dixon and Coles, Bivariate Poisson, and Weibull Count Copula goals models. By applying a rolling time-based validation approach, we evaluated their real-world predictive performance using the Ranked Probability Score (RPS).

We then optimized key parameters, including the lookback window, finding that using around four seasons of historical data strikes the best balance between capturing enough information and staying responsive to recent trends. Finally, we applied Dixon-Coles time weighting, tuning the decay factor to further improve prediction accuracy, reducing the RPS even more.

Key Takeaways

More advanced models often outperform the basic Poisson goals model (although not always), particularly in handling low-scoring biases and goal correlation.
Optimizing historical data usage prevents outdated matches from degrading predictions.
Applying time-weighting enhances accuracy, ensuring the model remains adaptable to recent team performances.

If you are interested in finding out what else the penaltyblog package can do then you can read the documentation here.

Thanks for reading!

Calculating Expected Threat in Python Using Linear Algebra

2025-01-08T19:30:00+00:00

Introduction

Imagine you're watching a soccer match and your team's midfielder has the ball at the halfway line. How dangerous is this position? What about if they dribble forward 10 yards? Or make a pass to the wing? Expected Threat (xT), originally developed Sarah Rudd and popularised by Karun Singh, attempts to answer these questions by quantifying the offensive value of every position on the pitch.

Unlike simpler metrics such as expected goals (xG) that only measure shot quality, xT evaluates both immediate shooting opportunities and the potential for creating future scoring chances. This makes it useful for analyzing buildup play and measuring contributions from those players who don't directly create shots.

Computational Approach

Karun Singh's implementation of expected threat relies on iterative calculations that gradually converge to the final xT values. While this approach works, we can actually take the more elegant approach of using linear algebra to calculate the xT values directly. By reformulating the problem as a system of linear equations, we can achieve the following benefits:

Calculate all values in a single step rather than multiple iterations
Eliminate the need to check for convergence
Improve processing speed by replacing nested loops with efficient matrix operations
Reduce code complexity
Express the model in standard linear algebra notation

In this article, I'll show how to use linear algebra to calculate xT efficiently, and walk through the process in Python, providing a step-by-step guide for implementing this approach in practice.

Understanding Expected Threat

Let's start off by understanding the theory behind xT, which can be described by the following equation:

$\mathrm{xT}_{x,y} = \underbrace{s_{x,y} \cdot g_{x,y}}_{\text{shot threat}} + \underbrace{m_{x,y} \cdot \sum_{(z,w) \in G} T_{(x,y)\to (z,w)} \cdot \mathrm{xT}_{z,w}}_{\text{move threat}}$

At its core, xT breaks down the value of any position on the pitch into two components: the immediate threat of scoring (shot threat) and the potential to create better opportunities through movement of the ball (move threat). Let's examine each component in more detail:

Shot Threat Component

The first term represents the immediate threat of scoring from position $(x,y)$:

$s_{x,y}$ represents the probability of taking a shot from this position
$g_{x,y}$ represents the probability of scoring if a shot is taken

Move Threat Component

The second term captures the potential threat from moving the ball:

$m_{x,y}$ is the probability of moving the ball (instead of shooting)
$T_{(x,y)\to (z,w)}$ represents the probability of moving the ball from position $(x,y)$ to position $(z,w)$
$\mathrm{xT}_{z,w}$ is the expected threat at the destination position

So, the equation attempts to measure a fundamental concept in soccer: the value of a position comes both from the immediate scoring opportunity and the potential to create better opportunities through moving the ball. For example, when Phil Foden carries the ball from the wing into the penalty area, he's moving from a low-xT position (around 0.01) to a much more dangerous position (around 0.3).

The recursive nature of this equation, where xT appears on both sides, is what typically necessitates an iterative solution. However, by recognizing xT as a system of linear equations, we can solve it directly using matrix algebra that can be solved efficiently using Python's linear algebra libraries.

Data

To calculate xT, we need event data capturing how the ball moves around the pitch. While the model can incorporate various types of ball movements, for simplicity we'll focus on three fundamental events in this article:

Shots
Goals
Passes

Data Structure

And for each event, we require the following information:

Starting position $(x, y)$
Ending position of passes $(\text{end_x}, \text{end_y})$
Event type (shot, goal, or pass)

Here's a sample of what my data looks like - note that these were events collected for the English Premier League between 2017 and 2023, giving us around 3,000,000 relevant events to fit the model to. Unfortunately, I can't share the data here but there are plenty of free data sources that you can use to replicate this.

x	y	end_x	end_y	event_type
45.8	21.8	53.3	26.7	pass
52.3	11.3	45.6	8.6	pass
38.4	50.1	40.0	64.8	pass
2.3	23.6	32.4	17.3	pass
62.9	9.4	63.8	1.6	pass

Table 1: Example of the data used here to calculate xT

Calculating xT

The Grid Approach

The first step in calculating xT is dividing the pitch into a grid. This transforms the continuous space of the soccer / football pitch into discrete zones that we can analyze. Each zone in the grid will have its own xT value, representing the expected threat when the ball is in that position.

Iterative vs Matrix Approach

The original methodology calculates xT through iteration:

Initialize each zone with base shooting probabilities
Update xT values by considering contributions from neighboring zones through ball movements
Repeat the process until xT values stabilize (typically requiring 5 iterations)

While effective, this method requires tracking convergence and is computationally more intensive for larger datasets.

Instead, we can reformulate the problem as a matrix equation:

$$\mathbf{X} = \mathbf{S} + \mathbf{M}\mathbf{T}\mathbf{X}$$

Where:

$\mathbf{X}$: vector of xT values we want to find
$\mathbf{S}$: vector representing direct shooting threat for each zone
$\mathbf{M}$: Diagonal matrix containing movement probabilities for each zone
$\mathbf{T}$: Transition matrix capturing probabilities of moving between zones

This approach allows us to solve for all xT values simultaneously, eliminating the need for iteration.

Choosing Grid Resolution

The choice of grid resolution for the pitch depends on the trade-offs between computational efficiency and spatial precision. Some common resolutions and their pros and cons are outlined below:

Resolution	Advantages	Disadvantages
$12 \times 8$ (Coarse)	More data per zone, Faster computation, more stable estimates	Less spatial detail
$16 \times 12$ (Fine)	Better spatial precision, finer tactical insights	Sparser data, Longer computation
$20 \times 15$ (Very Fine)	Highly detailed spatial patterns	Even sparser data, computationally intensive

Table 2: Grid resolution trade-offs

For this article, we'll use a $16 \times 12$ grid. This resolution strikes a balance between capturing detailed spatial patterns, such as distinguishing the penalty spot from the edge of the box, and maintaining sufficient data in each zone for reliable estimates. Additionally, the computational requirements are manageable with this resolution.

Next, let's dive into the Python implementation, showing how to construct the required matrices efficiently.

Computing Transition Probabilities

Before we can calculate xT, we need to understand how players use each zone of the pitch. This involves computing three key probabilities:

Required Probabilities

Shot Probability ($s_{x,y}$): The likelihood of a shot being taken from each zone, calculated as the ratio of shots taken to all events in that zone.
Goal Probability ($g_{x,y}$): The conversion rate of shots from each zone, determined as the ratio of goals scored to shots taken from that zone.
Movement Probability ($T_{(x,y)\to(z,w)}$): The likelihood of moving the ball from zone $(x, y)$ to zone $(z, w)$, based on observed transitions in the data.

Mapping to Grid Coordinates

To calculate xT, we need to map raw pitch coordinates onto our $16 \times 12$ grid system. Here’s how to map any position on the pitch to its corresponding grid cell:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from mplsoccer import Pitch


def map_coordinates_to_grid_vectorized(x, y, grid_width=16, grid_height=12):
    """
    Vectorized function to convert pitch coordinates (x, y) into grid cell indices,
    handling missing values.

    Parameters:
        x (pd.Series or np.array): X-coordinates on the pitch (normalized to 0-100).
        y (pd.Series or np.array): Y-coordinates on the pitch (normalized to 0-100).
        grid_width (int): Number of grid cells across the width of the pitch.
        grid_height (int): Number of grid cells across the height of the pitch.

    Returns:
        tuple: Two arrays (grid_x, grid_y) representing grid cell indices. Missing values
               will be represented as None.
    """
    PITCH_WIDTH = 100
    PITCH_HEIGHT = 100

    # Handle missing values (NaN or None)
    valid = ~pd.isna(x) & ~pd.isna(y)

    # Initialize arrays with None for invalid coordinates
    grid_x = np.full_like(x, None, dtype=object)
    grid_y = np.full_like(y, None, dtype=object)

    # Compute grid indices only for valid rows
    grid_x[valid] = np.floor(x[valid] * grid_width / PITCH_WIDTH).clip(0, grid_width - 1).astype(int)
    grid_y[valid] = np.floor(y[valid] * grid_height / PITCH_HEIGHT).clip(0, grid_height - 1).astype(int)

    return grid_x, grid_y

df = pd.read_csv("path/to/my/data.csv")

# Map starting coordinates to grid cells
df["grid_x"], df["grid_y"] = map_coordinates_to_grid_vectorized(df["x"], df["y"])

# Map ending coordinates to grid cells, handling missing values
df["grid_x_end"], df["grid_y_end"] = map_coordinates_to_grid_vectorized(df["end_x"], df["end_y"])

x	y	end_x	end_y	grid_x	grid_y	event_type
45.8	21.8	53.3	26.7	7	2	pass
52.3	11.3	45.6	8.6	8	1	pass
38.4	50.1	40.0	64.8	6	6	pass
2.3	23.6	32.4	17.3	0	2	pass

Table 3: Mapping coordinates to grid cells

By mapping our continuous pitch coordinates into discrete grid zones, we’ve converted our raw event data to a structured format suitable for xT calculations. For instance, a position at (50, 50) would map to grid zone (8, 6) in our $16 \times 12$ grid.

Next, we need to calculate three core matrices:

pass_count: Stores the number of passes originating from each grid zone.
shot_count: Records the number of shots taken from each grid zone.
goal_count: Tracks the number of goals scored from each grid zone.

These matrices will be used to calculate the probabilities ($s_{x,y}$, $g_{x,y}$, and $T_{(x,y)\to(z,w)}$) used in the xT computation.

Populating Event Count Matrices

Using our grid coordinates, we'll now populate the matrices that count the number of passes, shots, and goals in each grid zone. The following code achieves this by iterating through the event data and incrementing the corresponding grid cell:

# Initialize the empty grids
GRID_HEIGHT, GRID_WIDTH = 12, 16

pass_count = np.zeros((GRID_HEIGHT, GRID_WIDTH))
shot_count = np.zeros((GRID_HEIGHT, GRID_WIDTH))
goal_count = np.zeros((GRID_HEIGHT, GRID_WIDTH))

# Filter events by type
pass_df = df.query("event_type == 'pass'")
goals_df = df.query("event_type == 'goal'")
shots_df = df.query("event_type == 'shot'")

# Group by grid coordinates and count events
pass_counts = pass_df.groupby(["grid_y", "grid_x"]).size()
shot_counts = shots_df.groupby(["grid_y", "grid_x"]).size()
goal_counts = goals_df.groupby(["grid_y", "grid_x"]).size()

# Fill in the grids using the grouped data
for (y, x), count in pass_counts.items():
    pass_count[y, x] = count

for (y, x), count in shot_counts.items():
    shot_count[y, x] = count

for (y, x), count in goal_counts.items():
    goal_count[y, x] = count

Visualizing Event Count Matrices

To ensure the data looks correct, let’s visualize the count matrices as heatmaps. Each heatmap provides a spatial representation of the frequency of each event type across the pitch.

Shot Count heatmap

statistic = shot_count
y, x = statistic.shape
x_grid = np.linspace(0, 100, x + 1)
y_grid = np.linspace(0, 100, y + 1)
cx = x_grid[:-1] + 0.5 * (x_grid[1] - x_grid[0])
cy = y_grid[:-1] + 0.5 * (y_grid[1] - y_grid[0])
stats = dict(statistic=statistic, x_grid=x_grid, y_grid=y_grid, cx=cx, cy=cy)

pitch = Pitch(pitch_type='opta', line_zorder=2, pitch_color='#22312b', line_color='#efefef')
fig, ax = pitch.draw(figsize=(8, 4.125))

pcm = pitch.heatmap(stats, ax=ax, cmap='viridis')

Figure 1: Shot Count heatmap

Goal Count heatmap

statistic = goal_count
y, x = statistic.shape
x_grid = np.linspace(0, 100, x + 1)
y_grid = np.linspace(0, 100, y + 1)
cx = x_grid[:-1] + 0.5 * (x_grid[1] - x_grid[0])
cy = y_grid[:-1] + 0.5 * (y_grid[1] - y_grid[0])
stats = dict(statistic=statistic, x_grid=x_grid, y_grid=y_grid, cx=cx, cy=cy)

pitch = Pitch(pitch_type='opta', line_zorder=2, pitch_color='#22312b', line_color='#efefef')
fig, ax = pitch.draw(figsize=(8, 4.125))

pcm = pitch.heatmap(stats, ax=ax, cmap='viridis')

Figure 2: Goal Count heatmap*

Pass Count heatmap

statistic = pass_count
y, x = statistic.shape
x_grid = np.linspace(0, 100, x + 1)
y_grid = np.linspace(0, 100, y + 1)
cx = x_grid[:-1] + 0.5 * (x_grid[1] - x_grid[0])
cy = y_grid[:-1] + 0.5 * (y_grid[1] - y_grid[0])
stats = dict(statistic=statistic, x_grid=x_grid, y_grid=y_grid, cx=cx, cy=cy)

pitch = Pitch(pitch_type='opta', line_zorder=2, pitch_color='#22312b', line_color='#efefef')
fig, ax = pitch.draw(figsize=(8, 4.125))

pcm = pitch.heatmap(stats, ax=ax, cmap='viridis')

Figure 3: Pass Count heatmap

Calculate Event Probabilities

Now that we have the event counts for each grid cell, we can calculate the probabilities of each event type occurring in a given cell. We’ll handle cases where division by zero might occur by replacing NaN or infinite values with 0.

Notice that we also scale the move probability by 0.79. At the moment, our move probabilities presume a 100% chance that a pass successfully moves the ball to its target location. This is clearly not true as passes can be misplaced or intercepted, so we scale the move probability by the average pass accuracy in the dataset. This prevents us from overestimating the value of a pass when we solve for xT values using the linear algebra approach. There are more sophisticated ways to handle this, but for now, this approach does a good job of balancing the value of a pass versus that of a shot.

with np.errstate(divide='ignore', invalid='ignore'):
    move_probability = pass_count / (pass_count + shot_count)
    shot_probability = shot_count / (pass_count + shot_count)
    goal_probability = goal_count / shot_count
    goal_probability = np.nan_to_num(goal_probability, nan=0.0, posinf=0.0, neginf=0.0)
    move_probability *= 0.79

Pass Transition Matrices

We are now ready to calculate the pass transition matrix, which describes the probability of moving the ball from one grid zone to another. This matrix will have dimensions equal to the total number of grid cells (e.g., $192 \times 192$ for a $16 \times 12$ grid). Each element at row $i$ and column $j$ represents the probability of transitioning from grid zone $i$ to grid zone $j$ based on the observed transitions from our event data.

We start off by aggregating the event data by the source and target grid zones. This will give us a count of the number of passes from each source zone to each target zone, which we will use to calculate the transition probabilities between grid zones.

These probabilities are then assigned to the appropriate indices in the corresponding sub-matrix of t_matrices, mapping the transition probabilities from each starting grid zone to all possible ending zones. This process encodes the spatial patterns of ball movement across the pitch into a format that can be used more easily for the xT computation.

# Aggregate passes by start and end grid locations
pass_end_aggs = (
    pass_df
    .groupby(["grid_x", "grid_y", "grid_x_end", "grid_y_end"])
    .size()
    .reset_index(name="count_end")  # Count passes ending at each location
)

pass_start_aggs = (
    pass_df
    .groupby(["grid_x", "grid_y"])
    .size()
    .reset_index(name="count_start")  # Count passes starting from each location
)

# Merge the start and end aggregates and calculate pass probabilities
pass_aggs = pass_end_aggs.merge(pass_start_aggs, on=["grid_x", "grid_y"], how="left")
pass_aggs["pass_prob"] = pass_aggs["count_end"] / pass_aggs["count_start"]

# Drop unnecessary columns
pass_aggs = pass_aggs.drop(columns=["count_end", "count_start"])

# Initialize the transition matrices
GRID_HEIGHT, GRID_WIDTH = 12, 16
t_matrices = np.zeros((GRID_HEIGHT, GRID_WIDTH, GRID_HEIGHT, GRID_WIDTH))

# Populate the transition matrices
for _, row in pass_aggs.iterrows():
    start_x, start_y = int(row["grid_x"]), int(row["grid_y"])
    end_x, end_y = int(row["grid_x_end"]), int(row["grid_y_end"])
    t_matrices[start_y, start_x, end_y, end_x] = row["pass_prob"]

The xT Surface Map

Now that we have the pass transition matrices, we can finally calculate the expected threat (xT) for each grid zone. Rather than using the original convergence method, we will take the more efficient approach of using linear algebra to solve for xT in one step.

The code above constructs a system of linear equations to represent how xT flows across the pitch. Here's how it works at a high level:

Input probabilities: the probability matrices are flattened into vectors, representing the likelihood of events at each grid zone.
Matrix Setup ($\mathbf{A}$): the identity matrix ($\mathbf{A}$) is initialized, representing the default relationship where each grid cell depends only on itself. This is then adjusted by subtracting scaled transition probabilities from t_matrices, which describe how xT flows from one grid zone to others during possession.
Payoff Vector ($\mathbf{b}$): the payoff vector ($\mathbf{b}$) is constructed by combining shooting and goal probabilities, reflecting the immediate scoring threat for each grid cell.
Solving the System: The equation $\mathbf{A} \cdot \mathbf{x} = \mathbf{b}$ can now be solved using NumPy's np.linalg.solve to find $\mathbf{x}$, the expected threat (xT) for each grid zone.
Reshaping the Result: Finally, the solution vector $\mathbf{x}$ is reshaped back into a 2D grid format, aligning with the original pitch layout.

Our output now represents how both movement and shooting probabilities contribute to the scoring potential at every location on the pitch, producing the xT surface map.

N_CELLS = GRID_WIDTH * GRID_HEIGHT

# Flatten input probabilities
shot_prob_flat = shot_probability.flatten()
goal_prob_flat = goal_probability.flatten()
move_prob_flat = move_probability.flatten()

# Initialize matrix A as an identity matrix
A = np.eye(N_CELLS)

# Compute adjustments to A for movement probabilities and transitions
for i in range(N_CELLS):
    # Map flat index to grid coordinates
    y, x = divmod(i, GRID_WIDTH)

    # Subtract the scaled transition matrix from A
    A[i] -= move_prob_flat[i] * t_matrices[y, x].flatten()

# Create vector b (shooting payoff)
b = shot_prob_flat * goal_prob_flat

# Solve the linear system A * x = b
xt_grid_flat = np.linalg.solve(A, b)

# Reshape the result into the grid format
xt_grid = xt_grid_flat.reshape(GRID_HEIGHT, GRID_WIDTH)

Let's now visualize the xT surface map, to check the expected threat at every location on the pitch.

statistic = xt_grid

y, x = statistic.shape
x_grid = np.linspace(0, 100, x + 1)
y_grid = np.linspace(0, 100, y + 1)
cx = x_grid[:-1] + 0.5 * (x_grid[1] - x_grid[0])
cy = y_grid[:-1] + 0.5 * (y_grid[1] - y_grid[0])
stats = dict(statistic=statistic, x_grid=x_grid, y_grid=y_grid, cx=cx, cy=cy)

pitch = Pitch(pitch_type='opta', line_zorder=2, pitch_color='#22312b', line_color='#efefef')
fig, ax = pitch.draw(figsize=(8, 4.125))

pcm = pitch.heatmap(stats, ax=ax, cmap='viridis')

for i, row in enumerate(statistic):
    for j, val in enumerate(row):
        ax.text(cx[j], cy[i], f'{val:.2f}', ha='center', va='center', color='white', fontsize=6)

Figure 4: xT Grid

The results look sensible, with the highest expected threat being in the zones closest to the opponent's goal and the xT decreasing as you move away from the goal. Let's compare this to the xT grid created using the original convergence method shown on the Soccermatics website. There are some minor differences in values, most likely caused by the different datasets used, but the general shape of our xT surface map is reassuringly similar.

Figure 5: Soccermatics xT Grid calculated using the convergence method

Calculating Player xT

Now that we’ve calculated the xT grid for every cell on the pitch, we can take the analysis a step further by evaluating individual player contributions. By mapping player's actions onto the xT grid, we’ll quantify the offensive impact of each player, quantifying their ability to create and capitalize on scoring opportunities.

There are several ways to approach this analysis. For instance, we could focus solely on successful on-ball events, consider only events with positive xT values, or even calculate the net xT by subtracting the xT of negative events from positive ones. For simplicity, we’ll focus on successful passes with positive xT values. This allows us to quantify each player’s offensive impact while avoiding penalizing players for negative events. Additionally, it accounts for context — teams like Manchester City, known for their high possession and intricate passing style, often make backward passes to maintain control without any negative intent. Including such passes as negative contributions could unfairly skew the analysis.

For this article, we’ll also focus exclusively on passes rather than including goals. While it’s perfectly valid to incorporate goals, our interest here lies in evaluating players involved in the build-up play leading to scoring opportunities. Goals can often be better analyzed using a separate expected goals (xG) model, which is specifically designed to quantify the quality and likelihood of scoring from individual shots. By isolating passes, we can concentrate on the contributions players make during the progression and creation phases of an attack.

Let's take a look at the top players by xT for the English Premier League 2023/24 season.

# Load the data
player_df = df = pd.read_csv("path/to/player/data.csv")

# Map the coordinates to the grid
player_df["grid_x"], player_df["grid_y"] = map_coordinates_to_grid_vectorized(player_df["x"], player_df["y"])
player_df["grid_x_end"], player_df["grid_y_end"] = map_coordinates_to_grid_vectorized(player_df["end_x"], player_df["end_y"])

# Calculate the xT difference per action
player_df["xt_diff"] = player_df.apply(
    lambda row: xt_grid[int(row["grid_y_end"]), int(row["grid_x_end"])] - xt_grid[int(row["grid_y"]), int(row["grid_x"])],
    axis=1
)

# Filter for positive xT differences and aggregate by player
top_players = (
    player_df.query("xt_diff > 0")
    .groupby("player_id", as_index=False)["xt_diff"]
    .sum()
    .sort_values("xt_diff", ascending=False)
    .head(10)
)

	Name	xT
1	Pascal Groß	21.5
2	Rodri	21.3
3	Bruno Fernandes	18.7
4	Martin Ødegaard	18.4
5	Trent Alexander-Arnold	17.3
6	Kieran Trippier	15.2
7	Declan Rice	15.1
8	Bruno Guimarães	14.3
9	Morgan Gibbs-White	13.6
10	James Ward-Prowse	13.2

Table 4: Top 10 players by xT for the English Premier League 2023/24 season

Conclusion

By using linear algebra, we’ve introduced an efficient and elegant way to quantify player contributions to their team’s offensive efforts. This approach not only simplifies the computation of xT but also opens the door for more scalable analyses using larger datasets.

It’s also worth noting that this article has been intentionally simplified to (hopefully) make it more accessible and easy to understand. There are numerous ways to refine and expand upon this approach, which I'm planning on exploring in a future blog post.

Estimating Goal Expectancy From Bookmaker's Odds

2022-12-02T19:30:00+00:00

Introduction

I've explored lots of different ways of predicting football matches so far on this blog that have all been backwards looking - they take in historical data and try and use it to predict how many goals will be scored.

This goal expectancy is then used to work out the probabilities for the match, e.g. what's the probability the home team wins, or that it's a draw, what are the prices for the over / under markets etc.

This time we're going to do things the opposite way round and use the bookmaker's odds to predict goal expectancy.

Data

Let's start off by getting some bookmaker's odds from football-data.co.uk. This is really easy to do using the scraper from the penaltyblog python package, which can be installed via pip if you don't already have it.

One we have the data, we filter out the columns we don't need to make it a little easier to work with. The psh, psd and psa columns represent Pinnacle's home, draw and away odds for each fixture.

import penaltyblog as pb

df = pb.scrapers.FootballData("ENG Premier League", "2022-2023").get_fixtures()

cols = [
    "date",
    "team_home",
    "team_away",
    "goals_home",
    "goals_away",
    "psh",
    "psd",
    "psa",
]
df = df[cols]

print(df.head(5))

date	team_home	team_away	goals_home	goals_away	psh	psd	psa
2022-08-05	Crystal Palace	Arsenal	0	2	4.50	3.65	1.89
2022-08-06	Bournemouth	Aston Villa	2	0	3.93	3.58	2.04
2022-08-06	Everton	Chelsea	0	1	6.04	4.06	1.63
2022-08-06	Fulham	Liverpool	2	2	11.20	6.22	1.28
2022-08-06	Leeds	Wolves	2	1	2.39	3.33	3.30

The Overround

The odds in the psh, psd and psa columns are currently in decimal notation. Let's take the first fixture as an example and convert the odds into probabilities instead by taking the reciprocal.

example = df.iloc[0].copy()
example["psh"] = 1.0 / example["psh"]
example["psd"] = 1.0 / example["psd"]
example["psa"] = 1.0 / example["psa"]

print(example)

date	2022-08-05 00:00:00
team_home	Crystal Palace
team_away	Arsenal
goals_home	0
goals_away	2
psh	0.222222
psd	0.273973
psa	0.529101

Hmmm, something's not right here. Our fixture only has three possible outcomes (either the home team wins, the away team wins or it's a draw) so the probabilities should add up nicely to 1.0 but example["psh"] + example["psd"] + example["psa"] adds up to 1.025. Somehow, we're out by 2.5%.

The reason is the overround - this is an extra bit of margin the bookmaker has factored into the odds to help guarantee them a profit regardless of the fixture's outcome.

There are a whole load of different ways we can try and remove this overround and get back to the original odds before the bookmaker skewed them in their favour. We're going to use the power method included in the penaltyblog package.

import pandas as pd

def overround(row):
    odds = [
        row["psh"],
        row["psd"],
        row["psa"],
    ]

    odds = pb.implied.power(odds)

    return pd.Series(odds["implied_probabilities"])

df[["psh", "psd", "psa"]] = df.apply(overround, axis=1)

print(df.head())

date	team_home	team_away	goals_home	goals_away	psh	psd	psa
2022-08-05	Crystal Palace	Arsenal	0	2	0.214017	0.265241	0.520742
2022-08-06	Bournemouth	Aston Villa	2	0	0.246553	0.271239	0.482208
2022-08-06	Everton	Chelsea	0	1	0.157612	0.237040	0.605349
2022-08-06	Fulham	Liverpool	2	2	0.079856	0.147753	0.772391
2022-08-06	Leeds	Wolves	2	1	0.411107	0.293087	0.295806

Brilliant, we've now got the implied probabilities for the each fixture without the overround messing them up.

Goal Expectancy

Now we've got the data we need, how do we convert it into goal expectancies?

If you remember back to previous articles, we used the predicted number of goals to work out the probabilities for a home win, away win or a draw occurring.

This time though, we've got the probabilities for the outcomes but not for the goals scored. So, in theory all we need to do is try different combinations of home and away goals to see which ones give us those probabilities back.

We could do this through brute force by trying lots and lots of different score lines but that's slow and tedious so we're going to use scipy's minimizer to speed things up.

Instead of randomly trying lots of score lines, scipy will look at the error between our guesses versus the bookmaker's probabilities to work out what score lines to try out next and hopefully find us the answer more quickly.

To do this, we're going to need a function to measure the error between us and the bookmakers. We're going to use something called Mean Squared Error (MSE) as shown below.

from scipy.stats import poisson
import numpy as np

def _mse(params, home, draw, away):
    exp_params = np.exp(params)

    mu1 = poisson(exp_params[0]).pmf(np.arange(0, 16))
    mu2 = poisson(exp_params[1]).pmf(np.arange(0, 16))

    mat = np.outer(mu1, mu2)

    pred = np.array([
        np.sum(np.tril(mat, -1)), # home
        np.sum(np.diag(mat)), # draw
        np.sum(np.triu(mat, 1))]) # away

    obs = np.array([home, draw, away])

    mse = np.mean((pred - obs)**2)

    return mse

The _mse function takes an argument called params, which is the home and away goals to test and then exponentiates them. These values are then plugged into a Poisson distribution to get the probabilities for each team scoring 0-15 goals. We then take the outer product of these two sets of probabilities to get a matrix giving the probability of each possible scoreline.

This matrix is a bit like a spreadsheet, so in the example below we have a probability of 0.01041474 for nil-nil, 0.1207623 for 3-0, 0.00436915 for 1-2 and so on.

The sum of all the probabilities on the diagonal of this matrix gives us the overall probability of a draw, the sum of all the probabilities above the diagonal is the probability of an away win and the sum of all the probabilities below the diagonal is the probability of a home win.

We then just take the difference between our estimated probabilities and the bookmaker's probabilities, square them (to ensure they are not negative) and return the average back to the minimizer.

The job of the minimizer is then to find out what combination of expected home and away goals returns the lowest error. Let's set up the minimizer next.

from scipy.optimize import minimize

def goal_expectation(home, draw, away):
    options = {
        "maxiter": 1000,
        "disp": False,
    }    

    res = minimize(
        fun=_mse,
        x0=[0.5, -0.5],
        args=(home, draw, away),
        options=options,
    )

    output = {
        "home_exp": np.exp(res["x"][0]),
        "away_exp": np.exp(res["x"][1]),
        "error": res["fun"],
        "success": res["success"],
    }      

    return output

Our goal_expectation function takes the bookmaker's odds as inputs and returns the number of home and away goals predicted by the bookmaker.

The first part of the function just sets up some options to tell the minimizer to give up if it's not found a decent answer after 1000 attempts and not to print out a whole load of messages to the screen while it's doing it.

Next, we run the minimizer by passing in the function to minimise the error for. We also pass in some starting parameters for the home and away goals as x0 to give the minimizer a head start. Finally, we pass in the bookmaker's probabilities and the options we set up. We wrap things up by formatting the output from the minimizer to just return the information we're interested in.

Let's give it a go!

output = list()
for _, row in df.head().iterrows():
    res = goal_expectation(row["psh"], row["psd"], row["psa"])

    tmp = {
        "team_home": row["team_home"],
        "team_away": row["team_away"],
        "home_exp": res["home_exp"],
        "away_exp": res["away_exp"],       
        "success": res["success"],
        "error": res["error"]
    }

    output.append(tmp)

output = pd.DataFrame(output)    
print(output)

	team_home	team_away	home_exp	away_exp	success	error
0	Crystal Palace	Arsenal	0.837309	1.470508	True	8.111775e-12
1	Bournemouth	Aston Villa	0.921695	1.406204	True	6.030584e-10
2	Everton	Chelsea	0.727779	1.693048	True	6.227654e-12
3	Fulham	Liverpool	0.680463	2.510449	True	2.981267e-11
4	Leeds	Wolves	1.187333	0.960074	True	2.442655e-09

The success column shows lots of True values, meaning the minimizer successfully found a solution each time and the error column is full of lots of tiny numbers meaning our predictions should be pretty accurate 😃

Conclusions

This technique should give you decent estimates but there's still some room for improvement. If you go back to my article on the Dixon and Coles model, I discussed how the Poisson distribution can struggle somewhat with the probabilities for low scoring games and how Dixon and Coles used an adjustment factor to try and alleviate this.

We can also include this adjustment into our loss function to try and reduce our error even further. I haven't added it in here as I wanted to keep this article simple but you can try the Dixon and Coles adjusted version in my penaltyblog package by using the pb.models.goal_expectancy function.

Addendum

Just to make clear for anybody who is unsure, the goal expectancy discussed here is different to expected goals. Goal expectancy is how many goals the bookmaker is expecting based on their 1x2 odds, whereas expected goals is how many goals a team is expected to score based on the shots they have taken. They both have similar names but quite different meanings.

Penaltyblog Python Package Updated to v0.5.1

2022-11-04T19:30:00+00:00

Introduction

My penaltyblog python package has recently been updated to v0.5.1 so let's take a look at some of the new features.

Python 3.7 Support

By popular request, penaltyblog is now compatible with Python 3.7. The main reason for doing this is to allow it to run on Google Colab, which at the time of writing is still stuck on what is now a fairly old version of Python.

penaltyblog isn't included in Colab by default but can easily be installed via pip by running the command below within one of your notebook's cells

!pip install penaltyblog==0.5.1

Once it's installed, you can then import penaltyblog as normal and use all of its functions.

import penaltyblog as pb

understat = pb.scrapers.Understat("ENG Premier League", "2022")
fixtures = understat.get_fixtures()
fixtures.head()

Bayesian Hierarchical Goals Model

Another exciting update is the addition of a new goals model based on Bayesian hierarchical modelling. I explained the theory behind this approach in a previous article so I won't go into the theory here but this model is now included in the package.

The hierarchical model follows the same API as all the other goals models meaning that you also have the ability to optionally apply a decay weighting to the data so that more recent fixtures are considered more important when fitting the model.

Here's a quick example to get you started

import penaltyblog as pb

fd = pb.scrapers.FootballData("ENG Premier League", "2021-2022")
fixtures = fd.get_fixtures()

fixtures["weights"] = pb.models.dixon_coles_weights(fixtures["date"], 0.001)

model = pb.models.BayesianHierarchicalGoalModel(
    fixtures["goals_home"],
    fixtures["goals_away"],
    fixtures["team_home"],
    fixtures["team_away"],
    fixtures["weights"],
)

model.fit()

prediction = model.predict("Man City", "Chelsea")

print(prediction)

Bayesian Bivariate Poisson Goals Model

As well as the hierarchical model, I've also added in a Bayesian bivariate Poisson model as well. I'll probably write a separate article at some point to explain the theory behind the modelling so again I won't go into too many details here.

However, as I've mentioned in previous articles there is a common issue with Poisson-based models where they treat both team's scores as independent from each other.

This doesn't reflect reality though where each team's goals scored / conceded are likely not independent. For example, if the score is 0-0 with 15 minutes to go then the underdog may settle for a draw and not push to score. Or if a team goes a goal down early on they may park the buss to prevent a more humiliating score line.

The bivariate model attempts to account for this by modelling the underlying Poisson distributions as a bivariate function. So instead of having seperate Poisson distributions for the home and away teams, we have one combined distribution.

Here's another quick example to get you started. Notice how similar the code is to the hierarchical example above - all we have to do is change one word to switch out the model, making it easy to try out different approaches.

import penaltyblog as pb

fd = pb.scrapers.FootballData("ENG Premier League", "2021-2022")
fixtures = fd.get_fixtures()

fixtures["weights"] = pb.models.dixon_coles_weights(fixtures["date"], 0.001)

model = pb.models.BayesianBivariateGoalModel(
    fixtures["goals_home"],
    fixtures["goals_away"],
    fixtures["team_home"],
    fixtures["team_away"],
    fixtures["weights"],
)

model.fit()

prediction = model.predict("Man City", "Chelsea")

print(prediction)

So which model should you use 🤷

Unfortunately, there's no simple answer here. If you want something fast and reliable then go with the Dixon and Coles model. Otherwise, it you've got more time / computational power then try out the Bayesian models. The only real way of knowing though is backtesting them on your data to find out which ones work best for your particular use case.

So Fifa

The scrapers in penaltyblog have also been updated to include So Fifa. The get_players function essentially scrapes the front page of the website, which contains top-level player data. You can control the number of pages to scrape and how the data should be sorted to make it easier to just get the top-ranked players if that's all you're interested in.

import penaltyblog as pb

sofifa = pb.scrapers.SoFifa()

player_info = sofifa.get_players(max_pages=2, sort_by="potential")
print(player_info.head())

You can then use the get_player function to get more detailed stats about players you're interested in based on So Fifa's player ID.

import penaltyblog as pb
from time import sleep

sofifa = pb.scrapers.SoFifa()

player_info = sofifa.get_players(max_pages=1, sort_by="value")

players = list()
for id_ in player_info.index[:5]:
    tmp = sofifa.get_player(id_)
    players.append(tmp)
    sleep(1)

players = pd.concat(players)
print(players)

Remember to scrape nicely though, please don't crash someone's website by scraping too much / too fast.

What's Next?

My TODO list has plenty more modelling approaches to try and more websites to scrape but if there's anything else you think would be good to include then let me know.

Thanks for reading!

Ten Years of pena.lt/y/blog

2022-10-18T19:30:00+00:00

Introduction

Today marks the blog's ten year anniversary and what a journey it's been!

It all started whilst I was sat in Manchester Airport waiting for a flight to Dubrovnik to present at a scientific conference. I'd been watching Match of the Day the night before and was pondering some comment Alan Shearer had made that baffled me at the time. I can't remember what it was that he'd said but I do remember thinking it couldn't possibly be true.

When I got to my hotel in Dubrovnik, Alan Shearer's comment was still bugging me so I connected to the hotel's internet and started doing some Googling. I soon stumbled upon Simon Gleave's Scoreboard Journalism blog and was inspired to have a go at football analytics myself.

Career Changing

Within a year of starting the blog, I was contacted by a start-up company called Onside Analysis. Based largely on my blog, they offered me a job developing predictive models for the gambling industry and to try and help them expand into working with football clubs. I accepted the role and left the world of scientific reasearch / clinical statistics behind forever to become a data scientist.

Sadly, the role at Onside Analysis didn't last long as they were bought out by another company and I moved on as redundancy was looming. However, I still work within data science now and a large part of my career change was down to this blog.

The Highs and the Lows

Over the years, I've made friends, met interesting people and worked with football clubs and governing bodies all through this blog, which has given me some fantastic (and not so fantastic) experiences.

Some of the highs include my work being discussed by Rafa Benitez on his personal blog, being featured in an article by John Burn-Murdoch in the Daily Telegraph and receiving feedback from a large governing body that Sir Alex Ferguson was impressed by a piece of analysis I'd done for them "despite Sir Alex not normally liking that football analytics sort of thing" 😆

I've also presented at three Opta Pro conferences, as well as writing a fourth presentation that somebody else gave there on my behalf.

There have been some lows as well. I don't think I'll ever forget the humiliating taxi ride back to the train station after pretty much being laughed out of the room at a Premier League football club. They were a regular lower half team at the time and as part of a discussion about helping them with some work they challenged me to prove myself by recommending some attackers that were suitable transfer targets for them.

I enthusiastically made some suggestions based on a model I'd been developing and promptly was laughed at by the entire room, who took great pleasure in telling me none of my suggestions would ever be good enough for the Premier League. They then cancelled the rest of the day's meetings we had planned and I left for home never to be contacted by them again.

I can look back on it now with a wry smile but at the time I was quite disappointed as I'd taken a day off work, paid for an expensive train ticket and spent a lot of time preparing for the meeting only to be mocked. I think history is on my side though as all my suggestions went on to have much more successful careers than the player they eventually went on to sign. ¹ 😂

What's Next?

I released a new update to the penaltyblog python package last week, which adds in some new models and scrapers so keep an eye out on the blog for an article on those over the next few weeks.

After that, I've got a giant list of ideas to write about when I can find the time and data for them.

Here's to the next ten years of pena.lt/y/blog!

Footnotes

Because people are bound to ask, here's the players I recommended plus the teams they played for at the time, bearing in mind that the team in question regularly finished in the bottom half of the table:

Arkadiusz Milik (Ajax)
Michy Batshuayi (Marseille)
Wissam Ben Yedder (Toulouse)
Nabil Fekir (Lyon)

Milik moved to Napoli later that year and Batshuayi moved to Chelsea. Fekir almost moved to Liverpool in 2018 but the deal fell through due to injury and Ben Yedder went on to have a highly successful career with Monaco and France.

I have no regrets about my recommendations 😄

Scraping Football Data Using the penaltyblog Python Package

2022-08-05T19:30:00+00:00

Introduction

If you're interested in football analytics or trying to predict football results and odds for betting markets then you are going to need data. This article will walk through how to use the penaltyblog python package to scrape football (soccer) data from different websites and how to join data from different sources together.

Installing `penaltyblog`

The first thing you need to do is install the penaltyblog package from pypi using pip. Provided you've got Python 3 and pip already installed then you just need to run the command below in a terminal. If you've not got Python installed yet, then head over to Anaconda and install that first.

pip install penaltyblog

The Scrapers

penaltyblog contains scrapers for a number of different websites, included Understat, ESPN and football-data.co.uk. For consistency, each of the scrapers takes the same arguments when it is created. This is a standardized competition name and season. For example:

import penaltyblog as pb

understat = pb.scrapers.Understat("ENG Premier League", "2021-2022")
fb = pb.scrapers.FootballData("ENG Premier League", "2021-2022")
espn = pb.scrapers.ESPN("ENG Premier League", "2021-2022")

It doesn't matter which data source your are scraping, competition names always comprise the country's three letter code followed by it's name, and the season is start_year-end_year. The scraper then maps this to whatever the data source is calling the competition so you don't need to remember whether you are scraping La Liga, LaLiga, La Liga Primera Division or whatever else a particular website calls it.

You can get a list of available competitions for each scraper by calling its list_competitions() function

import penaltyblog as pb

understat = pb.scrapers.Understat.list_competitions()

['DEU Bundesliga 1',
 'ENG Premier League',
 'ESP La Liga',
 'FRA Ligue 1',
 'ITA Serie A',
 'RUS Premier League']

Fixtures

Let's go ahead and scrape ourselves fixtures from Understat for the English Premier League

import penaltyblog as pb

understat = pb.scrapers.Understat("ENG Premier League", "2021-2022")
under_fixtures = understat.get_fixtures()
under_fixtures.head()

	understat_id	datetime	team_home	team_away	goals_home	goals_away	xg_home	xg_away	forecast_w	forecast_d	forecast_l	season	competition	date
id
1628812800---brentford---arsenal	16376	2021-08-13 19:00:00	Brentford	Arsenal	2	0	1.888180	1.023850	0.6289	0.2287	0.1424	2021-2022	ENG Premier League	2021-08-13
1628899200---burnley---brighton	16378	2021-08-14 14:00:00	Burnley	Brighton	1	2	1.795480	1.685300	0.3894	0.2877	0.3229	2021-2022	ENG Premier League	2021-08-14
1628899200---chelsea---crystal_palace	16379	2021-08-14 14:00:00	Chelsea	Crystal Palace	3	0	1.187090	0.321701	0.6405	0.2822	0.0773	2021-2022	ENG Premier League	2021-08-14
1628899200---everton---southampton	16380	2021-08-14 14:00:00	Everton	Southampton	3	1	2.388630	0.580601	0.8359	0.1234	0.0407	2021-2022	ENG Premier League	2021-08-14
1628899200---leicester---wolverhampton_wanderers	16381	2021-08-14 14:00:00	Leicester	Wolverhampton Wanderers	1	0	0.668082	1.327140	0.1683	0.2750	0.5567	2021-2022	ENG Premier League	2021-08-14

If you look at the table above (you may need to scroll the table horizontally if you're on a small screen) then you'll notice that the column names have a consistent style to them, e.g. all in lowercase, formatted as snake case etc to make them easier to work with.

Where possible, the column names are also consistent between data sources. For example, the column for the home team is always called team_home whatever the site you're scraping. This may seem trivial but it's a huge time saving not having to try and remember what each different data source calls the same things.

Columns are always named as team_home, goals_home, xg_home etc so that if you print out the column names then related columns, such as team_home and team_away, appear next to each other. No more searching through a giant list of columns to try and find the one you want.

The data also comes with an id column as the dataframe's index so every row has a unique key associated with it comprising the timestamp plus the team names.

Merging Data Sources

Combining scraped data from multiple data sources can be tricky but the penaltyblog scrapers can help with this too. As a somewhat contrived example, let's try and combine Understat's xG scores with football-data.co.uk's betting odds for Bet365.

The first thing we need to do is scrape the data from both sites.

import penaltyblog as pb

under = pb.scrapers.Understat("ENG Premier League", "2021-2022")
ufix = under.get_fixtures()
ufix = ufix[["team_home", "team_away", "xg_home", "xg_away"]]

fb = pb.scrapers.FootballData("ENG Premier League", "2021-2022")
fbfix = fb.get_fixtures()
fbfix = fbfix[ ["team_home", "team_away", "b365_h", "b365_d","b365_a"]]

In theory, we should be able to merge the two datasets by joining on the team names. Let's give it a go and see what happens...

merged = ufix.merge(fbfix, on=["team_home", "team_away"], how="inner")
print(merged.shape)

Oh dear, we only have 240 fixtures instead of the 380 we would expect for a full season of Premier League fixtures 😕.

Unfortunately, both data sources use different team names. For example, Understat uses Manchester City whereas football-data uses Man City so we can't join on them since they don't match.

To get around this problem, all of the penaltyblog scrapers come with the ability to remap team names. penaltyblog doesn't have mappings for all the world's football teams (yet..) but there's enough for this example and you can easily extend the mappings with your own team names.

The mappings themselves are just a standard python dictionary, with the key as the team name you want to end up with and the value as a list of possibles choices to remap. The example below maps both Man Utd and Man United to Manchester United

{
  "Manchester United": ["Man Utd", "Man United"],
}

Let's try our join again but with the example team name mappings included. Notice how we can now simply join using the id column since it's unique per fixture and will be identical across both datasets now we've mapped the team names.

import penaltyblog as pb

mappings = pb.scrapers.get_example_team_name_mappings()

under = pb.scrapers.Understat("ENG Premier League", "2021-2022", mappings)
ufix = under.get_fixtures()

fb = pb.scrapers.FootballData("ENG Premier League", "2021-2022", mappings)
fbfix = fb.get_fixtures()

merged = ufix.merge(fbfix, left_index=True, right_index=True)
print(merged.shape)

Success! With just 12 lines of code (including the blank lines) we've scraped both Understat and football-data.co.uk then merged the two data sets together using the unique id the scrapers automatically created for us.

What Else Can the Scrapers Do?

Depending on the data source, the scrapers have additional functions for collecting extra data. For example, the Understat scraper can get you shot data, including the XY coordinates, and the ESPN scraper can get you player and team level data too.

import penaltyblog as pb

under = pb.scrapers.Understat("ENG Premier League", "2021-2022")
shots = under.get_shots("16376")

print(shots.head())

	competition	season	datetime	minute	result	x	y	x_g	player	h_a	player_id	situation	shot_type	match_id	team_home	team_away	goals_home	goals_away	date	player_assisted	last_action
id
1628812800---brentford---arsenal	ENG Premier League	2021-2022	2021-08-13 19:00:00	10	MissedShots	0.913	0.539	0.053	Frank Onyeka	h	9681	OpenPlay	Head	16376	Brentford	Arsenal	2	0	2021-08-13	None	Aerial
1628812800---brentford---arsenal	ENG Premier League	2021-2022	2021-08-13 19:00:00	11	ShotOnPost	0.908	0.315	0.118	Bryan Mbeumo	h	6552	OpenPlay	RightFoot	16376	Brentford	Arsenal	2	0	2021-08-13	Ivan Toney	Throughball
1628812800---brentford---arsenal	ENG Premier League	2021-2022	2021-08-13 19:00:00	21	Goal	0.874	0.698	0.052	Sergi Canos	h	1078	OpenPlay	RightFoot	16376	Brentford	Arsenal	2	0	2021-08-13	Ethan Pinnock	BallRecovery
1628812800---brentford---arsenal	ENG Premier League	2021-2022	2021-08-13 19:00:00	27	MissedShots	0.812	0.478	0.066	Sergi Canos	h	1078	OpenPlay	RightFoot	16376	Brentford	Arsenal	2	0	2021-08-13	Frank Onyeka	Pass
1628812800---brentford---arsenal	ENG Premier League	2021-2022	2021-08-13 19:00:00	29	MissedShots	0.892	0.357	0.081	Bryan Mbeumo	h	6552	OpenPlay	RightFoot	16376	Brentford	Arsenal	2	0	2021-08-13	Kristoffer Ajer	Chipped

Take a look at the penaltyblog documentation for more details and examples of each scraper.

What's Next?

There's plenty of other websites to add to the scrapers, with FBRef and whoscored next up on the TODO list but let me know if there's any others that should be included too.

After that, it's back to the modelling to add more techniques for predicting football results and betting markets.

Thanks for reading!

Predicting Football Results Using Bayesian Modelling with Python and PyMC3

2021-08-25T19:30:00+00:00

Introduction

So far in this series of articles we started off building a basic model for predicting football results using two independent Poisson variables inspired by M. J. Maher. This simple model worked surprisingly well, but had some issues correctly predicting low scores and draws due to the independence of the two Poissons.

In the second part of the series, we used ideas from Dixon and Coles to effectively add some dependance back in to our model via rho, which helped improve its performance with those tricky low scores and draws. We also added in a decay weighting to the data so older fixtures were considered less important that more recent ones when fitting the model.

Next, we're going to implement Baio and Blangiardo's Bayesian hierarchical model to see how well that performs compared with our models so far.

If you've not read through those previous articles yet, I suggest starting there first as it will provide you with the background to what we're doing here.

Let's get started!

The Data

The aim of the model we're building is to predict the number of goals each football (soccer) team will score when they play each other. As always, let's start off by downloading some historical fixtures from football-data.co.uk. Baio and Blangiardo's publication uses the Italian Serie A but we're going to see how their model fares on the English Premier League.

The code below uses the penaltyblog python package to download three season's worth of data into a pandas dataframe and renames the columns to match the syntax used by Baio and Blangiardo. The home_team and away_team columns contain the team names, and yg1 and yg2 are the number of goals scored by the home team and away team, respectively. We've also kept the date column, and the result of each fixture.

import pandas as pd
import penaltyblog as pb

fixtures = pd.concat(
    [
        pb.footballdata.fetch_data("england", 2018, 0),
        pb.footballdata.fetch_data("england", 2019, 0),
        pb.footballdata.fetch_data("england", 2020, 0),
    ]
).rename(
    columns={
        "HomeTeam": "home_team",
        "AwayTeam": "away_team",
        "FTHG": "yg1",
        "FTAG": "yg2",
        "FTR": "result"
    }
)

fixtures = fixtures[["Date", "home_team", "away_team", "yg1", "yg2", "result"]]

fixtures.head()

	Date	home_team	away_team	yg1	yg2	result
0	2018-08-10	Man United	Leicester	2	1	H
1	2018-08-11	Bournemouth	Cardiff	2	0	H
2	2018-08-11	Fulham	Crystal Palace	0	2	A
3	2018-08-11	Huddersfield	Chelsea	0	3	A
4	2018-08-11	Newcastle	Tottenham	1	2	A

We need to give each team a unique ID, as this will make the data easier to handle later on.

import numpy as np

n_teams = len(fixtures["home_team"].unique())

teams = (
    fixtures[["home_team"]]
    .drop_duplicates()
    .sort_values("home_team")
    .reset_index(drop=True)
    .assign(team_index=np.arange(n_teams))
    .rename(columns={"home_team": "team"})
)

df = (
    fixtures.merge(teams, left_on="home_team", right_on="team")
    .rename(columns={"team_index": "hg"})
    .drop(["team"], axis=1)
    .merge(teams, left_on="away_team", right_on="team")
    .rename(columns={"team_index": "ag"})
    .drop(["team"], axis=1)
    .sort_values("Date")
)

df.head()

	Date	home_team	away_team	yg1	yg2	result	hg	ag
0	2018-08-10	Man United	Leicester	2	1	H	15	12
505	2018-08-11	Watford	Brighton	2	0	H	21	3
932	2018-08-11	Bournemouth	Cardiff	2	0	H	2	5
881	2018-08-11	Huddersfield	Chelsea	0	3	A	10	6
290	2018-08-11	Fulham	Crystal Palace	0	2	A	9	7

The hg column now contains the ID for the home team and the ag column is the ID for the away team. So for example, Manchester United are team 15 and Brighton are team 3.

Next, we split the data in to a train and test data set. The train data is used to fit the model, while we hold back 250 fixtures to test the model's performance on. We also extract the team IDs and goals scored from the train data into numpy arrays as it makes them a little easier to work with later on.

TEST_SIZE = 250
train = df.iloc[:-TEST_SIZE]
test = df.iloc[-TEST_SIZE:]

goals_home_obs = train["yg1"].values
goals_away_obs = train["yg2"].values
home_team = train["hg"].values
away_team = train["ag"].values

Defining the Model

The model itself is the same as the previous Poisson models we've created. We are still assuming that the number of goals a team scores is a function of their attack strength combined with the opposition's defence strength. In other words, teams with better attacks should score more and teams with worse defences should concede more. We also include home field advantage.

So the model still looks like this:

goals_home = home_advantage + home_attack + defence_away

goals_away = away_attack + defence_home

Where it changes this time though is how we implement it as we're going Bayesian...

Bayesian Statistics

Bayesian statistics is a very deep subject area, which I would certainly not be able to do justice to here so if you really want to dig into the details then I whole heartedly recommend reading Statistical Rethinking by Richard McElreath and Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan by John Kruschke. I'm by no means an expert in the subject but I've learnt a huge amount from these two books alone.

But at a very high level for us building our model (and apologies to any real Bayesians reading this), the biggest difference here using a Bayesian approach is that we get a distribution of values for each model parameter rather than the single values we had previously.

Bayesian modelling also lends itself well to using a technique known as hierarchical modeling (also known as multilevel modelling or mixed-effects modelling). This allows us to account for relationships between variables by considering them to come from a common distribution.

What that means for us is that we can avoid the problems the independent Poisson variables caused in our previous two models. By using a hierarchical model containing two conditionally independent Poisson variables, correlation between the two is naturally taken into account as our observable variables are mixed at an upper level in the model.

In other words, we don't need to bother applying the Dixon and Coles adjustment to our model's output anymore. Hurrah!

Building the Model

We're using the awesome PyMC3 library in Python to build our model, as shown below.

The home parameter refers to the advantage teams get from playing at home. This is a fixed effect that assumes the advantage is constant across all teams and across the entire time the dataset covers. We assign an uninformative flat prior to it, meaning that its value comes from a uniform distribution ranging anywhere from -infinity to +infinity.

atts and defs represent each team's attack and defence ratings. Although each team has its own unique ratings, they are estimated from a common distribution. Whilst each football team is clearly different to each other, they are all playing in the same league so we make the reasonable assumption that their ratings come from the same distribution. This allows us to pool related data together, which can often improve model estimates. Our prior here is that the ratings are normally distributed.

It's worth pointing out that we are subtracting the mean ratings from both the attack and defence ratings to ensure identifiability. There are countless combinations of parameters that could potentially give us the same results so this forces the model to give us a reproducible output and keeps the parameters in a realistic range.

The final chunk of code, where we calculate home_theta and away_theta, looks very similar to our previous models. We just sum up the ratings and home advantage to calculate the goal expectancy for each team and apply that to a Poisson distribution.

import pymc3 as pm
import theano.tensor as tt

with pm.Model() as model:
    # home advantage
    home = pm.Flat("home")

    # attack ratings
    tau_att = pm.Gamma("tau_att", 0.1, 0.1)
    atts_star = pm.Normal("atts_star", mu=0, tau=tau_att, shape=n_teams)

    # defence ratings
    tau_def = pm.Gamma("tau_def", 0.1, 0.1)
    def_star = pm.Normal("def_star", mu=0, tau=tau_def, shape=n_teams)

    # apply sum zero constraints
    atts = pm.Deterministic("atts", atts_star - tt.mean(atts_star))
    defs = pm.Deterministic("defs", def_star - tt.mean(def_star))

    # calulate theta
    home_theta = tt.exp(home + atts[home_team] + defs[away_team])
    away_theta = tt.exp(atts[away_team] + defs[home_team])

    # goal expectation
    home_points = pm.Poisson("home_goals", mu=home_theta, observed=goals_home_obs)
    away_points = pm.Poisson("away_goals", mu=away_theta, observed=goals_away_obs)

Fitting the Model

Drum roll... now we fit the model by calling PyMC3's sample function. We're sampling 2000 times across six chains, which gives us 12,000 samples in total to play with. We've also set tune to 1000, which means each chain will throw away 1000 samples before starting as these initial samples can sometimes be more uncertain.

with model:
    trace = pm.sample(2000, tune=1000, cores=6, return_inferencedata=False)

Let's check how well the model has done by looking at it's fitted parameters. We'll start off with the home advantage.

with model:
    pm.plot_trace(trace, var_names=["home"]);

As I mentioned above, with Bayesian models we get a distribution of possible parameter values rather than a single point estimate. Reassuringly, we've got a smooth curve here centred somewhere around 0.3, with is pretty close to the home advantage we saw for our Poisson models - our Dixon and Coles model in the last article gave us a value of 0.294.

We can also take a look at the attack and defence ratings.

with model:
    pm.plot_trace(trace, var_names=["atts", "defs"]);

Again, we've got nice looking curves which is reassuring but that looks a little busy with 20 different attack and defence curves so let's replot the data a little differently.

import arviz as az
import matplotlib.pyplot as plt

atts = (
    pd.DataFrame(az.stats.hdi(trace["atts"]), columns=["lower_hdi", "upper_hdi"])
    .assign(median=np.quantile(trace["atts"], 0.5, axis=0))
    .merge(teams, left_index=True, right_on="team_index")
    .drop(["team_index"], axis=1)
    .rename(columns={"team": "Team"})
    .assign(lower=lambda x: x["median"] - x["lower_hdi"])
    .assign(upper=lambda x: x["upper_hdi"] - x["median"])
    .sort_values("median", ascending=True)
)

plt.errorbar(
    atts["median"],
    atts["Team"],
    xerr=(atts[["lower", "upper"]].values).T,
    fmt="o",
)

defs = (
    pd.DataFrame(az.stats.hdi(trace["defs"]), columns=["lower_hdi", "upper_hdi"])
    .assign(median=np.quantile(trace["defs"], 0.5, axis=0))
    .merge(teams, left_index=True, right_on="team_index")
    .drop(["team_index"], axis=1)
    .rename(columns={"team": "Team"})
    .assign(lower=lambda x: x["median"] - x["lower_hdi"])
    .assign(upper=lambda x: x["upper_hdi"] - x["median"])
    .sort_values("median", ascending=False)
)

plt.errorbar(
    defs["median"],
    defs["Team"],
    xerr=(defs[["lower", "upper"]].values).T,
    fmt="o",
)

These last two plots show us the median value for each team's attack and defence rating (the dot in the middle), plus the high density interval or HDI (the horizontal line). You can think of the HDI as being the Bayesian equivalent to a 95% confidence interval.

Again, the results pass the eye test - Manchester City and Liverpool have the highest attack rating (meaning the model thinks they score the most goals) and the lowest defence rating (meaning they concede the fewest goals). Plus, we have Huddersfield and Norwich being terrible at both ends of the pitch.

The width of the HDI represents the model's uncertainty for the parameter, where the wider the bar the more uncertain we are of the parameter's true value. Typically, it tends to be wider here for teams that have been promoted / relegated as we have less data available for them in the data set.

Predicting Results

So far, so good. We've fit our model and its parameters look feasible so the next stage is to predict some results.

Things are slightly different with this model though as we don't have a specific value for each of our parameters. Instead, we have a distribution of values - in fact, we have 12,000 values for each parameter (remember that's how many samples we got back when we fit the model).

To simplify things though we'll just collapse our distributions down to a single value by taking the average of each parameter, which we can then use to calculate the goal expectation in a similar way to how we did it for our previous Poisson models.

This makes me sad though as one of the major advantages of Bayesian models is having these distributions so it seems a shame to throw them away and just keep their average. However, I seem to get the same results doing this compared with other approaches I've tried so let's go with it.

The chunk of code below shows how to do this. The first few lines just pull out the average parameters from the model's output for a given home and away team. We then use these values to calculate the goal expectation. If you've read the previous two articles in this series then there's nothing particularly unexpected here.

def goal_expectation(trace, home_team_id, away_team_id):   
    # get parameters
    home = np.mean(trace["home"])
    atts_home = np.mean([x[home_team_id] for x in trace["atts"]])
    atts_away = np.mean([x[away_team_id] for x in trace["atts"]])
    defs_home = np.mean([x[home_team_id] for x in trace["defs"]])
    defs_away = np.mean([x[away_team_id] for x in trace["defs"]])

    # calculate theta
    home_theta = np.exp(home + atts_home + defs_away)
    away_theta = np.exp(atts_away + defs_home)

    # return the average per team
    return home_theta, away_theta

We then just need to pass in the model's output and the relevant team IDs to get our predicted number of goals. Let's give it a try for Manchester City Vs Manchester United.

goal_expectation(trace, 14, 15)

(2.40817399509211, 0.9015500181975004)

As a Manchester City fan, this makes me happy as it has City winning by 2.4 goals to 0.9 goals :)

Let's flip it around and have Manchester United as the home team.

goal_expectation(trace, 15, 14)

(1.2138865328696127, 1.7885438632930635)

Nice, even without the home advantage Manchester City is still winning.

Is Our Bayesian Model Any Good?

Now it's time to assess whether these predictions are actually any good or not. We'll do this by predicting the probability of home win / draw / away win for each fixture in our test data set and compare it to what actually happened using Rank Probability Scores to measure the error. I explained why we use Rank Probability Scores in the last article so take a look here if you want to know more.

To do this, we're going to need to convert our predicted score lines to win / draw / loss probabilities. This is exactly the same as how we did it in the previous articles. We just use the Poisson distribution to get the probability of scoring 0-10 goals for each team, then take the outer product of the two to get the probabilities for each possible scoreline.

The sum of the upper triangle of the resulting matrix is the probability an away win, the sum of the diagonal is the probability of a draw and the sum of the lower triangle is the probability of a home win.

from scipy.stats import poisson

def win_draw_loss(home_expectation, away_expectation, max_goals=10):
    h = poisson.pmf(range(max_goals+1), home_expectation)
    a = poisson.pmf(range(max_goals+1), away_expectation)    
    m = np.outer(h, a)

    home = np.sum(np.tril(m, -1))
    away = np.sum(np.triu(m, 1))
    draw = np.sum(np.diag(m))

    return home, draw, away

And here's the function to calculate the Rank Probability Score of our model on our test data. It's pretty simple, we just loop through the fixtures and decide whether the actual result was a home win, draw or away win. We then get the probabilities for each outcome from our model and calculate the Rank Probability Score for our predictions versus what actually happened. Finally, we return the average error.

def calculate_rps(df):
    rps = list()
    for idx, row in df.iterrows():
        if row["result"] == "H":
            outcome = 0
        elif row["result"] == "D":
            outcome = 1
        elif row["result"] == "A":
            outcome = 2

        h, a = goal_expectation(trace, row["hg"], row["ag"])
        predictions = win_draw_loss(h, a)

        rps.append(pb.metrics.rps(predictions, outcome))

    return np.mean(rps)

Let's give it a go and see what happens.

calculate_rps(test)

0.21976530250997778

We've got our average error but is this any good or not? Let's run our Dixon and Coles model over the same data and compare the error to see which is best.

def calculate_rps_dc(dc, df):
    rps = list()
    for idx, row in df.iterrows():
        if row["result"] == "H":
            outcome = 0
        elif row["result"] == "D":
            outcome = 1
        elif row["result"] == "A":
            outcome = 2

        predictions = dc.predict(row["home_team"], row["away_team"]).home_draw_away

        rps.append(pb.metrics.rps(predictions, outcome))

    return np.mean(rps)

train["weight"] = pb.poisson.dixon_coles_weights(train["Date"],  0.001)

dc = pb.poisson.DixonColesGoalModel(
    train["yg1"],
    train["yg2"],
    train["home_team"],
    train["away_team"],
    train["weight"]
)

dc.fit()

calculate_rps_dc(dc, test)

0.21589324705157467

Oh dear, the average error for our Bayesian model is actually slightly higher than for our Dixon and Coles model for this particular set of data :(

Why is this?

The first reason is that our Dixon and Coles model has a decay weighting built in to it. This means that it considers more recent data to be more important that older data when estimating the model's parameters. Currently, our Bayesian model lacks this so it considers all data to be equally important.

We know that team's strengths are dynamic and vary over time due to form, injuries, transfers etc so this is definitely a limitation of our current Bayesian approach.

Overshrinkage

The second reason is something called overshrinkage. This a well known problem associated with hierarchical models where more extreme values get pulled back towards the overall average of the data.

For us, it means that the best teams, such as Liverpool and Manchester City, will not appear as strong as they should be whilst the really bad teams, such as Huddersfield, will appear better than they should. Essentially, our parameter estimates get regressed towards the mean, which reduces the model's performance.

Baio and Blangiardo discuss this issue in their publication and propose a solution where by they restructure their model to put teams into one of three different groups - top teams, mid-table teams or bottom teams. The model then uses this grouping within the hierarchical model so that parameters are estimated from a distribution of similar quality teams to try and reduce the overshrinkage.

We've covered a lot of ground in this article already though so I'm going to save digging any further into the overshrinkage for a future blog post.

Conclusions

We've built a Bayesian model using Python and PyMC3 that's capable of predicting football results. Its performance is pretty close to our previous Dixon and Coles model despite being a simpler model in many ways - it doesn't have a decay weighting included yet and it doesn't require the rho adjustment either, plus we've not accounted for the overshrinkage issue - so we've got a pretty good base model here to build on.

Thanks for reading!

Predicting Football Results Using Python and the Dixon and Coles Model

2021-06-24T19:30:00+00:00

Introduction

In the last article, we built a model based on the Poisson distribution using Python that could predict the results of football (soccer) matches. Whilst the model worked fairly well, it struggled predicting some of the lower score lines, such as 0-0, 1-0, 0-1.

In this article we'll look at how Dixon and Coles added in an adjustment factor to improve the model's performance.

All the code here builds on my last article so if you've not read that yet then I highly recommend going back and reading that one first.

Okay, let's get coding!

The Data

Just like last time, our model will work by predicting the number of goals scored / conceded by each team when they play each other based on their historical performances. Let's grab the same data as last time from football-data.co.uk.

import pandas as pd

df = pd.read_csv("https://www.football-data.co.uk/mmz4281/1718/E0.csv")
df[["Date", "HomeTeam", "AwayTeam", "FTHG", "FTAG"]].head()

	Date	HomeTeam	AwayTeam	FTHG	FTAG
0	11/08/2017	Arsenal	Leicester	4	3
1	12/08/2017	Brighton	Man City	0	2
2	12/08/2017	Chelsea	Burnley	2	3
3	12/08/2017	Crystal Palace	Huddersfield	0	3
4	12/08/2017	Everton	Stoke	1	0

Building the model

The model itself doesn't change from last time. We are still assuming that the number of goals a team scores is a function of their attack strength combined with the opposition's defence strength. In other words, teams with better attacks should score more and teams with worse defences should concede more. We also include home field advantage.

So the model still looks like this:

goals_home = home_advantage + home_attack + defence_away

goals_away = away_attack + defence_home

Finding the Optimal parameters

Now we finally get to the part where things start to change. Dixon and Coles recognised that the Poisson model was struggling with the 0-0, 1-0 and 1-0 score lines and proposed adding in an extra parameter (known as $\rho$ or rho) to the model that provides dependance between these scores. This is done in such a way that ensures the marginal distribution remain Poisson shaped.

def rho_correction(goals_home, goals_away, home_exp, away_exp, rho):
    if goals_home == 0 and goals_away == 0:
        return 1 - (home_exp * away_exp * rho)
    elif goals_home == 0 and goals_away == 1:
        return 1 + (home_exp * rho)
    elif goals_home == 1 and goals_away == 0:
        return 1 + (away_exp * rho)
    elif goals_home == 1 and goals_away == 1:
        return 1 - rho
    else:
        return 1.0

Next, we need to tweak our log_likelihood function from the last article to calculate the new rho_correction and add it on to our log-likelihood.

def log_likelihood(
    goals_home_observed,
    goals_away_observed,
    home_attack,
    home_defence,
    away_attack,
    away_defence,
    home_advantage,
    rho,
):
    goal_expectation_home = np.exp(home_attack + away_defence + home_advantage)
    goal_expectation_away = np.exp(away_attack + home_defence)

    home_llk = poisson.pmf(goals_home_observed, goal_expectation_home)
    away_llk = poisson.pmf(goals_away_observed, goal_expectation_away)
    adj_llk = rho_correction(
        goals_home_observed,
        goals_away_observed,
        goal_expectation_home,
        goal_expectation_away,
        rho,
    )

    if goal_expectation_home < 0 or goal_expectation_away < 0 or adj_llk < 0:
        return 10000

    log_llk = np.log(home_llk) + np.log(away_llk) + np.log(adj_llk)

    return -log_llk

We also need make some minor updates to the code we wrapped around scipy's optimizer to add in rho. Compared with the version from the last article, we've added an extra value to params for rho and updated _fit to pass rho through to the log_likelihood function. We're also printing out res["fun"], which is the value of the log likelihood our optimised set of parameters gives us.

from pprint import pprint
import numpy as np
from scipy.optimize import minimize
from scipy.stats import poisson


def fit_poisson_model():
    teams = np.sort(np.unique(np.concatenate([df["HomeTeam"], df["AwayTeam"]])))
    n_teams = len(teams)

    params = np.concatenate(
        (
            np.random.uniform(0.5, 1.5, (n_teams)),  # attack strength
            np.random.uniform(0, -1, (n_teams)),  # defence strength
            [0.25],  # home advantage
            [-0.1], # rho
        )
    )

    def _fit(params, df, teams):
        attack_params = dict(zip(teams, params[:n_teams]))
        defence_params = dict(zip(teams, params[n_teams : (2 * n_teams)]))
        home_advantage = params[-2]
        rho = params[-1]

        llk = list()
        for idx, row in df.iterrows():
            tmp = log_likelihood(
                row["FTHG"],
                row["FTAG"],
                attack_params[row["HomeTeam"]],
                defence_params[row["HomeTeam"]],
                attack_params[row["AwayTeam"]],
                defence_params[row["AwayTeam"]],
                home_advantage,
                rho
            )
            llk.append(tmp)

        return np.sum(llk)

    options = {
        "maxiter": 100,
        "disp": False,
    }

    constraints = [{"type": "eq", "fun": lambda x: sum(x[:n_teams]) - n_teams}]

    res = minimize(
        _fit,
        params,
        args=(df, teams),
        constraints=constraints,
        options=options,
    )

    model_params = dict(
        zip(
            ["attack_" + team for team in teams]
            + ["defence_" + team for team in teams]
            + ["home_adv", "rho"],
            res["x"],
        )
    )

    print("Log Likelihood: ", res["fun"])

    return model_params

Let's run the code and see what the model's parameters look like.

model_params = fit_poisson_model()
pprint(model_params)

'Log Likelihood:' 1050.8007455859752

{'attack_Arsenal': 1.4475806767563462,
 'attack_Bournemouth': 0.9564134429072025,
 'attack_Brighton': 0.6846752119668259,
 'attack_Burnley': 0.6983313339318459,
 'attack_Chelsea': 1.2572225786245756,
 'attack_Crystal Palace': 0.9493623391542453,
 'attack_Everton': 0.9377478892934925,
 'attack_Huddersfield': 0.48934382402256327,
 'attack_Leicester': 1.189901145845123,
 'attack_Liverpool': 1.5643592069690049,
 'attack_Man City': 1.786002855268045,
 'attack_Man United': 1.3309393567717684,
 'attack_Newcastle': 0.7670321482410185,
 'attack_Southampton': 0.7651647511032018,
 'attack_Stoke': 0.7195758814775219,
 'attack_Swansea': 0.46645544795474325,
 'attack_Tottenham': 1.4273383910058157,
 'attack_Watford': 0.9338520316567531,
 'attack_West Brom': 0.5837325635190684,
 'attack_West Ham': 1.0449689235308348,
 'defence_Arsenal': -0.9058108004220589,
 'defence_Bournemouth': -0.758464198660216,
 'defence_Brighton': -0.8946000243344525,
 'defence_Burnley': -1.2266994646301788,
 'defence_Chelsea': -1.2203046589466593,
 'defence_Crystal Palace': -0.8536543328557012,
 'defence_Everton': -0.8097571454511648,
 'defence_Huddersfield': -0.8264395373062519,
 'defence_Leicester': -0.7548204664366124,
 'defence_Liverpool': -1.1755563808950777,
 'defence_Man City': -1.515837101040477,
 'defence_Man United': -1.5183987188316523,
 'defence_Newcastle': -1.0403853901493407,
 'defence_Southampton': -0.8498059001250292,
 'defence_Stoke': -0.6625948352647537,
 'defence_Swansea': -0.865693751408734,
 'defence_Tottenham': -1.272001837824072,
 'defence_Watford': -0.7091269693877006,
 'defence_West Brom': -0.8701857668132795,
 'defence_West Ham': -0.6432122607551816,
 'home_adv': 0.29444537714494584,
 'rho': -0.1285194379447786}

Reassuringly, this look pretty similar to the parameters we got in the last article.

Even better, our log likelihood is 1050.80, which is lower than the 1052.34 we get if we re-run the model from the previous article. This means the new model fits the data better :)

But What Does rho Actually Do?

So how has adding rho into the model managed to improve things?

If you remember all the way back to the last article, I mentioned that one of the issues with our Poisson model was that it predicted each team's goals independently from each other.

In reality though, each team's goals scored are not independent from each other. For example, if the score is 0-0 with five minutes to go then the underdog may settle for a draw and not push to score. Or if a team goes a goal down early on they may park the buss to prevent a more humiliating score line.

Whilst goals are generally Poisson distributed, there are situations where their Poisson-ness breaks down somewhat. Dixon and Cole's adjustment using rho is an attempt to fix this by adding some dependance between the score lines.

As an example, let's look at the difference between the probabilities of of the 0-0, 1-0 and 0-1 scores before and after applying the Dixon and Coles adjustment.

We start off by getting the model's parameters and using them to calculate the number of goals we expect each team to score. Then we use this to get the probability of 0 and 1 goals being scored (the Dixon and Coles adjustment only affects those score lines of 0 and 1 goals so we won't bother going any higher).

We then take the outer product of the two sets of probabilities to create a matrix, apply the Dixon and Coles adjustment and print out how the probabilities changed.

home_team = "Arsenal"
away_team = "Chelsea"

home_attack = model_params["attack_" + home_team]
home_defence = model_params["defence_" + home_team]
away_attack = model_params["attack_" + away_team]
away_defence = model_params["defence_" + away_team]
home_advantage = model_params["home_adv"]
rho = model_params["rho"]

home_goal_expectation = np.exp(home_attack + away_defence + home_advantage)
away_goal_expectation = np.exp(away_attack + home_defence)

home_probs = poisson.pmf([0, 1], home_goal_expectation)
away_probs = poisson.pmf([0, 1], away_goal_expectation)

probability_matrix_before = np.outer(home_probs, away_probs)

probability_matrix_after = probability_matrix_before.copy()
probability_matrix_after[0, 0] *= 1 - home_goal_expectation * away_goal_expectation * rho
probability_matrix_after[0, 1] *= 1 + home_goal_expectation * rho
probability_matrix_after[1, 0] *= 1 + away_goal_expectation * rho
probability_matrix_after[1, 1] *= 1 - rho

print(probability_matrix_after - probability_matrix_before)

array([[ 0.01377996, -0.01377996],
       [-0.01377996,  0.01377996]])

The top-left value has increased by 0.01377996, which has been taken from the top-right value. This means the Dixons and Coles adjustment has increased the likelihood of a 0-0 score line occurring at the expense of 1-0.

The opposite has also occurred in the bottom row, where the bottom-right value has increased at the expense of the bottom-left value. This means the probability of a 1-1 score line has increased whilst 0-1 has decreased.

So Dixon and Cole's model increases the probability of low score draws compared with our basic Poisson model. It's not a massive change to the probabilities but it's certainly enough to improve the model's overall performance.

Time Decay

Now that we've sorted out the issue with the dependance between scores, we have a second problem we need to handle.

At the moment we're considering team's attack and defence ratings to be static. But in reality they actually change over time. Think about how Sheffield United finished 9th in 2019/2020 yet were bottom of the league in 2020/2021. If we were to train our model using both season's worth of data then we'd probably end up overrating how many goals we'd expect Sheffield United to have scored in 2020/2021.

We could solve this by only training our model over a short time window to ensure we just use the most recent fixtures for a team. But then we'd be at risk of the model reacting to short term trends in form rather than a team's true ability.

So how do we determine how much data to train the model on?

Dixon and Coles proposed using exponential decay to down-weight the importance of fixtures based on how long it is since they were played. This means the model can be trained on lots of historical data, with it placing more importance on the most recent fixtures but without completely losing the influence of older fixtures.

Here's the function they suggested:

$$\phi(t) = exp(-\xi t)$$

And here's its implementation in python, where xi controls the strength of the down-weighting and t is the time since the fixture was played.

def dc_decay(xi, t):
    return np.exp(-xi * t)

Lets plot out a few different values of xi to see what the down-weighting looks like, where the lower the weight the less important the fixture becomes.

import matplotlib.pyplot as plt
import seaborn as sns

sns.set()

t = np.array(range(1000))
xis = [0.001, 0.003, 0.005]

for xi in xis:
    plt.plot(t, dc_decay(xi, t), label=str(xi))

plt.legend()
plt.xlabel("Number of Days Ago")
plt.ylabel("Weight")

Optimising the Time decay

So the larger the value of xi then the more extreme the down-weighting is. But what value should xi be?

Ideally, you would optimize your model in a similar way to how we found our attack / defence parameters above and work out the exact value of xi that gives you the best performance. To keep the code a little simpler here though, we're just going to loop through a few different values to see the effect of tweaking xi.

Whilst we're still using log likelihood to optimise the model itself, we're going to need to use a different metric to compare the values of xi to find out which one is best. We'll do this using Rank Probability Scores.

Ranked Probability Scores were original proposed by Epstein back in 1969 as a way to compare probabilistic forecasts against categorical data. Their main advantage here is that as well as looking at accuracy, they also account for distance in the predictions e.g. how far out inaccurate predictions are from what actually happened. This means we can measure the accuracy of our win / draw / loss probabilities against each match's outcome and measure the overall error.

To keep things simple, we'll download the penaltyblog python package and use the rps function from there - the idea of this article is to focus on the Dixon and Coles model itself rather than get sidetracked with implementing different metrics.

Install the penaltyblog package by running the command below in your terminal.

pip install penaltyblog

We'll then use the penaltyblog package to download more data from football-data.co.uk, giving us five season's worth of fixtures to play with.

import penaltyblog as pb
import pandas as pd

df = pd.concat([
  pb.footballdata.fetch_data("england", 2016, 0),
  pb.footballdata.fetch_data("england", 2017, 0),
  pb.footballdata.fetch_data("england", 2018, 0),
  pb.footballdata.fetch_data("england", 2019, 0),
  pb.footballdata.fetch_data("england", 2020, 0),   
])

The next thing we need to do is to update our log_likelihood function to apply the weighting. It's a fairly simple change here, all we do is pass in the weight for the fixture as an additional argument and then multiply the log likelihood by the weight.

def log_likelihood(
    goals_home_observed,
    goals_away_observed,
    home_attack,
    home_defence,
    away_attack,
    away_defence,
    home_advantage,
    rho,
    weight
):
    goal_expectation_home = np.exp(home_attack + away_defence + home_advantage)
    goal_expectation_away = np.exp(away_attack + home_defence)

    home_llk = poisson.pmf(goals_home_observed, goal_expectation_home)
    away_llk = poisson.pmf(goals_away_observed, goal_expectation_away)
    adj_llk = rho_correction(
        goals_home_observed,
        goals_away_observed,
        goal_expectation_home,
        goal_expectation_away,
        rho,
    )

    if goal_expectation_home < 0 or goal_expectation_away < 0 or adj_llk < 0:
        return 10000

    log_llk = weight * (np.log(home_llk) + np.log(away_llk) + np.log(adj_llk))

    return -log_llk

Now we need to update the code that wraps the call to the optimizer. The main change here is that we pass in our value for xi and create weights for the fixtures based on how long ago they were played.

def fit_poisson_model(df, xi=0.0001):
    teams = np.sort(np.unique(np.concatenate([df["HomeTeam"], df["AwayTeam"]])))
    n_teams = len(teams)

    df["days_since"] = (df["Date"].max() - df["Date"]).dt.days
    df["weight"] = dc_decay(xi, df["days_since"])

    params = np.concatenate(
        (
            np.random.uniform(0.5, 1.5, (n_teams)),  # attack strength
            np.random.uniform(0, -1, (n_teams)),  # defence strength
            [0.25],  # home advantage
            [-0.1],  # rho
        )
    )

    def _fit(params, df, teams):
        attack_params = dict(zip(teams, params[:n_teams]))
        defence_params = dict(zip(teams, params[n_teams : (2 * n_teams)]))
        home_advantage = params[-2]
        rho = params[-1]

        llk = list()
        for idx, row in df.iterrows():
            tmp = log_likelihood(
                row["FTHG"],
                row["FTAG"],
                attack_params[row["HomeTeam"]],
                defence_params[row["HomeTeam"]],
                attack_params[row["AwayTeam"]],
                defence_params[row["AwayTeam"]],
                home_advantage,
                rho,
                row["weight"],
            )
            llk.append(tmp)

        return np.sum(llk)

    options = {
        "maxiter": 100,
        "disp": False,
    }

    constraints = [{"type": "eq", "fun": lambda x: sum(x[:n_teams]) - n_teams}]

    res = minimize(
        _fit,
        params,
        args=(df, teams),
        constraints=constraints,
        options=options,
    )

    model_params = dict(
        zip(
            ["attack_" + team for team in teams]
            + ["defence_" + team for team in teams]
            + ["home_adv", "rho"],
            res["x"],
        )
    )

    return model_params

Once we've fit our model, we'll use it to calculate the probabilities of a win / draw / loss occurring for each fixture and then measure the rank probability score of our predictions compared with what actually happened.

We will also split the data up so that we train on the first fours seasons and predict on the final season. This means our model is predicting on different data to that it was trained on to help avoid overfitting.

Here's the updated predict function that returns home win, draw, away win probabilities.

def predict(params, home_team, away_team):
    home_attack = params["attack_" + home_team]
    home_defence = params["defence_" + home_team]
    away_attack = params["attack_" + away_team]
    away_defence = params["defence_" + away_team]
    home_advantage = params["home_adv"]
    rho = params["rho"]

    home_goal_expectation = np.exp(home_attack + away_defence + home_advantage)
    away_goal_expectation = np.exp(away_attack + home_defence)

    home_probs = poisson.pmf(range(10), home_goal_expectation)
    away_probs = poisson.pmf(range(10), away_goal_expectation)

    m = np.outer(home_probs, away_probs)

    m[0, 0] *= 1 - home_goal_expectation * away_goal_expectation * rho
    m[0, 1] *= 1 + home_goal_expectation * rho
    m[1, 0] *= 1 + away_goal_expectation * rho
    m[1, 1] *= 1 - rho    

    home = np.sum(np.tril(m, -1))
    draw = np.sum(np.diag(m))
    away = np.sum(np.triu(m, 1))

    return home, draw, away

And finally, here's the function that calculates the ranked probability score from those probabilities.

def calculate_rps(params, df):
    rps = list()
    for idx, row in df.iterrows():
        if row["FTR"] == "H":
            outcome = 0
        elif row["FTR"] == "D":
            outcome = 1
        elif row["FTR"] == "A":
            outcome = 2  

        predictions = predict(params, row["HomeTeam"], row["AwayTeam"])
        rps.append(pb.metrics.rps(predictions, outcome))

    return np.mean(rps)

All we have to do now is loop through some values for xi and see what happens. Warning - you may want to go and make a coffee at this point as it's going to take a while to loop through all this. At the moment the code is very much written for simplicity rather than speed but I'll show you a faster way to do this bit later on via the penaltyblog python package.

xis = [0, 0.00005, 0.0001, 0.001, 0.0025,]
rps = list()
for xi in xis:
    train = df.iloc[-380:]
    test = df.iloc[-380:]

    train["days_since"] = (train["Date"].max() - train["Date"]).dt.days
    train["weight"] = dc_decay(xi, train["days_since"])

    params = fit_poisson_model(train, xi)
    rps.append(calculate_rps(params, test))

plt.plot(xis, rps)
plt.ticklabel_format(useOffset=False)
plt.xlabel("xi")
plt.ylabel("RPS")

Rank probability scores are a measure of the error in our predictions so the smaller the value the better the model is performing. It looks like we are doing best with an xi of roughly 0.001. Smaller xi values are likely providing too much influence to the older data, whilst larger xi values are giving too much influence to the latest data.

In Dixon and Cole's paper, the authors report using an xi value of 0.0065. However, their time units are half weeks whilst ours are in days. If we divide 0.0065 by 3.5 (the number of days in a half week) then we get 0.00186, which is in line with our value of 0.001.

Conclusions

This has been a long article with a lot of code so thanks for reading it all the way to the end!

With relatively minor modifications, we've upgraded our Poisson model to use the Dixon and Coles adjustment to improve its predictions for the 0-0, 0-1 and 1-0 score lines.

We've also added in time decay so that we down-weight the influence of older data when we fit the model and give greater influence to more recent fixtures.

This isn't the end of the road though. There's plenty of other things we can do to improve our model, which I'll cover in future articles.

Addendum

I mentioned above that the function we were using to optimize was slow as the code was written for simplicity rather than speed. I've neatened up the code and added it my penaltyblog python package if you want to try out a faster version.

Using penaltyblog we can repeat everything we've done above in just a few lines of code and and have it run in minutes rather than hours.

pip install penaltyblog

import penaltyblog as pb
import pandas as pd
import numpy as np


def calculate_mean_rps(dc, df):
    rps = list()
    for idx, row in df.iterrows():
        if row["FTR"] == "H":
            outcome = 0
        elif row["FTR"] == "D":
            outcome = 1
        elif row["FTR"] == "A":
            outcome = 2

        predictions = dc.predict(row["HomeTeam"], row["AwayTeam"]).home_draw_away

        rps.append(pb.metrics.rps(predictions, outcome))

    return np.mean(rps)

df = pd.concat([
  pb.footballdata.fetch_data("england", 2016, 0),
  pb.footballdata.fetch_data("england", 2017, 0),
  pb.footballdata.fetch_data("england", 2018, 0),
  pb.footballdata.fetch_data("england", 2019, 0),
  pb.footballdata.fetch_data("england", 2020, 0),   
]).reset_index()

rps = list()
for xi in [0, 0.0001, 0.001, 0.02,]:
    train = df.iloc[:-300]
    test = df.iloc[-300:]

    train["weight"] = pb.poisson.dixon_coles_weights(train["Date"], xi)

    dc = pb.poisson.DixonColesGoalModel(
        train["FTHG"],
        train["FTAG"],
        train["HomeTeam"],
        train["AwayTeam"],
        train["weight"],
    )

    dc.fit()
    rps.append(calculate_mean_rps(dc, test))

print(rps)

[0.2214667529243473, 0.22109214369155702, 0.21811162166084946, 0.22953214528180002]

Predicting Football Results With the Poisson Distribution

2021-06-18T19:30:00+00:00

Introduction

This article is going to walk through building a model to predict football results using the Poisson distribution. As we'll find out, the model is fairly simplistic and struggles at points. However, it provides us a with great base to improve from over the next few articles in this series.

Let's get started!

The Data

Our model will work by predicting the number of goals scored / conceded by each team when they play each other based on their historical performances. To work that out we're going to need some data so let's grab some football results from the awesome football-data.co.uk website.

import pandas as pd

df = pd.read_csv("https://www.football-data.co.uk/mmz4281/1718/E0.csv")
df[["Date", "HomeTeam", "AwayTeam", "FTHG", "FTAG"]].head()

	Date	HomeTeam	AwayTeam	FTHG	FTAG
0	11/08/2017	Arsenal	Leicester	4	3
1	12/08/2017	Brighton	Man City	0	2
2	12/08/2017	Chelsea	Burnley	2	3
3	12/08/2017	Crystal Palace	Huddersfield	0	3
4	12/08/2017	Everton	Stoke	1	0

We've now got the team names, the number of goals scored by the home team (FTHG) at full time and the number of goals scored by the away team (FTAG) at full time, which is everything we need to get started.

Home Advantage

Let's dig into the data to get a better understanding of what we're trying to model. We'll start off simple by looking at the average goals scored.

df[["FTHG", "FTAG"]].mean()

FTHG    1.531579
FTAG    1.147368
dtype: float64

On average, the home team scores around a third of a goal more than the away team so to it looks like our model will need to handle home advantage.

Next, we'll plot out the distribution of goals scored by the home and away teams.

import matplotlib.pyplot as plt
import seaborn as sns

sns.set()

max_goals = 10
plt.hist(
    df[["FTHG", "FTAG"]].values, range(max_goals), label=["Home", "Away"], density=True
)
plt.xticks([i - 0.5 for i in range(max_goals)], [i for i in range(max_goals)])
plt.xlabel("Goals")
plt.ylabel("Proportion of matches")
plt.legend(loc="upper right", fontsize=13)
plt.title("Number of Goals Scored Per Match", size=14, fontweight="bold")

The shape of that distribution looks rather similar to the Poisson distribution. We can confirm this by overlaying the number of goals we'd expect from the Poisson distribution based on the average goals scored that we calculated above.

import numpy as np
from scipy.stats import poisson

home_poisson = poisson.pmf(range(10), df["FTHG"].mean())
away_poisson = poisson.pmf(range(10), df["FTAG"].mean())

max_goals = 10
plt.hist(
    df[["FTHG", "FTAG"]].values, range(max_goals), label=["Home", "Away"], density=True
)

plt.plot(
    [i - 0.5 for i in range(1, max_goals + 1)],
    home_poisson,
    linestyle="-",
    marker="o",
    label="Home Poisson",
)

plt.plot(
    [i - 0.5 for i in range(1, max_goals + 1)],
    away_poisson,
    linestyle="-",
    marker="o",
    label="Away Poisson",
)


plt.xticks([i - 0.5 for i in range(1, max_goals + 1)], [i for i in range(max_goals)])
plt.xlabel("Goals")
plt.ylabel("Proportion of matches")
plt.legend(loc="upper right", fontsize=13)
plt.title("Number of Goals Scored Per Match", size=14, fontweight="bold")

It's not perfect, but the two sets of data look pretty close to each other. This is good as it means we can approximate the number of goals scored using the Poisson distribution - this assumption is going to form the basis of our predictive model.

Building the model

To plot the Poisson distributions above, we used the average number of goals scored home and away, but to predict actual matches we're going to need to know how many goals we expect each individual team to score against each other.

For this, we're going to assume that the number of goals a team scores is a function of their attack strength combined with the opposition's defence strength. In other words, teams with better attacks should score more and teams with worse defences should concede more.

We also need to remember to account for the home advantage too.

So our model is going to look like this:

goals_home = home_advantage + home_attack + defence_away

goals_away = away_attack + defence_home

So far, so good. But where do we get the values for those model parameters from???

We're going to use good old trial and error to test out a whole bunch of different numbers and see what works best. We are not going to do it by hand though as that will be tedious. Instead, we'll let the computer do the hard work for us.

Finding the Optimal parameters

For our computer to find the best set of attack and defence parameters, we need to give it a value to optimise. For a problem like this, the usual approach is Maximum Likelihood Estimation for identifying the parameter values that maximise the likelihood that the process described by the model produces the data that were actually observed. In other words, it will work out what set of attack and defence parameters get closest to reproducing those historical fixtures we downloaded.

Unfortunately, for this example calculating the likelihood involves multiplying lots of small numbers together, which can cause issues with floating point precision and underflows.

Instead, we'll maximise the log-likelihood as we can just sum the logs of the likelihoods togethers rather than multiplying and avoid all those awkward precision issues.

And actually, instead of maximising the log-likelihood, we're going to minimize the negative of the log-likelihood because Python's scipy library comes with an optimizer that minimizes rather than maximizes. If this is confusing, making the negative of the log-likelihood as small as possible is equivalent to making the log-likelihood itself as big as possible.

Now we just need the function to calculate the log- likelihood.

def log_likelihood(
    goals_home_observed,
    goals_away_observed,
    home_attack,
    home_defence,
    away_attack,
    away_defence,
    home_advantage,
):
    goal_expectation_home = np.exp(home_attack + away_defence + home_advantage)
    goal_expectation_away = np.exp(away_attack + home_defence)

    if goal_expectation_home < 0 or goal_expectation_away < 0:
        return 10000    

    home_llk = poisson.pmf(goals_home_observed, goal_expectation_home)
    away_llk = poisson.pmf(goals_away_observed, goal_expectation_away)

    log_llk = np.log(home_llk) + np.log(away_llk)

    return -log_llk

That function is less complicated than it initially looks. First, we put the attack, defence and home_advantage parameters into the model we defined above to get the number of goals we expect from the two teams playing each other. If the number of goals is less than zero then we return a really big number to tell the optimizer that the parameters it is testing are rubbish because teams can't score less than zero goals in real life.

If we have zero or more goals, we use the Poisson distribution we work out the likelihood of our parameters giving us the actual number of goals scored. We then convert that to its log and add them together. Finally, because we are minimizing instead of maximizing, we return the negative of the log-likelihood.

Now we need to wrap that code to pass into the optimizer.

from scipy.optimize import minimize


def fit_poisson_model():
    teams = np.sort(np.unique(np.concatenate([df["HomeTeam"], df["AwayTeam"]])))
    n_teams = len(teams)

    params = np.concatenate(
        (
            np.random.uniform(0.5, 1.5, (n_teams)),  # attack strength
            np.random.uniform(0, -1, (n_teams)),  # defence strength
            [0.25],  # home advantage
        )
    )

    def _fit(params, df, teams):
        attack_params = dict(zip(teams, params[:n_teams]))
        defence_params = dict(zip(teams, params[n_teams : (2 * n_teams)]))
        home_advantage = params[-1]

        llk = list()
        for idx, row in df.iterrows():
            tmp = log_likelihood(
                row["FTHG"],
                row["FTAG"],
                attack_params[row["HomeTeam"]],
                defence_params[row["HomeTeam"]],
                attack_params[row["AwayTeam"]],
                defence_params[row["AwayTeam"]],
                home_advantage,
            )
            llk.append(tmp)

        return np.sum(llk)

    options = {
        "maxiter": 100,
        "disp": False,
    }

    constraints = [{"type": "eq", "fun": lambda x: sum(x[:n_teams]) - n_teams}]

    res = minimize(
        _fit,
        params,
        args=(df, teams),
        constraints=constraints,
        options=options,
    )

    model_params = dict(
        zip(
            ["attack_" + team for team in teams]
            + ["defence_" + team for team in teams]
            + ["home_adv"],
            res["x"],
        )
    )

    return model_params

model_params = fit_poisson_model()

That's a big chunk of code so let's step through it to see what it's doing.

First of all, we extract the teams from those historical fixtures we are fitting the model to as we're going to need to use them later on to index into the model's parameters with.

Next, we create the default model parameters. The optimizer will use these as a starting point so the closer we can get them to the actual values then the easier (and therefore quicker) it will be for the optimizer to do its job.

We then create a function called _fit, which is a wrapper around the function for calculating the log-likelihood. This loops through our historical fixtures, calculates the log-likelihood for each one and then adds them all up. It's this final number that the optimizer is using to work out what the best set of parameters is.

Just a small warning here that this function is rather slow. I've tried to keep the code as simple as possible by just using a for loop here but there are ways to rewrite this to speed it up massively, which I'll point you towards later.

We then set up some additional options for the optimizer to tell it to give up if it has not found a solution after 100 attempts and to hide the default output.

Next, we set up a constraint to tell the optimizer that the sum of the attack parameters must equal the total number of teams. There are countless combinations of parameters that could potentially give us the same results so this forces the optimizer to give us a reproducible output and keeps the parameters in a realistic range by making them average around 1.

After all that, we finally run the optimizer and return the parameters it has decided are the best fit.

Let's take a look at the parameters and check they look sensible

from pprint import pprint

pprint(model_params)

{'attack_Arsenal': 1.4472981861257104,
 'attack_Bournemouth': 0.9586519453041221,
 'attack_Brighton': 0.6704348922643556,
 'attack_Burnley': 0.7124518794812125,
 'attack_Chelsea': 1.2560897724037843,
 'attack_Crystal Palace': 0.9524125006581355,
 'attack_Everton': 0.9329845923970115,
 'attack_Huddersfield': 0.4800186720177229,
 'attack_Leicester': 1.1770358138298533,
 'attack_Liverpool': 1.5606892885829151,
 'attack_Man City': 1.782353073898856,
 'attack_Man United': 1.3383146669687525,
 'attack_Newcastle': 0.8007419035953682,
 'attack_Southampton': 0.7572219737186204,
 'attack_Stoke': 0.7139419884005881,
 'attack_Swansea': 0.4779863899533932,
 'attack_Tottenham': 1.4314120522353588,
 'attack_Watford': 0.9392553101070111,
 'attack_West Brom': 0.5799340702765279,
 'attack_West Ham': 1.0307710277807003,
 'defence_Arsenal': -0.904617084909674,
 'defence_Bournemouth': -0.7555648468890336,
 'defence_Brighton': -0.8891290180796632,
 'defence_Burnley': -1.213060870426372,
 'defence_Chelsea': -1.2124734969715354,
 'defence_Crystal Palace': -0.8593840991466305,
 'defence_Everton': -0.8071894597751467,
 'defence_Huddersfield': -0.8236837487885708,
 'defence_Leicester': -0.7605866654074676,
 'defence_Liverpool': -1.1893484944870034,
 'defence_Man City': -1.5087555344502186,
 'defence_Man United': -1.5123413324174197,
 'defence_Newcastle': -1.0231224233993557,
 'defence_Southampton': -0.8496091601384902,
 'defence_Stoke': -0.6570589447421037,
 'defence_Swansea': -0.8588218644055267,
 'defence_Tottenham': -1.254167039595323,
 'defence_Watford': -0.7084540000589359,
 'defence_West Brom': -0.8557753688634717,
 'defence_West Ham': -0.6434205443618255,
 'home_adv': 0.2888264579592506}

That looks reasonable to me, Huddersfield and West Brom both have low attack parameters reflecting that they didn't score many goals while Manchester City and Chelsea have much higher attack parameters. West Ham and Bournemouth both have poor defence parameters, which again matches what actually happened in our historical data.

Our home advantage parameter is also pretty close to what we got by just looking at the difference in average goals scored home and away.

Predicting Scores

Now for the fun bit, let's start making some predictions :)

def predict(home_team, away_team, params, max_goals=10):
    home_attack = params["attack_" + home_team]
    home_defence = params["defence_" + home_team]
    away_attack = params["attack_" + away_team]
    away_defence = params["defence_" + away_team]
    home_advantage = params["home_adv"]

    home_goal_expectation = np.exp(home_attack + away_defence + home_advantage)
    away_goal_expectation = np.exp(away_attack + home_defence)

    home_probs = poisson.pmf(list(range(max_goals + 1)), home_goal_expectation)
    away_probs = poisson.pmf(range(max_goals + 1), away_goal_expectation)

    probability_matrix = np.outer(home_probs, away_probs)

    return probability_matrix

The start of this function looks similar to the log_likelihood function we used when fitting the model. We start off by getting the parameters and using them to calculate how many goals we expect the home and away team to score against each other.

We then use that with the Poisson distribution to get the probability of each team scoring 0, 1, 2, 3 etc goals.

Then take the outer product of those two sets of probabilities to get a matrix containing the probability of all the possible scores.

Let's give it a go and get the probabilities for all scores up to four goals!

probs = predict("Man City", "Stoke", model_params, 4)
pprint(probs)

array([[0.01041474, 0.00470398, 0.00106231, 0.00015994, 0.00001806],
       [0.04283444, 0.01934684, 0.00436915, 0.0006578 , 0.00007428],
       [0.0880862 , 0.03978549, 0.00898487, 0.00135272, 0.00015274],
       [0.1207623 , 0.05454416, 0.01231786, 0.00185452, 0.00020941],
       [0.12416985, 0.05608323, 0.01266543, 0.00190685, 0.00021531]])

The columns show the probability of Manchester City scoring a given number of goals, whilst the rows are the probabilities for Stoke. For example, the probability of 0-0 is 0.01041474 and the probability of 3-1 is 0.05454416.

We can now use this matrix to get the probabilities for various betting markets. For example, the sum of the diagonal of the matrix gives us the probability of a draw, the sum of the lower triangle is the probability of a home win and the sum of the upper triangle is the probability of an away win.

# draw
np.sum(np.diag(probs))

0.04081627039389976

# home win
np.sum(np.tril(probs, -1))

0.5531558007746874

# away win
np.sum(np.triu(probs, 1))

0.01276037463224738

Note that those probabilities don't add up to one as we've only calculated the probabilities for up to 4 goals. By the time you get up to 10 goals then you're pretty much at 1.0.

Something Smells a Bit Fishy

We now have a model and can make predictions but are those predictions any good??

If we look at the average probabilities over our data set then our model expects around 46% of games to finish as a home win, 23% as a draw and 31% as an away win.

However, if we look at what actually happened that season then 46% of games ended as a home win, 26% as a draw and 28% as an away win - that's pretty good but we're clearly under-predicting draws and over-predicting away wins. If you're planning on using this model to beat the bookie then you're likely to be disappointed at this stage.

The problem is that we're predicting the number of goals each team scores independently from each other. But goals in a football match are not independent. For example, the team playing away may settle for a draw if the game is 0-0 in the second half and attack less. A team that's gone three goals down may park the bus to try and avoid a more humiliating score line.

So there are situations where teams aren't necessarily playing to score and that negatively impacts our Poisson assumption.

Conclusions

All of this is not to say our model is useless though, it actually gives us a really good base model to build upon.

In the next article in this series, we'll look at how Dixon and Coles propose an adjustment to our Poisson model to try and alleviate these issues with under-predicting draws and improve our accuracy.

Addendum

I mentioned above that the function we were using to optimize was a little slow as the code was written for simplicity rather than speed.

I've neatened up the code and added it my penaltyblog python package if you want to try out a faster version. This package also includes some other useful functions for working with football and betting data.

It's as simple as:

pip install penaltyblog

df = pb.footballdata.fetch_data("England", 2017, 0)
df[["Date", "HomeTeam", "AwayTeam", "FTHG", "FTAG"]].head()

pois = pb.poisson.PoissonGoalsModel(df["FTHG"], df["FTAG"], df["HomeTeam"], df["AwayTeam"])
pois.fit()
probs = pois.predict("Man City", "Man United")
print(probs.home_draw_away)

[0.5645798158431763, 0.23488303272721173, 0.20053715077601225]

Have fun!

Which Young Players Will be Stars?

2021-05-19T19:30:00+00:00

Introduction

Over the years, I've shared results from my PlayerRatings model with lots of people, including a number of professional football clubs, and everybody always comes back with the same question - who are the best young players?

Let's take a look!

The Best Players Aged 21 or Under

Let's start off with the top 25 players aged 21 or under at the time of writing this article:

	Name	Team	Age	TransferMarkt Value £
1	Matthijs de Ligt	Juventus	21	75.00m
2	Phil Foden	Manchester City	20	70.00m
3	Alphonso Davies	Bayern Munich	20	75.00m
4	Jadon Sancho	Borussia Dortmund	21	100.00m
5	Erling Håland	Borussia Dortmund	20	110.00m
6	Perr Schuurs	Ajax	21	8.00m
7	Sergiño Dest	Barcelona	20	25.00m
8	Ryan Gravenberch	Ajax	18	28.00m
9	Mitchel Bakker	Paris Saint Germain	20	10.00m
10	Pedri	Barcelona	18	70.00m
11	João Félix	Atlético Madrid	21	80.00m
12	Callum Hudson-Odoi	Chelsea	20	35.00m
13	Mason Greenwood	Manchester United	19	50.00m
14	Kai Havertz	Chelsea	21	50.00m
15	Bukayo Saka	Arsenal	19	50.00m
16	Ferrán Torres	Manchester City	21	50.00m
17	Justin Kluivert	RB Leipzig (loan from Roma)	21	14.00m
18	Moussa Diaby	Bayer Leverkusen	21	38.00m
19	Jonathan David	Lille	21	30.00m
20	Jens Petter Hauge	Milan	21	10.00m
21	Óscar Mingueza	Barcelona	21	10.00m
22	Aurélien Tchouaméni	Monaco	21	25.00m
23	Antony	Ajax	21	25.00m
24	Myron Boadu	AZ Alkmaar	20	14.00m
25	Owen Wijndal	AZ Alkmaar	21	15.00m

So that looks pretty reasonable to me. You may not agree with the exact list but there's nothing too controversial in there. They are all players who are getting regular first team minutes in some of the biggest leagues around Europe. Many of them are even full internationals despite being still being 21 or under.

They are also all already at big teams so if you are looking to recruit players with potential, then you are too late. These players have already been discovered and are probably on big contracts.

Let's try searching a bit younger and see if we can find the future stars before they hit the big time.

The Best Players Aged 17 or Under

Below I've got the top 50 players aged 17 or under at the time of writing this article. A lot of these players won't be getting first team minutes yet and it may well be a few years until they make a name for themselves. Some of them may not even make it in the game.

So, since it's going to take a while for this list to mature I'm going to keep the players anonymised to start with to avoid the temptation of judging the predictions too early. If / when players become well known, get bought up by bigger clubs or their value on Transfermarkt goes to £10.00m or above I'll reveal their names.

A few of the players already meet this criteria so are de-anonymised to start off with.

	Name	Team	Age	TransferMarkt Value £
1	Florian Wirtz	Bayer Leverkusen	17	45.00m
2	Youssoufa Moukoko	Borussia Dortmund	16	10.00m
3	Anonymous	-	17	-
4	Anonymous	-	17	-
5	Anonymous	-	17	0.8m
6	Anonymous	-	17	1.8m
7	Anonymous	-	17	-
8	Anonymous	-	15	-
9	Anonymous	-	17	0.54
10	Anonymous	-	17	4.5m
11	Anonymous	-	17	-
12	Anonymous	-	16	-
13	Anonymous	-	17	0.18m
14	Anonymous	-	17	-
15	Jude Bellingham	Borussia Dortmund	17	35.00m
16	Anonymous	-	17	-
17	Anonymous	-	17	1.35m
18	Anonymous	-	17	0.9m
19	Anonymous	-	17	-
20	Anonymous	-	17	-
21	Anonymous	-	17	-
22	Anonymous	-	17	-
23	Anonymous	-	17	-
24	Anonymous	-	17	3.15m
25	Anonymous	-	17	0.45m
26	Anonymous	-	17	-
27	Anonymous	-	17	-
28	Anonymous	-	17	-
29	Anonymous	-	17	-
30	Anonymous	-	17	1.08
31	Anonymous	-	17	-
32	Anonymous	-	17	-
33	Anonymous	-	17	-
34	Anonymous	-	17	-
35	Anonymous	-	17	-
36	Anonymous	-	17	-
37	Anonymous	-	16	-
38	Anonymous	-	17	-
39	Anonymous	-	16	-
40	Anonymous	-	16	-
41	Anonymous	-	17	0.27m
42	Anonymous	-	17	1.35m
43	Anonymous	-	17	-
44	Anonymous	-	17	-
45	Anonymous	-	16	-
46	Anonymous	-	17	-
47	Anonymous	-	16	-
48	Anonymous	-	17	0.585m
49	Anonymous	-	17	-
50	Anonymous	-	17	1.08m

Conclusions

I have no idea what a good hit rate is here, one out of the 50 going on to have a good career? Five? Ten? More?

The model is already flagging the players who are well known (to me at least), such as Florian Wirtz and Jude Bellingham, which is reassuring. However, the model only has information on youth football for many of the players so I'm really intrigued as to how predictive these types of fixtures will actually be!

Predicting Player Career Trajectories

2021-05-03T19:30:00+00:00

tl;dr

I've updated the my Player Ratings model to show football players' potential career trajectories.

Introduction

Back in 2016, I gave a presentation at the Opta Pro Forum discussing how to rate / rank footballers using a metric somewhat related to a plus/minus score. The presentation went down well with the audience and probably generated more feedback than all my other Opta Pro Forum appearances combined. A few years on, I've finally got around to refining the idea further to come up with version 2.0 of the algorithm and smooth out some of its wrinkles.

How Does it Work?

Although I've given the model a complete overhaul, the principles behind it are the same so I'm not going to repeat myself too much but the model is essentially looking at how well teams perform when a player is in the team compared when they are out of the team.

Unfortunately though, we can't just naively add up a teams goal difference with and without a specific player to calculate their plus/minus score. For example, stick an average player in Real Madrid's first team and they'd likely come out with a much better plus/minus than they would playing for Sheffield United.

So we need to account for the strength of a player's team mates and their opposition - this makes things get a bit more complicated though because to calculate the current player's score we now need to know the score of the other players around them, but to calculate those other players' scores we need to know the score of the current player. Circular!

Plus there are relatively few substitutions in football so we often don't get much data around how well teams perform with different combinations of players. To help alleviate this, the model incorporates a number of statistical priors that are used to inform the predictions where there is a lack of data.

As an example, imagine watching a footballer play for the first time - there’s a probability they may be the next Lionel Messi, there’s a probability they may be the next Lee ~~Badbuy~~ Bradbury and there’s a probability they may be average and be somewhere between the two.

This is essentially how the model works. Based on all the available data for each player, the model constructs a set of priors and uses them in conjunction with the observed data to estimate the player’s true talent compared with all the other footballers in the world. As the model gains more information about a player over the course of their career, the influence of these priors diminishes relative to the observed data.

Career Trajectories

If we calculate these scores over the course of a player's career, we can then visualise their career trajectory. As an example, here's the UK's future Prime Minister Marcus Rashford.

Figure 1: Marcus Rashford's career trajectory to date

The blue line shows Rashford's rating by the model over the course of his career. The scale of the rating is somewhat arbitrary as the number doesn't refer to anything specific, such as goals scored etc. Rather, it's used to compare footballers against each other and Rashford's current rating puts him comfortably in the top 200 footballers in the world. This is pretty impressive as there are currently around 100,000 players he's being compared against.

This chart is only looking at what has happened in his career so far though. Wouldn't it be more useful to get an idea about how the rest of his career is going to look?

Predicting how a player's career is going to pan out is tricky though. We have no idea about what will happen with injuries, managers, teams, transfers, motivation etc so we can't just give a single prediction. Instead, we need to provide a range of realistic possibilities that cover our uncertainty. The model does this be comparing the player with historical players to find those with the closest matching career trajectories to date.

As an example, here is Marcus Rashford again (in blue) alongside the closest matching career trajectories (in grey). Now, this isn't to say that Rashford will follow any particular one of those grey lines. All footballers are unique snowflakes and will all have their own unique career trajectory but it does give us a realistic range in which the player's career could develop.

Figure 2: Marcus Rashford's potential career trajectories

There's quite a wide range of potential trajectories there for Rashford, which shows how uncertain players' careers can be. At the top end, he is closest to Ivan Rakitic but at the bottom end there is Manuel Fernandes.

To be clear though, that doesn't mean Marcus Rashford and Ivan Rakitic or Manuel Fernandes are similar in how they play, but that they are similar (or could be similar) in the impact they have on their team's results.

Erling Braut Haaland

Here's another example, this time for the in-demand Erling Braut Haaland.

Figure 3: Erling Haaland's potential career trajectories

We can already see that Haaland is at the upper end of all the similar trajectories, showing what a generational talent he potentially is. There are very few players who've careers have started as strongly as his so we don't have many careers to compare him with.

The most similar career is actually Mesut Özil's, who the model considers to have been one the greatest players I have data on. Again, this doesn't mean Haaland and Özil are similar in how they play, but that they are similar in the impact they have on their team's results.

Neymar

At the opposite end of the career spectrum is Neymar who the model has been thoroughly unimpressed with recently. I don't follow Paris Saint Germain particularly closely but at one point Neymar was vying with Messi to be ranked the number one player in the world overall. However, it seems his impact has waned significantly over the past few years.

Figure 4: Neymar's potential career trajectories

Andrew Robertson

And as a final example, here's Liverpool's Andrew Robertson. The reason for including Robertson is that it's a nice example showing a player that has probably reached his peak.

Although Robertson has steadily improved over the course of his career, based on the trajectories of similar players, it's likely that Liverpool will start to see him decline from here onwards. A slight positive from this though is that the expected drop off in impact looks to be fairly gradual compared with Neymar!

Figure 5: Andrew Robertson's potential career trajectories

Conclusions

This is a fairly short post just to introduce the idea of visualising players' potential career trajectories.

I'll expand on this topic in future articles but in the meantime, thanks for reading!

Sharing xG Using Multi-touch Attribution Modelling

2019-11-23T19:30:00+00:00

Introduction

A question that often comes up in data science is how you determine the performance of different marketing channels. For example, somebody might land on your website by clicking an advert on Google. They may find something they want to buy but leave the site only to return a few minutes later via an affiliate link with a discount code. They then add something to their shopping basket but never complete the purchase so you send them an email to remind them and finally they convert.

Which channel drove the conversion here, was it the final channel which they converted from? Was it the first one as it brought them onto the site in the first place? Or was it one of the other ones in-between that helped guide them to the conversion?

Okay, this is a blog about football analytics so why I am writing about marketing? Well, consider this next example - Aymeric Laporte tackles the opposition's striker to win the ball and passes it to Fernandinho. Fernandinho then plays a short ball to Bernardo Silva, who then knocks it out wide to Raheem Sterling. Raz runs down the wing, beats the fullback and crosses the ball to Sergio Agüero who then scores. Which player was responsible for the goal here, was it Laporte for winning possession in the first place, was it Agüero since he scored or was it one of the players in-between who helped move the ball down the pitch to Agüero so he could score the goal?

Hopefully you've spotted that both examples are effectively the same problem - how do you determine the value of all the events leading up to a conversion?

Heuristics

Traditionally, this has been 'solved' using simple heuristics. For example the last player who touches the ball is awarded the goal or the last marketing channel a customer interacts with gets the credit for their purchase. Depending on the metric being measured, people will occasionally give credit to the first event in the sequence instead, or perhaps share everything out equally because it sounds fairer. If they are feeling really adventurous they may even apply some sort of curve to it but there's no real scientific rationale being used, it's typically just somebody's personal preference.

Multi-touch Attribution Models

A more scientific approach taken from the world of marketing analytics is to use multi-touch attribution modelling to quantify the importance of each event in the sequence and assign a fractional amount of credit to it based on how much it drives the final outcome.

There are lots of different ways of doing this, including using Markov chains. These are mathematical systems that can be used to model the probability of sequences transitioning from one event to another. For example, the probability of a customer clicking through to a company's home page from a tweet, followed by the probability of that being their last interaction (and therefore failing to convert) or the probability they move onto some other interaction with the company.

We can apply this same principle to football, e.g. if Sergio Agüero is in possession of the football then what is the probability he passes, what is the probability he scores, what is the probability the sequence of possession ends with him?

Attributing xG Using Markov Chains

To apply multi-touch attribution modelling to football and Expected Goals (xG) I created a dataset of possession sequences from the Premier League where each sequence contained the players involved plus a True / False flag terminating the sequence to designate whether it ended with a shot or not and used it to train a Markov Chain.

[Ederson, Laporte, Stones, Sterling, True]

Figure 1: Example possession sequence used to train the Markov chain

The trained Markov Chain was then used to simulate possessions by picking a starting player and taking a random walk through the probabilities until it hit a True / False event. From here we can calculate the importance of each player in terms of shot generation - essentially, each possession sequence's propensity to lead to a shot changes as different players become involved. These differences in shot propensity can then be used to reattribute the xG from a given shot across all the players in the possession leading up to it - the more important a player is the more xG is awarded to them even if they didn't take the shot.

Results

The table below shows the attributed xG (axG) for Manchester City's 2018/2019 season. The first thing to note is that the most attacking players, such as Agüero and Jesus, have lower axG compared with xG. This is to be expected as traditional xG models will credit them with 100% of the value of the shot whereas axG takes some of that xG and reassigns it to the players involved in the build up play. They still come out with the highest axG scores overall though as these are the players taking the majority of the shots generating the xG so their presence in the possession sequences is important in terms of shot generation.

name	xg	axg	axg:xg
sergio agüero	19.92	15.86	0.8
raheem sterling	13.14	12.2	0.93
david silva	8.07	8.79	1.09
bernardo silva	6.74	7.78	1.15
gabriel jesus	8.76	7.22	0.82
leroy sané	5.52	5.84	1.06
riyad mahrez	5.72	5.37	0.94
ilkay gündogan	4.52	4.74	1.05
aymeric laporte	3.36	4.23	1.26
fernandinho	1.74	2.58	1.48
kevin de bruyne	1.99	2.26	1.14
kyle walker	0.45	1.66	3.69
nicolás otamendi	1.39	1.4	1.01
phil foden	1.72	1.06	0.62
john stones	0.51	0.92	1.8
oleksandr zinchenko	0.19	0.9	4.74
vincent kompany	0.35	0.59	1.69
danilo	0.44	0.54	1.23
benjamin mendy	0.2	0.36	1.8
fabian delph	0.08	0.29	3.62
ederson	0	0.23

Table 1: Manchester City axG 2018/2019

Looking at the ratio of xG to axG shows that the biggest beneficiaries for Manchester City are their defenders, particularly the fullbacks. Kyle Walker has a 3.7 fold increase in xG credited to him and Oleksandr Zinchenko has a 4.7 fold increase. Laporte and Fernandinho also have noticeable increases too reflecting their importance in City's build up play.

The Importance of Full Backs

It's not just Manchester City's full backs who do well when we reattribute xG, it's pretty common across all other teams too as shown in the table below. These players are typically out wide where they can't take many shots but are important for getting the ball into the danger zones for the attacking players. It's still small volumes of xG compared with attackers but once we start accounting for fullbacks' involvement in the build up play then their xG numbers increase noticeably.

team_name	name	xg	axg	axg:xg
Burnley	charlie taylor	0.09	0.73	8.11
Arsenal	stephan lichtsteiner	0.02	0.15	7.5
Bournemouth	simon francis	0.03	0.22	7.33
West Ham	arthur masuaku	0.08	0.42	5.25
Manchester City	oleksandr zinchenko	0.19	0.9	4.74
Everton	lucas digne	0.46	2.16	4.7
Manchester City	kyle walker	0.45	1.66	3.69
Arsenal	sead kolasinac	0.45	1.63	3.62
Tottenham	kieran trippier	0.4	1.41	3.52
Bournemouth	diego rico	0.08	0.28	3.5
Liverpool	trent alexander-arnold	0.66	2.07	3.14
Manchester United	ashley young	0.47	1.42	3.02
West Ham	pablo zabaleta	0.17	0.51	3
Watford	josé holebas	0.6	1.79	2.98
Liverpool	andrew robertson	1.18	2.83	2.4
Newcastle United	javier manquillo	0.07	0.16	2.29
Burnley	matthew lowton	0.33	0.75	2.27
Leicester	ricardo pereira	1.35	2.81	2.08
Leicester	ben chilwell	1.05	2.14	2.04
Bournemouth	nathaniel clyne	0.07	0.14	2
Manchester United	luke shaw	1.1	2.15	1.95

Table 2: Fullback axG 2018/2019

Top 25 Players by axG

team_name	name	xg	axg	axg:xg
Liverpool	mohamed salah	18.83	18.99	1.01
Manchester United	paul pogba	15.6	17.06	1.09
Arsenal	pierre-emerick aubameyang	20.84	16.92	0.81
Manchester City	sergio agüero	19.92	15.86	0.8
Fulham	aleksandar mitrovic	15.14	15.52	1.03
Wolverhampton Wanderers	raúl jiménez	15.46	14.48	0.94
Leicester	jamie vardy	15.59	13.93	0.89
Chelsea	eden hazard	10.5	13.57	1.29
Brighton	glenn murray	11.87	13.11	1.1
Tottenham	harry kane	13.64	12.66	0.93
Everton	gylfi sigurdsson	11.94	12.41	1.04
Bournemouth	joshua king	13.07	12.34	0.94
Newcastle United	salomón rondón	11.86	12.32	1.04
Manchester City	raheem sterling	13.14	12.2	0.93
Liverpool	sadio mané	14.54	11.7	0.8
Arsenal	alexandre lacazette	12.3	11.23	0.91
Liverpool	roberto firmino	11.96	11.04	0.92
Burnley	ashley barnes	11.25	10.83	0.96
Bournemouth	callum wilson	12.24	10.18	0.83
Southampton	danny ings	9.51	10	1.05
Watford	troy deeney	9.56	9.71	1.02
Everton	richarlison	9.37	8.91	0.95
Manchester City	david silva	8.07	8.79	1.09
Manchester United	romelu lukaku	10.18	8.57	0.84
Manchester City	bernardo silva	6.74	7.78	1.15

Table 3: Top 25 Players by axG 2018/2019

Conclusions

It's worth clarifying that this is not an expected possession value (EPV) model. It's taking the output from a shots-based xG model and redistributing it across the players involved in the build up play based on the propensity of a shot occurring from that particular group of players.

In many ways, the output of the model is closer to a Shapley Value in that it's looking at all the different combinations of players in the possession sequences to quantify how much each player contributed to the propensity of a shot occurring. In fact, this is something I want to play around with further to see what other uses it has.

Whilst the approach described here is perhaps not as complex as some EPV models, it has a couple of advantages. First of all, it's quick to process the data, but most importantly it's easy to explain to stakeholders. This isn't some complicated and uninterpretable black box that senior management need to take a leap of faith to trust, multi-touch attribution is just sharing out xG more fairly based on the probabilities of shots occurring during the sequence of play and for me that's a big win. A simpler approach that people can relate to often has a bigger impact in a business than a bigger model that's much more complicated to get buy in for.

Multi-touch attribution has pretty much achieved this in marketing analytics now, it's a significant improvement from the simple heuristics without being so complex it scares the C-Suite off. Perhaps it could also play a similar role in football as a step up from xG without ostracizing the more data-reluctant coaches?

VAR, What is it Good For?

2019-08-22T19:30:00+00:00

Introduction

As Edwin Starr famously said, VAR, what is it good for? Well, let's find out...

Player Speed

To get an idea of what VAR can and can't do we need to know how fast the players are moving on the pitch. According to this article in The Express and this one by Sky Sports, it's not uncommon for players to be able to reach speeds of 33km/hour upwards.

What About the VAR Cameras?

VAR has access to the broadcast footage of the match so the exact resolution of the cameras will vary depending on the broadcaster but in the case of Sky Sports and BT here in the UK they record at a resolution of 50 frames per second

What Does This Mean?

50 frames per second means an image is captured by the camera every 0.02 seconds
A footballer running at 33km/h is moving 18cm every 0.02 seconds
Therefore, a footballer can potentially move 18cm between two frames being captured

Now imagine VAR is assessing an offside decision by looking at the earliest frame where the ball was clearly played forwards. Chances are the ball was not played exactly at that moment the frame was captured and actually occurred at some point between the current frame and the previous one 0.02 seconds ago. Since we know that players can move 18cm between frames the frame being viewed by VAR is potentially out by up to 18cm.

It gets worse than this though as we haven't accounted for the defender moving. As a worst case scenario the defender could be sprinting in the opposite direction to the attacker, whilst trying to play the offside trap for example.

This means the image being viewed by VAR to determine whether a player is offside or not has potentially occurred late enough after the ball has been played for the defender and attacker to have moved apart by 36cm.

Therefore if the attacking player is shown to be offside by 36cm or less then VAR has no absolutely no idea whether they were really offside at the time the ball was actually played. This is a fairly hefty margin of error, which effectively renders VAR unsuitable for any close offside decisions, particularly those where a player has an arm offside.

VAR making a decision it doesn't have the accuracy to do

Conclusions

VAR, what is it good for? Certainly not offside decisions!

Addendum

Apparently, in some situations VAR has access to super slow motion footage recorded at 120 frames per second. This reduces the potential error to around 15cm, which is still to large for those close offside calls.

Opta Pro Forum 2018

2018-11-21T19:30:00+00:00

Introduction

I presented a poster back in 2017 at the Opta Pro forum looking at using tracking data to evaluate footballer’s decisions in and around the box, which I've previously written about on Opta Pro’s blog if you want to find out more about the original idea. Opta then kindly invited me back in 2018 to expand on the work and talk about it on the main stage.

Rather than turn up in 2017 with just a poster, I’d created a web app that could animate the tracking data so I could present some real world examples and make it more exciting than just me stood next to a giant piece of paper. I decided to do something similar this time around and do without any slides. Instead, I just talked whilst the web app animated the tracking data Opta had provided. This was rather scary to say the least as you never know when your computer is going to update or crash, or even be compatible with the conference’s audio-visual facilities. Plus, I had no slides to remind me what to say and had to get my timings right so I was talking over the correct section of the match. Thankfully, it all seemed to go smoothly and my computer behaved itself.

Demo

Opta have published videos of some of the presentations from that day but as far as I’m aware mine was never released (it could be said have a face for blogs rather than vlogs 🙂) so I’ve added a quick run through of some of the features below. Please note that as part of the agreement to using the tracking data all analyses had to be anonymised, which is why team names and player names are all removed in the video.

Shapes

After logging in and selecting a match, the first thing I did in the video above was to add a bounding box around each team’s outfield players. This really helps highlight the changes in a team's shape over the course of a game. The demo also calculates the area of the boxes surrounding each team and the ratio between the two so you can quickly see how much of the pitch each team is controlling.

Alternatively, instead of just drawing a box around the players we can be a bit more sophisticated and use a convex hull (at 00:29 in the video). This is the simplest polygon that can be drawn around the players and personally I feel it shows up changes in a team’s shape more clearly than the bounding box, especially if you have a player drifting out of position.

Next up was the central box (at 00:42 seconds into the video), which is an idea that came about when I ran through an early version of my presentation with some of the analysts from Brighton and Hove Albion. They suggested plotting a box between the central midfielders and central defenders to monitor the space between them. Apparently, this is an area of the pitch that Tony Pulis is very keen on so if it's good enough for Tony, it’s good enough for me to include..

Any finally in this section was the Voronoi (at 00:52 seconds into the video). Its become somewhat of a cliché that somebody always presents a Voronoi at the Opta Pro Forum and this year it was my turn. Cliché or not, I really like them and apart from being somewhat hypnotic they are a great way of visualising the space around each player.

Offensive

I then moved onto talking about some ideas around offensive play, particularly using expected goals (xG). The idea being that if you could track the expected goals for each attacking player’s location on the pitch then you could look at whether players were making the correct decision in who they were choosing to pass to.

The first overlay here at 1:10 in the video shows the expected goals for each attacking player if they were to take a shot from where they were stood based on distance, angle, location and pressure from opposition players near to them. If a player passed to a team mate with a 2% chance of scoring when there’s somebody else with a 10% chance of scoring you could argue that they’ve not made the optimal decision.

This is somewhat simplistic though as it doesn’t account for the difficulty of making that pass in the first place. This is solved by the xgoals (pass) overlay at 1:23 in the video that shows the combined probability of the player with the ball successfully passing to a team mate and that team mate scoring from their current location based on distance, angle, location, opposition pressure etc.

There’s also a line of sight overlay 1:47 in the video that draws a triangle from the player with the ball to the opposition’s goal posts and calculates the number of opposing players inside that triangle blocking the attacker’s view of the goal.

Defensive

The next section of my presentation moved onto looking at some defensive ideas. The first one was just plotting the offside line to be able to quickly identify players drifting offside. It’s not particular exciting but certainly comes in useful.

What's more interesting though is to plot a line through each team’s defenders (at 2:18 in the video). You can then measure how straight the defensive line is and whether particular players are getting pulled out of position. You can also measure the distance between the defenders and look at how evenly spaced out they, which is the dispersion metric in the video - the higher the dispersion the more uneven the distance between neighbouring defenders is.

Match Summaries

As well as watching the animation to see these metrics changing in realtime, it’s often useful to get a view of how they changed over the full game so I also talked through a few summaries of the data generated by the app (shown at 2:46 in the video).

The first chart shows the area of the home team’s bounding box with blue lines showing when they conceded shots. As you can clearly see they tended to concede shots when their area was the smallest. This is somewhat to be expected as teams naturally compact when they are defending and are pushed back but it would be interesting to analyse the differences between those decreases in area where a shot wasn't conceded with those where it was.

The second chart shows the expected goals for one of the attackers during the match based on his location on the pitch and the defending players around him. During the first half he was constantly in areas where he was threatening the goal, whereas in the second half there is only three occasions where he was potentially in a goal scoring position. As an analyst, you probably wouldn’t want to go and show this chart directly to a coach but it certainly gives you something you should go and analyse the causes of.

Conclusions

Most of the ideas I talked about were intentionally fairly simple as I wanted the presentation to be approachable to the analysts in the audience without getting bogged down in technical explanations. There’s loads of potential to expand on these ideas though. For example I didn’t get time to talk about the distance between the defenders / midfielders or midfielders / attackers and how this correlates with shots. I also skipped over talking about predicting a player’s probability of scoring from a future position rather than their current one so you can analyse the value of players’ runs or through balls etc.

So how was my presentation received? I honestly have no idea as outside a couple of questions from the audience I had no feedback whatsoever. I tried to talk with analysts from a few different clubs after my presentation to get their feedback but was pretty much blanked by all of them. I can only presume teams are either not interested in tracking data, are too scared to talk about it in case they give something away or the ideas I presented just stink.

Personally, I feel there’s a huge amount of insight to be gained from working with tracking data. It’s much more difficult to work with though and it’s much harder to put an engaging presentation / blog together when everything has to be anonymised. Being able to talk about say David Silva’s movement on the pitch would be much more interesting to an audience than talking about Anonymous Player X.

It’s also difficult as an outsider to the football industry as there is no public access to tracking data that I know of. I’m lucky enough to have accumulated a few matches worth of TRACAB data through HackMCFC and the Opta Pro Forum but I still have nowhere near enough to attempt the majority ideas I have. And, even if I did, I’m not sure if I could ever publicly write about it.

Finally, because I want to end on a positive note, I've throughly enjoyed presenting at the Opta Pro Forum each time I’ve been there. It’s always a pleasure to catch up with the guys from Opta and all the bloggers I’ve got to know over the years, and I'm really looking forward to 2019's Forum.

Thanks for reading!

Automated Feature Engineering

2018-05-25T19:30:00+00:00

Automated Feature Engineering Using Deep Feature Synthesis

I recently gave a presentation at the MancML about using Deep Feature Synthesis to automate Feature Engineering and how it fits into a machine learing pipeline - the slides are belowfor anyone interested in findng out more.

Deep Feature Synthesis from Martin Eastwood

Deep Learning in R Using Keras

2017-09-11T19:30:00+00:00

Deep Learning in R Using Keras

I recently gave a presentation at the Manchester R Users Group on Deep Learning in R using Keras. The slides are below for anybody who's interested, but be warned R's Keras package is evolving rapidly so they may soon start to go out of date!

Deep Learning In R from Martin Eastwood

An Alternatve to Radars for Visualising Football Data

2017-07-17T19:30:00+00:00

Introduction

I've long been critical of the use of radar plots for visualizing football data and was recently challenged by a reader of this blog to come up with a better alternative so here we go.

Radars

First of all, why are radar plots so bad?

Well, as Luke Bornn's recent tweet so excellently shows, the shape of the plot is controlled by the order of the variables - swap a few of them around and suddenly the shape you are looking at is completely different, making visual comparisons difficult and prone to error. Essentially, the shape you are looking at and judging is meaningless as it's determined purely by the author's choice of layout.

A reminder, blatantly plagiarized from @stat_sam, of why radar plots are misleading. Eye focuses on area, not length. pic.twitter.com/Dk3gcn1GD1
— Luke Bornn (@LukeBornn) 17 May 2017

Another major issue is that the area of the radar's shape does not increase linearly. If you double the size of a value on a bar chart then the bar representing it doubles in area too - this is a good thing as it means we can compare data easily.

However, double the size of your values in a radar plot and the area increases by the square of the values instead. This magnifies small differences, making them appear much bigger than they really are and distorting any visual comparisons we make. This is a bad thing, a very bad thing.

The Alternatives

Neil Charles showed some player comparisons at the Opta Pro Forum a couple of years back. These were based on bullet graphs but with the central bar replaced with a strip plot to try and highlight the distribution of the data.

Gylfi, because he's topical. First four boxes raise serious questions (which others inc. @mixedknuts have answered in detail) #Swans #EFC pic.twitter.com/yzowruuywW
— Neil Charles (@neilcharles_uk) July 13, 2017

I remember being really impressed with these at the time but always felt that the central strip plot didn't quite work as it was one-dimensional and the overlap of the points obscured the true distribution of the data. If you look at the example above you can see it all kind of merges into one grey line down the centre of the plot.

Another idea that has been gaining a little bit of traction recently are Joyplots. These are great as they show the distribution of the data really clearly provided they are plotted at a reasonable size. However, as soon as you shrink them to fit few onto a page or overlap them then it obscures the data making visual comparisons tricky.

Crosby, McDavid, and the Greatest Player in the World... pic.twitter.com/VbZAm372ck
— Ryan Stimson (@RK_Stimp) July 14, 2017

Swarm Plot

After much consideration I settled on going with a swarm plot. The swarm plot takes the central strip used by Neil Charles but instead of flattening it into a single dimension where points can obstruct each other, it applies just enough jitter to each point to separate them and prevent any overlaps.

This can create a chart that looks a little like a swarm of bees, hence the name, but importantly it also means that you can clearly see the distribution of the underlying data without any overlapping points obscuring each other.

I kept Neil Charles' approach of coloring the data by quintiles so you can quickly see what group the player falls within. For example, if a player is in the red section then they are in the bottom 20% of players for that particular metric (bad), if they are in the yellow then they are in the middle 20% (average) and if they are dark green then they are in the top 20% of players (yay!).

I've also added a player rating in too, which is just the average of all the quintiles. I'm not fully convinced of the merits of this but I've shown this style of chart to a few people and pretty much everybody asked what the player's average was so I've included it on the chart.

Conclusions

As ever, let me know what you think.

Opta Pro Forum 2017

2017-03-11T19:30:00+00:00

Analyzing Footballer’s Decisions in and Around the Penalty Box

I wrote up the poster presentation I gave at the 2017 Opta Pro Forum for the Opta Pro blog looking at using machine learning to quantify footballer's decisions. You can read all about it here

Opta Pro Forum 2016

2016-12-20T19:30:00+00:00

A Mathematical Approach to Evaluating Footballers

With the 2017 Opta Pro Forum rapidly approaching, I thought it was about time I transcribed my presentation from the previous event discussing a mathematical model I’d been developing for rating and ranking footballers.

Introduction

An obvious starting point for developing a model like this was the plus / minus score you often see in American sports. This simply measures a team’s goal difference when a player is on the pitch compared with off it, where the greater the plus / minus score the better the player’s impact.

Whilst this may seem like a good idea, it’s unfortunately heavily biased towards the better teams. For example, take Sergio Agüero and put him in Stockport County’s first team and his plus / minus score would inevitably drop due to the quality of the players around him.

You can try and improve on this by moving to the adjusted plus / minus, which tries to account for the talent of the other players on the pitch. However, this doesn’t work well for football as the sport’s relative lack of substitutions doesn’t provide enough data to model all the combinations of players accurately enough and you end up with large errors that make the results meaningless.

Going Bayesian

An alternative approach is to use Bayesian Statistics, which allow you to incorporate prior beliefs into your mathematical model to help inform the predictions where there is a lack of data, such as for young players who haven’t played much before.

Imagine watching a footballer play for the first time - there’s a probability they may be the next Lionel Messi, there’s a probability they may be the next Bebé and there’s a probability they may be average and be somewhere between the two.

This is essentially how the model works. Based on all the available data for each player, the model constructs a set of Priors and uses them to estimate the player’s true talent compared with all the other footballers in the world. As the model gains more information about a player over the course of their career, the influence of these Priors diminishes and the credible intervals narrow around the player’s true talent level as the model gains more confidence in its predictions.

Talent Distribution

Figure One shows the distribution of player ratings in the English Premier League. To put these ratings into context they were calculated in February 2016 and at the lower end of the scale was Fabriccio Coloccini, a once decent footballer whose legs were rapidly failing him. In the middle was Ryan Mason who’s pretty much the epitome of averageness for the Premier League, and at the top end was Mesut Özil, one of the highest rated players to have ever graced the Premier League.

Figure One: Distribution of player ratings in the English Premier League

Player Career Trajectories

If you take these ratings and plot them out by age you can get an idea of what the average Premier League player’s career look like (Figure Two). Players start to break into the first team aged around 18, then steadily improve until the age of 25. They then plateau until around the age of 28, at which point the effects of age start to creep in and the player starts to decline before dropping out of the Premier League by the time they hit 35.

Figure Two: Average career of a Premier League Footballer

To put this into perspective, Figure Three shows Wayne Rooney’s career trajectory up until February 2016. The model rated Wayne Rooney as being massively superior to the average footballer from a very young age but it also suggests that he has been in serious decline since his mid-twenties and that he’s probably playing like a footballer five years older than he really is.

Figure Three: Wayne Rooney’s career trajectory

Figure Four below also shows the career trajectory for Micah Richards who signed for Aston Villa back in the summer of 2015. On the surface, it looked like a pretty good deal as Villa were getting an England international at his peak age for free. However, Richards’ profile shows that he has been in decline since the age of 25, and that by the time his four-year contract is up he’s likely to be a very average Championship defender at best.

Figure Four: Career trajectories for Wayne Rooney, James Milner, Micah Richards, and Michael Owen

Validating the Model

Whenever I discuss this concept with people their first question is inevitably to ask who the top players in the world are, and at the time of writing the presentation for Opta back in February 2016 the results were:

Lionel Messi
Thomas Müller
Cristiano Ronaldo
Manuel Neuer
Busquets
Toni Kroos
Neymar
Robert Lewandowski
Jordi Alba
David Alaba

Admittedly, it doesn’t take a complex mathematical model to know that Lionel Messi is the greatest footballer in the world but it’s reassuring to see that the results look feasible, and it hopefully provides some confidence in the model’s utility. This is important because these types of ratings are highly subjective and difficult to validate as there’s no right or wrong answer - ask 100 football fans who the top players in the world are and you’re likely to get 100 slightly different answers.

False Negatives

The next step in terms of validating the model was to look at false negatives. If the model’s ratings were to be trusted, then you’d expect the best players in the world to have shown up as some of the best players in their age group when they were younger. If not, then the model is potentially missing something and ranking players lower than they should be.

To investigate this, I looked back at how the top players in the world were ranked on their 21st birthday compared with all the other players aged 21 or under at the time (Figure Five shows the top ten). Again, the results looked feasible with all the players ranked highly amongst their peers. The lowest ranked was Jordi Alba, who was playing at Gimnàstic de Tarragona in the Spanish 2nd tier at the time.

Figure Five: Rankings of the top ten players world-wide when aged 21

False Positives

We can also test the model by looking forwards in time to see whether youngsters rated highly tend to go on and have successful careers. As an example, Figure Six shows the rankings on the 1st February 2016 for the top ten ranked 21-year-olds on the 1st February 2010.

Figure Six: Rankings of the top ten 21-year-olds on the 1st February 2010

So back on the 1st February 2010, Holger Badstuber was ranked as the number one player worldwide aged 21 or under and by the 1st February 2016, he was ranked 229th best footballer in the world overall. To put this ranking into perspective, there are around 75,000 active players in the model’s database so it puts Badstuber in the top 1% of all footballers worldwide and in the top 10% of all footballers across the big five European leagues (England, Spain, Germany, Italy and France).

I’ve only shown a snapshot of data here as it’s not possible to display comparisons for all 75,000 players. However, for the current top 5,000 players in the world the average difference in rank at age 21 to now is around 150 places suggesting the model has decent predictive power for identifying which youngsters are likely to go on to have successful careers.

Player Ratings and Team Quality

These player ratings are also highly correlated to how well a team performs over the course of a season, where the higher a team’s average player ratings them the more points they are likely to achieve (Figure Seven).

Figure Seven: Correlation between average team rating and points achieved per season

For those of you with a mathematical background, the r-squared value here is 0.77, meaning that 77% of the variability in the number of points a team achieves per season can be explained by the average rating of their players.

An interesting use of this is to predict the impact of injuries on a team’s expected points. For example, Sergio Agüero missed 12 matches of the 2015 / 2016 English Premier League due to injury and was predominantly replaced by Wilfried Bony. This reduced Manchester City’s average team rating by the equivalent of three points over the additional minutes played by Bony during Agüero’s injures.

You can also extend this idea to assess the value of potential transfers on your team. For example, if you are in the position to sign a new player who will increase your predicted points next season by two points then how much is that worth to you as a club in terms of transfer expenditure?

Player Ratings and League Position

A team’s average player rating also correlates with their final league position (Figure Eight).

Figure Eight: Correlation between average team rating and league position

If you’ve not seen this type of chart before, it’s known as a Box and Whiskers plot. The thick black line across the centre of each box shows the average, the vertical lines show the most common range of values and the dots are outliers.

So, to finish first in the English Premier League your team needs an average player rating of 125, although the champions have ranged between 115 to 130.

To finish in the top four and reach the Champions League teams need an average rating of 118, although in 2011 / 2012 Tottenham managed fourth with a rating of 103.

To avoid relegation, teams need a rating of 105. The outlier here is Liverpool in 2009 / 2010 who managed to finish in seventh place despite having the second highest rated team that season, with players like Javier Mascherano, Fernando Torres and Steven Gerrard in their squad.

Finally, the teams that get relegated have average ratings of around 101, noticeably below the League’s average player rating of 108.

Interestingly though, there is a large overlap between the relegation and safety groups. This shows that the worst teams don’t always get relegated, perhaps due to reasons like bad luck, injuries or employing Tony Pulis…

Applications for Football Teams

I’m not advocating that teams should be signing players purely on the output of a mathematical model but using data like this can help football teams in many ways, such as identifying potential transfer targets, exploring player’s career trajectories and quantifying the potential impact of injuries.

Appendix

2016 Opta Pro Forum from Martin Eastwood

A Footballer Recommendation Engine

2016-06-30T19:30:00+00:00

Introduction

With the transfer window well under way I thought I'd discuss my footballer recommendation engine for identifying potential transfer targets.

Recommendation Engines

Recommendation engines have become increasingly popular over the past few years as a way for companies to personalize content. Whether it's Amazon recommending books, Twitter suggesting people to follow or Netflix suggesting what films to watch, they are typically generating recommendations using versions of a technique called Collaborative Filtering.

Collaborative Filtering takes information about users' behaviors and uses it to calculate their personal preferences. We then assume that if users exhibit the same behaviors they'll likely agree about other things. For example, if a group of people like watching the same films as you, then you'll probably like watching the films they've seen that you've not.

There are countless different approaches for carrying out collaborative filtering, each of which add their own unique flavor to the recommendations they produce. Recommendation engines have been extensively studied over the past couple of decades and many of the algorithms are well documented but the key to making your recommendations successful is often finding an approach that works for your particular domain. For example, quantifying the similarities across sports data and recommending footballers requires a different approach to how you would recommend books or films.

As an aside, Amazon have some interesting patents discussing recommendations that are well worth a read if you are interested in the topic. Patents are normally fully of lexicon-mangling legalese but to their credit Amazon's are actually really accessible and easy to read.

Recommendations

Okay, let's take a look at some of the recommendations.

Pablo Zabaleta

I'm a Manchester City fan so let's start there as there's likely to be plenty of transfers this summer - in fact, the next transfer looks like the departure of Pablo Zabaleta to Roma. With Zabaleta gone City's only other senior right fullback is 33-year-old Bacary Sagna so unless there's some crazy change in tactics from Pep Guardiola it seems reasonable to assume they'll be looking for a replacement.

In his prime Zabaleta was one of the greatest fullbacks to have graced the Premier League so let's look at recommendations for similar players to him. We don't want similar players to last season's Zabaleta though as his legs were clearly slowing down and he was struggling to keep up with the game. Instead, let's look at the Pablo Zabaleta who was Manchester City's player of the year back in 2012/2013 and see which modern day players compare with him.

Figure One: Players Similar To Pablo Zabaleta

Reassuringly, the top ten players recommended are all right fullbacks. At no point do I define player's positions in the algorithm, the recommendation engine learns this implicitly from the data and incorporates it into the recommendations. Also, to my eye they all look to be attacking fullbacks too, which matches with Zabaleta's style of bursting down the wing on the overlap to support the attack.

The players are ranked by how similar the recommendations are to the real thing, ranging from zero where there is no similarity to one where they are identical to each other. The top recommendation here is Paris Saint-Germain's Gregory van der Wiel with a similarity of 0.87, making him a very close match. Typically, the recommendations don't really go above 0.8 that often so it's quite rare to get such a close match.

Yaya Touré

Another player who is (hopefully) on his way out is Yaya Touré. Yaya has been immense over the years for Manchester City, but like Zabaleta his legs are fading fast and he is struggling / can't be bothered (delete as appropriate) to keep up with the game so who's his closest like-for-like replacement?

Figure Two: Players Similar To Yaya Touré

The first thing to notice here is that the closest match only has a similarity of 0.65. This is low - Yaya really is a unique player and there is just nobody around who is similar to him. Interestingly though, the top recommendation is Ilkay Gündogan who Manchester City have already signed this summer. So while there is no like-for-like Yaya replacement out there, City have already managed to buy the closest match there is. Top marks to City's scouting department there!

Riyad Mahrez

Let's take a quick look at a few other interesting players too. Leicester City have done a good job of holding on to their title-winning team so far so who would you sign if you want Riyad Mahrez but can't tempt him away from the KP Stadium?

Figure Three: Players Similar To Riyad Mahrez

Paulo Dybala is the closest match, with a decent similarity of 0.81. Yeah, good luck trying to convince him to leave Juventus. Second in the list is Nathan Redmond, who's recently been snapped up by Southampton for a bargain £10 million. Third in the list is 19-year-old Milot Rashica who's been getting decent reviews playing in Vitesse's midfield and is rumored to be a target for Napoli. There's also Leroy Sané on there who's hopefully on his way to Manchester City this summer, and Ousmane Dembele who's recently moved to Borussia Dortmund for the crazy low fee of €15 million, so it's a pretty strong list.

Neymar

How about you want to sign Neymar for your team but can't afford the real thing? Figure Four: Players Similar To Neymar

The closest match to Neymar is Liverpool's Philippe Coutinho. The similarity is below 0.8 so it's not super high but it's still a fairly reasonable match. Second on the list is Ajax's Amin Younes with a virtually identical score to Coutinho - by the way, it's pretty amazing how often Ajax players crop up in these recommendations. You can pretty much pick any elite player and guarantee Ajax have someone in their teens / early twenties who profiles like them!

Lionel Messi

And finally, because people are bound to ask - here's Lionel Messi. Figure Five: Players Similar To Lionel Messi

Conclusions

*Insert usual disclaimer that I'm not advocating signing players based purely on data science / analytics / machine learning / statistics and that the goal should be to combine all of the above with the domain knowledge from professionals inside the game yada yada yada.*

Collaborative filtering is a useful technique for taking large amounts of data and filtering it down to a single value - in this case how similar different players are to each other. This information can then help refine shortlists for potential transfer targets and be used alongside more traditional scouting.

It can also be used to identify interesting youngsters. For example, there is currently a 20-year-old out there with a similarity to Gareth Bale of 0.81, a 21-year-old goalkeeper with a similarity to Manuel Neuer of 0.88 and two players in their early twenties with similarities of 0.85 to Sergio Agüero.

Much like Yaya Touré, many of the real elite players are pretty unique in what they do and there are very few of these youngsters out there who profile like them, at least outside of Ajax anyway!

Expected Goals and Uncertainty

2016-04-29T19:30:00+00:00

Introduction

My Twitter feed seems to be increasingly taken up with discussions of Expected Goals in football, yet there always seems to be something important missing from the discussion, and that's uncertainty.

What Are Expected Goals

Due to the way most expected goals models have been developed, when we refer to expected goals we are typically talking about the probability of an average player scoring a goal from an average shot taken from a certain situation on the pitch - for example taking a header at location x, y, from a chance originating from a through ball.

While this provides a reasonable estimate as to what happens over the long term for that type of shot, it fails to accurately represent the individual shot we are attempting to measure.

Pierre-Simon Laplace suggested back 1814 in his Philosophical Essay on Probabilities that if you know the precise location and momentum of every atom in the universe, their past and future values can be calculated. But in the real world we don't have that level of detail, just a bunch of data scraped off the internet. We have no idea of the ball's momentum, whether it's spinning, what the position of the defenders are, how well the goalkeeper is positioned, whether the attacking player is off balance, the direction of the wind, whether the attacking player is nursing an injury, how good the attacking player is at shooting, and so on.

As well as the uncertainty due to this lack of information, there is also the uncertainty of the model itself. We are not training the model on every shot that's ever been taken, rather we are using a subset of shots and that in itself adds to our uncertainty. This is known as the sampling error, and is the difference between the true value for a statistic and the value we've estimated from our sample of data.

No two shots taken from the same situation are ever exactly the same, and even if they were we cannot be certain enough of the estimate from our model to assign a single, absolute value to the output.

Uncertainty

This doesn't necessarily mean that expected goals are useless, but that more care needs to be taken of their usage. Instead of a single value, expected goals need to convey their uncertainty using techniques such as confidence intervals.

Confidence intervals provide an estimated range likely to contain the true value at a set confidence level. So instead of just saying a shot is worth 0.25 expected goals, we would say say the shot was worth 0.25 ± 0.1 expected goals at the 95% confidence level. This essentially means that we are 95% confident the true value of expected goals for that particular shot lies somewhere in the range of 0.15 - 0.35.

Sure, it's not so snappy but it conveys so much more information then the single value does. There is huge variability in expected goals through lack of information, sample sizes and the general randomness of football. By just using the central estimate you are missing out on a lot of information, and potentially sharing misleading numbers too.

For example, take a look at Figure One below showing the cumulative expected goals by minute from a match between Norwich City and Sunderland.

Figure One: Cumulative Expected Goals By Minute

The lines show the central estimate for the total expected goals surrounded by the 95% confidence intervals. Around minute 68 Sunderland's central estimate is higher than Norwich's suggesting they have been the better team in terms of expected goals. If you look closer though, due to the high variance associated with Sunderland's shots the bottom of their confidence interval actually stretches below the bottom of Norwich's, suggesting its's feasible for Sunderland to actually have a lower expected goals total than Norwich at that point of the game.

Even when you move towards larger sample sizes and look at expected goals by team over the course of a season (Figure Two), you still see huge variability that you'll potentially be mislead by when you ignore the uncertainty. Plus, there's some really interesting information in there, like how Crystal Palace's goals against total has a narrower range than West Bromwich Albion's. Who'd have though that??

Figure Two: Expected Goals By Team For 2014/2015

Conclusions

The variance associated with expected goals, especially at the level of individual matches (let alone individual shots!!), is such that the uncertainty needs to be clearly accounted for. Without this information, expected goals are at best an inaccurate measure and at worst misleading or wrong.

Embrace the uncertainty and include confidence intervals!

Ranking Football Teams Using Google's Page Rank Algorithm

2016-03-21T19:30:00+00:00

Introduction

I've discussed various techniques for ranking football teams on my blog before, such as using Massey Ratings to account for strength of schedule, but I've not covered the most famous ranking algorithm of them all yet, Google's PageRank.

Google PageRank

The PageRank algorithm (Figure One) was initially developed by Larry Page and Sergey Brin back in the mid nineties whilst working on a research project at Stanford University. When Page and Brin later founded Google, PageRank became the cornerstone for how their search engine ranked webpages and determined the most relevant set of results for a user's query.

$PR(A) = (1-d) + d(PR(T1)/C(T1) + ... + PR(Tn)/C(Tn))$

where:

PR(A) is the PageRank of page A

PR(Ti) is the PageRank of pages Ti which link to page A

C(Ti) is the number of outbound links on page Ti

d is a damping factor ranging between 0-1*

Figure One: Google's PageRank Algorithm

Google's search algorithm has evolved considerably over the years since then, with updates such as Panda, Hummingbird and RankBrain brought in to help deal with content farms and to better understand ambiguous queries. However, PageRank still remains the central method for determining a web page's rank in the search results.

How Does It Work?

The PageRank algorithm essentially counts links on the web and treats them like votes of support. The more links there are leading to a specific web page then the more votes there are for that page being of high quality. Not all votes are counted equally though. Votes coming from pages themselves considered high quality count for much more than from pages with few links leading to them.

This puts us in a bit of a tricky situation though as it means the rank of a web page is dependant on the ranks of all the pages linking to it, which are themselves dependant on the ranks of all the pages pointing to them and so on. Plus, when you factor in that two web pages can both link to each other then you end up with enough circularity to make this initially seem an impossible calculation.

It turns out that this problem can be solved fairly easily though through brute-force iteration. We start off by giving each webpage a default score we can use to start calculating its PageRank and then iteratively move through the system updating all the pages' ranks based on the ranks of the all pages linking back to them until the whole system converges and the ranks settle down to their true values (or at least close enough we don't mind the remaining error).

If you want to be more elegant though, you can actually skip this brute force approach and solve the whole system using linear algebra but I'm going to leave that for a future article.

How Does This Apply To Football?

Instead of using using links as votes of support from one web page to another we can use goals as votes of support from one team to another, where the more goals a team concedes then the stronger their vote of support for the opposition that scored against them.

A handy feature of the PageRank algorithm is that web pages only get one vote that ends up being shared out equally between all the other pages they are linking to. When we apply this to football it means that the more goals you score against a team, the greater the share of their vote you receive.

Results

Table One below shows the rankings for the top 100 European teams over the past year based on domestic league and European fixtures (domestic cup competitions are not currently included) as calculated using Google's PageRank algorithm.

Rank	Team
1	paris saint-germain
2	fc barcelona
3	bayern münchen
4	atlético madrid
5	borussia dortmund
6	real madrid
7	sevilla fc
8	manchester city
9	arsenal fc
10	olympique lyon
11	valencia cf
12	vfl wolfsburg
13	tottenham hotspur
14	juventus
15	sl benfica
16	villarreal cf
17	cska moskva
18	athletic bilbao
19	chelsea fc
20	shakhtar donetsk
21	as roma
22	celta vigo
23	inter
24	ssc napoli
25	bayer leverkusen
26	as monaco
27	bor. mönchengladbach
28	stade reims
29	as saint-étienne
30	lazio roma
31	fc lorient
32	zenit st. petersburg
33	fk krasnodar
34	1899 hoffenheim
35	toulouse fc
36	acf fiorentina
37	everton fc
38	hellas verona
39	malmö ff
40	sampdoria
41	olympique marseille
42	lille osc
43	rsc anderlecht
44	celtic fc
45	rb salzburg
46	dinamo moskva
47	olympiakos piräus
48	real sociedad
49	afc ajax
50	liverpool fc
51	montpellier hsc
52	manchester united
53	fc porto
54	fc augsburg
55	psv eindhoven
56	club brugge kv
57	leicester city
58	asteras tripolis
59	west ham united
60	sassuolo calcio
61	southampton fc
62	girondins bordeaux
63	crystal palace
64	lokomotiv moskva
65	brøndby if
66	molde fk
67	dinamo kiev
68	fc københavn
69	fenerbahçe
70	paok saloniki
71	ogc nice
72	fc schalke 04
73	werder bremen
74	stoke city
75	terek grozniy
76	sporting cp
77	espanyol barcelona
78	kaa gent
79	bsc young boys
80	torino fc
81	galatasaray
82	dnipro dnipropetrovsk
83	hamburger sv
84	stade rennes
85	levante ud
86	fc twente
87	rosenborg bk
88	gd estoril
89	ac milan
90	fc basel
91	fc nantes
92	hannover 96
93	panathinaikos
94	az alkmaar
95	sunderland afc
96	sm caen
97	rubin kazan
98	fk ural
99	1. fsv mainz 05
100	standard liège

Table One: Top 100 European Teams As Ranked By The Google PageRank Algorithm

The initial results seem pretty feasible, with the top five spots comprising Paris Saint-Germain, Barcelona, Bayern München, Atlético Madrid and Borussia Dortmund.

Olympique Lyon are somewhat of a surprise though in position ten but they beat Paris Saint-Germain earlier in the season and have been playing in the Champions League so perhaps they are better than my pre-conceptions? FK Krasnodar also appear higher than I was expecting but looking back at their results they did quite well in the Europa League this season, including beating Borussia Dortmund, so it's perhaps not unreasonable for them to appear in the top half of the rankings too.

The are a number of ideas for improving this concept further, such as adding a decay into the data so more recent results carry greater importance in the rankings or adding in home field advantage (which is currently missing), so no doubt there'll be an update to this blog in the future once I've refined things.

It’s Not Just The Money: Quality of Signings Suggest China Could Become a Major Power in Club Football

2016-02-29T19:30:00+00:00

Introduction

John Burn-Murdoch and the Financial Times have used my PlayerRatings model to analyse the recent flux of players moving to teams in the Chinese Super League - you can read the full article here

Frequency of Draws in Football

2015-12-12T19:30:00+00:00

Introduction

There has been some discussion and misunderstanding around the low frequency of draws in football on my Twitter feed recently so I thought I'd just give a quick recap around why the probabilities of draws are so low.

The Bookmakers

Just to give you an idea around how infrequently draws occur in football, the figure below shows Bet365's odds of a draw for all English Premier League matches going back to the 2009/2010 season. The maximum probability they gave during this time was 33.3%, the minimum was 9.1% and the mean was 26.1% with a standard deviation of 4.1%. You can actually see that 33.3% value as it's so rare it stands out from all the other points on the chart. In fact, since 2009/2010 only eight matches have even gone above 31%.

Why So Low?

Bet365's odds of a draw occurring are really narrow at 26.1% ± 4.1%. Is this range really plausible?

Well, first of all let's start off by making the assumption that the betting industry know what they are doing and that their odds are reflective of what actually happens in football. Presuming this narrow range of draw probabilities is correct, let's work through some examples and find out why they are so low.

Goals scored in football matches pretty much follow a Poisson distribution as shown in the figure below. Sure, it's not perfect, but as you can see it's very close and the bits where it doesn't quite match up can be accounted for in our calculations later on.

Since we know the probability distribution that describes goal scoring it's actually fairly trivial to calculate the probability of a team scoring 0, 1, 2, 3 goals etc based on how many goals they score on average. For example, the figure below shows the probabilities for each goal from 0-10 for a match where the home team scores two goals on average and the away team scores one goal on average.

And since we now know the probabilities of how many goals each team will score, we can calculate the probabilities of each individual score line that could feasibly happen and put it into a table as shown in the figure below.

To find out the probability of a draw, we can then sum up the probabilities of all the tied score lines (which I've highlighted in yellow for clarity) giving us a 21.1% chance of a draw, rising to 24.7% once we account for football's divergence from the true Poisson distribution.

This is pretty close to Bet365's average but still seems low overall so let's do the same again but for two identical teams both expected to score two goals each. Surely the chances of a draw will increase massively right?

Actually no, even with two identical teams the draw probability is still only around 25%, with home and away win probabilities of 37.5% each. Yes, even for two identical teams the least likely outcome is still the draw.

Think about it, of all the possible score lines that could happen in a match comparatively few lead to ties - there are much more score lines that result in a home or away win so the probability of a draw is always low and a draw will never be the most likely outcome because of this.

The chance of a draw does increase though for teams that score fewer goals as there is greater potential for both teams to fail to score, resulting in more nil-nils. If we change the example above from teams averaging two goals each to one goal each then the draw percentage increases from 25% to nearer 30%.

There is a limit to how low scoring even the worst teams are though, which is why there is a natural limit on how high the bookmakers draw probabilities will ever reach and why it's extremely rare you'll ever see anything above Bet365's previous maximum of 33.3%.

Conclusions

Football is difficult to forecast accurately and draws can provide a quick way to sense check a model's feasibility - if you are seeing draw probabilities regularly breaking the 31% barrier then your model's accuracy is likely very poor and does not accurately reflect the true chances of seeing a draw. Plus, if your draw probabilities are inaccurate then your home / away win probabilities must be inaccurate too...

Be very wary of high draw probabilities!

Updated Massey Ratings

2015-11-24T19:30:00+00:00

Introduction

I've have explained the theory behind Massey Ratings on the blog before (here and here) but incase you haven't come across them, they are a way of rating teams that takes into account the strength of the schedule they've faced. Teams get an overall rating that can also be split into separate defensive and offensive ratings to allow you to see where team's strengths and weaknesses are.

I've had quite a few requests to update the results from the previous posts so here are the team ratings for the English Premier League so far this season.

Overall Ratings

Figure One: EPL Massey Ratings

Defensive Ratings

Figure Two: EPL Defensive Massey Ratings

Offensive Ratings

Figure Three: EPL Offensive Massey Ratings

Conclusions

At the time of writing this article Leicester City were top of the English Premier League after thirteen matches; however, Massey only has them as sixth best, suggesting their schedule may have been somewhat easier than other teams so far.

Massey actually rates Tottenham Hotspur as the best team in the league, slightly above both Arsenal and Manchester City so their promising start to the season is potentially even better than the league table shows.

Poor Chelsea though are rated as sixth worst overall, similar in overall ability to Swansea City. Their attack is doing okay, with a rating sandwiched somewhere between Southampton and Liverpool. It's their defence that is letting them down, with a rating even worse than Aston Villa's!

I'll be posting updated Massey ratings throughout the season so it will be interesting to track how they change as the football season progresses.

Mathematically Optimising Your Fantasy Football Team: Redux

2015-08-07T19:30:00+00:00

Introduction

Due to popular request, here's an updated version of a blog I wrote around this time last year where I used mathematical optimisation to select the best fantasy football team possible.

The Results

The team the linear solver selected using the latest Premier League data is shown in the table below – this is the squad with the highest possible number of points that can be achieved using the constraints we are working within (e.g max three players from one team, can only spend £100 million etc).

Position	Team	Points	Name	Cost(£)
Goalkeeper	Swansea	151	Fabianski	5.00
Goalkeeper	Liverpool	149	Mignolet	5.00
Defender	Chelsea	179	Ivanovic	7.00
Defender	Chelsea	177	Terry	7.00
Defender	Everton	142	Jagielka	5.50
Defender	Liverpool	142	Clyne	5.50
Defender	Southampton	140	Bertrand	5.50
Midfielder	Chelsea	233	Hazard	11.50
Midfielder	Liverpool	162	Henderson	7.00
Midfielder	Spurs	160	Chadli	7.00
Midfielder	Crystal Palace	132	Bolasie	6.50
Midfielder	Swansea	129	Ki Sung-yueng	5.50
Forward	Spurs	191	Kane	9.50
Forward	West Brom	148	Berahino	6.50
Forward	Leicester	130	Ulloa	6.00

Limitations

I'm going to point out the obvious limitations here before I get an inbox filled with grumpy messages. First of all, the new football season hasn't started so I’m using the points totals from last season. This means all the players at the promoted teams and any new signings to the Premier League will have zero points and so will not get selected. I’ll be running this script each week throughout the season for my own use, so as these players gain points they will start to get selected by the linear solver if they perform well enough.

Also, this method doesn't account for injuries, suspensions, strength of the opposition, captains, substitutions, etc. However, it's worth noting that last season I finished in the top 1.5% of all players in the Premier League's competition using this idea to help inform my decisions. I didn't follow its recommendations rigidly but the data certainly guided my transfers and helped me finish as high up as I did.

Finally, as I noted last year, it still seems that spending your money evenly across the squad works better than splashing a heap of money on a couple of star players and then making up the numbers with a load of budget players. Apart from the Chelsea players, it's a fairly run-of-the-mill squad that doesn't look too impressive at first glance but it's actually the highest scoring squad you could have purchased.

Good luck!

Appendix

All code is available on GitHub

Expected Goals And Support Vector Machines

2015-07-13T19:30:00+00:00

tl;dr

Support Vector Machines provide a viable approach for calculating expected goals
Forecasting expected goals conceded seems more difficult than expected goals scored
Expected goals have lots of variability associated with them so calculate confidence intervals rather than just point estimates

Introduction

I've written about expected goals on this website before but I've changed approach recently so I thought I'd write up some of the different ideas I've been playing around with.

My previous expected goals model was very basic. It was essentially a non-linear regression using an exponential decay curve fitted against shot xy coordinates. Sure, there were different curves for foot shots, headed shots etc but that was about as sophisticated as it got.

An alternative approach though is to consider expected goals as a binary problem. Shots either end up in the net and score a goal or they don't. There is no other outcome, no middle ground, no half way, it's either a goal or it's not a goal. So, since this gives us just two distinct outcomes to predict, we can view expected goals as a classification problem rather than regression.

What's the difference? Well, classification and regression are inherently linked but to put it simply classification forecasts whether something will happen where as regression forecasts how much something will happen.

This means we are interested in predicting the probability a shot produces a goal rather than how much of a goal a shot is worth (which is what my previous model did). It may sound a somewhat subtle difference between the two but it opens up the opportunity to use a number of different modelling techniques.

Support Vector Machines

Support vector machines (SVM) are models often used in machine learning for data classification. They have the ability to analyse data sets and identify patterns that can then be used to forecast classes for new data points.

SVMs do this by identifying the hyperplane that separates the different categories within a training data set by the largest margin. Okay, that sounds fairly complex so as a simple example, imagine having a collection of apples and oranges sat on a table top. You could separate the different fruit by drawing a line between them so that all the apples and oranges were on opposite sides of the line. Next, imagine the fruit are now floating at different heights in the air. That simple line we drew before isn't going cut it so instead we slide a sheet of paper between the fruit to separate the apples from the oranges. As we add in more and more dimensions the sheet of paper becomes too simple to represent the separation between the fruit so we move to using a hyperplane.

SVMs also have the added benefits that they can handle non-linear data and calculate probabilities rather than just output binary predictions. So, in terms of expected goals we've got a technique that can handle the non-linearities in the shot distances and provide probabilities of shots resulting in goals. Sounds promising!

The Data

Support Vector Machines rely on a technique known as supervised learning. This means we have to train the model using a set of shots labelled with whether they lead to a goal or not. This training set was created from approximately 30,000 shots taken from the English Premier League's 2012/2013 and 2013/2014 seasons. This included open-play shots taken with head or feet, as well as free kicks and penalties.

The model was then validated using shots taken during the English Premier League's 2014/2015 season. The reason for separating the data out like this is that it helps avoid overfitting and provides a much more realistic estimation of how well the model really works. We want to create a model that generalises well and can forecast future shots. If we test the model on the same data we trained it on then we run the risk of optimising towards noise in the data and creating something that only forecasts well over the dataset it was trained with.

Initial Results

Let's start off simple and see if the preliminary results look feasible. In the test dataset there were 942 goals scored, while the trained SVM predicted 939 goals. This means we're out by just 3 goals, which is around 0.3% of the total. That looks like a pretty good start but we need to be careful here. We could be making really bad forecasts and just be lucky that all the errors are stacking up nicely and masking how bad we are.

We can find out the error by looking at the root mean square error (RMSE). This metric aggregates the differences between the model's forecasts and what actually happened to provide a single value representing the average error we made. For our SVM this came out at 0.269.

On its own this RMSE doesn't really mean much. Is 0.269 good, bad or average? RMSE isn't really meant to be used on its own though, its strength lies in providing a simple metric that can be used to compare the predictive power of different models. So what do we compare with? Well, a recent article on Deadspin criticising expected goals suggested that ignoring shot location and just using the average conversion rate gives equivalent results so let's use that as our baseline to compare against.

This naïve model, which is effectively just measuring total shots, predicted 978 goals for the test data set, with an RMSE of 0.294. So, on both metrics the SVM out performed the baseline meaning we are adding something worthwhile to the predictions.

Are You Certain About That?

So far all the forecasts have been individual point estimates. We can do better than that though and provide a range of values that our expected goals forecast likely falls within.

A common way to do this for SVMs is Bootstrapping. Instead of using our training set to create a single model, we repeatedly resample the shots to create thousands of different combinations of our data and train different SVMs with them. We can then take the average prediction from our set of SVMs as the point estimate and use the distribution of all the individual predictions to calculate a confidence interval representing the uncertainty associated with our forecast.

Doing this with 1,000 SVMs each trained on 30,000 resampled shots gives us our expected goals forecast above of 939 goals, with a 95% confidence interval of 820 - 1057 goals (approximately 12% in either direction, Figure One).

Figure One: Distribution of expected goals

Penalties

Another quick check of the SVM's feasibility is how well it performs on penalties. In the test data set there were 83 penalties taken of which 20 were missed, making a penalty worth 0.76 goals on average. Using the Bootstrapped SVM, a penalty is valued at 0.75 expected goals, with a 95% confidence interval of 0.75 - 0.79. The confidence interval looks pretty narrow and neatly encompasses the expected value so again the our SVM approach looks to be on the right track.

I previously posted an example from my exponential decay model showing that a headed shot from taken from the penalty spot had a value of around 0.08 expected goals. Moving this example to the Bootstrapped SVM gives 0.081 expected goals, with a 95% confidence interval of 0.076 - 0.085. The exponential decay model worked reasonably well at the time (with some limitations) so it's reassuring to see the results are still in line with each other.

Expected Goals By Team

This all looks good so far but we're not generally interested in predicting the total expected goals for a league - expected goals per team are of much more interest. Figure Two below shows expected goals scored versus actual goals scored for the English Premier League 2014/2015 season. The line denotes the linear regression between the two values and the shaded region the 95% confidence interval for the regression. For those interested, the r² for the regression was 0.822 and the RMSE was 7.21. For comparison, the naïve model had an r² of 0.707 and an RMSE of 9.16, both of which were inferior to the Bootstrapped SVM.

Figure Two: Expected versus actual goals for

Figure Three below shows the same plot but for expected goals conceded versus actual goals conceded. Notice how much wider the shaded region is, meaning we have much less certainty forecasting goals conceded. This matches what Will Gurpinar-Morgan previously reported (incidentally if you haven't Will's article on expected goals it's well worth a read as he goes into the uncertainty aspect in more detail than I have here). As well as the confidence interval widening, the r² dropped to 0.521 and the RMSE increased to 16.33. For comparison, the r² (0.486) and RMSE (20.57) of the naïve model also worsened, and again were inferior to the SVM.

Expected Goals By Player

And just for completion, Figure Four shows expected goals versus actual goals at the player level. As expected, since we're slicing the data to a much more granular level, the r² is slightly lower than for team goals at 0.786, with an RMSE of 1.70. It looks better than I initially expected though as the 95% confidence interval looks surprisingly narrow.

Discussion

This article is already way too long so I'm going to draw it too a close here (congratulations if you made it all the way through!) but I am planning on writing more about expected goals and SVMs in future posts.

The standout points for me though are that SVMs seems a viable approach to calculating expected goals and comfortably outperform the naïve model that ignores shot locations. The SVM model also handles both foot and head shots, as well as free kicks and penalties, which my previous exponential decay model could not.

The fact that expected goals conceded forecasts are so much poorer than those for expected goals scored is intriguing and warrants further study. Presumably, there is some factor having a noticeable impact on whether the defending team concedes that the model does not currently account for and which has less of an impact on whether the attacking team scores.

Finally, since there is a reasonable amount of uncertainty associated with expected goals forecasts I strongly advise calculating confidence intervals rather than just point estimates as it adds much more context to the metric.

Thanks for reading!

PlayerRatings And False Negatives

2015-04-23T19:30:00+00:00

Introduction

My last article looked at how well my PlayerRatings model predicted which young players would go on to have successful careers. The results looked really exciting, with the majority of players aged 21 or under who were flagged by the model as having high potential going on to become world class players.

Although it's fantastic that the model is correctly predicting these players' careers, there's something else we need to test and that's false negatives - how many talented players did the model miss and incorrectly flag as being unlikely to be make the grade.

False Negatives

False negatives here refers to those players who were initially ranked lowly by their PlayerRating score but went on to become world class players anyway. One way to identify these players is to work backwards from the top players of today to see how they were rated early on in their careers compared with their peers.

The table below shows the top 25 players in the world today as judged by their PlayerRating score (incidentally, I'd be interested to hear any feedback on this list. Does it seem feasible? Are there any major names you think are missing? Anyone who doesn't deserve to be in the Top 25?).

Player	Rank
Lionel Messi	1
Cristiano Ronaldo	2
Sergio Ramos	3
Thomas Müller	4
Cesc Fàbregas	5
Rafinha	6
Bastian Schweinsteiger	7
Manuel Neuer	8
Mesut Özil	9
Philipp Lahm	10
Wayne Rooney	11
Piqué	12
Karim Benzema	13
Marcelo	14
Iker Casillas	15
Iniesta	16
Jérôme Boateng	17
Javier Mascherano	18
Toni Kroos	19
Dani Alves	20
Ángel Di María	21
Maicon	22
David Silva	23
Arjen Robben	24
Vincent Kompany	25

Table1: The World's top 25 players by PlayerRating

Results

Okay, now we've identified the best players in the world, let's take a look at how they ranked on their 21st birthday compared with all the other players aged 21 or under at that time.

If a player's ranking was considerably lower at age 21 compared with their peers, yet they've still gone on to become world class, it suggests their initial PlayerRating score was too low and so can be classified as a False Negative.

Player	Rank At Age 21
Lionel Messi	1
Cristiano Ronaldo	2
Sergio Ramos	4
Thomas Müller	1
Cesc Fàbregas	1
Rafinha	16
Bastian Schweinsteiger	2
Manuel Neuer	26
Mesut Özil	1
Philipp Lahm	4
Wayne Rooney	1
Piqué	21
Karim Benzema	3
Marcelo	283
Iker Casillas	2
Iniesta	28
Jérôme Boateng	8
Javier Mascherano	9
Toni Kroos	6
Dani Alves	71
Ángel Di María	33
Maicon	123
David Silva	7
Arjen Robben	1
Vincent Kompany	5

Table2: Player's ranks at age 21 compared with all other players aged 21 and under at the time

Discussion

Again, the results look pretty impressive - the mode (or most frequently occurring rank) was position one, with a median rank of five suggesting that the overwhelming majority of today's world class players were correctly flagged as having high potential by the time they were 21.

The results aren't perfect though, especially for Maicon, Marcelo, Dani Alves and perhaps Ángel Di María but there is an obvious connection between these players - they all started their careers playing for South American teams.

Unfortunately, since we are going back 8-10 years in many cases, there was less data publicly available documenting the earliest stages of these player's careers, hence their reduced ratings. As I've mentioned in previous articles, minutes-played is an important factor in determining how confident the PlayerRating model is in its predictions. Without an adequate volume of data, particularly in the early years of a player's career, the model tends to be cautious in its recommendations.

Going forwards though, this becomes a non-issue as all the match data needed to calculate PlayerRatings is now freely available (albeit with a fair bit of hard work...) but it does help reinforce the point that any model is only ever as good as the data available to train it on.

Backtesting PlayerRatings

2015-04-09T19:30:00+00:00

Introduction

Following my last article discussing my PlayerRating model for quantifying footballers, JackIO challenged me to test the model by looking at what young players it recommended. Rather than looking at what players score highly now though, I thought it would be more interesting to go back in time and look at what players it rated ten years ago. This way we can see whether the PlayerRating model was accurate and whether these players actually went on to have successful careers or not.

A Decade Ago

Okay, let's go back a decade to April 2005. Tony Christie and Peter Kay were at number one in the music charts, Tony Blair and George Bush were at war with Iraq and Bolton Wanderers were five points away from the Champions League with just seven matches to play.

So who were the top players to sign back then? The table below shows the top twenty rated players aged under 21 on the 1st April 2005. I've also added in the player's values at that point according to transfermarkt, the peak value they reached, how many full international caps they've had to date and the team they were playing for at the time.

Player	Nationality	Age	Team	Value (£m)	Peak (£m)	Caps
Alexis Ruano Delgado	Spain	19	Málaga	1.75	8.8	0
Wesley Sneijder	Netherlands	20	AFC Ajax	10	39.6	113
Bastian Schweinsteiger	Germany	20	Bayern Munich	6	35.2	109
Llorente	Spain	20	Atlético Madrid	0.5	26.4	23
David Clarkson	Scotland	19	Motherwell FC	0.15	0.8	2
Nigel de Jong	Netherlands	20	AFC Ajax	4	18.5	80
Michael Rensing	Germany	20	Bayern Munich	0.75	4	0
Gonzalo Rodríguez	Argentina	20	Villarreal CF	3	12.3	6
Cristiano Ronaldo	Portugal	20	Manchester United	20.1	105.6	119
Glen Johnson	England	20	Chelsea FC	5	15	54
Christian Lell	Germany	20	Bayern Munich	0.6	3.5	0
Javier Mascherano	Argentina	20	River Plate	2	26.4	111
Iniesta	Spain	20	Barcelona	11	61.6	102
Leighton Baines	England	20	Wigan	1	20	30
Wayne Rooney	England	19	Manchester United	27	57	103
Fernando Torres	Spain	20	Atlético Madrid	22	44	110
Obafemi Martins	Nigeria	20	Inter Milan	8.8	14	39
Andreas Ottl	Germany	20	Bayern Munich	0.4	2.2	0
Piotr Trochowski	Germany	20	Hamburger SV	0.75	8.8	35
Per Mertesacker	Germany	20	Hannover 96	4.4	16.7	104

Reassuringly, of these 20 players 16 went on to receive full international caps for their countries and the overwhelming majority (if not all depending on your football geekiness) are recognizable names such as Ronaldo, Iniesta, Schweinsteiger, Sneijder etc. There's also a nice mixture of positions suggesting the model isn't necessarily biased to certain positions.

A somewhat surprise inclusion on the list is David Clarkson who's had somewhat of a journeyman career to date. Looking back at his playing history though he broke into Motherwell's first team shortly after his 17th birthday and went on to score 14 goals the following season, so his inclusion back in 2005 seems valid. I don't know enough about Scottish football though to know whether that was a fluke season but his time afterwards in English football was far from successful, culminating with him being part of the Bristol Rovers squad that was relegated to non-league football at the end of the 2013/2014 season. If anyone knows more about him I'd be intrigued to hear!

Another point worth highlighting is that all the players on the list are aged 19 or 20 and one reason for this is that minutes-played is an important factor in how the model determines a player's true rating - the more game time a player has then the more confidence the model has in its estimation of the player's true ability. Also, the majority of players in the top 20 are seemingly already at big teams by this age. It's alright if you're Chelsea and you have the finances to Hoover up all the young talent from other teams but for smaller teams looking to find promising players you've potentially missed the boat by that age.

Because of this, I also took a quick look at what players aged 18 or under the model recommended too. I fully expected these results to look worse because these players have likely had less game time meaning there is less data for the PlayerRating model to work with but even so the results still look promising.

Player	Nationality	Age	Team	Value (£m)	Peak (£m)	Caps
Vincent Kompany	Belgium	18	RSC Anderlecht	10	30	60
Yoan Gouffran	France	18	Caen	1	2.5	0
Aiden McGeady	Ireland	18	Celtic	1	8	75
Tony McMahon	England	18	Middlesbrough	0.04	1.3	0
Sergio Ramos	Spain	18	Sevilla FC	6	40	126
Cesc Fàbregas	Spain	17	Arsenal FC	6.6	48.4	94
Igor Akinfeev	Russia	18	CSKA Moscow	1	17.6	78
Tom Huddlestone	England	18	Derby County	0.3	10.6	4
Yoann Gourcuff	France	18	Rennes	1	21.1	31
Fernando Gago	Argentina	18	Boca Juniors	0	20	57
Quincy Owusu-Abeyie	Ghana	18	Arsenal FC	0.5	2.5	28
Manuel Neuer	Germany	18	Schalke 04	0.06	40	58
Christian Fuchs	Austria	18	SV Mattersburg	0.18	6.6	65
Edin Džeko	Bosnia-Herzegovina	18	Zeljeznicar	0	28.2	71
James Morrison	Scotland	18	Middlesbrough	0.7	6.6	36
Ibrahim Afellay	Netherlands	18	PSV Eindhoven	0.1	12.3	49
Ryan Babel	Netherlands	18	AFC Ajax	1	15.2	43
Marco Motta	Italy	18	Atalanta	2	6	1
Charles NZogbia	France	18	Newcastle Utd	0.5	10.6	2
Stéphane Mbia	Cameroon	18	Rennes	0.2	13.2	52

Anyone fancy signing an 18-year-old Manuel Neuer for £60k? How about Christian Fuchs for £180k? Tony McMahon for £40k? Who? To be fair to Tony McMahon, back in 2005 he was aged 18, had represented England at under-16, under-17 and under-19 level, captained Middlesbrough to the FA Youth championship, was playing regular first team football and had appeared in the UEFA Cup. It's an impressive start to anyone's career, and who knows what would have happened had he not knackered his knees and broken his leg from pretty much that point onwards.

I know the results of all this are certainly not perfect (or even scientific) and there are most likely some really good players who were ranked outside the top 20 and so haven't made this list (e.g a very young Lionel Messi at position 28) but even so the results still so far look feasible, especially considering the young age of the players. Had anyone taken a punt on this group of players then they could have put together a really exciting squad for a tiny price that would have made a hefty long term profit. Sounds good to me!

Conclusions

Okay, let's get one thing straight before my inbox is filled with people moaning - I'm not advocating teams should sign players based just on numbers let alone my PlayerRating model - I firmly believe football scouting should be a mixture of the qualitative and quantitative. Also, I know this article is somewhat subjective about what classes as successful / not successful and just because Transfermarkt says Manuel Neuer was worth £60k back in 2005 doesn't mean he was actually available for that price. I'm pretty sure Shalke would have twigged he was rather good at football by then and not wanted to sell him off quite so cheaply!!!

However, considering the paucity of data available for young players and the high attrition rate of young players making the grade I think the results of the PlayerRating model look really promising - I haven't tweaked anything here, these are the actual players the model would have rated back then based purely on the data available up until April 2005. I've still got lots more ideas for improving the PlayerRating system further but even so I'm really excited by the results so far.

PlayerRatings And Team Quality

2015-03-09T19:30:00+00:00

Introduction

Last week I introduced PlayerRatings, a mathematical model I’ve been working on over the past few months to quantify the ability of individual footballers.

Aggregable

One of the nice characteristics of this approach is that player ratings can be aggregated together to create team ratings that correlate strongly with their team’s performance. For example, Figure One below show the relationship between the average rating of a team’s players and how many points the team achieved in the English Premier League for the past eight seasons.

The r² value for this correlation is 0.73, essentially meaning that 73% of the variance associated with how many points a team achieves is captured by the average ratings of its players. To put this number into context, Total Shot Ratio (TSR), which is widely used among the analytics community to assesses a team's performance, has an r² value of around 0.68 when correlated with points.

This means that something as simple as taking a team’s average PlayerRating potentially provides us with more information than its Total Shot Ratio does. There is also scope to potentially improve this further as there is no doubt a more elegant solution than just taking the average, e.g factoring in substitute appearances, injuries, opposition quality etc.**

Baselines

Since PlayerRatings correlate well with points we can use this data to set approximate baselines for what quality of squad is needed to win the league, qualify for Europe, avoid relegation etc (Figure Two).

For example, over the past few seasons the Premier League champions have had an average PlayerRating of around 141 +/- 11, teams reaching the Champions League through positions two, three or four have averaged a PlayerRating of 133 +/- 11, teams in positions five to seventeen have averaged a PlayerRating of 111 +/- 7 and teams relegated have averaged a PlayerRating of 106 +/- 4.

So, as it stands this season, the only teams whose PlayerRatings look good enough to win the league are Chelsea and Manchester City (no surprises there then). At the bottom of the table though Hull, Sunderland, Aston Villa, Queens Park Rangers, Leicester, Crystal Palace and West Bromwich Albion all have team ratings that fall within the typical range of relegation candidates.

This is reassuring as it matches the clear lack of parity we see in the English Premier League - there are much fewer teams capable of winning the league than there are at risk of relegation, which matches previous observations I've made. I'm really interested in the distributions of these different groups so I'm planning at looking at this in more detail over the coming weeks.

Conclusions

As well as allowing individual players to be quantified, PlayerRatings can also be aggregated together to create team ratings that correlate more strongly with points achieved over the course of a season than TSR. These team ratings could be then be used to assess squad quality or evaluate whether potential signings are worthwhile. For example, will that expensive new striker on a five-year contract actually increase your squad's average rating and push it closer to the level required to challenge for Europe?

Addendum

**To be a bit more detailed, it’s actually the average of all the team’s players who started a league fixture that season weighted by the number of minutes they played in total.

PlayerRating: A Bayesian Method For Evaluating Football Players

2015-02-26T19:30:00+00:00

Introduction

I originally submitted the idea behind this article to the recent Opta Pro Forum and although it was turned down I thought I’d write it up anyway incase anyone else was interested in the results.

Plus/Minus

The premise of my abstract was that while the plus/minus score is popular in the analysis of many sports, such as NHL and MLB, it hasn't taken off in football. And there is a good reason for that - it’s hard to do.

For anyone who hasn’t come across them before, plus/minus scores measure a team's goal difference while an individual player is on the pitch. Players with a positive score are considered to have a favourable effect on the team’s overall performance while those with a negative score are causing the team to perform worse.

It’s a simple concept that sounds feasible enough but it has a big flaw in that it treats all players equally so is biased towards players on good teams. Think what would happen if you put me into Barcelona’s first team, they’d probably still win more matches than they’d lose and I’d have a positive plus/minus making me look like a great footballer. In reality, I’d have been flailing around hopelessly and would have been lucky to have even touch the ball let alone made a positive contribution.

The Adjusted Plus/Minus

One solution to this bias is the adjusted plus/minus. This incorporates a linear regression in to the calculation to account for the effect of all the other players on the pitch during the match in order to avoid a player’s score being inflated by his team mates.

However, as Howard Hamilton has previously shown on his blog, the adjusted plus/minus doesn’t work well for football. With only three substitutes per team and 38 league matches per season there is little data available to cover all the possible combinations of players. Plus some players, such as goalkeepers in particular, play a large portion of the available minutes making it virtually impossible to distinguish the true effect of removing them from the team. And with football being such a low scoring game there is a lot of noise in the data increasing the regression’s prediction errors.

As an example, here’s the current top 10 best players as rated by adjusted plus/minus scores for the English Premier league so far this season.

Player

Joleon Lescott

Ahmed El Mohamadi

Tom Heaton

Jonas Olsson

Ashley Williams

Gareth Barry

Wes Morgan

Victor Moses

Oussama Assaidi

Steven NZonzi

Now, I don’t know about you but I’m pretty sure Joleon Lescott is not the league’s best player. And what about some of the Premier League’s big stars? Well, Cesc Fàbregas is rated as the 86th best player, Diego Costa is 199th and Sergio Agüero is down in position 256. In fact the error for Sergio Agüero’s plus/minus score is so high that we can’t even tell whether he has a positive or negative rating. Yep, according to the adjusted plus/minus Sergio Agüero may actually be having a negative effect on Manchester City’s performances this season. Okaaay then.

Regularised Adjusted Plus/Minus

The next step from here is to try and reduce the errors by moving from a standard linear regression to a ridge regression. I’m not going to go into too much detail as again Howard Hamilton has a great article on this but the idea is that ridge regression helps minimise the errors associated with the player’s plus/minus scores. As with everything in life though, there is no such thing as a free lunch and by making the regression behave better we incorporate some bias into the results. But is it worth it? Nope, the results using ridge regression still have too much error to be useful. Hands up if you think Chris Smalling is the Premier League’s best player. Nobody? Right, let's move on then.

Next Steps

So now what? One of the major problems we have is a lack of data for many of the players so let's take a more Bayesian approach and add in something called a Prior. These are basically probability distributions covering some aspect of what we want to predict that expresses our uncertainty before we account for the evidence. Where we don’t have much data this Prior helps inform our predictions but as we accumulate more data the Prior’s influence decreases and the real evidence holds more weight.

Okay, that probably sounds a bit complicated if you don’t have a maths background so here’s an example: imagine you’re watching a footballer play for the first time, there is a chance the player may be as good as Lionel Messi, there is a chance they may be as bad as Tom Cleverley, and there is a chance they may be somewhere in-between and be average. As the game progresses you see them play and form your conclusion as to whether they're any good or not.

This is essentially how my PlayerRating model works. Based on preliminary data it constructs a set of Priors and estimates the probability of the player being world class, average or stealing a living in the sport. As the player’s career progresses the model gains more data about them and the estimates iteratively move away from the Prior towards the Player’s true rating.

PlayerRating

The PlayerRating model works by combining a number of factors for each player into a single rating. This rating is typically very small so to try and keep things a bit more understandable they get rescaled to centre them around 100 and make them look a little bit like a percentage. It’s not really a percentage but since everyone is familiar with that kind of number it's hopefully a bit less scary.

So what do the results look like? Well, as a starting point here is the current top twenty rated players:

Player	Rating
Lionel Messi	174
Cristiano Ronaldo	173
Sergio Ramos	163
Cesc Fàbregas	162
Thomas Müller	160
Philipp Lahm	158
Bastian Schweinsteiger	157
Rafinha	157
Manuel Neuer	156
Wayne Rooney	155
Mesut Özil	155
Piqué	155
Karim Benzema	153
Iker Casillas	151
Iniesta	150
Javier Mascherano	148
Toni Kroos	148
Jérôme Boateng	147
Ángel Di María	145

And just for the fun of it here are Lionel Messi and Cristiano Ronaldo’s careers to date:

What Next?

This is still really early stages and the work is far from finished but I wanted to get something up on the blog as it will encourage me to keep working on it and to document its progress. The next step is to dig through the data further to gain a better understanding of where this approach is working / not working so well and start to refine things. For example, goalkeepers are currently treated the same as outfield players and I suspect their ratings may be improved by having their own set of Priors.

After that there are lots of other things I want to take a look at, such as how well the ratings predict the trajectory of the player’s remaining career, how to extract confidence intervals, what's the effect of swapping an individual player out of a team and so on. My todo list is growing at a rapid rate!

At some point I’m also going to need to optimise things if I decide to continue with this idea as the ratings are pretty intense to compute. Currently, they are updated in monthly intervals and each month takes around twelve hours to process so it’s not exactly quick to tweak parameters and see the effect! There are some obvious steps to speed things up, such as distributing the processing across multiple cores or computers etc that'll provide some easy wins but no doubt the underlying maths can be optimised too. Plus, it’s all in R which doesn’t help so it may be time to dust off my C++ compiler for bits of the code...

Anyway, let me know what you think. Good idea? Bad idea? Waste of time? Do the results look feasible? Ruining football with numbers (again)?!?! (actually, if you think the last one you really don’t need to let me know!!!)

Mathematically Optimising Your Fantasy Football Team: Update

2015-01-21T19:30:00+00:00

Introduction

At the start of this season I showed how to use linear programming to mathematically optimise your fantasy football team in order to get as many points as possible for your transfer budget.

The winter transfer window is now well under way, meaning we all get a bonus wildcard to play in the Premier League's Fantasy Football that gives unlimited transfers for a week. Since I've been inundated^* with requests over the past few days for an update here is the 'best' squad you can currently buy for your £100 million.

The Optimised Squad

Position	Team	Points	Name	Cost(£)
Goalkeeper	Man Utd	89	de Gea	5.80
Goalkeeper	Swansea	80	Fabianski	5.20
Defender	Chelsea	108	Terry	6.70
Defender	Southampton	103	Bertrand	5.80
Defender	Southampton	94	Clyne	5.80
Defender	Everton	93	Baines	7.10
Defender	Southampton	89	Fonte	5.60
Midfielder	Arsenal	144	Sánchez	11.50
Midfielder	West Ham	116	Downing	6.60
Midfielder	Spurs	111	Eriksen	8.20
Midfielder	Swansea	104	Sigurdsson	6.70
Midfielder	Spurs	102	Chadli	6.60
Forward	QPR	109	Austin	6.50
Forward	Spurs	90	Kane	5.90
Forward	West Brom	88	Berahino	5.50

^*I had four requests, watch out readers this blog's going big time :-)

Massey Ratings For Football Part Two

2014-12-04T19:30:00+00:00

Introduction

In part one I introduced Massey Ratings and how they can be used to rank football teams in a way that accounts for their strength of schedule. Next, we’ll take a look at how Massey Ratings can be extended further to look at team’s attack and defence strength separately.

Massey Ratings

The idea behind Massey Ratings is that they rate teams such that the difference between any two teams is equal to the expected margin of victory between them. For example, if a team rated -1.0 played a team rated +1.0 then we’d expect the average goal difference between them to be two goals.

Since Massey Ratings look at goal difference rather than goals scored or conceded they account for a team’s overall strength and combine both their attack and defence strengths together into a single value. This means with a bit of mathematics we should be able to decompose a Massey Rating to split out these two constituent parts.

Attack And Defence

In part One we originally defined the Massey Rating as shown below in Equation One:

$y=ra–rb$

where y is the margin of victory for fixture, ra is the rating of team a and rb is the rating of team b. Let’s take this a step further and define the total goals a team should score in a match as Equation Two below:

$ya=oa–db$

where ya is the number of goals team a is expected to score, oa is team a’s attack strength and db is team b’s defence strength.

Extending this further we can say the total goals a given team should score over the course of a season is therefore equal to its attack strength multiplied by the number of matches played minus the sum of the defence strength of all its opponents. Since we know what the team’s overall rating are, how many matches they’ve played, how many goals were scored and who their opponents were we’re getting pretty close to getting what we need.

Decompose The Massey Matrix

Next we need to decompose the Massey Matrix we created in Part One into it’s diagonal and off-diagonal elements to give us two new matrices, G and P, which we use in Equation Three below:

$(G–P)r=p$

where G is total games played, P is the number of pairwise matchups each team has played, r are the team’s Massey Ratings and p is a vector of the team’s goal differentials.

From here, Ken Massey uses some clever algebra to derive the equivalent of Equation Four below:

$(G+P)d=Gr–f$

where G is total games played, P is the number of pairwise matchups each team has played, d is the defensive rating and f is the number of goals scored.

If you are interested in finding out more about the mathematics behind this then I heartily recommend taking a look through Ken Massey’s thesis where he explains it in much more detail than I’ve gone in to here.

Calculating The Ratings

Finally, we can now solve this linear system to get the attack and defence ratings for each team.

Figure One: Defensive Massey Ratings

Figure Two: Offensive Massey Ratings

It’s no surprise that Manchester City and Chelsea rate high for offensive strength but Everton are somewhat surprisingly rated third best offensive team even though they only rank mid-table in the league. Everton may only have a goal difference of +2 at the moment though but they are actually joint third highest goal scorers in the Premier League. They are performing well offensively, it’s their defence that is letting them down and is actually ranked worse than relegation-threatened Burnley’s.

QPR also rate pretty high in terms of attacking strength for a team in the relegation zone. Looking at their results for this season though they managed to score two against Manchester City, scored against Chelsea and are one of the few teams to actually get a goal against Southampton so they are performing well offensively against the league’s stronger teams. Like Everton though, their defence is performing poorly and dragging down their overall performance.

What’s that at the bottom of the offensive chart in red? Why it’s Aston Villa whose attack is so poor it actually gets a negative rating! I’ve mentioned in my last two articles about how Aston Villa’s Pythagorean and Massey Ratings show them to be seriously over-placed in the league and once again here’s another metric showing how poor they are. Bizarrely, Villa are somehow in twelfth place having managed a pitiful eight goals from fourteen matches. Although they are mid-table in the league and their defensive rating is pretty good, from an offensive point of view Aston Villa’s numbers suggest they are perhaps rather fortuitous to be so far away from the relegation zone…

Further Improvements

So far the Massey Ratings have considered each match a team plays equally but Ken Massey suggests they can be improved further by weighting matches based on their importance. For example, playing a cup match against a team from a lower division is probably less relevant to calculating the ratings than say a league match against a close rival. By weighting matches appropriately we can reduce the influence less relevant matches have on a team’s ratings and potentially improve their accuracy.

Example Code

If you are interested in having a go with Massey Ratings then I’ve put some example R code on GitHub. You’ll need to add your own data though as I’ve stripped out the section where it connects to my database for security reasons.

Massey Ratings For Football Part One

2014-11-27T19:30:00+00:00

Introduction

We all know the league table can lie and one of the common causes of this is strength of schedule. Take Southampton, at the time of writing they are currently second in the Premier League twelve matches in yet still haven’t played Chelsea, Manchester City, Manchester United or Arsenal. Without wishing to be dismissive of Southampton, who undoubtedly are a very talented team, there’s a pretty decent chance that they’d currently be lower down the league table had these fixtures come up earlier in the season instead of Leicester, Hull or Aston Villa.

Massey Ratings

So if we can’t rely on the league table to tell us which teams are performing best what do we do? One alternative is to use Massey Ratings. This is a method devised by Ken Massey back in 1997 for his honours thesis that rates teams based on what opposition they’ve played. The system was originally designed for American Football but it can be adapted to football fairly trivially.

The idea behind Massey Ratings is that they rate teams such that the difference between any two teams is equal to the expected margin of victory between them, as shown in Equation One below:

$y=ra–rb$

where y is the margin of victory for fixture, ra is the rating of team a, rb is the rating of team b

Error

In an ideal world we’d have enough data that we could calculate true ratings for each team but with players moving from one team to another and with football seasons typically lasting just 38 matches we never have sufficient data for that so we have to settle for approximating ratings based on previous match results. This means we need to modify equation one to add in an error term to allow us to account for any unexplained variation in the outcome of games (Equation Two below).

$y=ra–rb+e$

where y is the margin of victory, ra is the rating of team a, rb is the rating of team b and e is the remaining error in the model.

So far so good, but how do we know what ra and rb should equal? Well, to start with we want that error term we added into Equation Two to be as small as possible so we use a technique called Least Squares to find the optimal set of ratings for each team in order to minimise e based on the past data we have.

The Matrix

Things get slightly trickier here but let’s say our past data comprises m matches involving n teams. We know what the margin of victory was for each match and who won but not the ratings for each team so we have m equations we need to solve to find the n unknown rating values, which we can write as Equation 3 below:

$y=Xr+e$

Where y is the the margin of victory, r is the rating we are trying to find, e is the remaining error and X is an m x m sized matrix of coefficients where each row represents a matchup containing a 1 for the winning team and -1 for the losing team. Unfortunately though, this gives us a very sparse matrix that is likely to be highly over-determined making it difficult to find a unique solution to the system.

The Massey Matrix

Thankfully Massey discovered that you can modify the matrix such that the diagonal elements equal the number of games each teams has played and the off-diagonal elements equal the negation of the number of matchups teams have played against each other giving Equation Four below:

$p=Mr$

where M is the modified Massey Matrix, p is a vector of the score differentials and r is the vector of unknown scores.

We are getting closer now but the matrix still doesn’t necessarily have a unique set of Ratings so Massey modifies it further to set the bottom row to zero and the corresponding element of p to zero too. This constraint creates a full rank matrix for us and forces the ratings to sum to zero.

Massey Ratings For The English Premier League

Finally, using some linear algebra we can solve the system and get the ratings for each team, shown below in Figure One.

Figure One: EPL Massey Ratings

It’s no surprise that Chelsea are ranked far ahead of anybody else in first place but Southampton do actually get ranked in second place, showing that even accounting for their easier schedule to date they deserve to be second in the league at the moment.

Interestingly, Swansea get ranked fourth rather than their current position of seventh in the league. However, Swansea have already played five of the six teams above them so their Massey Rating shows they are performing better than their raw points tally would suggest.

At the bottom of the table it’s not looking good for Aston Villa. I showed in my last article how their Pythagorean meant they were over performing being even as high as they are and this is now backed up by their Massey Rating ranking them in one of the relegation spots.

Next Steps

In my next article I’ll show how we can take Massey Ratings a step further and decompose teams’ overall ratings into separate ratings for both attack and defence. I’ll also add some example code too so you can have a go calculating them yourself.

In the meantime, if you are interested in finding out more about the maths behind Massey Ratings then take a look at Ken Massey’s honours thesis which goes into the theory in much more depth than my brief overview here.

English Premier League Pythagorean

2014-11-04T19:30:00+00:00

I’ve not posted this for a while so here is the latest Pythagorean for the English Premier League.

Football Pythagorean

If you’ve seen this before, it’s an adaptation of the baseball Pythagorean that allows you to estimate how many points a team would be expected to achieve on average based on the number of goals they have scored and conceded. It’s a simple equation but it is surprisingly accurate.

Take a look at my previous blog posts if you want to find out more about the theory behind it, how it was tested and what the equation itself actually looks like.

The Season So Far

Figure One below below shows the difference between how many points teams have achieved in the English Premier League and how many points their Pythagorean record predicts they should have.

Figure One: EPL Pythagorean Results So Far

Worryingly for Aston Villa they’ve currently got five points more than would be expected based on their goal record. These points were all from their crazy start to the season in which they were undefeated in their first four matches, somehow coming out with ten points even though they scored just four goals. However, they appear to have regressed somewhat since then managing just a single goal and zero points from their last six matches. It’s not looking good…

Chelsea are also up five points more than expected but in contrast things could not be looking better. They are playing well and gaining more points than their goal scoring record suggests. All the signs of potential champions – a good team that are exceeding their expected points. If you want to win the league you have to be good and lucky!

Predicting Football Using R

2014-11-02T19:30:00+00:00

I recently gave a presentation to the Manchester R Users' Group discussing how to predict football results using R. My presentation gave a brief overview of how to create a Poisson model in R and apply the Dixon and Coles adjustment to it to account for dependance in the scores.

The slides are below for anybody interested and contain enough example R code to get you started. Unfortunately, there are no slide notes though but hopefully the slides should be descriptive enough to get you going!

Example code from the presentation can be found at my GitHub account

Predicting Football Using R from Martin Eastwood

Expected Goals: Foot Shots Versus Headers

2014-08-28T19:30:00+00:00

Introduction

My last article on expected goals introduced the concept of using exponential decay to estimate the probability of scoring based on the shooter’s distance from the goal. The article received lots of feedback (thanks everyone!!), with a couple of common comments standing out that I wanted to address.

Simplifying The Model

One common theme was whether the model was at risk of over-fitting and this is certainly something I was concerned about myself. In fact, I have since simplified the model to the equation below to help minimise this risk:

$expg=exp(-distance/a)$

Figure 1: Simplified Expected Goals Equation

As well as reducing the complexity of the model and making it easier to calculate the expected goals, the new equation has fewer parameters so the potential for overfitting is lower. The correlation between actual / expected goals has fallen slightly from 0.98 to 0.97 but the advantages of the simpler equation far outweigh such a minimal change.

Headers Versus Foot Shots

Another common question was whether it was important to split out headers and foot shots into separate models as the previous articles have so far ignored headers due to lack of data.

To investigate this I have been busy all summer collecting more shot data. I’m up to 45,000 shots in total now, including around 7,500 headers so I’m at the point where I’m happy to start the preliminary work comparing foot / headed shots although I certainly want more headers before drawing any definite conclusions.

I’ve run through all the curve fitting again for both headers and foot shots and plotted the resulting probability curves in Figure Two below.

Figure 2: Expected Goals: Shots Versus Headers

As you can see, headers have a noticeably lower chance of leading to a goal. The gap between head and foot shots appears largest around the ten metre mark, where foot shots have pretty much twice the probability of scoring. By 22 metres the chance of scoring from a header is virtually zero, while foot shots don’t reach this level until around 40 metres out.

Conclusions

But is this difference significant and do we actually need to bother creating separate expected goals models for headers and foot shots?

Well, if we compare the two probability curves against each other then the p value comes out at 0.064. Typically we take p values of 0.05 or lower to signify significance so by that count there is no real difference between the two.

However, p values should never be about some absolute cut off where <= 0.05 equals significance and everything else can just be ignored.

Having a value close to significance is suggestive that there may be a real difference there, especially when there is still a limited data size for headers so it’s certainly possible that headers and foot shots will warrant separate models. Luckily with the current equation this is really simple to do as we just need to alter the value of a as shown below in the appendix. This is an area I’ll be exploring in more detail as I add more headers to my database.

Appendix: Using the Expected Goals Model

To use the expected goals model you just need two numbers:

x = distance from goal in metres along x axis

y = distance from centre of goal in metres along y axis

These can then be used to calculate the total distance the shot is taken from:

$distance=sqrt(x^2+y^2)$

The expected goals for the shot is then just:

$expected goals=exp(-distance/a)$

where a = 4.4 for headers and 7.1 for foot shots

Example

Here’s an example for a player taking a header from the penalty spot.

x = 11 as penalty spots are roughly 11 metres from the goals (equal to 12 yards)

y = 0 as penalty spots should be level with the centre of the goal

$distance=sqrt(11^2+0^2)=11$

$expected goals=exp(-11/4.4)=0.08$

So on average, a header from the penalty spot would be worth around 0.08 goals.

Easy, just don’t forget you need to use negative distance inside the exponential!

Mathematically Optimising Your Fantasy Football Team

2014-07-24T19:30:00+00:00

Introduction

The Premier League Fantasy Football is back ready for the new season so I thought I’d run through an example of how linear programming can help you select your team. If you haven’t come across linear programming before it’s a mathematical optimisation technique for that can be used to maximise the total number of points your team is worth within a set of constraints, e.g. staying within budget and not signing too many players from the same team.

Collecting The Data

The first thing we are going to need to do is scrape some data to optimise our team with so let’s fire up R. We are going to need the names of all the players that are available, what team they play for, how much they cost to sign and most importantly how many points they are worth. Conveniently, we can exploit of structure of the Premier League’s website to get the data and use it as a pseudo API.

DISCLAIMER: there is a fine line between scraping someone’s web site and creating a denial-of-service attack so make sure you spread out your calls to the website. Trying to scrape all the data in quick succession can put unnecessary strain on the site’s servers. If you scrape somebody’s data please ensure you do it in a way that does not impact the service they are providing!

#load libraries
library(lpSolve)
library(stringr)
library(RCurl)
library(jsonlite)
library(plyr)
# scrape the data
df = ldply(1:521, function(x){
# Scrape responsibly kids, we don't want to ddos
# the Fantasy Premier League's website
Sys.sleep(2.5)
url = sprintf("http://fantasy.premierleague.com/web/api/elements/%s/?format=json", x)
json = fromJSON(getURL(url))
json$now_cost = json$now_cost / 10
data.frame(json[names(json) %in%
c('web_name', 'team_name', 'type_name', 'now_cost', 'total_points')])
})

Constraints

Now we have the data we need to think about the constraints we will have to build into the linear system. For example, we can only spend a maximum of £100 million, we cannot have more than three players from the same team and are restricted to two goalkeepers, five defenders, five midfielders and three forwards.

#Create the constraints
num_gk = 2
num_def = 5
num_mid = 5
num_fwd = 3
max_cost = 100
# Create vectors to constrain by position
df$Goalkeeper = ifelse(df$type_name == "Goalkeeper", 1, 0)
df$Defender = ifelse(df$type_name == "Defender", 1, 0)
df$Midfielder = ifelse(df$type_name == "Midfielder", 1, 0)
df$Forward = ifelse(df$type_name == "Forward", 1, 0)
# Create vector to constrain by max number of players allowed per team
team_constraint = unlist(lapply(unique(df$team_name), function(x, df){
ifelse(df$team_name==x, 1, 0)
}, df=df))
# next we need the constraint directions
const_dir <- c("=", "=", "=", "=", rep("<=", 21))

The Objective

We also need to create the vector defining our objective, which is to maximise the number of points the team is worth within the constraints we are setting.

# The vector to optimize against
objective = df$total_points

Solving The Matrix

Finally, we put all the constraints into a matrix and let R solve the linear system to create our mathematically optimised team selection.

# Put the complete matrix together
const_mat = matrix(c(df$Goalkeeper, df$Defender, df$Midfielder, df$Forward,
df$now_cost, team_constraint),
nrow=(5 + length(unique(df$team_name))),
byrow=TRUE)
const_rhs = c(num_gk, num_def, num_mid, num_fwd, max_cost, rep(3, 20))
# And solve the linear system
x = lp ("max", objective, const_mat, const_dir, const_rhs, all.bin=TRUE, all.int=TRUE)
print(arrange(df[which(x$solution==1),], desc(Goalkeeper), desc(Defender), desc(Midfielder), desc(Forward), desc(total_points)))

The Results

The team the linear solver selected is shown in the table below – this is team with the highest possible number of points that can be achieved using the constraints we are working within.

Position	Team	Points	Name	Cost (£)
Goalkeeper	Everton	160	Howard	5.5
Goalkeeper	Crystal Palace	144	Speroni	5
Defender	Everton	180	Coleman	7
Defender	Chelsea	172	Terry	6.5
Defender	Arsenal	157	Mertesacker	6
Defender	Arsenal	155	Koscielny	6
Defender	Southampton	149	Fonte	5.5
Midfielder	Man City	241	Yaya Touré	11
Midfielder	Liverpool	205	Gerrard	9
Midfielder	Crystal Palace	131	Puncheon	6
Midfielder	Stoke	126	Sidwell	5.5
Midfielder	West Ham	125	Noble	5.5
Forward	Arsenal	187	Giroud	8.5
Forward	Liverpool	179	Lambert	7.5
Forward	Aston Villa	106	Weimann	5.5

Limitations

Now, before the internet gets grumpy and starts trolling me (whenever I’ve mentioned using mathematics for fantasy football people seem to get very irate) there are a few obvious limitations worth pointing out. First of all the new football season hasn’t started so I’m using the points totals from last season. This means all the players at the promoted teams and any new signings to the Premier League will have zero points and so will not get selected. I’m planning on running this script regularly throughout the coming season though to help guide my transfers, so as these players gain points they will start to get selected by the linear solver if they perform well enough.

Also, we’ve set the constraints to optimise for the best squad. You may want to spend all your money on the best possible first eleven and go for budget substitutes instead. For example, the table below shows what happens if you optimise for eleven players playing 1-3-4-3 at a total price of £82 million (this leaves enough to buy four substitutes at £4.5 million each).

Position	Team	Points	Name	Cost (£)
Goalkeeper	Everton	160	Howard	5.5
Defender	Everton	180	Coleman	7
Defender	Chelsea	172	Terry	6.5
Defender	Southampton	149	Fonte	5.5
Midfielder	Man City	241	Yaya Touré	11
Midfielder	Liverpool	205	Gerrard	9
Midfielder	Liverpool	178	Lallana	8.5
Midfielder	Stoke	126	Sidwell	5.5
Forward	Arsenal	187	Giroud	8.5
Forward	Liverpool	179	Lambert	7.5
Forward	Southampton	152	Rodriguez	7.5

Interestingly (for me at least) is that the cost of the players is fairly evenly spread across the team. Typically, when I select my fantasy football teams I tend to splash the cash on the big name strikers and then go for cheap defenders. However, based on these results though that’s looking like a bad decision so this season I’m going to follow the data and actually sign some decent defenders. Wish me luck…

Appendix

All code is available on GitHub

Expected Goals And Exponential Decay

2014-04-22T19:30:00+00:00

Introduction

In my last article on expected goals I showed how to incorporate the distance from goal along the Y axis into the expected goal model using Pythagoras' Thereom.

This all worked pretty well, giving us an r squared value of 0.95. However, while the r squared value was good there was still a flaw in the model we need to fix.

Better than Ronaldo

Eagle-eyed readers will have noticed that the fit of the curve broke down for very short distances, meaning the probability of scoring from zero metres was actually slightly above one. And as reader Benjamin Lindqvist commented, not even Ronaldo will score more than 100% of the time, not even from the goal line. Benjamin also had a good suggestion to improve this, adding an exponential decay function into the model to make it behave better around zero

Exponential Decay

If you aren’t familiar with exponential decay it basically means that a value decreases at a rate proportional to its current value. It’s a phenomenon that crops up fairly frequently in science and the natural world. For example, air pressure decays exponentially as you go higher up into the Earth’s atmosphere and radioactivity decreases exponentially over time.

A general equation for exponential decay is shown in Figure 1, where Y(t) is the value at time t, a is the starting value, k is the decay constant and t is time.

$y(t)=ae^{kt}$

Figure 1: Exponential Decay

So how do we apply this to football? Well, the first thing to do is replace time with metres and assume that the probability of scoring a goal decreases exponentially based upon the distance from goal the shot is taken from.

Next we need to find the correct value for the decay constant as this controls the shape of the curve. Rather than doing this manually through trial and error, we can use something such as R’s optim function to find it for us. We can also tweak the equation to add in a multiplier for the independent variable and an intercept as found in a traditional regression model giving us the fit shown in Figure 2.

Figure 2: Shots Versus Distance From Goal

Notice how the orange line now hits the Y axis just below 1.0? This fixes the problem we had before where it was possible to score more than one goal from a single shot. In fact, if you’re standing on the goal line the model now predicts around 0.96 expected goals, so very likely to score but with a small chance of screwing up (yes Edin Džeko I’m looking at you).

The new curve fit also pushes the r squared value up to 0.9883, meaning 98.83% of the variance for the probability of scoring from a shot can be accounted for using just distance from goal along the X and Y axes.

The final equation (Figure 3) is slightly more complicated now but it’s still pretty simple to use.

$expg=e^{-d/4.79}*0.921985+0.036212$

Figure 3: Expected Goals Equation Incorporating Exponential Decay

where:

$d=sqrt(dx^2+dy^2)$

Figure 4: Equation for d

and dx, dy are the difference between the x coordinates and y coordinates in metres for the shot location and the goal location.

As ever, let me know what you think!

Expected Goals: The Y Axis

2014-04-16T19:30:00+00:00

Introduction

Expected goals are one of the hot topics in the football analytics community at the moment and it’s a topic I’ve previously written a number of articles discussing how to calculate them. If you haven’t read those pieces yet it’s probably worth taking a quick look to set the context for the rest of this article.

The Story So Far

A few week’s back I published a simple equation for calculating expected goals that received a lot of positive feedback from readers as it was easy to use and was pretty accurate based on its r squared value of 0.86. This effectively means the equation is capable of explaining 86% of the variance in the shots data I have collected from Squawka.

For such a basic equation this is a really good result. I’d purposely tried to keep things simple so that the equation was easy enough for non-mathematicians to use in order to try and encourage its adoption by other people. Rather than keep these sort of things to myself I’d much rather share them around and see them get used elsewhere.

One of the restrictions I’d set myself for this was to only use the distance the player shooting was from the goal along the X axis so that the equation only needed data along one dimension. However, I received a lot of messages through Twitter and on the blog asking about the Y axis so let’s take a look…

The Y Axis

So the first question to ask was whether the Y axis was even worth bothering with, after all the r squared value when just using distance along the X axis was already 0.86 which only left around 14% of the variance in the data to account for.

Well, it turns out that how far away you are from the goal along the Y axis does have an impact (Figure 1). Unsurprisingly the further away you are then the less likely you are to score. Before you ask, the r squared value is 0.88 (I have learnt now to include r squared values for pretty much all charts otherwise I get bombarded by requests for them :-)).

Figure 1: Shots Versus Distance From Goal Along Y Axis

Adding The Y Axis Into The Equation

Okay, we know the Y axis has an effect on expected goals but how do we factor this into my previous equation? There are a number of mathematical techniques we can use to solve for multiple dimensions. However, I am keen to try and make this as simple as possible so that the lay-person can use it so let’s keep it basic and go with Pythagoras’ Theorem, a topic most people have touched on at High School at some point.

If we know the xy coordinates of the player taking the shot and the xy coordinates of the goal then using Pythagoras’ Theorem we can calculate the total distance between the two points. Figure two shows the equation for this where dx is the distance between the two x coordinates, dy is the distance between the two y coordinates and AB is the total distance the player is from the goal.

$AB=sqrt(dx^2+dy^2)$

Figure 2: Calculating the distance between two points

I did this for all 17,000 shots I have collected so far from Squawka (excluding penalties) to get their total distances from goal and calculated the probability of scoring from different distances based on the number of shots taken versus goals scored (Figure 3).

Figure 3: Shots Versus Total Distance From Goal

As previous, I’m using a power curve to fit the line through the data and as you can see it’s a pretty good fit. So what is the effect of adding in the Y axis? Well the r squared value has changed from 0.86 to…

drumroll

0.95

Yep, including both the x and y axis into the expected goals model accounts for 95% of the variance in the data. This barely leaves any room for the shooting player’s talent to have any effect or even for defensive pressure to play a part.

At first I thought this seemed a bit odd but thinking about it in more detail it actually seems logical. It doesn’t make much difference whether you are shooting from five metres out against a strong defence or a weak one, you still have the same chance of scoring from that particular position.

However, playing against a strong defence will likely mean you will get into that good position less often so your overall expected goals will be lower. Conversely, better players will be able to get into those good positions more often than weaker players so their overall expected goals will be higher.

In other words, at the individual shot level expected goals seems to be all about a player’s position in respect to the goal when they shoot. Other factors, such as player talent, defensive pressure etc are probably not visible until you start looking at larger samples, such as expected goals per fixture or even per season.

Anyway, here’s the final equation:

$ExpG=Distance^{-1.33796}*10^{0.4720605}$

Let me know what you think!

English Premier League Pythagorean Update

2014-04-04T19:30:00+00:00

Introduction

I’ve not posted an update on the Pythagorean for the English Premier League (EPL) for a while so the latest figures are below.

Football Pythagorean

In case you haven’t seen it before, my football Pythagorean is an adaptation of the baseball pythagorean that allows you to quickly estimate how many points a team would be expected to achieve on average based on the number of goals they have scored and conceded. It’s a pretty simple little equation but it is surprisingly accurate.

Take a look at my previous blog posts here about it if you want to find out more about the theory behind it, how it was tested and what the equation itself actually looks like.

The Season So Far

Figure One below shows the difference between the actual points each Premier League team has achieved this season and how much my Pythagorean predicts they should have on average. For teams in green the difference is positive meaning they have more points than expected while those teams in red have less points than expected based on the number of goals they have scored and conceded.

Figure One: EPL Pythagorean Results So Far

Once again Tottenham are way ahead of where they would be expected to be, with an astonishing 15 points extra. Either Spurs are doing something fantastically efficient this season or they are extremely lucky to be where they are in the league. Take those 15 points away and they drop down to 10th place just ahead of Stoke. This season has been a bit of a write off for Spurs compared with pre-season expectations but it could / should have been so much worse based on their Pythagorean.

Down at the other end of the table Hull should probably be feeling quite pleased with themselves as they are looking a pretty safe bet to avoid relegation even with their Pythagorean of -5.

Poor Swansea though have the lowest Pythagorean in the league. On average teams with their goal record would expect to have achieved roughly nine more points than their current total. In fact if Swansea and Tottenham both had the average points their goals suggest then the Swans would actually be the higher placed of the two teams!

Let’s see what happens if / when regression towards the mean starts to kick in…

Expected Goals Updated

2014-03-01T19:30:00+00:00

Introduction

When I introduced my Expected Goals model a few weeks back a number of people commented on the bump in the curve where I had included penalty shots in the data set used to fit the model. The reason I’d originally left penalties in was I felt their number was too few to have an impact on the fit of the model and at the time I hadn’t actually tracked which shots were and were not from penalties.

Since that decision seemed to cause quite a kerfuffle I have since gone back to the raw data, removed all the penalties and refitted the curve. While I was at it I also added in more shots I had collected and rescaled all the co-ordinates to use a larger pitch (105 x 68m) as Claus Moeller had suggested my estimate of Premier League pitch size was too small.

As expected, the difference in the fit of the curve is very small (Figure 1) but it has pushed the r squared value up to 0.86 from 0.84, meaning that 86% of the variance in goal scoring is due to the distance from the goal the shot is taken from and just 14% is due to other reasons, such as player talent, defensive pressure, goalkeeper etc.

Figure 1: Shots Versus Distance From Goal

The equation for expected goals is now updated to -1.014718 for the coefficient and 0.05082859 for the intercept so for my previous example a shot from 8 metres gives:

$8^{-1.014718}*10^{0.05082859}=0.1362846$ expected goals

Actual Goals Versus Expected Goals

2014-02-15T19:30:00+00:00

Introduction

Since my last article about how to calculate expected goals one question has come up more than any other and that is about the correlation between expected goals and actual goals so here you go:

Figure 1: Shots Versus Distance From Goal

Figure 2: Expected Goals Away Versus Actual Goals Away

The correlations look pretty good, 0.86 for goals for and 0.72 for goals away. I’m not sure yet why the correlations differ slightly for home / away and whether it means anything or is just down to noise in the data but I’ll keep an eye on that as I collect more shots over the course of the season.

Another question that popped up a few times was whether my expected goals correlated with actual goals better than Total Shot Ratio (TSR) does and the answer is yes it does.

This is to be expected really as expected goals account for shot location while TSR considers all shots to be equal when clearly they are not – a shot from one metre out is vastly more likely to lead to a goal than a shot from 20 metres out.

Figure 3: TSR Versus Actual Goals For

Figure 4: TSR Versus Actual Goals Away

There is still a heap of work to do to improve / optimise / characterise the expected goals model further but it is a promising start for it so far. I’ll post more updates as as I progress with the model’s development over the coming weeks.

Expected Goals For All

2014-02-12T19:30:00+00:00

Introduction

It seems that everybody has their own expected goals models for football nowadays but they all seem to be top secret and all appear to give different results so I thought I post a quick example of one technique here to try and stimulate a bit of chat about the best way to model them.

The Data

Over the past few weeks I have tediously collected several thousand xy co-ordinates for shot locations from Squawka and converted them into approximate distances from goal in metres, assuming that an average football pitch is 100m x 65m.

Goals Versus Distance

Figure 1 below shows the relationship between the probability of scoring a goal and how far away from the goal line the shot is taken from.

Figure 1: Shots Versus Distance From Goal

There seems to be a little bit of noise in the data, particularly around the 12-13m mark but overall I was pleasantly surprised how neat the data looks – there seems to be a pretty clear non-linear relationship between the likelihood of scoring and how far away from the goal the shot is taken from.

So how do we model this relationship? Obviously we cannot just stick a linear regression through the graph it as the relationship is clearly not linear so one possibility is to use a polynomial instead of a straight line (Figure 2).

Figure 2: Fitting a Polynomial

Unfortunately, this does not give particularly good results as low order polynomials (the orange line) do not fit tightly enough to the non-linearity in the relationship while higher-order polynomials (the red line) start to fit to the noise in the data leading to problems with over-fitting.

So what do we do now? Well, looking closer the shape of the curve appears exponential so one option is to fit a Power function to it. We can do this pretty easily by taking the log of the data, fitting a linear regression against it and plotting this against our non-logged data (Figure 3).

Figure 3: Power Curve

This gives an extremely good fit with the data and seems a plausible choice. We know goal scoring is Poisson distributed so it would seem natural to fit expected goals using an exponential shaped curve since Poisson and exponential distributions are inherently linked – the exponential distribution in fact describes the time taken between individual events occurring in a Poisson process.

If we calculate the r squared value for the fit of the Power curve then we get a value of 0.84, meaning 84% of the variance in goal scoring can be attributed to how far away the player taking the shot is from the goal. This is pretty impressive as it leaves just 16% attributed to other reasons, such as the angle of the shot, goalkeeper positioning, defensive pressure, the shooting player’s talent etc.

Before you ask, I’ll be looking at whether adding these additional factors into the model can improve it or whether the added complexity is not worth chasing the 16% for in the coming weeks.

Using the Expected Goals Model

But how do we use the model? Although everybody else’s models seem to be top secret I’m going to give mine away. The coefficient for the regression is $-1.036884$ and the intercept is $0.05950286$.

To put this into action all you need to do is raise the distance away from the goal in metres to the power of the coefficient and multiply by 10 to the power of the intercept. For example, a shot from 8 metres gives:

$8^{-1.036884} * 10^{0.05950286} = 0.132771$ expected goals

So how about we give it a proper test and try it out on this season’s English Premier League to date? The results are shown in Table 1 and overall give a root mean square error of 8.2 goals, which seems a pretty reasonable starting point for developing the model further from.

	Team	Goals	expG	Residual
1	Man City	68.00	46.90	21.10
2	Liverpool	63.00	42.56	20.44
3	Arsenal	48.00	35.26	12.74
4	Chelsea	48.00	42.56	5.44
5	Man Utd	41.00	35.26	5.74
6	Southampton	37.00	29.05	7.95
7	Everton	37.00	33.31	3.69
8	Newcastle	32.00	30.32	1.68
9	Swansea	32.00	26.89	5.11
10	Tottenham	32.00	31.21	0.79
11	WBA	30.00	31.17	-1.17
12	West Ham	28.00	26.69	1.31
13	Aston Villa	27.00	24.20	2.80
14	Stoke	26.00	25.29	0.71
15	Sunderland	25.00	25.63	-0.63
16	Hull	25.00	23.95	1.05
17	Fulham	24.00	24.39	-0.39
18	Cardiff	19.00	24.67	-5.67
19	Norwich	19.00	27.61	-8.61
20	Crystal Palace	18.00	25.03	-7.03

Table 1: Expected Goals For The English Premier League To Date

You can also see a pretty clear pattern in that the teams at the top of the league have generally over-performed the goal expectancy while those towards the bottom end have under-performed it. This would seem reasonable as we are predicting average goal expectancy and the top teams are obviously above average so should perhaps do better with their chances, while the lower teams are below average so would be expected to perform worse?

What Next?

I’m not claiming this to be the only way of calculating expected goals, or even the best way but hopefully it will encourage more discussion of how to calculate expected goals rather than a lot of secret black boxes all giving different results.

I hope to write more about expected goals over the coming weeks in order to test this equation to see how well it really works, to hopefully improve it further and to try and understand what the metric can and cannot tell us.

In the meantime, feel free to use my equation to calculate expected goals, all I ask is that you don’t try and pass the equation off as your own (you know who you are!!) and that if you use it then please acknowledge me and link back to my site.

Be warned though it’s a work in progress so is subject to change as and when I improve things…

Enjoy!

Comparing Players Using Cluster Analysis

2014-02-10T19:30:00+00:00

Introduction

As there were a couple of presentations at the recent Opta Pro Forum talking about identifying player similarities I thought I’d give a quick example of how to do something similar using k-means cluster analysis.

The Data

All the data used in the analysis was taken from public websites, such as whoscored, squawka, transfermarkt etc and painstakingly matched together to try and get as much information on each player as possible.

The first stage of analysis was to normalize the data so it was all in the same range to avoid biasing the clustering. If you think about how many goals a typical player scores per match compared with how many passes they play then the scale is quite different. Since k-means clustering uses Euclidean Distance the clusters formed are influenced strongly by the magnitudes of the variables, especially by outliers. By normalizing all data into the same range this bias can be avoided.

Principal Component Analysis

While normalizing the data, I also performed Principal Component Analysis (PCA) on it too. This step isn’t essential but it is a handy way of reducing the dimensions in the data down to a more manageable size by squashing all the data together into new variables known as principal components.

These principal components are created in such as way so that the first one accounts for as much as the variance in the data as possible, the second one then accounts for as much of the remaining variance and so on.

As you can see in Figure 1 below, the first component represents pretty much 70% of all the variance in the data with each additional component accounting for less and less. This means we can represent pretty much all the information in the data without losing much using just five components, and around about 80% using just two components.

Figure 1: PCA scree plot showing amount of variance accounted for by each principal component

Clustering The Players

The next step was to then run the k-means clustering algorithm on the data. As shown in Figure 2 the players split relatively neatly into five distinct coloured clusters when plotted by the first two principal components.

Figure 2: Players split into different clusters by colour

Goalkeepers

As a quick test we can look at the grey cluster located at the bottom of the image in more detail to see which players are contained within it (Figure 3). If you click the image to zoom in on it you can see it’s done a pretty good job of pulling out the goalkeepers from the rest of the players. This is to be expected since goalkeeper’s stats should be pretty distinct from outfield players but it’s reassuring to check the technique passes this first simple test before we move on.

Figure 3: The grey cluster up close

Vincent Kompany

Now that we have separated out the goalkeepers we can take a look at how well the technique copes with outfield players, starting with Manchester City’s central defender Vincent Kompany located at the centre of Figure 4. The results are pretty good, with Kompany surrounded by players predominantly considered to be defenders. As you move up the image the players start to get a bit more attacking with people like David Luiz, Phil Jones and Fabien Delph starting to appear

Figure 4: Clustering of Vincent Kompany

Adnan Januzaj

Next up is Adnan Januzaj, one of the few Manchester United players to be having anything resembling a decent season this year. Again the results look pretty plausible (Figure 5), with Januzaj surrounded by predominatly attacking midfielders. There are a couple of slightly surprising results in there though, such as Manchester City’s strikers Álvaro Negredo and Edin Džeko.

Figure 5: Clustering of Adnan Januzaj

Mikel Arteta

Finally, I added in Arsenal’s midfielder Mikel Arteta (Figure 6). This one was probably the most surprising of all the players I’ve looked at as there seems to be quite a mix of players around Arteta, including both offensive and defensive players, although perhaps this is actually representative of Arteta’s role at Arsenal?

Figure 6: Clustering Mikel Arteta

Next Steps

For a first go the results are pretty promising but there are plenty of ways the technique could be improved. At the moment I have used all the data I had available for each player but I suspect more specific results could be obtained by filtering the data.

For example, there may be specific attributes of a player you want to match on e.g. looking for attackers by just their creative output may be more useful than including their tackles, interceptions etc, which may be of minor importance to their role.

Finally, all the data used here are aggregated. A really interesting next step would be to include xy co-ordinates for shot locations, interceptions, passes etc to cluster players based on the locations of their actions on the pitch (donations of xy data will be gratefully accepted :)).

EPL 2013/2014: Football Pythagorean So Far

2014-01-20T19:30:00+00:00

Introduction

Welome back! Now that I'm no longer part of Onside Analysis I'm free to start blogging again so let's start off by taking a look at how my football Pythagorean is doing for the English Premier League so far this season.

Football Pythagorean

In case you haven’t seen it before, my football pythagorean is an adaptation of the baseball Pythagorean that allows you to quickly estimate how many points a team would be expected to achieve on average based on the number of goals they have scored and conceded. It’s a pretty simple little equation but it is surprisingly accurate!

The Season So Far

Figure One below shows the difference between the actual points each Premier League team has achieved this season and how much my Pythagorean predicts they should have on average. For teams in green the difference is positive so they actually have more points than expected while those in read have gained less points than would be expected based on the number of goals they have scored and conceded.

Figure One: EPL Pythagorean Results So Far

The stand out team here is obviously Tottenham, who have somehow managed to end up with eleven points more than would be expected based on their goals. Spurs’ Pythagorean has looked pretty big for a while now so I suppose you could look at this two ways – either they have developed an extremely effective and efficient system or they have been lucky to get as may points as they have. I’ll let you decide on the answer to that one…

Interestingly, Manchester City are pretty close to their expected points total despite their enormous goal difference. One reason for this is that my football Pythagorean is not linear so as you score more goals they become less valuable to help account for high scoring matches, such as most of City’s home games this season! This helps prevent over-prediction of expected points for teams scoring heavily – having a good goal difference is obviously helpful but whether you win by one goal or five goals you still only get three points from the match.

As it stands though Manchester City are in second place behind Arsenal who have acquired six points more than expected, meaning that typically we would not expect Arsenal to be top based on their results so far this season.

How Will The Season End?

As well as looking at how teams are doing so far, we can also extrapolate the results and predict how the teams will end up at the end of the season (Table One). This is a very simplistic prediction, for example it does not take into account strength of schedules, but it is fairly accurate – the r squared value for Pythagorean predicted points versus actual points across multiple leagues worldwide was 0.938 with an average error of less than four points – so it should give a reasonable estimate of how the Premier League will finish next May.

	Team	Points
1	Manchester City	84.50
2	Arsenal	83.59
3	Chelsea	80.99
4	Liverpool	73.75
5	Everton	72.20
6	Tottenham Hotspur	65.89
7	Manchester United	62.56
8	Newcastle United	59.32
9	Southampton	54.42
10	Aston Villa	41.52
11	Hull City	40.97
12	West Bromwich Albion	40.74
13	Swansea City	39.66
14	Stoke City	36.85
15	Norwich City	35.78
16	West Ham United	33.91
17	Sunderland	32.28
18	Crystal Palace	31.30
19	Fulham	30.65
20	Cardiff City	29.29

Table One: Pythagorean Predicting Final Standings For the English Premier League 2013/2014

UEFA Champions League – Route To The Final

2013-09-30T19:30:00+00:00

Introduction

With the UEFA Champions League group stage now underway I took a quick look at what it typically takes for teams to reach the final.

I started off by looking at how well teams from the major six domestic leagues (England, France, Italy, Spain, Germany and Portugal) performed in the UEFA Champions League based on what position they qualified in domestically (Figure 1) as this affects at what point they enter the competition…

Find out more by reading the full article on the Onside Analysis blog here.

Analysing Football Teams Using Cluster Analysis and Principal Component Analysis

2013-08-30T19:30:00+00:00

Introduction

The amount of football data available is growing rapidly – with every passing week of the season more matches are played and even more data gets collected. This is great as it allows us to increase our understanding of the game but it also means we quickly end up with more information than could ever be analysed manually.

Instead, we can use techniques such as cluster analysis and principal component analysis (PCA) to critically analyse these large sets of football data to identify important patterns and relationships that can help explain a team’s performances.

Find out more by reading the full article on the Onside Analysis blog here.

Anouncement

2013-06-19T19:30:00+00:00

Introduction

You may have noticed that my blogging has slowed down over the past few weeks and the reason is that I have joined Onside Analysis as a computational statistician.

I am really excited about my new role as it means that I will be working on football analysis full time instead of trying to squeeze it in around my day job, family, sleep etc. I am not sure exactly what this means for my blog here though but the plan is that I will be contributing to the Onside Analysis blog so keep an eye out on that if your interested in what I have been writing about so far.

I’ll also still be around on Twitter so please keep in touch :)

Betting With The Eastwood Index And Kelly Criterion

2013-05-23T19:30:00+00:00

Introduction

I demonstrated in my last post that the odds calculated using the Eastwood Index were slightly more accurate than the bookmakers over the course of the football season. My next goal is to work out the optimal way of using this edge to make a profit, starting off with the Kelly Criterion.

The Kelly Criterion

The first point of call for any staking plan is the Kelly Criterion, a method developed by John Larry Kelly Jr to determine the optimal bet size based on how far the odds are perceived to be in your favour.

The equation used to calculate the Kelly Criterion is shown in Figure 1 where $p$ is your expected probability of winning, $b$ is the odds offered and $f$ is the Kelly Criterion or recommended percentage of your bankroll to bet.

$f=(pb-1)/(b-1)$

Figure 1: Kelly Criterion

Let’s run through a quick example using Fulham’s last match of the season against Swansea City. Bet365 offered Fulham to win at odds of 4.75, which is equivalent to an expected win probability of around 21%, while let’s say you think the probability of Fulham winning is actually closer to 24%.

$f = (0.24 * 4.75 – 1) / (4.75 – 1)$

$f = 0.14 / 3.75$

$f = 0.0373$

$f = 3.73%$

So according to the Kelly Criterion we should be willing to risk 3.73% of our bankroll on this bet.

Applying the Kelly Criterion to the Eastwood Index

So what is the best way of applying the Kelly Criterion to the Eastwood Index? There are numerous different strategies that could be used but to start off with I’ve gone purely with value bets.

For each match I calculated the Kelly Criterion based on the Home, Draw and Away odds from Bet365 and looked for the outcome where the recommended bet was the largest. The reason for this was that the larger the recommended bet then the greater the difference between my probabilities and the bookmaker’s odds so the greater the potential value of the bet.

Figure Two shows the results over the course of the season. Starting off with a bankroll of £100 there was a slight loss over the first half of the season followed by pretty steady growth to finally finish with £114 in the bank. This gave a return on investment (ROI) of 14% for the Eastwood Index based on the 2012–2013 premier League season.

Figure 2: Bankroll over 2012-2013 Season

Fractional Kelly

Being relatively risk averse, I’ve found that using a fractional Kelly Criterion is preferable for me. Although using the full Kelly Criterion is optimal for maximizing growth of the bankroll long term, there is more risk of being caught out by variance and an unlucky streak wiping out your bank balance.

Betting a fractional value, such as half the recommended amount slows down growth but helps protects you from volatility. As a comparison, take a look at Figure Three where I bet a full Kelly Criterion on each match and you can see that at its peak the ROI reaches 73%, yet at the end of the season variance has pulled the bankroll down below its starting value causing a loss overall.

Figure 3: Volatility Betting The Full Kelly Criterion

Early Season Dip

One intriguing aspect of Figure Two is why the bankroll grew so much more in the second half of the season compared with the first? Partly this may be due to random variance but another possibility is betting on recently promoted teams.

At the moment promoted teams take over the EI ratings of the equivalent relegated team so the team promoted as champions take the EI rating of the team relegated third from bottom, the team promoted second take over the rating of the team relegated second from bottom and the team promoted in the playoffs gets the ratings of the team finishing bottom.

These ratings will not be exactly correct for the promoted teams but should over time move towards the right levels to reflect the team’s performances. Looking through the history of the bets made though those involving a promoted team during the first half of the season lost money overall while those not involving promoted teams made a profit.

By the end of the season I had made a profit out of the promoted teams, suggesting that the team’s ratings had sufficiently corrected themselves. This means though that I could improve the ROI even further by avoiding bets on the promoted teams early on in the season or improving the way the promoted teams ratings are handled, such as correcting their EI ratings faster.

Conclusions

This is only a quick look at a very simple strategy for placing bets using the Eastwood Index; there are still plenty of improvements that can be made to improve results further. Yet even with this relatively naive approach the ROI is 14%, which is much more than I would have made sticking my money into a bank account.

Applying the Eastwood Index to betting is also a great ways to identify the strengths and weaknesses of the model as the ROI gives a clear indicator of what works, what doesn’t work and what can be optimized further.

Addendum

I was asked on Twitter what the ROI works out at per bet for the Eastwood Index without betting the Kelly Criterion. Using a fixed bet of £1 per bet gave an overall profit for the season of £17 over 380 matches, which works out at an ROI of around 4.5% per bet.

Did The Eastwood Index Beat The Bookmakers?

2013-05-21T19:30:00+00:00

Introduction

It’s the end of the season so it’s time to review how the Eastwood Index performed over the year and how it compared with the bookmakers.

Ranked Probability Scores

One of the most important aspects to me is how accurate the forecasts were and I’ve assessed this using Ranked Probability Scores, as recommended by Constantinou and Fenton. I’ve discussed Ranked Probability Scores on the blog before but for people new to them they measure the difference between the forecasts and what really happened. Scores range between 0–1 and represent the amount of error in the predictions so lower Ranked Probability Scores are better and signify greater accuracy.

Comparison With Bookmakers

Looking at Figure 1 you can see that the Eastwood Index has consistently outperformed the bookmakers all season – and this isn’t just one bookmaker that the Eastwood Index has beaten but the combined knowledge of the industry as I’ve aggregated multiple bookmakers’ odds together and stripped out the overround to make the comparison as tough as possible.

Figure 1: Eastwood Index Vs Aggregated Bookmakers

Interestingly, the difference in accuracy seems to be greatest as both ends of the season. I expected the start of the season to be difficult to forecast as new teams have been promoted, players have been bought and sold, and managers may have changed clubs but the Eastwood Index seems to have coped with these variables better than the bookmakers’ odds have.

Over the course of the season the bookmakers’ forecasts improved until there was very little difference between them and the Eastwood Index but I was somewhat surprised to see how far out their accuracy drifted over the final few weeks of the season.

In theory these should be the easiest matches to forecast as we have the most information but in reality they can be tricky as team’s motivations change. For example, Manchester United have been playing their reserve goalkeeper so he gets enough appearances to earn his winners medal while Swansea’s players may as well have been on holiday since they won the league cup.

These changes seem to have thrown the bookmakers’ odds out quite noticeably while the Eastwood Index’s accuracy has remained constant. In fact, it suggests that bookmakers may be over-compensating for these apparent end-of-season effects as the Eastwood Index does not currently take them into account and has not struggled because of it.

Conclusions

Overall, I am pleased with the Eastwood Index’s debut season. I was slightly reticent to publish the forecasts at first in case the model did not hold up but it has remained accurate throughout the year. The next stage of its development is to identify any patterns as to where its forecasts differ from the bookmakers and how that could be combined with various staking strategies as well as looking at expanding to cover other leagues too.

EI Match Probabilities for the English Premier League

2013-05-17T19:30:00+00:00

We have finally reached the end of the season so for the last time in 2012-2013 here are the Eastwood Index’s (EI) probabilities for the English Premier League.

Once the season is over and done with I’ll be looking back at how the EI has performed and how well it’s predictions compare with the bookmakers so look out for that next week!

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Chelsea	Everton	52	28	20
Liverpool	QPR	71	18	11
Man City	Norwich	77	14	9
Newcastle	Arsenal	23	32	45
Southampton	Stoke	42	31	27
Swansea	Fulham	46	30	24
Tottenham	Sunderland	67	21	13
West Brom	Man United	13	30	57
West Ham	Reading	49	29	23
Wigan	Aston Villa	43	31	27

EI Match Probabilities for the English Premier League

2013-05-10T19:30:00+00:00

Here are the latest match probabilities for the English Premier League calculated using the Eastwood Index (EI).

Somewhat surprisingly, Liverpool are only just favorites to beat Fulham with the odds so close that a draw would seem the likely outcome.

Down at the bottom of the table Newcastle versus QPR and Norwich versus West Brom look likely to finish tied, while Sunderland are slight favorites against Southampton meaning Wigan desperately need to take points off Arsenal to stand any chance of avoiding relegation.

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Aston Villa	Chelsea	21	32	47
Stoke	Tottenham	24	32	44
Everton	West Ham	67	20	12
Fulham	Liverpool	31	33	36
Norwich	West Brom	37	32	31
QPR	Newcastle	32	32	35
Sunderland	Southampton	49	29	22
Man United	Swansea	76	15	9
Arsenal	Wigan	71	18	11
Reading	Man City	8	28	64

MLS Player Salaries: 2013

2013-05-10T19:30:00+00:00

Introduction

The latest Major League Soccer (MLS) salaries were released recently by the MLS Players’ Union so I thought I would post a quick summary of the data.

Average Salary By Team

The first thing I was interested in was average salaries per team and whether there had been any changes compared with previous seasons (Figure 1).

The trend over the past few years has been pretty constant, with LA Galaxy and New York Red Bulls having the highest outgoings on wages, which again continues for 2013.

Toronto have typically followed in a distant third place but this season sees them overtaken by Seattle following the addition of Obafemi Martins to their roster.

click the legend headers to show / hide each season’s data and hover the data points for more information

Number of Players

Next I looked at the number of players currently playing in each position. The results are pretty much the same as 2012, with a marginal gain in the number of forwards and defenders registered for this season (Figure 2).

click the legend headers to show / hide each season’s data and hover the data points for more information

Average Salary By Position

Next I looked at average salary by position and it is probably no surprise that forwards receive the most remuneration (Figure 3). In fact, the higher up the field you are, the more money you earn, with goalkeepers earning the least followed by defenders, midfielders, attacking midfielders and then forwards.

The only player outside of this trend are defensive midfielders who earn even less than goalkeepers. In terms of salary this appears to be the least appreciated position by quite a large margin. If you are out to make money then you are much better off specializing as either a clear-cut defender or midfielder rather than something perhaps between the two. Or even better, learn to score goals…

click the legend headers to show / hide each season’s data and hover the data points for more information

The Big Earners

Although the average salaries show that forwards earn noticeably more than any other position, the actual value is skewed by a few high-profile players earning big bucks. Table 1 shows the top ten earners in the MLS compared with the overall league average. Of the ten players, eight are forwards and two are midfielders. The highest paid defender is Toronto’s Darren O’Dea, ranked 18th overall and the highest paid goalkeeper is Portland’s Donovan Ricketts, ranked just 41st overall.

Club	Last Name	First Name	Pos	Base Salary	Compensation
NY	Henry	Thierry	F	$3,750,000	$4,350,000
LA	Keane	Robbie	F	$4,000,000	$4,333,333
NY	Cahill	Tim	M	$3,500,000	$3,625,000
LA	Donovan	Landon	F	$2,500,000	$2,500,000
MTL	Di Vaio	Marco	F	$1,000,008	$1,937,508
SEA	Martins	Obafemi	F	$1,600,000	$1,725,000
TOR	Koevermans	Danny	F	$1,250,000	$1,663,323
VAN	Miller	Kenny	F	$1,114,992	$1,132,492
SEA	Montero	Fredy	F	$700,000	$856,000
DAL	Ferreira	David	M-F	$625,000	$730,000
			League Average	$141,903	$159,849

Since these star players are skewing the averages, we can analyse the median salary instead using box and whiskers plots (Figure 4). These show the distribution of the different salaries for each position where the thick line across the center of the box is the median salary, the top and bottom of the box are the 75th and 25th percentiles and the whiskers represent 1.5x the interquartile range. Outliers outside of this range are then plotted as dots.

Looking at the median salaries there is actually very little difference between the outfield players. The average MLS player’s salary is also clearly nothing like the league’s star players, in fact if we remove the top twenty earners then the overall league average falls from $159,849 to $113,516 with a median of $83,000 and a mode of $46,500, which is the league minimum for first teamers (roster positions 1-24).

Conclusions

This is only a quick overview of the data and there is still a lot more to explore so feel free to get in touch if there is anything in particular you want to have a look at.

The Eastwood Index, MLS and Parity

2013-05-07T19:30:00+00:00

Introduction

I showed in my last post how Major League Soccer (MLS) is a much more closely matched league than the English Premier League (EPL), with the wage cap and draft system increasing the parity between teams.

The Eastwood Index

This high level of parity can also be seen using the Eastwood Index (EI), a rating system designed to calculate odds of match outcomes when different teams play each other.

The Eastwood Index rates teams so that the average rating is 2000 and the higher the rating the better a team is compared with the rest of the league.

EI ratings increase when teams win matches or draw against superior opposition and decrease when teams lose matches or draw against weaker opposition. The size of the gain or loss in ratings is linked to the quality of the opposition so that beating a superior team is worth more than winning against a lower ranked team.

The change in EI rating is also weighted by the goal difference in the match so that the greater the difference in goals scored or conceded then the greater the change in ratings. Home advantage is also included in the calculations so that the home team is expected to perform better when playing at home compared with away.

Major League Soccer EI Ratings

Currently, the highest rated team in MLS is LA Galaxy, with an EI of 2506 (Table 1) while the lowest is Toronto FC, with an EI of just 1303 (Table 2). Outside of this, the majority of teams are fairly evenly matched in MLS and are rated around 1880 – 2300 demonstrating the parity in the league.

Position	Club	EI Rating
1	New York Red Bulls	2225
2	Sporting Kansas City	2374
3	Houston Dynamo	2210
4	Montreal Impact	2052
5	Columbus Crew	2082
6	Philadelphia Union	1809
7	New England Revolution	1610
8	Chicago Fire	2063
9	Toronto FC	1303

Table 1: MLS Eastern Conference EI Ratings

Position	Club	EI Rating
1	FC Dallas	2150
2	LA Galaxy	2506
3	Real Salt Lake	2271
4	Portland Timbers	1804
5	Colorado Rapids	1871
6	Chivas USA	1433
7	San Jose Earthquakes	2267
8	Vancouver Whitecaps	1690
9	Seattle Sounders FC	2392

Table 2: MLS Western Conference EI Ratings

It is still a bit early in the season to draw too many conclusions but if we combine the two MLS conferences together then Columbus currently come out as mid-table, with an EI rating of 2082. This is just 424 lower than the top team (LA Galaxy) and 569 higher than the bottom of the league (Toronto FC), and close to theoretical league average EI of 2000.

MLS Compared with EPL

Compare this with the EPL (Table 3) and you can see an immediate difference in the level of parity. Taking the average of West Ham and Stoke to be the middle of the table then a mid placed EPL team’s EI is below the theoretical average at 1634, just 264 better than QPR at the bottom of the table and a gigantic 1431 away from Manchester United. The top of the EPL has been very much a league-within-a-league for a while now, with average teams vastly more likely to be relegated than they are of ever winning anything or even reaching the European qualification spots.

Position	Club	EI Rating
1	Manchester United	3064
2	Manchester City	2909
3	Chelsea	2598
4	Arsenal	2627
5	Tottenham Hotspur	2514
6	Everton	2351
7	Liverpool	2291
8	West Bromwich Albion	1883
9	Swansea City	1797
10	West Ham United	1520
11	Stoke City	1747
12	Fulham	1806
13	Aston Villa	1704
14	Southampton	1611
15	Sunderland	1741
16	Norwich City	1607
17	Newcastle United	1814
18	Wigan Athletic	1650
19	Reading	1397
20	Queens Park Rangers	1369

Table 3: EPL EI Ratings

Parity

Compared with the EPL, MLS is a very evenly matched league where the margins between the top and bottom of the conferences are small, making it a really exciting league to follow as virtually any team is in with a chance of reaching the playoffs at the start of the season.

EI Match Probabilities for the English Premier League

2013-05-03T19:30:00+00:00

Here are the latest match probabilities for the English Premier League calculated using the Eastwood Index (EI).

Please note that I have not included next week’s mid-week matches yet as the odds for those will change depending on how this weekend’s matches finish. I’ll try and add those on Tuesday once I have all the data available.

Edit – Table 1 is now updated with the odds for this week’s mid week matches.

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Fulham	Reading	59	25	16
Norwich	Aston Villa	44	30	26
Swansea	Man City	15	31	54
Tottenham	Southampton	68	20	12
West Brom	Wigan	54	27	19
West Ham	Newcastle	37	32	32
QPR	Arsenal	13	30	57
Liverpool	Everton	44	30	26
Man United	Chelsea	60	24	16
Sunderland	Stoke	45	30	25
Man City	West Brom	72	18	10
Wigan	Swansea	41	31	28
Chelsea	Tottenham	47	30	23

How Much Does Luck Affect MLS?

2013-05-02T19:30:00+00:00

Introduction

Following my recent article for Betting Expert quantifying how large a role luck plays in the English Premier League (EPL) I thought it would be interesting to look at Major League Soccer (MLS) too.

MLS Structure

MLS is structured differently to the EPL as it has followed other North American sports in implementing wage caps and player drafts. Unlike the current salary free-for-all in the EPL, MLS clubs are currently limited to spending a maximum of $2.95 million in wages over their first 20 roster spots, with up to three additional designated players paid (partially) outside of this salary cap.

MLS also has a draft system that takes place each January during which teams can sign players graduating from college or otherwise signed by the league. The draft is split into three rounds and is designed to give priority to the league’s weaker teams allowing them first choice of players ahead of the more successful teams.

MLS is also a shorter season than the EPL with teams playing just 34 matches compared with the EPL’s 38. This is important as the more matches that are played then the more opportunity talent has to overcome luck.

Overall, this all works towards increasing parity in the MLS and making it a more evenly balanced league, which in turn should enhance the role luck plays.

Results

Using my adaptation of Tom ‘Tango’ Tiger’s baseball equation I calculated the average win rate in MLS going back to 2004 and the variance of the win rate. I them calculated the variance expected due to luck and subtracted one from the other to get the amount of variance attributed to talent.

Luck accounts for around 35% of a team’s win rate in the EPL and I was expecting MLS to be higher, but it initially came out at a staggering 82% for MLS. Instinctively this seems too high and I suspect it is inaccurate due to the changes in MLS’s structure over the years. For example, back in 2004 there were only ten teams and one conference while there are currently 19 teams and two conferences. There have also been changes to the level of the salary cap and the number of designated players allowed over this time period too.

So I went back and reprocessed the results using just the 2010–2012 data. Although this reduces the sample size considerably it leaves us with data more representative of the current state of MLS. And the results this time? Luck accounted for around 57% of a team’s win percentage compared with just 43% for talent.

So compared with the EPL, the structure of MLS does appear to increase parity and enhance the influence luck has in deciding the league champions. In fact, being lucky is probably the more important of the two, although luck on its own is not enough – you need to be a talented team with luck on its side to win the MLS Cup.

How Much Does Luck Affect Football?

2013-04-30T19:30:00+00:00

Introduction

I’ve written a new article for Betting Expert quantifying how much luck affects football. Take a look here as it is probably more than you are expecting!

What Is A Meaningful Sample Size?

2013-04-28T19:30:00+00:00

Introduction

I had an article published at Betting Expert last week looking at how to determine statistically how much data you need to make accurate predictions.

Find out more by reading the rest of this article here.

EI Match Probabilities for the English Premier League

2013-04-26T19:30:00+00:00

It’s Friday again so here are this weekend’s match probabilities for the English Premier League.

I was a little surprised to see Manchester United come out as favourites against Arsenal, even they they are away from home. but the odds are so close though that it looks like a potential draw. However, it all depends on Sir Alex Ferguson’s squad selection, with the league won will he rest the bigger stars and let some of the second-string players reach enough appearances to be eligible for a winners medal? Anders Lindegaard, for example, still needs to play another two matches this season to claim his medal.

Other possible draws include Southampton Vs West Brom and Newcastle Vs Liverpool, while you’d hope Reading Vs QPR will not be a draw as a single point is useless for either team.

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Man City	West Ham	78	13	9
Everton	Fulham	58	25	17
Southampton	West Brom	39	31	29
Stoke	Norwich	47	29	24
Wigan	Tottenham	20	32	48
Newcastle	Liverpool	35	32	33
Reading	QPR	44	30	26
Chelsea	Swansea	65	22	13
Arsenal	Man United	31	32	37
Aston Villa	Sunderland	40	31	29

EI Match Probabilities for the English Premier League

2013-04-19T19:30:00+00:00

It’s been a busy day but I’ve finally got the probabilities for this weekend’s matches completed.

There are some pretty close games, with QPR Vs Stoke, Tottenham Vs Man City and Liverpool Vs Chelsea all looking like potential draws. Plus you could maybe throw Sunderland Vs Everton and even West Ham Vs Wigan into that group too.

The only clear favourties are Manchester United and Norwich so it’s going to be a tricky week to call.

Home Team	Away Team	Home(%)	Draw (%)	Away (%)
Fulham	Arsenal	26	32	42
Norwich	Reading	53	27	20
QPR	Stoke	37	32	31
Sunderland	Everton	28	33	39
Swansea	Southampton	49	29	22
West Brom	Newcastle	45	30	25
West Ham	Wigan	41	31	28
Tottenham	Man City	31	33	36
Liverpool	Chelsea	36	32	32
Man United	Aston Villa	79	12	9

EI Match Predictions for the English Premier League

2013-04-12T19:30:00+00:00

Here we go with this week’s predictions from the Eastwood Index (EI)!

The EI doesn’t hold out much chance for Wigan or West Ham getting three points off the two Manchester clubs this week, with the lowest odds I think the EI has ever produced. Interestingly, both of Fulham’s matches look like possible draws.

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Arsenal	Norwich	70	19	11
Aston Villa	Fulham	35	32	33
Everton	QPR	70	18	11
Reading	Liverpool	22	32	46
Southampton	West Ham	50	29	21
Newcastle	Sunderland	48	29	23
Stoke	Man United	10	29	61
Arsenal	Everton	51	28	21
Man City	Wigan	77	14	9
West Ham	Man United	7	27	66
Fulham	Chelsea	32	32	36

EI Match Predictions for the English Premier League

2013-04-05T19:30:00+00:00

A couple of weeks back I demonstrated how the EI is more accurate than the bookies based on rank probability scores but a few people have asked if I can do something a bit simpler so Figure 1 shows how often the EI picked the winner as being the favourite compared with aggregated bookmaker’s odds. It’s pretty close but the EI seems to have a small but reasonably constant margin over the bookmaker so far this season.

Figure 1: EI Versus Bookmakers

Last week turned out to be a pretty good week with the EI managing to correctly predict the winner in eight out of the ten matches played. I’d made a few minor tweaks before posting last week’s odds to try and enhance the the way draws and away wins are calculated so hopefully the EI will be able to maintain its edge over the bookmakers.

Anyway, here are this week’s odds:

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Reading	Southampton	38	32	30
Norwich	Swansea	41	31	28
Stoke	Aston Villa	50	29	21
West Brom	Arsenal	26	32	42
Liverpool	West Ham	66	22	12
Tottenham	Everton	49	29	22
Chelsea	Sunderland	66	21	13
Newcastle	Fulham	44	30	26
QPR	Wigan	39	31	30
Man United	Man City	51	28	21

Understanding Total Shot Ratio in Football

2013-04-02T19:30:00+00:00

Introduction

The use of Total Shot Ratio (or TSR) seems to have slowly been gaining ground so I thought it would be worth analyzing the statistic in more detail to see what it can and cannot do.

What is Total Shot Ratio?

Put simply Total Shot Ratio is the proportion of shots taken by one team compared with another. It can be calculated by dividing the number of shots taken by a team by the total shots overall (Figure 1).

$TSR=ShotsFor/(ShotsFor+ShotsAway)$

Figure 1: Total Shot Ratio

It is often used as a surrogate for dominance as the presumption is that the team taking the majority of the shots will be controlling the match and possibly limiting the opposition’s ability to shoot at goal.

Total Shot Ratio Data

Using data taken from the football-data.co.uk website I calculated the Total Shot Ratios for all matches from the English Premier League going back to the 2001-2002 season, giving a total of 8360 data points, which are normally distributed (Figure 2).

Figure 2: Distribution of Total Shot Ratios

The average Total Shot Ratio is always 0.5, because for every value above 0.5 you always an equivalent value below it for the opposition. For example, if the home team’s Total Shot Ratio is 0.75 then the away team’s ratio must be 0.25.

$(0.75 + 0.25) / 2 = 0.5$

The standard deviation, which is a measure of the dispersion of the data around the average value, was 0.166.

Total Shot Ratio: Correlation With Goals Scored

Since Total Shot Ratios are being used to show dominance in a match it makes sense to assess the correlation with the number of goals scored. The higher a team’s Total Shot Ratio is then the more shots it is having compared with the opposition so the expectation would be that they would score more goals. However, this does not seem to be the case as the relationship between the two is extremely weak (Figure 3; r2=0.079).

Figure 3: Correlation Between Total Shot Ratio and Goals Scored

Total Shot Ratio: Correlation With Goal Difference

So how about the relationship between Total Shot Ratio and goal difference instead? Since teams with higher Total Shot Ratios are thought to be dominating matches, perhaps they are more likely to have a higher goal difference in the match as they may also be less likely to concede goals? Again though the correlation is weak (Figure 4, r2=0.11).

Figure 4: Correlation Between Total Shot Ratio and Goal Difference

Total Shot Ratio: Correlation With Match Outcomes

Additionally, the relationship between Total Shot Ratio and match outcome is also poor (r2=0.066) suggesting that Total Shot Ratio also has very little influence on the likelihood of a team winning a particular match. Just because you are taking a greater proportion of the shots does not mean you are any more likely to win.

Total Shot Ratio: Correlation With Points Per Season

Although the match-by-match correlations above are weak there is the suggestion of a trend so it may be that Total Shot Ratio is heavily luck driven in the short term and that we need more matches before we can see the overall effects of a higher ratio. For example, looking at the correlation between Total Shot Ratio and points over an entire season shows a pretty decent relationship between the two (Figure 5). This suggests that long term possessing a higher Total Shot Ratio is in fact associated with fewer matches being lost per season.

Figure 5: Correlation Between Total Shot Ratio and Points

Total Shot Ratio: How Much Data is Enough?

So if Total Shot Ratio is only becoming meaningful over longer periods of time then how much data do we actually need before it becomes a useful metric? To look at this I calculated the overall Total Shot Ratio per season by team and then randomized the order of each match that season. I then looked at how the deviation changed over course of a season compared with the overall ratio, e.g. after five matches, ten matches etc (Figure 6).

Figure 6: Deviation in Total Shot Ratio by Sample Size

As more data is used to calculate the Total Shot Ratio it moves closer towards its true value and the deviation decreases as the effect of any outlier matches becomes less influential. With fewer matches being used to calculate the Total Shot Ratio there is more dispersion and variability in the calculated value. Interestingly, there is still a reduction in the deviation moving from 30 matches to 38 matches, suggesting that we may need at least a full season’s worth of data to get an accurate measure of a team’s Total Shot Ratio.

Total Shot Ratio: Calculated Sample Size

Another option to find out how much data we need is to calculate the sample size required to identify specific differences in Total Shot Ratio. There are a number of different methods for this but the commonly used t-test sample size estimation suggests that to be 95% certain that two teams with a difference in Total Shot Ratio of 0.1 are actually different from each other takes 45 matches.

So, to be statistically certain that a team with a Total Shot Ratio of 0.6 actually has a higher ratio than a team with a Total Shot Ratio of 0.5 rather than it just being down to random variability requires over a season’s worth of matches to be played.

As the differences become smaller then the number of matches required increases even further – to identify a difference in Total Shot Ratio of 0.05 takes nearly five season’s worth of matches!

Conclusions

In the short term, Total Shot Ratio appears to virtually meaningless in terms of goals scored or match outcomes as its variability is so high.

Over the long term though, skill outweighs luck and Total Shot Ratio becomes increasingly correlated with outcomes. However, it may take a long time for this to occur and may be less accurate than other statistics available if you are interested in predicting performance.

Finally, this article is not intended to say “do not to use Total Shot Ratio” as it is still an interesting metric. Rather, make sure that you are aware of its abilities and limitations if you are planning on using it for analysis.

EI Match Predictions for the English Premier League

2013-03-28T19:30:00+00:00

After last week’s international matches, domestic football is finally back so here are this weekend’s match predictions using my EI predictive model. Let’s see if it can keep up its good form and continue to beat the bookmakers!

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Sunderland	Man Utd	10	29	61
Arsenal	Reading	74	16	10
Man City	Newcastle	70	18	11
Southampton	Chelsea	20	32	48
Swansea	Tottenham	27	32	40
West Ham	West Brom	30	32	37
Wigan	Norwich	43	30	26
Everton	Stoke City	60	24	16
Aston Villa	Liverpool	28	30	42
Fulham	QPR	59	25	16

How Accurate Are The EI Football Predictions?

2013-03-21T19:30:00+00:00

Introduction

Unfortunately time caught up with me last week and I was unable to post any predictions from my Eastwood Index. However, since then I have been busy validating the results to see how accurate the predictions really are using the 296 matches played in the English Premier League so far this season.

Ranked Probability Scores

I have previously discussed the problems of trying to determine the accuracy of probability-based models and Jonas posted a suggestion in the comments section recommending the use of ranked probability scores, which turned out to be a really interesting idea.

Ranked probability scores were originally proposed by Epstein back in 1969 as a way to compare probabilistic forecasts against categorical data. Their main advantage over other techniques is that as well as looking at accuracy, they also account for distance in the predictions e.g. how far out inaccurate predictions are from what actually happened.

They are also easy to calculate. The equation for ranked probability scores is shown in Figure 1 for those of a mathematical disposition, where $K$ is the number of possible outcomes, and $CDF_{fc}$ and $CDF_{obs}$ are the predictions and observations for prediction $k$.

Interpreting Ranked Probability Scores

Ranked probability scores range between 0–1 and are negatively orientated meaning that the lower the result the better. For simplicity, you can think of them representing the amount of error in the predictions where a score of zero means your predictions are perfect.

The Results

I started off looking at how well I would have done if I had just guessed at random for each match in the English Premier League this season rather than using the Eastwood Index and obtained a ranked probability score of 0.231.

Next, I looked at how well the bookmaker’s odds predicted matches. To do this I aggregated the odds from multiple bookmakers, partly to reduce the comparisons needed and partly because aggregating data often improve predictions and I wanted to give the Eastwood Index the toughest test possible. This gave a ranked probability score of 0.193 for the bookmakers.

Finally I calculated the score for the Eastwood Index and got…

drum roll please

a ranked probability score of 0.191. Okay, it is not much lower than the bookmakers but it does mean that so far this season the Eastwood Index has been more accurate at predicting football matches than the combined odds of the gaming industry which is really pleasing for me.

Conclusions

Most importantly though this suggests that the Eastwood Index works. I had originally set myself the target of being able to compete with the bookmakers as I consider them to be gold standard prediction for football. These are large companies employing professional odds compilers to generate their odds so for me to be able to beat them, even by a small amount, using a bunch of equations is a big success for the Eastwood Index.

It is still early days and it is still a relatively small number of predictions (n=296) so I will be continuing to monitor the results to check the accuracy doesn’t change over time. It is a fantastic start though and great inspiration to continue developing and improving the Eastwood Index further!

Is Brendan Rogers Improving Liverpool?

2013-03-13T19:30:00+00:00

Introduction

As well as using my EI Index to predict future matches, it can also be used to look back at how team’s performances have changed over time. An interesting example is Liverpool, who sacked Kenny Dalglish at the end of the 2011–2012 season to bring in Brendan Rogers from Swansea City.

The green line in Figure One shows the weekly EI rating for Kenny Dalglish’s Liverpool team over the course of the 2011–2012 season, with the black line showing the moving three-match average. Up until around Christmas time Liverpool were making decent progress in terms of EI, improving from a rating of 2127 up a peak of 2247 following their 3-1 victory against Newcastle United.

However, Liverpool’s form plummeted soon after that, with 11 losses out of their remaining 19 matches dragging Liverpool’s EI back down rapidly. Their worst performances in terms of EI were losses against Bolton Wanderers and Wigan Athletic, both of which Liverpool’s EI ratings suggested they should have had a good chance of winning. Despite a small flurry at the end of the season, Liverpool still finished with an EI lower than they started with.

In contrast, the red line in Figure One shows Liverpool’s weekly EI ratings under Brendan Rogers, with the black line again showing the moving three-match average.

The first few matches of the season did not go particularly well for Rogers and Liverpool’s EI dropped even lower than under Dalglish. The obvious narrative here is that Liverpool may have needed time to adjust to Rogers tactical changes but it’s also worth noting they had a tough start to the season, with fixtures against Manchester City, Arsenal and Manchester United all within the opening few weeks.

Since then, Liverpool has shown a pretty steady increase in EI over the rest of the season. There have been a few drops along the way due to unexpected losses against teams such as Aston Villa and Stoke but their EI rating is currently on course to exceed Dalglish’s peak by the end of the season.

To put these numbers into context, Chelsea are currently in fourth position with an EI of 2464 while Tottenham Hotspur finished fourth last season with an EI of 2329. While Liverpool’s EI isn’t quite that high yet, if they can maintain their current rate of improvement then their EI rating suggests they have a decent chance of challenging for a Champion’s League place next season.

addendum

In case anyone wonders why Brendan Rogers’ starting EI is lower than Kenny Dalglish’s final EI – Brendan Rogers lost his first match against West Bromwich Albion so the difference between the two is the loss in EI caused by that particular match.

EI Match Predictions for the English Premier League

2013-03-08T19:30:00+00:00

Here we go again!

Last Week was another success for the EI, with seven out of the ten predicted favourites winning their matches. It is still a bit early to be drawing too many conclusions but so far that is 14 out of 19 for the EI, which seems a pretty good start to me!

Jonas commented on my recent post discussing the difficulties of assessing probability-based models to suggest trying Ranked Probability Scores which looks like a really good idea so look out for that once I have a bit more data to play with.

It is only a small gameweek this week due to the FA cup but here are the predictions anyway.

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Norwich	Southampton	48	24	28
QPR	Sunderland	36	27	37
Reading	Aston Villa	42	26	32
West Brom	Swansea	46	25	29
Newcastle	Stoke	49	24	27
Liverpool	Tottenham	38	27	35

EI Match Predictions for the English Premier League

2013-03-01T19:30:00+00:00

Since last week’s predictions turned out to be so popular I thought I would continue testing the EI index in public so here are this week’s predictions. Fingers crossed they turn out well again!

Home Team	Away Team	Home (%)	Draw (%)	Away (%)
Chelsea	West Brom	60	20	21
Everton	Reading	64	17	19
Man United	Norwich	76	10	14
Southampton	QPR	49	24	27
Stoke	West Ham	56	21	23
Sunderland	Fulham	42	26	32
Swansea	Newcastle	42	26	32
Wigan	Liverpool	31	27	42
Tottenham	Arsenal	44	25	31
Aston Villa	Man City	14	25	61

EI Match Predictions for the English Premier League

2013-02-28T19:30:00+00:00

The EI

Last week was a big test for my new EI Index – it had finally reached the point where I was confident it was working well enough to post its predictions in public.

For those people who haven’t come across it before, the EI index is a mathematical system I have been developing for ranking football teams and predicting the outcomes of matches. Using the EI it is possible to predict the odds for each team winning, drawing or losing its match.

Results

So how did the EI do?

Well, this is the tricky bit as the EI is a probability model. For linear models it is relatively simple to assess accuracy as you get an R-squared value showing you how well your predictions match the observed result. The higher the R-squared then the better you did.

For a probability-based model though you cannot do this. An obvious alternative is to just look at whether the model’s predicted favourites won their matches. And on this measure the EI performed fantastically well by correctly matching seven of its nine predictions, giving it an overall success rate of (78%).

But we need to be careful here as this can be a misleading way of looking at accuracy. Just because Manchester City had a 54% probability of beating Chelsea doesn’t mean they will win the match purely because it is the most probable result. Instead, it means that if this match was played 100 times then Manchester City would be expected to win 54 of them and not win 46 of them.

Rather than looking at the accuracy of the predicted favourite winning we really need to look at the accuracy of the predicted probabilities. Are teams ranked with a 50% chance of winning actually winning 50% of the time? Are teams ranked with a 25% chance of winning actually winning 25% of the time?

This can only be done by making lots and lots of predictions so over the coming weeks I will keep making them until I have enough of predictions to get an estimate of how good they are.

Overall though it is a very exciting start for the EI, bring on next week!

EI Match Predictions for the English Premier League

2013-02-21T19:30:00+00:00

For a bit of fun, here is a trial run at predicting this weekend’s EPL matches using my EI ratings. I haven’t compared these with anyone else’s odds yet but they generally look about what I would have expected.

Poor QPR don’t seem to have much chance holding out against Manchester United, even playing at home they are only rated at having a 9% chance of winning.

It looks like it could be a good weekend for Arsenal to bounce back from their Champions League defeat as they have a massive 67% chance of beating Aston Villa.

Personally, I am surprised Newcastle are rated quite so highly against Southampton. I wonder if this may be due to Newcastle being so strong last season while Southampton only have this season’s data for generating EI ratings from? If so, it may be that I need to go back and tweak the equation weightings slightly to account for situations like this.

Match	Home (%)	Draw (%)	Away (%)
Fulham Vs Stoke	46	25	29
Arsenal Vs Aston Villa	67	15	17
Norwich Vs Everton	29	27	43
QPR Vs Man United	9	23	68
Reading Vs Wigan	43	25	31
West Brom Vs Sunderland	47	24	28
Man City Vs Chelsea	54	22	24
Newcastle Vs Southampton	53	22	25
West Ham Vs Tottenham	19	27	54

Introducing the Eastwood Index

2013-02-21T19:30:00+00:00

Introduction

My past couple of articles have focused on Elo ratings and how they can be applied to football teams to rank them against each other and to estimate win probabilities.

Problems with Elo Ratings

On the whole the Elo system works okay but it was not designed with football in mind and so there are some issues with it, for example it can only handle two distinct outcomes – winning and losing.

Elo ratings try to get around this problem by considering each draw to be half a win and half a loss. However, this means that the win probabilities calculated using the Elo equation are actually the probability of winning or drawing versus the probability of losing or drawing, which isn’t particularly useful.

For a game such as chess, which Elo ratings were originally developed for, this may not be too much of an issue as tied matches are comparatively rare but in football draws are a common occurrence so we really need to be able to model three outcomes – win, loss and draw.

The Eastwood Index

So instead of combining draws with wins and losses, we need to be able to calculate their probabilities individually. To do this I have been developing my own ranking system, which for want of a better name I am currently calling the Eastwood Index, or EI for short (it feels rather pretentious to be naming it after myself so if anyone has any better names for it then feel free to let me know!)

The Eastwood Index allows football teams to be ranked using a mathematical rating system that evaluates relative strength based on previous performances weighted so that more recent matches have a greater impact on a team’s ranking.

Methodology

Teams EI ratings are scaled so that the average rating is 2000. The higher the rating the better a team is compared with the rest of the league.

EI ratings increase when a team wins a match or draws against superior opposition. Conversely, EI ratings decrease when teams lose matches or draw against weaker opposition. The size of this increase or decrease in ratings is linked to the quality of the opposition. For example, beating a superior team is worth more than winning against a lower ranked team.

The change in EI rating is also weighted by the score line so that the greater the difference in goals scored or conceded then the greater the change in ratings. Home advantage is also included in the calculations so that the home team is considered to perform better at home compared with away.

So far this all sounds similar to an Elo rating. However, the Eastwood Index has a major advantage over Elo in that it is multinomial, meaning it can function with multiple outcomes. This makes it possible to accurately calculate the probabilities of teams winning, drawing or losing matches.

A further advantage of the Eastwood Index is that it is does not rely on the Logistic distribution the same way Elo ratings do. The use of the Logistic distribution in Elo ratings originates from chess where it was considered to predict chess outcomes reasonably well. Football and chess are different games with different outcomes so instead the Eastwood Index uses custom curves developed using football data. This means that predictions for football should be more accurate using the EI compared with Elo ratings.

Example

The underlying mathematics for the EI is completely different to how an Elo rating is calculated but rather than wade through a list of equations it is simpler to show how it works using the recent Liverpool versus Swansea match.

Prior to the game Liverpool had an EI rating of 2151 compared with Swansea’s rating of 1891. Team performances are considered to be normally distributed around their rating so on any given day a team may play above or below their true skill level. By comparing the distribution curves for the two teams we can then calculate the probabilities of each outcome of the match before it is played.

Although both teams have similar ratings Liverpool has the home advantage giving them overall a 52% chance of a win compared with a 25% chance of Swansea winning and a 23% chance of a draw (Figure 1).

Figure 1: Predicting Liverpool Versus Swansea City

We can also use these probabilities to calculate the expected points from the match. If these two teams were to play the same match repeatedly then on average Liverpool would be expected to earn (0.52 * 3) + (0.23 * 1) = 1.79 points while Swansea would be expected to earn (0.25 * 3) + (0.23 * 1) = 0.98 points.

Once we know the actual result we can then update the EI for each team based on their current ratings and the score line, which was Liverpool 5 – 0 Swansea. Since Liverpool already had a higher EI rating and had beaten somewhat lesser opposition they would expect only a small rise in their EI but taking into account their high score in the match Liverpool’s rating moves up to 2183 while Swansea’s falls to 1859.

Conclusions

The EI Index offers a potentially superior way of rating football teams compared with other ranking systems, with the advantage that it can predict wins, losses and draws, and uses mathematics specifically designed to accurately model football data.

I will be discussing the EI in more detail in future posts and showing how it can be used to analyse and predict football matches.

As ever, get in touch if you have any comments of questions!

Understanding Elo Ratings Part Two

2013-02-07T19:30:00+00:00

Introduction

Now that we understand the theory behind Elo Ratings, let’s take a look at how to calculate them and how to make them more relevant to football.

Calculating Elo Ratings

The equation for calculating a team’s Elo rating is shown below in Figure 1, where $Ra_{new}$ is the team’s new Elo rating after a match, $Ra_{old}$ is the team’s previous Elo rating before the match and $k$ is a weighting factor. $Sa$ is the outcome of the match normalised to the range 0–1 so that 0 is a loss, 0.5 is a draw and 1 is a win.

$Ra_{new}=Ra_{old}+k(Sa-Ea)$

Figure 1: Elo Rating equation

$Ea$ is the expected probability of the team winning the match and is calculated using the equation in Figure 2 where $Rb-Ra$ is the difference in Elo ratings between the two teams.

$Ea=1/1+10^{(Rb-Ra)/400}$

Figure 2: Expected win probability equation

Win Expectancy

The calculation for $Ea$ is actually slightly different from the original Elo equation as it uses a logistic distribution for player performances rather than a normal distribution. The use of the logistic distribution stems the chess community, who suggested that it fit player performances better than the normal distribution did. In effect, the differences between the two are relatively minor, with the logistic curve skewing more performances to the tails of the distribution, meaning players are slightly more likely to over- or under-perform (Figure 3).

Figure 3: Comparison of logistic and normal distributions

Weighting Factor

The constant $k$ in the equation controls how many points are gained or lost each match. Increasing k will apply more weight to recent matches while lowering it will allow historic matches to have more of an effect on a team’s Elo rating. Therefore, using an inappropriate rating for $k$ may lead to inaccurate Elo ratings being calculated.

Eloratings.net is a website that applies Elo ratings to international football. They use a weighting of 60 for a world cup final, 50 for continental championship finals and major intercontinental tournaments, 40 for World Cup and continental qualifiers, 30 for all other tournament matches and 20 for international friendly matches. However, since none of these ratings apply directly to domestic football and since Eloratings.net does not explain how they were determined I decided to calculate my own.

Using Least Squares I optimized the value of $k$ to minimize the error of the predicted outcomes versus the actual match results using data from the English Premier League. Overall, the most accurate predictions were obtained using a value of 15 for $k$.

Figure 4: Effect of k on error of Elo prediction

Goal Difference

Another modification we can do to make the Elo ratings more applicable to football is to take into account the number of goals scored so that beating the opposition by two goals for example is better than wining by just one.

We can do this by scaling $k$ by the goal difference so that the larger the difference the more points are gained by the victor and the more lost by the loser. There are a number of ways this can be done but in my method each additional goal a team scores becomes increasingly less important. For example, going from 1–0 to 2–0 is much more critical in terms of winning a game than going from 8–0 to 9–0.

Eloratings.net used a similar approach where their scaling reduces the weightings for goal differences of two and three. However, for goal differences of four upwards their scale (intentionally or unintentionally) becomes linear and from then on applies equal weightings to each additional goal scored. Instead, I have used a sigmoid function to smoothly reduce the weightings of each goal scored to create the curve shown in Figure 5, which is then used to produce the scaling factors shown in Table 1

Figure 5: Goal difference scaling factor smoothed using a sigmoid

Goal Difference	Scaling Factor
10.00	2.99
9.00	2.88
8.00	2.77
7.00	2.64
6.00	2.49
5.00	2.32
4.00	2.11
3.00	1.85
2.00	1.51
1.00	1.00

Table 1: Goal difference scaling factors

Home Advantage

If two teams with equal Elo ratings play each other then in theory they should both have an identical chance of winning the match; however, in football the home team always has a noticeable advantage.

Looking back at the 2011–2012 English Premier League season, home wins accounted for 47% of results compared with just 24% for away wins. The remainder of the results are draws, which Elo ratings consider to be half a win, so including these gives us a final win expectancy of 61% for the home team and 39% for the away team.

To account for this we can give the home team’s Elo a temporary boost of 75 points. For two equally matched teams this then raises the win expectancy for the home team from 50% to 61%, matching what we see in the English Premier League.

Relegations and Promotions

Another issue to consider is how to deal with relegations and promotions. We could calculate Elo ratings for each tier of the league so that a team already has a rating when it gets promoted or alternatively we could award each promoted team the average Elo rating of 1500. A nice feature of Elo ratings is that they are self-correcting so although these arbitrary ratings may not be accurate they would gradually alter to the correct level.

This does have the unfortunate side effect of skewing the other team’s Elo values though. The gain and loss of Elo points is zero sum, meaning that for every Elo point a team gains another team has to lose one. So adding in teams with different Elo ratings would distort the values of other the team’s ratings by altering the overall number of points available in the league.

The simplest way to deal with this problem is to give the promoted teams the equivalent relegated team’s Elo rating. So the best promoted team takes the Elo rating of the best relegated team, the second best promoted team takes the Elo rating of the second best relegated team, and so on. This then keeps the correct number of Elo points in the league and maintains the parity in points between teams.

Conclusions

Elo ratings are a really quick and easy way to compare teams directly and calculate win expectancies. While techniques like the Pythagorean Expectation looks at how teams perform over a long period of time, Elo ratings can be used to look at teams on a match–by–match basis.

Understanding Elo Ratings

2013-01-31T19:30:00+00:00

What are Elo Ratings?

The Elo rating system was originally devised by its creator Arphad Elo as a way to calculate the average skill levels of two chess players. Although the system was created specifically for chess it has also been adapted to many other games and sports, including international football.

How Do They Work?

The fundamental principle behind Elo ratings is that the performance of a team in each match can be considered a random variable sampled from a normally distributed population centred on the team’s true skill level. Although performances will vary from match-to-match, the true skill level of the team is likely to only change slowly over time so can be considered to be the mean value of all their performance values.

For example, Figure one shows a team with an Elo rating of 1500. On any given day their actual performance could vary from anywhere below 1000 to above 2000. But over a reasonable period of time their performances will average out to 1500.

Figure 1: Possible performances for a team with Elo of 1500

Why are Elo Ratings Useful?

Elo ratings have no units and taken in isolation their specific values are of little interest. However, they become useful when comparing teams together as they can be used to determine the expected outcome between two teams based on the difference between their Elo ratings.

The range used for Elo ratings is somewhat arbitrary with Elo himself suggesting they should be scaled so that a difference of two hundred points equates to the higher ranked team having a win probability of 75%. In addition, Elo ratings are generally scaled so that an average team has a rating of 1500.

Predicting Match Results Using Elo

Plotting two team’s Elo distributions together gives a nice way of visualizing their expected performances. Figure 2 shows Team 1 with an Elo rating of 1100 compared with Team 2 with an Elo of 1500. The most likely outcome is that both teams will play to their average ratings and so Team 2 will win overall as they have the higher ranking. However, both team’s performance distributions overlap each other, so it is possible for Team 1 to out perform Team 2 and win the match.

Figure 2: Comparision of two team’s Elo performance probabilities

The more these performance distributions overlap then the greater the chance of the lower placed team winning the match. The actual probability of victory can then be calculated from these two distributions by subtracting one from the other to get the normal difference distribution between the two (Figure 3).

Figure 3: Probability of Elo differential occurring

The centre of this new distribution is equal to the difference between the two ratings (1500 – 1100), meaning the most likely outcome is that Team 2 play like a team with an Elo rating of four hundred higher than Team 1. As we move further to the left the difference between the two teams decreases until we reach a negative differential at which point Team 1 actually start to play better than Team 2, albeit with a low probability of occurrence.

The actual probability of this occurring can be plotted using cumulative frequency to show the overall chance of winning based on the Elo differential (Figure 4). So for our example above, Team 1 with its differential of -400 actually has around a 9% chance of winning the match while Team 2 has a 91% chance of winning.

Figure 4: Probability of winning based on Elo differential

So now we understand the theory behind Elo ratings, my next post will look at how they can be calculated and applied to football teams.

Predicting Football Matches Using Shot Data Part Two

2013-01-25T19:30:00+00:00

Introduction

Having found that the correlation between goals scored and shots on target was the strongest of the various shooting variables I had available to me, I decided to see how well they could predict the outcome of a football match.

Creating The Model

The obvious approach would have been to just do a linear regression for goals scored against number of shots on target and then predict the average number of goals each team would be expected to score. This doesn’t provide much insight though. The average score line might be of interest if each team was going to play each other 20 or 30 times a season but for a single game it is pretty much irrelevant.

What is of more use is to predict the actual odds for each possible outcome between the teams. In other words what is the probability of each team winning, drawing or losing?

To do this I looked at how many shots on target each team achieved and conceded each match compared with the league’s average to estimate how many they would be expected to have against each other. This was then mapped to the distribution of their shot on targets over the season so far and their shot conversion rate used to calculate the probabilities of the different number of goals they could score. Each match was then played one million times as part of a Monte Carlo simulation to see what the likely outcomes was.

Are the Predictions Accurate?

One difficulty with a model like this is to assess its accuracy. With a traditional linear model you can just look at the $r2$ value to see how well you predictions match the actual results. The higher the $r2$ value then the better your model is.

But with a probability model this doesn’t work. For example take the situation where the probability model predicts Team A have a 75% chance of beating Team B. Even if the model has calculated these odds perfectly then Team A will still lose 25% of the time, making it look like the prediction was incorrect.

One alternative is to identify what the most probable outcome for each match was – win, draw or loss – and compare that with what actually happened to see if they match. To do this I applied the model retrospectively to all the matches from the 2011–2012 English Premier League season and overall the proportions of outcomes predicted did match closely what actually happened (Figure 1).

Figure 1: Proportion of outcomes predicted compared with actual results for 2011–2012 English Premier League season

Another test we can do is to compare the Shot on Target model with other models to see how well they compare. Again I picked the most probable outcome from my odds and this time compared it with those from Bet365 for the entire 2011–2012 English Premier League season. I also randomly guessed the outcome for each match by chance to see how the model compared with pure luck too.

Prediction Results

Overall, the Shot on Target model’s most probable outcome correctly matched what actually happened for 43% of the matches tested compared with 52% for Bet365 and 33% from randomly guessing.

Interestingly, even the bookies only managed to get the odds correct for around half the matches so the Shot on Target model is doing pretty well at 43% and isn’t that far behind the professional odds compilers. Also, this is only the first stage of the model, there are still plenty of ways it can be tweaked to try and improve its accuracy further.

What Is The Chance of Bradford City reaching Wembley?

2013-01-22T19:30:00+00:00

With League Two’s Bradford City only one match away from playing at Wembley in the League Cup final I thought it would be interesting to see what the chances were of them getting this far.

It has been an unbelievable cup run for Bradford City as they have had to play against teams in higher divisions nearly every step of the way. The first round of the cup pitted them against League One team Notts County who they managed to defeat in extra time. This was then followed a couple of weeks later with a 2–1 victory away at Championship side Watford.

The third round was somewhat kinder to them as they played at home against Burton Albion, a team from their own division. However, having seen off Burton the fourth round then took them to Premier League side Wigan Athletic who they managed to defeat on penalties.

The quarter final again put Bradford against another Premier League team, with Arsenal this time Bradford’s victims. Next up was Aston Villa who lost 3–1 to Bradford at the Valley Parade although Villa did come away with an away goal, which could prove to be critical for them.

To work out the probability of Bradford’s cup run I collected the odds for each match from www.oddsportal.com and removed the overround to get the true odds. The overround is the bookies profit margin created by offering odds lower than the actual true odds of the event occurring. To remove it we just need to scale the odds by the excess so that they add up to exactly 100%.

Once we have the true odds we can them work out the cumulative probability of the cup run by multiplying the odds together (note that bookies odds generally refer to what happens over the first ninety minutes of the match so Bradford City beating Notts County in extra time is actually classed as a draw rather than an away win).

$Prob(Cup Run) = prob draw with Notts County * prob beating Watford * prob…$

Overall, the probability of Bradford City’s current cup run so far is 0.008%. If we take into account tonight’s match then the chances of Bradford’s cup run taking them all the way to Wembley is around 0.001% or 1 in 100,000. It’s not quite a lottery win, which is around 100 times less likely again, but it is a fantastic achievement for Bradford City and is likely to be a once in a lifetime experience for their fans.

Predicting Football Matches Using Shot Data

2013-01-21T19:30:00+00:00

Introduction

Previously on this blog I have discussed my attempts at using the Poisson distribution to predict the number of goals scored in football matches. So far, the results have been disappointing as the mathematical model I constructed under-predicted the number of draws that occurred. This is something I intend to go back and address at some point by adding in the Dixon and Coles adjustment but in the meantime I thought I would try predicting the outcome of matches using shots instead.

Shot Data

There were a number of reasons for working with shots instead of using goals directly. First of all, shots and goals are inherently linked together. For every goals scored there has to be a shot taken. Secondly, not every shot taken leads to a goal, giving us a much larger data set to work with compared with just goals alone. Thirdly, the number of shots taken in a match is pretty much normally distributed (Figure 1) whereas the number of goals scored is closer to a Poisson distribution. This is useful as many statistical tests rely on a normal distribution of data.

Figure 1: Frequency of total shots in English Premier League matches 2009–2012

The first stage of developing the model was to determine what variables to use for it. Looking at data over a whole season showed a decent correlation between goals scored and total shots taken ($r2$=0.62), shots on target($r2$=0.76), shots blocked ($r2$=0.59) and shots wide ($r2$=0.32; Figure 2).

Figure 2: Correlation between goals scored and various shooting parameters for the 2011–2012 English Premier League Season

Variability

Unfortunately when you start looking at the data match-by-match the correlations become much weaker. Over the course of an entire season a lot of the variability in the data starts to even out but over a single match it is not the case and variables such as luck can play a much bigger role. For example it is likely that the teams with the most shots on target will score the most goals overall per season as skill would start to dominate over luck. However, this isn’t always the case for an individual game – we have all seen matches where a team has scored a lucky goal and then managed to hold on for the win even though the opposition has showered their goal with shots for ninety minutes.

Because of this I decided to exclude many of the variables as they have little value over a single match. Instead, I focussed on using just shots on target data as this had the highest correlation with goals match-by-match. As with the total number of shots taken, the data is also roughly normally distributed although it is skewed towards zero (Figure 3) as obviously no matter how bad a team is it cannot achieve less than zero shots on target in a match (although Blackburn Rovers come close by managing to go the entire match against Tottenham Hotspur in 2012 without taking even a single shot, let alone managing to get one on target!)

Figure 3: Frequency of Shots on Target in English Premier League matches 2011–2012

In my next post I will explain more about how the Shot on Target model works and discuss its accuracy.

Predicting The Premier League Using The Refined Pythagorean Equation

2013-01-18T19:30:00+00:00

Introduction

New article for Betting Expert looking at the current Premier League standings compared with the predictions from my refined version of the Pythagorean Expectation.

How Early In The Season Can Pythagorean Predictions Be Made?

2013-01-02T19:30:00+00:00

Introduction

The next stage for developing my refined version of the Pythagorean equation (known as the MPE) is to characterise how much data it actually needs to make accurate football predictions.

Methodology

To investigate this I selected Manchester City, Swansea City and Wolverhampton Wanderers from the English Premier League’s 2011–2012 season. The reason for choosing these teams was that they represented the top, middle and bottom of the league so I could test the MPE equation across teams of varying quality and league position.

I then used the MPE equation to predict the total points at the end of the season for each team week-by-week to see how the prediction changed throughout the year. Figure 1 shows the difference between the predicted points and the actual points achieved at the end of the season for each of the three teams.

Figure 1: Difference Between Predicted and Actual Points

Results

The prediction settled down very quickly for Manchester City and from match three onwards the root mean square (RMSE) of the error was just 1.96 points. This means that after just three games the MPE equation was accurately predicting how many points Manchester City would have at the end of the season to within two points.

For Swansea City the prediction was slightly more problematic as they didn’t score during their first four matches and the MPE equation needs goals to have been scored before a valid prediction can be made. Swansea City finally scored in their fifth match in a 3–0 victory over West Bromwich Albion and from then on the prediction steadily improved and was within three points of their actual total after their next six matches.

Wolverhampton Wanderers’ season was an interesting one to predict as they had a very misleading start with two wins and a draw in their first three matches giving a predicted point total of 83. At this point though it all went disastrously wrong for them and they lost their next five matches on the run by which time their predicted points had dropped all the way down to 30. Wolverhampton Wanderers eventually finished bottom of the league with 25 points.

Overall, the MPE equation appears to give stable results and the only real requirement is that goals have been scored. Based on the data in Figure 1 accurate predictions can be made early in the season as there is very little change in the predictions from week ten of the season onwards.

What Has Caused Dimitar Berbatov’s Recent Lack of Goals?

2012-12-15T19:30:00+00:00

Introduction

Up until week 12 of the season, Dimitar Berbatov was one of the English Premier League’s top goal scorers and goal creators. However, since then he has gone 450 minutes without registering either a goal or an assist, coinciding with Bryan Ruiz’s injury. Check out my guest article in which I analyse the effect the absence of Ruiz has had on Berbatov’s performances here

Using the Pythagorean Expectation Across Leagues Wordwide

2012-12-10T19:30:00+00:00

Introduction

I showed in my last post that my initial version of the Pythagorean Expectation (MPE) predicted total points for the English Premier League (EPL) pretty well, with an RMSE of approximately four points over the course of a whole season (see here for an explanation of using RMSE to measure the error of the predictions). The next stage for the equation’s development is to see whether it can be applied to other leagues too. Having one MPE equation that could be used globally across leagues is preferable to having to create specific equations for each league.

The Eredivisie

At the recommendation of Scoreboard Journalism's Simon Gleave I started with the Eredivisie, the top flight division in Holland. The reason for choosing the Eredivisie is that it is a unique league, with high rates of goal scoring and a number of results in recent years that appear as potential outliers. For example, in the 2009–2010 season Ajax scored 43 goals more than Twente and conceded three fewer yet still finished second to them in the league. At the other end of the table Willem II finished 15th in 2007–2008 with a goal difference of -9 while the two teams immediately above them had goal differences of -30 and -24, respectively. These sort of results make the Eredivisie difficult to predict and so provide a good stress test for the MPE equation.

Applying the MPE to the final Eredivise standings from 1999–2000 to 2011–2012 worked surprisingly well, with an overall RMSE of 4.35 points. It is slightly higher than the 4.08 previously obtained for the EPL but this is perhaps to be expected since the original MPE equation was generated using just data from the EPL.

To see whether the Dutch league needed its own version of MPE I recreated the equation based on just Eredivisie data and the overall error dropped to 4.21, a decrease of around 3%. Such a minor improvement suggests that the equation maybe stable across leagues and so we will not need league-specific versions.

To test this hypothesis further I collected 223 league tables from around the world and optimised the MPE against this larger data set. The reason for this was three-fold. Firstly, the original equation I published was created just from EPL data so any peculiarities to the EPL could bias results for other leagues.

Secondly, the previous data set was smaller so any outliers in the data could have a large effect on the finalised results. By using a larger data set the influence of any outliers will be minimised.

Thirdly, and perhaps most importantly, this gave enough data to cross-validate the equation by randomly splitting the league tables up into training and validation sets. Initially, the MPE had been trained and tested using the same data. Now it has been tested on different data to which it was optimised against, reducing the risk of Type III errors errors occurring.

Figure One shows the RMSE for the predictions for fifteen leagues randomly selected as a validation set. The overall RMSE across the entire validation set is 3.88 points and is plotted as the vertical dotted line. The overall RMSE is now reduced to below four points and this new version of the MPE equation appears suitable for use globally across different leagues.

Figure 1: Results For Validation of MPE Equation

The finalised MPE Pythagorean Expectation is shown in Figure 2. Based on the data shown here this new version of the MPE equation is suitable for use across multiple leagues worldwide, with an average error of less than 4 points per season.

$predicted points = (goalsfor^{1.2299}/(goalsfor^{1.16793} + goalsaway^{1.20053})) * 2.29761 * numberofgamesplayed$

Figure 3: MPE Equation

Applying the Pythagorean Expectation to Football: Part Two

2012-12-03T19:30:00+00:00

Introduction

In my previous article, I discussed how to apply the baseball Pythagorean expectation to football and how to measure the error of the predictions using RMSE. This second article will demonstrate how to optimize the equation further to improve its accuracy.

Accuracy

One of the major reasons for the error in the predictions is the occurrence of draws in football. The Pythagorean expectation only looks at wins and losses and presumes that if a team scores zero goals then it will achieve zero points. This is of course incorrect, it is perfectly feasible for a team to fail to score but still gain a point through a nil-nil draw so we need to take this into account.

Howard Hamilton of Soccermetrics has published an updated Soccer Pythagorean equation that does just that, and it does a good job of it. For the 2011–2012 season, Howard Hamilton reports an RMSE of 3.81 compared with the RMSE of 5.65 I reported for my previous version of the Pythagorean equation. The downside to Howard Hamilton’s equation though is that it is rather complicated. While the original Pythagorean equation is simple enough to be used by any football fan, Howard Hamilton’s equation requires a decent understanding of mathematics to use it.

Because of that, I thought I would tweak the original Pythagorean formula a bit further to try and improve its accuracy without adding too much extra complexity to it. One easy way to do this is to scale the points scored per match to take into account the occurrence of draws. Applying least squares to this reduces the RMSE for the 2011–2012 season to 4.04 points, just 6% higher than Howard Hamilton’s equation. This is based on only one season’s data though so to get a true idea of how well my enhanced Pythagorean expectation works (abbreviated to MPE) I optimized the equation based on a much larger data set and applied it to the last 10 English Premier League (EPL) seasons ( Figure 1).

Figure 1: MPE Prediction by Season in the EPL

The MPE works well, with an average residual (the difference between predicted points and actual points) of 4.08 points. This compares nicely with Howard Hamilton’s published value of 3.81 and is less than half of the error the original Pythagorean Expectation equation gave. It is also worth noting that Howard Hamilton’s RMSE of 3.81 is for just one season, and of the ten seasons analysed here using the MPE, two actually have an RMSE lower than 3.81.

Plotting the MPE predicted points versus actual points for the last ten EPL seasons shows visually how well the MPE equation works (Figure 2). The correlation between the predicted and actual points scored is excellent, with an an $r2$ value 0.938 (Figure 2).

Figure 2: MPE Predicted Points Versus Actual Points in the EPL

So based on the initial work so far I am pleased that the MPE version of the Pythagorean expectation gives results comparable to Howard Hamilton’s more detailed and advanced derivation but without quite as much added complexity. The final equation for anybody who wants to give it a try is shown below in Figure 3.

$predicted points = (goalsfor^{1.22777}/(goalsfor^{1.072388} + goalsaway^{1.127248})) * 2.499973 * numberofgamesplayed$

Figure 3: MPE Equation

Applying the Pythagorean Expectation to Football: Part One

2012-11-26T19:30:00+00:00

Introduction

The Baseball Pythagorean Expectation is a formula originally derived by Bill James to estimate how many games a baseball team could be expected to win over a season based on the number of runs they score and concede (Figure 1). Teams winning fewer games than their Pythagorean prediction are considered to have been unlucky while those outperforming the prediction are thought to have had luck on their side.

$wins = runs scored^2 / (runs scored^2 + runs allowed^2)$

Figure 1: The Baseball Pythagorean Expectation

The formula works well for baseball, giving predictions generally within three games of what actually happens. The Pythagorean expectation has also been applied successfully to other sports, including American football and basketball. However, so far the equation has not worked particularly well for predicting football matches.

Table 1 shows goals scored and conceded in the English Premier League (EPL) during the 2011–2012 season, along with the actual points and Pythagorean predicted points. Looking at the difference between predicted and actual points it is clear that the Pythagorean expectation is over-predicting at the top of the table and under-predicting at the bottom.

Team	GF	GA	Pts	Pythag Pts
Manchester City	93	29	89	104
Manchester United	89	33	89	100
Arsenal	74	49	70	79
Tottenham Hotspur	66	41	69	82
Newcastle United	56	51	65	62
Chelsea	65	46	64	76
Everton	50	40	56	70
Liverpool	47	40	52	66
Fulham	48	51	52	54
West Bromwich Albion	45	52	47	49
Swansea City	44	51	47	49
Norwich City	52	66	47	44
Sunderland	45	46	45	56
Stoke City	36	53	45	36
Wigan Athletic	42	62	43	36
Aston Villa	37	53	38	37
Queens Park Rangers	43	66	37	34
Bolton Wanderers	46	77	36	30
Blackburn Rovers	48	78	31	31
Wolverhampton Wanderers	40	82	25	22
			RMSE	8.4

Table 1: Pythagorean Expectation for the EPL 2011–2012

We can quantify this error by calculating the root-mean-square error (RMSE). This technique basically squares the difference between the predicted and actual points and then takes the square root of the average. It sounds complicated but all the squares and square roots do is make all the numbers positive. Imagine if we predicted just two values and were -10 points out for the first and +10 points out on the second. If we just averaged these two numbers then the average error would be zero, making it look like our prediction was perfect when it obviously was not. Instead, if we square the numbers first and then take the square root of the average we get the correct error of ten points. Doing this calculation for Table 1 gives us a RMSE of 8.4 points meaning that on average the Pythagorean expectation was eight points out for the 2011–2012 season.

The more accurate the predictions are then the lower the RMSE will be. One way to improve the prediction is to alter the exponent used in the equation. In other words, instead of raising goals scored and conceded to the power of two we use different values. Figure 2 shows what happens to the RMSE as the exponent is changed from 0.1–3. Looking at the chart, the RMSE is lowest using an exponent of 1.35, giving an average error of 5.75, nearly three points lower than before.

Figure 2: Effect of Altering Exponent on RMSE

The next logical step to improve the prediction further is to try using a different exponent for each part of the equation. This makes the formula harder to optimize but by applying a technique called least squares to it we come up with optimal exponents of 1.39, 1.43 and 0.98. Unfortunately this has little effect on the RMSE though, reducing it just 0.1 to 5.65 points.

So far the predictions are still nearly six points out but in part two of this article I will discuss why the error is high and show how to improve it further to increase the accuracy of the predictions.

Disparity in European Football Leagues

2012-11-20T19:30:00+00:00

Introduction

Having mentioned the effect disparity plays on determining the league champions in previous posts I thought it would be interesting to look at the actual levels of disparity currently present in football.

English Premier League

I started off looking at the English Premier League (EPL) over the past decade and plotted the points achieved each season as a Tukey Box-and-Whiskers plot (Figure 1). Looking at Figure 1, the spread of points across the league each season is broadly consistent. There have been a few years where individual teams have done particularly well, such as Chelsea in 2004-2005, or particularly badly, such as Derby County in 2008, but there are no obvious changes over time.

Figure 1: Points Scored in EPL Per Season

One noticeable feature, however, is that the median value for every season (the thick black line in the middle of each box) is lower than the overall average (plotted as the horizontal dotted line), suggesting the data is skewed. Looking at the 2010–2011 season as an example, half the teams scored less than 47 points while half scored 47 or more. In comparison, the average points scored that season was 51.5. This means that an average mid-table EPL team is closer to relegation than it is to winning the league. To put it into perspective, West Ham finished bottom that season scoring just 18.5 points less than the average while Manchester United won the league with 28.5 points more than the average.

Other European Leagues

A similar pattern can be seen across all the major league in Europe (Figure 2) where the median points achieved was also lower than the average. The median points for the Budesliga and Eriedivise were furthest from the average but it is worth bearing in mind that these two league player fewer matches than the EPL, La Liga and Ligue 1 so this is perhaps to be expected.

Figure 2: Points Scored in 2010-2011

La Liga and Ligue 1 both show two teams that are classified as statistical outliers. It is no surprise that the two outliers in La Liga are Real Madrid and Barcelona who both finished more than twenty points ahead of the rest of the league. In the case of Ligue 1, the champions Montpellier and relegated Arles-Avignon are both classed as outliers. A major the reason for this is how close the middle of Ligue 1 finished that season – Monaco were relegated with 44 points, only seven points less than Bordeux who finished in seventh place.

Since leagues play different numbers of matches it is difficult to compare them directly so I also looked at the difference in points scored per match by the top team and the middle team, and the middle team and bottom team (Table 1). The results show that La Liga was the most uncompetitive of the leagues, with the champions scoring 1.737 points more per match than the bottom team. The EPL came out as the most competitive league, with the lowest difference between the top and bottom teams. However, Ligue 1 appears the most balanced, with the smallest difference between the top and bottom of the league compared with the middle. Interestingly, the Eriedivisie appears unbalanced in the opposite way to most other leagues, with the bottom team further away from mid-table than the champions are from mid-table.

League	Top/Middle	Middle/Bottom	Total
EPL	0.868	0.368	1.237
Ligue 1	0.711	0.763	1.474
La Liga	1.289	0.447	1.737
Bundesliga	0.912	0.441	1.353
Eriedivisie	0.765	0.941	1.706

Analysis of André Villas-Boas Vs Harry Redknapp

2012-11-18T19:30:00+00:00

Introduction

Since taking over as manager of Tottenham Hotspur, André Villas-Boas has been trapped in former Spurs manager Harry Redknapp’s shadow. Every tactical decision or team selection Villas-Boas makes is seemingly compared with Redknapp’s previous achievements. And after Tottenham’s apparent slow start to the season, Villas-Boas has come under heavy criticism from the media whose narrative seems to be that Tottenham are performing poorly. But is this criticism fair and are Tottenham really performing any worse than last season under Harry Redknapp?

Find out by reading the rest of this article here.

Effect of Season Length on Deciding the League Champion

2012-11-12T19:30:00+00:00

Introduction

In my previous article I looked at the interplay between luck and skill in determining the league champions. There is another parameter though that also interacts with luck and that is the structure of the league itself. How many times have you heard the same tired, old cliché from football managers about how luck evens itself out over a season? But does it? Is a football season really long enough for the effects of chance to be cancelled out?

I used the same mathematical model as before to simulate 10,000 seasons of a league containing 20 teams. Skill levels were randomly assigned to each team from a normally distributed population with a mean of 0.5 and a standard deviation of 0.1. The length of the season was then altered to see how frequently the team with the highest skill level won the league depending on the number of matches played – teams either played each other once, twice, four times or eight times per season.

Number of Teams	Frequency Teams Meet	Mean Win %	Best Team Win %
20	1	60.2	32.11
20	2	75.3	45.9
20	4	82.2	48.5
20	8	85.2	50.8

Table One: Effect of Season Length on League Champions

Results

The results in Table One show that as the length of the season increases the probability of the team with the highest skill rating winning the league increases too. The Champions also win a greater percentage of their matches too. Therefore, the more matches that are played the less of an influence chance seems to play in determining the overall league champion.

The second row of Table One matches the structure of four of the major leagues in Europe – Premier League, Serie A, La Liga and Ligue 1 – which all contain 20 teams that play each other twice per season. The Eriedivisie and Bundesliga only contain 18 teams though, so what affect does this have? Rerunning the mathematical model with 18 teams gives a lower frequency for the best team winning the league of 28.8%. This suggests that the smaller size of these two leagues makes them somewhat more competitive as there are fewer matches for luck to be evened out.

The Scottish Premier League (SPL) is smaller again, containing just 12 teams. The structure of the league is fairly unique in Europe, with teams playing each other three times, either twice away and once at home or vice versa. The league then splits in half and teams play a further match against the remaining five teams in their half of the league. If we apply the mathematical model to this structure then we come out with a frequency of 19.3% for the best team winning the league. This means the SPL should be one of the most competitive leagues in Europe, yet it has only ever been won by two teams – Celtic and Rangers. The reason for this is likely due to the large disparity in talent between Glasgow’s two largest teams and the rest of the league cancelling out the effect of chance.

Interestingly, with Rangers now relegated from the SPL for financial irregularities, the league is the closest it has ever been. It was thought that without Rangers present in the SPL, Celtic would go on to dominate a very one-sided division. Yet with Hibernian currently sitting top of the league, the reduction in disparity from the loss of Rangers may actually make it the most competitive and exciting year in the SPL’s history.

How Often Does The Best Team Win The League?

2012-11-08T19:30:00+00:00

Introduction

How often does the best team win the league? Probably not as often as you think as it is not just talent that is required for success; a decent amount of luck is needed too.

Methodology

To investigate how big a role luck plays compared with ability I created a mathematical simulation based on the English Premier League (EPL) containing 20 teams that play each other twice per season. Each team was randomly assigned a skill level drawn from a normally distributed population with a mean of 0.5 and a standard distribution linked to the spread of talent across the league so that the disparity between the top and bottom clubs could be controlled. The simulation was then run for 10,000 seasons at various disparity levels and the number of times the team with the highest skill level won the league was measured.

Mean Skill Level	Disparity	Mean Win %	Best Team Win %
0.5	0	65.5	0
0.5	0.02	65.9	10.3
0.5	0.04	67.4	22.7
0.5	0.06	69	32.4
0.5	0.08	72.1	41.2
0.5	0.1	76.4	46.4

Table 1: Effect of Disparity on League Champions

The first row in Table One shows what would happen if all teams in the league were identical. Each team has a skill level of 0.5, meaning that they would each be expected to win 50% of their matches and lose 50% (ignoring draws to keep the model simple). Due to random chance though some teams will win more than 50% and some will lose more than 50%. You can see that the average number of matches won by the league champions over 10,000 seasons was 65.5% so in an evenly matched EPL you would just need to be lucky enough to win an extra 15% of matches to be champions.

As the disparity increases though, the influence of chance decreases and the best team goes on to win the league more often and more of their matches in the process. Take a look at this season’s EPL and while it is possible that QPR could go on to fluke wins in all their remaining matches and go on to win the league it would take a colossal amount of luck compared to say the amount of luck Manchester City would need to finish ahead of Manchester United since their skill levels are closer.

This leads to the question though of what is preferable, an evenly matched, competitive league in which luck is a major determining factor in winning or a league that is perhaps fairer as it has enough disparity that the best team is predominantly likely to win?

The Poisson Model So Far

2012-11-02T19:30:00+00:00

Introduction

In my last article I wrote about my experiences using the Poisson distribution to predict the outcome of football matches. The results so far have been rather disappointing so I thought I would have a look at where things were going wrong.

Probabilities

The first place I decided to look was at the probabilities generated for the matches predicted correctly compared with those predicted incorrectly. I suspected that maybe the model was struggling with matches between more evenly matched teams. For example, for last week’s match between Stoke and Sunderland the predicted outcome was a home win with a probability of 51%. This still leaves us with a 49% chance though that the game will finish with an away win or a draw instead making it potentially difficult to predict accurately.

Overall, the average probability for games correctly predicted was 64% compared with 56% in the games where the prediction failed. At first look it would therefore appear that the model does struggle somewhat with games between more closely matched teams. However, when you look at the variability in the data it is not possible to discern between the two percentages (Figure 1). In fact comparing the data sets using analysis of variance (ANOVA) gives a p-value of 0.32 suggesting no statistical difference between the two percentages based on the current data.

Figure 1: Average probabilities of matches correctly / incorrectly predicted by the Poisson model

Next I looked at which outcomes were being incorrectly predicted and a problem immediately became apparent. So far the model has predicted 50 matches of which 58% were predicted to be home wins, 34% as away wins and 8% as draws. Looking at what really happened though, of those 50 matches 42% were actually home wins, 30% away wins and 28% were draws (Figure 2). This suggests the model is under-predicting the likelihood of draws by quite a large margin and is actually predicting them as home wins.

Figure 2: Proportion of Match Outcomes - Poisson vs Actual

Conclusions

A quick Google revealed two possible fixes. Karlis and Ntzoufras recommend replacing the independent Poisson with a bivariate Poisson to add an element of correlation between the home and away team’s scores. However, even with this they still needed to inflate the diagonal of the score matrix to try and improve the prediction of draws, suggesting that moving to the bivariate Poisson is not necessarily much of an improvement. An alternative proposal by Dixon and Coles was to stick with the two independent Poisson calculations but add in an additional parameter to modify the probabilities of 0-0, 1-1, 1-0 and 0-1 scores occurring.

So where does this leave the current Poisson model? For me, it is time to move on to other ideas. The Poisson model is one the most widely used models for predicting football outcomes so I will return to it in the future to try out the Karlis and Ntzoufras and Dixon and Coles adjustments but I gave a few other ideas to write about first.

Using Poisson to Predict Football Matches

2012-10-29T19:30:00+00:00

Introduction

The Power Of Goals recently blogged about using the Poisson distribution to predict the outcome of football matches. I have been evaluating the predictive ability of the Poisson for the English Premier League (EPL) this season so I thought I would share my experiences too.

For anyone who is unaware, the number of goals scored by each team in a football match roughly follows a Poisson distribution. As you can see in Figure 1, it is not exact though as the Poisson distribution underestimates the likelihood of no goals being scored and overestimates one, two and three goals being scored. By four goals and upwards the Poisson starts to underestimate again. The actual difference between the Poisson and what is observed in the EPL is reasonably small though so it just requires a small fudge factor to bring the two into line.

Figure 1: Poisson Distribution vs Observed

To carry out the predictions I have written a script in R that scrapes the Premier League table directly from the BBC’s website. The script then calculates attack and defence coefficients for each team by comparing their goals scored and conceded with the overall EPL average home and away. The predicted number of goals scored in a particular match can then be calculated by scaling the EPL’s average goals by the two team’s attack and defence coefficients. This can then be mapped to the Poisson distribution to generate a probability matrix for each particular score line (Table 1). From this, the probabilities can be summed to find the odds that each match will end as a home win, draw or away win.

Goals	0	1	2	3	4	5	6	7	8
0	1.96	4.08	4.24	2.94	1.53	0.64	0.22	0.07	0.02
1	3.63	7.56	7.86	5.45	2.83	1.18	0.41	0.12	0.03
2	3.36	7.00	7.27	5.04	2.62	1.09	0.38	0.11	0.03
3	2.08	4.32	4.49	3.11	1.62	0.67	0.23	0.07	0.02
4	0.96	2.00	2.08	1.44	0.75	0.31	0.11	0.03	0.01
5	0.36	0.74	0.77	0.53	0.28	0.12	0.04	0.01	0.00
6	0.11	0.23	0.24	0.16	0.09	0.04	0.01	0.00	0.00
7	0.03	0.06	0.06	0.04	0.02	0.01	0.00	0.00	0.00
8	0.01	0.01	0.01	0.01	0.01	0.00	0.00	0.00	0.00

Table 1: Example Goal Probabilities (%)

Since the predictions are based on past performance this season, I waited until week five of the EPL to start testing it so I had at least a month’s worth of previous results to work with. The first week went well, with the model correctly predicting the outcome of six of the ten matches that weekend. Table 2 shows the predicted probabilities (%) of the home team winning each match. From this I also calculated the odds and compared mine with those available from Betfair to see how well they compared.

Home	Away	Prediction	Probability (%)	Odds	Betfair	Result
Swansea	Everton	HOME	56.3	1.78	3.35	AWAY
Chelsea	Stoke City	HOME	63.4	1.58	1.39	HOME
Southampton	Aston Villa	AWAY	49.2	2.03	3.1	HOME
West Brom	Reading	HOME	41.1	2.43	1.82	HOME
West Ham	Sunderland	HOME	35.7	2.80	2.24	DRAW
Wigan	Fulham	AWAY	40.1	2.49	3.25	AWAY
Liverpool	Man Utd	AWAY	75.6	1.32	2.82	AWAY
Newcastle	Norwich	HOME	82.9	1.21	1.84	HOME
Man City	Arsenal	AWAY	37.1	2.70	1.78	DRAW
Tottenham	QPR	HOME	41.1	2.43	1.51	HOME

Table 2: EPL Week 5 Predictions

Since then, the performance of the Poisson has between correctly predicting between 30-60% of matches each week (Figure 2). So far, the average accuracy is 46%, which is slightly higher than the 33% we could expect from randomly guessing each result.

Figure 2: Weekly Performance of Poisson Predictive Model

I am hopeful the model’s success rate will improve over the course of the season as it gets more data to work with. There are also further improvements that can be made as well. For example, the model currently considers the goals scored by each team to be independent events. However, it may be that the two should be correlated together as it would seem intuitive that the more goals one teams scores the less likely the opposition is to score. At the moment though I wouldn’t place too much faith in the Poisson model.

Influence Of Clean Sheets

2012-10-26T19:30:00+00:00

Introduction

To make much sense of the statistics available for football we need to have an understanding of their context so I am planning on starting off simple by looking at baselines for various events and statistics while I build up the information required to start a mathematical model.

Clean Sheets

While most football analytics seems to focus heavily on goals, I am going to start off with defending and the all important clean sheet. Clean sheets have been fairly consistent throughout the English Premier League’s (EPL) history, occurring in around 27% of matches between 1993 and 2011 (Figure 1). The data shows some variability around the mean with perhaps the slightest hint of an upwards trend, but in general the total number of clean sheets per season has remained constant.

Figure 1: Total English Premier League Clean Sheets

Home And Away

If we split the data by home and away then we can immediately see a significant difference (Figure 2; p>0.001). On average, the home team will keep a clean sheet 33% of the time while the away team will only manage it in 22% of their matches. Interestingly, both sets of data appear to follow broadly similar patterns with peaks and troughs occurring in the same years. I hope to explore this in more detail in the future.

Figure 2: Clean Sheets Home and Away

Clean sheets are valuable commodities as they guarantee you a minimum of one point. As the cliche goes, if you keep a clean sheet you cannot lose. Looking back over the EPL’s history shows that a clean sheet at home is actually worth 2.1 points on average. This means that over the course of a season obtaining a clean sheet in 33% of matches would be expected to generate 13.2 points. Away from home a clean sheet is of lower value, generating just 1.8 points each. Over the course of a season this would therefore bring in an additional 7.5 points.

The English Premier League

We can use these baselines to examine how teams are performing in terms of clean sheets home and away. Figure three shows the proportion of matches in which each team in the EPL obtained clean sheets for the 2011-2012 season. The teams in the upper right quadrant all acheived an above average number of clean sheets both home and away. In comparison, West Brom’s defence performed very well at home yet they struggled to obtain clean sheets away from the Hawthorns. Liverpool were the opposite of West Brom, keeping clean sheets away from Anfield but struggling at home. Bolton, Blackburn and Wolves all generated very low numbers of clean sheets home and away and were all relegated from the EPL. Norwich are an interesting exception as they possessed the worst away record for clean sheets yet managed to finish in a respectable 12th position last year.

Figure 3: Proportion of Matches With Clean Sheets Home and Away

League Position

If we carry out linear regression on 2011-2012’s data (Figure 4) we can see the correlation between the number of clean sheets a team kept over the season and their final league position. The r2 value of 0.72 for the regression shows that the two are strongly correlated with each other so any team not keeping clean sheets could be expected to finish lower down the league table. This does not bode well for current champions Manchester City, who have conceded goals in seven of their eight EPL matches this season.

Figure 4: Correlation of Final League Position to Number of Clean Sheets 2011/2012

penaltyblog

Why I Wrote My Own MCMC Sampler for Penaltyblog

Introduction

Moving Beyond Point Estimates

The Solution: Cythonized MCMC

How to use it

A Quick Note on the Arguments

Diagnostics

Prediction

The Hierarchical Model

Summary

Introducing the Opta API Connector for matchflow

Introduction

The "Why?": Handling the Hard Parts for You

Getting Started: A Quick Example

The Real Power: Building a Pipeline

Say Goodbye to "Magic Numbers"

Why matchflow? Cloud Storage & More

The "Killer Feature": Joining Streams

Pro Tip: Working with IP Whitelists

What Feeds Are Supported?

Conclusion

Shrinkage, Uncertainty, and Son Heung-min: Using Bayesian Methods to Identify Finishing Ability

Introduction

The Trouble with a Single Number

A Better Approach: Thinking in Probabilities

The Results: Ranking Europe's Elite Finishers

Understanding the Rankings: What Do the Numbers Mean?

Sense Check: The Elite (Lionel Messi)

Sense Check: The Not-so Elite (Jesús Navas)

Putting It Into Practice

Practical Applications for Clubs

Methodology Details

Conclusion

From Biased Odds to Fair Probabilities: Removing the Bookmaker's Overround

Introduction

What's New in penaltyblog?

A Deep Dive into Implied Probability Methods

The Basic Approaches

Iterative Approaches

Putting It All Together: A Practical Example

An Empirical Test: Which Method is Most Accurate?

Results

Conclusion

Penaltyblog v1.5.0: Faster Models, Smarter Queries, and a Sharper Edge

Introduction

Pitch - The Interactive Football Plotting Library

Example: Shot Map

Example: Layering Plots

Pitch Documentation

Blazingly Fast Goal Models

More Control Over Optimisation

Example: Dixon–Coles Model with Custom Optimisation

Improved FootballProbabilityGrid

1. Optional Normalization

2. Expanded Market Calculations

3. Push-Aware Asian Handicap Calculations

4. Richer Representation

5. Backwards Compatibility

Example: Predicting & Using the Grid

Flow Query DSL

How It Works

Supported syntax

Comparisons

Logical operators

Field access (dot notation)

Membership

NULL / missing checks

String transforms (inside comparisons)

Predicate-style string methods (standalone)

Regex flags

Variables from Python (@var)

Date & datetime literals

Common Patterns

Performance notes

Gotchas & limitations

Quick reference

Conclusions

How Accurate Are Soccer Odds? A Data Dive into 250 Million Betting Lines

Introduction

Improved `FootballProbabilityGrid`

Variables from Python (`@var`)

Improved Plan Visualization with `.plot_plan`