Way back in 2017 I was trying to learn about a new statistical technique, hierarchical Bayesian inference, using a python package called PyMC. One of the tutorials back then illustrated how to use it to predict football scores, and I thought it would be fun to apply it to rugby. Back then I had a model which looked at historical scores and tried to predict the outcome of future matches just based on those (at some point I’ll write up a more detailed and technical post about how the model works), but this year I’ve been playing around with updating things, and taking into account things like the players actually on the teamsheet each week.

Anyway, since the Six Nations started this week, it’s time for a slightly belated look at the data and the model’s predictions.

Before things started

Before the first match had been played I asked the model to predict two things, first, the outcomes of the matches, and second, the final table at the end of the tournament.

There were some surprises in there. The one which seems most demanding of scepticism is that Italy were being predicted to beat Scotland this week, but Scotland’s predicted to win against both France and England… Well, more on that later if you’ve not already seen the results.

Then the predicted final table was as follows:

This probably won’t be a source of any comfort to Welsh, and our perhaps Scottish, fans, but the model was pretty confident that France would win the tournament, and that England and Ireland would be the main challengers.

Well.

After week 1

It’s been a week of quite high-scoring matches, and upsets (though, as it turned out, predictable upsets). Let’s look at how the predictions have now changed.

Well, not a lot, but then the model was correct about the outcome of all three games this week, though it was perhaps a little less good on the margins (it had France win by 14 rather than 24, Italy by 4 rather than 3, and England by 17 rather than… 41).

That means I now have a revised prediction for the final table.

Scotland’s defeat by Italy has saved Ireland here; when I re-ran the predtions on Friday morning with just the France - Ireland game in hand things looked much better for Scotland.

Paths to Victory

One of the fun things my model allows me to do is to determine the ways in which each team can win the tournament, and the probabilities of those different paths to victory. At this stage this is based entirely on simulation, as there are billions of potential ways that the tournament could play out at this stage. Later, as we get towards the final week, it will be possible to exactly determine the possible states.

Scotland [Probability of winning: 4.1%]

Critical games for Scotland are as follows:

1. Ireland vs Scotland [Impact: +3.1%]
2. Scotland vs England [Impact: +2.3%]
3. Scotland vs France [Impact: +1.9%]
4. Wales vs Scotland [Impact: +1.6%]
5. Wales vs France [Impact: +1.1%]
6. France vs Italy [Impact: +0.2%]
7. Ireland vs Italy [Impact: +0.2%]
8. Italy vs England [Impact: +0.2%]
9. England vs Ireland [Impact: +0.1%]
10. Ireland vs Wales [Impact: +0.1%]

Here the critical games give you an idea of how much each game matters for the team’s chances of winning the tournament. For example, for Scotland, the most important game is their match against Ireland, which will increase their chances of winning the tournament by 3.1 percentage points if they win it, and decrease it by 3.1 percentage points if they lose it. There are no massive surprises in these; winning all your games will generally have the biggest effect on your chances, but it can be useful to see which games between your opponents are most important after those. I will admit to some scepticism about some of these numbers, but I suppose I have a few more weeks to work out if there’s a bug somewhere in my code.

France [Probability of winning: 60.9%]

Critical games for France are as follows:

1. France vs England [Impact: +9.3%]
2. Wales vs France [Impact: +7.0%]
3. Scotland vs France [Impact: +6.6%]
4. France vs Italy [Impact: +5.4%]
5. Italy vs England [Impact: +0.7%]
6. Scotland vs England [Impact: +0.6%]
7. Wales vs Italy [Impact: +0.3%]
8. Ireland vs Wales [Impact: +0.1%]
9. Ireland vs Scotland [Impact: +0.1%]
10. Ireland vs Italy [Impact: +0.1%]

England [Probability of winning: 21.2%]

Critical games for England are as follows:

1. France vs England [Impact: +9.7%]
2. Italy vs England [Impact: +4.6%]
3. Scotland vs England [Impact: +3.9%]
4. England vs Ireland [Impact: +2.7%]
5. Wales vs France [Impact: +1.8%]
6. Scotland vs France [Impact: +1.6%]
7. France vs Italy [Impact: +0.6%]
8. Wales vs Italy [Impact: +0.0%]
9. Ireland vs Scotland [Impact: +0.0%]
10. Wales vs Scotland [Impact: +0.0%]

Ireland [Probability of winning: 5.3%]

Critical games for Ireland are as follows:

1. England vs Ireland [Impact: +4.3%]
2. Scotland vs France [Impact: +1.3%]
3. Ireland vs Scotland [Impact: +1.3%]
4. Wales vs France [Impact: +1.2%]
5. Ireland vs Italy [Impact: +1.1%]
6. Ireland vs Wales [Impact: +0.5%]
7. France vs Italy [Impact: +0.5%]
8. France vs England [Impact: +0.2%]
9. Scotland vs England [Impact: +0.1%]
10. Italy vs England [Impact: +0.1%]

Italy [Probability of winning: 5.4%]

Critical games for Italy are as follows:

1. France vs Italy [Impact: +4.4%]
2. Ireland vs Italy [Impact: +2.0%]
3. Wales vs Italy [Impact: +1.9%]
4. Italy vs England [Impact: +1.5%]
5. Wales vs France [Impact: +0.3%]
6. Scotland vs France [Impact: +0.2%]
7. Scotland vs England [Impact: +0.1%]
8. France vs England [Impact: +0.1%]
9. Ireland vs Scotland [Impact: +0.0%]
10. Ireland vs Wales [Impact: +0.0%]

Wales [Probability of winning: 2.6%]

Critical games for Wales are as follows:

1. Ireland vs Wales [Impact: +3.0%]
2. Wales vs France [Impact: +1.9%]
3. Wales vs Italy [Impact: +1.4%]
4. Wales vs Scotland [Impact: +1.3%]
5. France vs Italy [Impact: +0.3%]
6. Scotland vs France [Impact: +0.2%]
7. Italy vs England [Impact: +0.1%]
8. England vs Ireland [Impact: +0.1%]
9. Ireland vs Scotland [Impact: +0.1%]
10. Ireland vs Italy [Impact: +0.1%]