Analysing the Six Nations - Week 3

It was all bound to come to an end eventually. After two weeks of correct predictions the model got not one but two wrong in week 3. Let’s have a look at what was predicted versus what actually happened, and spend some time looking in a little more detail at the predictions beyond the point estimate of the scoreline. Yes, it’s time for statistics: it was bound to happen eventually.

So last week the predictions were:

So we were expecting England, Wales, and France to win. However, somewhat unexpectedly Ireland managed to beat England 21-42 in Twickenham, while Scotland managed to beat Wales 23-26 in Cardiff. That left only the France-Italy game for the model to redeem itself, where a 33-8 victory for France at least saw the outcome correctly predicted.

To understand what went wrong we need to look at what the model actually tells us. It gives substantially more information than just a single guess at the scoreline for a given match. Instead, to arrive at a prediction, it performs 1000 simulations of the match, and then uses the median scoreline across those simulations as the point estimate for the predicted scoreline.

This means that we can in fact look at the distribution of predicted scorelines across those 1000 simulations, and see how likely the actual scoreline was according to the model.

England vs Ireland

Let’s start with the real shock. The model predicted a 37-25 victory for England, but instead Ireland won 21-42. Each possible scoreline is shown with England on the horizontal axis and Ireland on the vertical; the brighter the colour, the more the model’s simulations landed on that scoreline. The actual result is the white circle, and the model’s point prediction is the black cross.

You can see that the prediction was a long way off the reality: the dark purple of the actual score indicates that the scoreline was rare in the model’s simulations. Indeed, it lies two standard deviations away from the predicted scoreline — something expected in only around 2.5% of matches if the model were correct. So, a big result for Ireland, and the first miss for the model.

Wales vs Scotland

Wales v Scotland was predicted to be close, and in the end it was; while the result was the wrong way around, the light-coloured region on the plot covers a spread of scores which could have gone either way. So, taking what consolation I can, the model at least got pretty close — and Scotland won, which is also a consolation.

France vs Italy

However, France performed substantially better than the model anticipated, with the actual result lying around one standard deviation away from the prediction (something we would expect to see in around 16% of matches if the model is correct).

We can look at recent form for all six teams to get some idea of why the model predicted what it did: England are coming out of a very good run, where Ireland have been less consistent. Scotland have been a mixed bag, but Wales have been having something of a nightmare streak, so the model predicting a Wales win was rather surprising. France are consistently good, and while Italy have had some notable recent successes, France was presumably a test too far according to the model.

All of that brings us, however, to the current state of play. A month ago I would not have anticipated the outcomes being predicted today.

Week 4 Predictions

Let’s dive directly into the predictions for this weekend.

One of these is, I think, uncontroversial. If Wales beat Ireland at this stage it would be a shock.

However, the model is predicting a very close game between Scotland and France, with Scotland just edging it. This is a game where the model is predicting a lot of uncertainty, with the distribution of predicted scorelines being quite wide, however, the idea that Scotland are in with a chance of beating France, and derailing their Grand Slam bid, is a very exciting prospect.

Again, a close game between Italy and England is probably not something I had on my bingo card going into the competition, but a combination of a rather strong showing from Italy, and a rather underwhelming England make this one almost plausible.

So, it’s not clear to me if the model’s run of luck has now come to an end after a bad week 3, or if these surprising predictions are in fact going to be better than I think.

The most important upcoming games

With just two weeks to go we can start looking quite seriously at which games are critical for the final standings in the tournament. We can do this by calculating a quantity called the mutual information between the outcome of a given match and the final standings in the tournament. This weekend’s Scotland-France fixture tops the list: it’s the place where France’s run might start to come undone, and also offers a real opportunity for Scotland to cement themselves into second place. The next two most significant games also both involve the two most likely competitors for the first place finish (as things stand today).

1. 1 Scotland vs France · 7 Mar 25.7% impact
2. 2 France vs England · 14 Mar 21.5% impact
3. 3 Ireland vs Scotland · 14 Mar 16.0% impact
4. 4 Ireland vs Wales · 6 Mar 4.3% impact
5. 5 Italy vs England · 7 Mar 0.4% impact
6. 6 Wales vs Italy · 14 Mar 0.1% impact

Predicted final standings

So, in terms of teams still in with any real chance of topping the table we have France (75% chance), Scotland (12%), and Ireland (7%). To some extent, it’s all to play for, but the reality is that France are comfortably the favourites, and will be right up to their last game. This means that second has become the more interesting position to predict, and it’s really proving difficult to call between Ireland and Scotland. The most probable final standings have Ireland and Scotland finish on the same number of (league) points and the outcome decided on (game) points difference. So their game in week five is unlikely to be anything short of tense.

While Scotland managing a second place finish would certainly be a major turnaround compared to the state of play after week one, when the commentary mostly revolved around their coach’s job, the bigger upsets come lower down in the predictions. A fourth place finish for Italy would surely put an end to any speculation that they should be replaced by another country? But fifth place for England? Going into the tournament I was predicting they might win…

We shall see. This weekend’s matches may well be pivotal. I’ll be back next week with the final predictions, and with a look at all of the different paths to victory for each team, and the probabilities of each of those paths.