Wales were never world number one

There has been a lot of commentary around the official World rugby rankings, mostly relating to the lack of accuracy they display. Even World Rugby’s vice president Agustin Pichot has waded in, saying:

“It is a ranking that is badly done, and I said it the first day I arrived at World Rugby. It has no order, it is all mathematical and I would say that it is almost a matter of marketing."

Agustin Pichot [1]

Our experts agree that many of the official rankings systems are flawed, not just those relating to rugby,– both in the mathematical approach and in maintaining impartiality. Most official rankings just use a simplistic points exchange system to calculate the rank of each team, or player, following national or international games and tournaments.

We like to embrace a challenge, particularly when it comes to the rugby pitch. With significant domain expertise both in sports performance analysis and data science, the team has been working hard to develop a new ranking model that addresses the shortfalls of the official rankings. We believe we have something that Mr Pichot would be interested to see, well he did assure us he wanted to change it (the rankings system).

Warren Gatland, Welsh Coach is more philosophical about it all, he says of the recent promotion of Wales to Number 1 status;

“Lots of journalists out there are going ‘this is a joke’… it’s just a number - that’s all it is at the moment.”

Warren Gatland [2]

To put a stop to the ranking skeptics, we can prove our model works. We have used public domain, open source results data to retrospectively apply the model to all of the games played during the last four Rugby World Cup tournaments, 2003, 2007, 2011 & 2015.

Rugby World Cup
World Rugby Ranking Success
Insight Ranking Success

We have seen much greater accuracy over this period in the prediction of results from our ranking (compared against the official rankings). Plus, our model is sport agnostic, to be used for any sport variant and could expand for use to rank teams, or athletes, within or even between sports. The scalability of our model together with application variables across real world cases, with no limit to sport category opens a multitude of opportunities.

For example, we see that by using our standardised model, we can apply the rankings to inter-sport comparisons, or individual player analysis enabling the ranking of players across many sporting disciplines within one table. A sports persons super-ranking table. 

So, how does this work? The model takes many factors into account, not just the two teams competing against each other. To fully assess the teams, the dominance of one team over another needs to be considered. Therefore, a lower-ranked team can still improve their ranking, if they lose by a smaller margin than expected. This is then iterated throughout the competition to provide a real-time view of the teams’ rankings. Each time a “shock” result occurs, the rankings are adjusted across the network as it impacts all the teams across the whole pool or tournament

In what is deemed to be the tightest Rugby World Cup ever, the official rankings has shown the top 5 teams frequently moving up and down the table, with Ireland heading to the World cup as number one. This has
highlighted the narrow margins between the top teams and leaves commentators unable to confidently predict who will lift the Webb Ellis trophy at the end of the tournament.

Whilst we concur that the rankings are the tightest they have ever been in the professional age, they are not as fluid as you might believe based on the movements in the official rankings. Our model currently ranks New Zealand as the outright number one by a significant margin, maintaining a dominant stance over the next ranked teams. In fact, what we have seen is that the top of the table has been fairly static and the real changes are happening in the lower ranks - within the 6-10 positions.

Whether or not you agree with Agustin Pichot that the official rankings are ridiculous, is largely irrelevant. The rankings, even when using the existing, immature model, represent an impression based on the
current status of the professional rugby ecosystem.

What is less obvious in this model is the impact of any shock results to the rankings. The fact is that whilst data can tell a story, there are some situations that cannot be readily predicted; for example, the defeat of South Africa by Japan in one of the 2015 World cup opening games. As far as we are aware, no model anticipated this result, with South Africa consistently 10 places above Japan in both the official and our rankings. However, with our model, we were able to re-assess across the network to reflect the impact this result had on all team rankings -  giving us the ability to demonstrate greater accuracy for the remaining games played by Japan.

Even if you do come down on the side of the rankings being no more than a marketing tool, it has done it’s job. It’s opened a conversation – it’s got people talking - by raising awareness of the World Cup rankings and interest in rugby, it also instills desire in the teams to win so they take action to do so. Just like a good marketing stunt should.

A quick review of Twitter, for example, shows national newspapers, commentators, influencers and the general public all remarking on the “ridiculous” state of the rankings, once again using Mr Pichot’s term.

It’s topical, it’s relevant, it’s happening right now. And it’s going to be an interesting tournament, being played at some lovely venues across the host nation of Japan.