You do not usually have to go very far in the World of Tanks forum before running into a discussion of statistics, and more particularly of how one statistic or another can be used to evaluate a player’s skill. The simplest and most commonly used of all those statistics is the win rate. It seems fairly simple, surely the higher someone’s win-rate the better player they are. It is an easy paradigm and one we use in other sports – the folks who win the most in the (league/division/whatever) are the winners of that (league/division/whatever).
However, there is a crucial difference between ordinary sports and the win rate in World of Tanks: random teams. Unlike in ordinary team sports, where the win rate is for the team and the ability of individual players is measured by a series of other stats, in World of Tanks the win rate is assigned to individual players for performance in the team.
It should not take very long to see the implicit problem in this. In World of Tanks you are one member of a fifteen person team taking on another fifteen person team. Success in a match usually depends a great deal on the performance of the other fourteen members of your team.
Think about it like this: on a team in which everyone plays the same tank, with the same equipment and consumables, and the same-skilled crew in theory everyone contributes 6.667% to the team. Of course it is rarely that clear cut. In the sole Tier VII tank in a Tier VII match one should contribute a great deal more. Likewise the lowest tier tanks of any given match will generally contribute less. However, even at best most of the time you are massively reliant on the rest of your team.
Practically speaking I know that there are some games in World of Tanks where the team performance is so poor that you cannot help but lose, no matter the quality of your own game. The reverse is also sometimes true – it is possible to have a really poor game yourself yet your team will sometimes pull you through. These are games where your own performance does not matter.
In some games though an individual player can make a massive difference. The guy who notices a flank has collapsed and gets back into a defensive position just in time to prevent capture – thereby allowing the rest of his or her team to complete they own capture is just one example. A crucial difference in winning the match – and in this case the difference between victory and defeat. However, still worthless without the rest of the team.
That basically encapsulates my unease with solely relying on win rate, both to judge my own performance or others. This is despite the basic response to all of the above which is to say that things should average out over several thousand of matches played. and therefore win rate becomes more determined on those times when a player can make a positive or negative difference to the team.
That is a powerful argument. A win rate of 40% over 100 games sounds pretty bad. But 100 games is not really a very large sample size, and it is easily possible to go through a sticky patch where one gets loaded down with a fair number of poor teams. A win rate of 40% over 1000 games is worse – the odds of consistently getting poor teams over a more extended period of time are reduced. However, there are reasons why my unease remains.
The first is the size we are talking about. In World of Tanks the average win rate appears to be about 49% – the reason it is not 50% (which it otherwise should be) is because of the small number of drawn games. The vast majority of games fall just a few percentage points either side of that (see this graph, linked from this thread). Over the course of a thousand battles it only requires a greater than average amount of poor or good luck in 10 of them to change one’s win rate by 1%.
Now one can say that while win rate cannot perhaps determine whether player A or player B (both of which have fought one-thousand matches and have win rates of 52 and 53% respectively) is better, one can say that both players are almost certainly better than player C (a one-thousand match player with a win rate of 47%). To which I say … probably.
I don’t mean to be facetious about it: quite often a win rate difference of that much will reflect the relative quality of a player. However, there are plenty of instances when it will not. I will highlight what I believe are the two most common.
Take the above situation. Say the first players A and B have concentrated mostly on just one tank line. Perhaps they got into the KV-1 and have found it such fun they have mostly stayed there. The majority of their 1000 games is in the KV-1, perhaps even to the extent the crew is now working their third skill or perk. Particularly in matches in its own tier, that tank is going to be formidable. Now look at player C. Player C hasn’t concentrated on any particular tank, being rather something of a gadfly first playing one tank, then another, then another. Player C never establishes any rhythm, and never has an opportunity to learn all the ins and outs of their tanks as our first two players know the KV-1. Likewise most of his tanks have crews that have not even reached 100% in the primary qualification. On average Player C’s tanks are going to be performing a great deal more poorly than our KV-1 drivers, but not because the Player C is necessarily any worse as a player in terms of skill.
The second situation is rather different. Player D also has one thousand matches and also has concentrated on the KV-1. His win rate is 48%. It sounds like he would be a poorer player than players A and B. Indeed, one hundred matches ago he would have been. Player D took quite a while to understand some of the concepts of World of Tanks, but sometime around match 900 something ro somethings clicked in his head and he is now playing with great skill. This is not yet reflected in his win percentage however which has a considerable drag from the poor performance in those first 900 matches.
Both these situations (and others) mean I feel quite justified in not judging someone solely on the basis of their global win rate. It might be right a fair amount of the time, even most of the time, but all to often it will not tell the whole story.