How Elo ranks your games, and why stars don't

The trouble with stars

A score is a comparison against an imaginary scale. Is this game an 8 or a 9? The answer depends on the day, on the last thing you played and on how generous you feel; and when you are done, thirty games share the 9 and the list is not in order. Stars describe; they don't sort.

A duel is a comparison against something real: another game. "Of these two, which?" has an immediate answer, almost always the same one morning and evening, and every answer says something about the order. The trick is turning many small answers into one long list. That is what Elo is for.

What Elo does with one duel

Every game in your collection carries a number that starts at 1500. Before a duel, the system works out how much each one "should" win from the gap between their numbers: two equal games split the expectation 50/50; one sitting 200 points higher expects to win 76% of the time. After the duel, the winner goes up and the loser down in proportion to the surprise: winning as expected moves little; winning against the odds moves a lot.

How far it moves is set by one constant, K. In Rank Your Games, K is 64: winning a duel between equals is worth +32 to one side and −32 to the other. That is a high K by chess standards (FIDE uses 10 for grandmasters), and it is deliberate: your collection is not a tournament with hundreds of games per player, it is a list with a handful of duels per game, and it needs each one to count.

That is all Elo does. The rest — which two games it shows you, when a number can be trusted — is the app's doing, and that is where the problems were.

What we measured before changing anything

The app stores every duel with the numbers before and after it, so the whole history can be replayed. In August 2026 we replayed it: 2,203 duels across 39 real accounts. Three things came out of that.

1. Games had far too few duels on them

The average was 2.48 duels per game, and the cause was how pairs were formed: 59.7% of duels brought in a new game you didn't own, so the app was adding games faster than it measured them. The visible symptom: 46% of top-10 places were held by a game that had never faced another top-10 game, and one in four had got there on a single duel.

The fix was not to touch Elo but the pairs. The share of new games now falls as your collection grows (from 80% under ten games to 25% at twenty or more), and opponents are drawn with two biases: towards games with fewer duels, and, one time in three, a duel inside your top 15, which is where the order matters and where it was checked least. With that, top-10 games that had never met a peer went from 4.36 to 1.52 per account in the simulation.

2. Lowering K changes the order of nothing

The obvious hypothesis was that K=64 overshot: a game wins one duel, jumps to 1532 and slips above others measured in depth. We replayed the 2,203 duels at K=64, 32, 16 and 8 and counted how many top-10 places were held by a game with one duel or none:

There is a reason: K is a rescaling of each game's distance from 1500, and a monotone rescaling cannot reorder anything; it only squeezes the range (from 977–1936 at K=64 to 1357–1620 at K=8). Shrinking numbers towards 1500 by how many duels a game has fails for the same reason. The ordering error is not in the step size; it is in the missing duels. That is why K stays at 64 and is not a number open to debate: it is measured.

3. What does work: duels first, Elo second

If the problem is missing duels, the list should say so. Since version 2.0.8 the ranking sorts first by whether a game has reached two duels, and only then by Elo. A game with a single duel keeps its number but sits below every settled one, with a note: "still settling". In the replay, under-measured top-10 places went from 145 to 101, and the No. 1 changed in only 3 of the 39 accounts. The 101 that remain belong to accounts that don't yet have ten well-measured games, and no ordering fixes that: more duels do.

Re-ranking a game: five duels, not random ones

When you add a game or re-rank it, the app doesn't show you five random opponents. It opens near the median of your collection and, with each answer, keeps only the band the game can still belong to: a binary search. Five steps localise it to 1/32 of the collection, roughly ±3 places in a list of 100. And it cheats in the game's favour: lose to everything and it drops; beat everything and it rises; it never runs out of opponents halfway.

What this means for you

• There are no numbers to decide. You choose, and the order follows.
• A game with one duel is "still settling": the app tells you so instead of pretending it already knows where it goes.
• Duels between your favourites are the ones worth most. When the app pits two of your top games against each other, that is not an accident.
• If you turn the Elo on (it is off by default, under You → Language and colour → Diagnostics), remember the number only compares within your collection. Your 1700 and someone else's have nothing to do with each other.