How Analysis Tools Separate Luck from Skill in Backgammon

To a newcomer, backgammon can look like pure luck. The more you play, the more skill you see in it, yet the dice never go away. Sometimes you want to strip the luck out and ask a clean question: was move A better than move B? Is one player really stronger than another, or did they just roll better tonight? Analysis programs such as GNU Backgammon and eXtreme Gammon give a practical answer. This page explains the idea behind it. For the general argument about luck and skill, see luck or skill.

Equity: a number for every position

A strong program can estimate, for any position, the expected result for the player on roll. In a single game this is called equity; in a match it is usually expressed as match winning chances. A position worth +0.5 in a game is clearly in your favor; one worth 30% in a match means you win that match about three times in ten from here, with good play on both sides.

A simple equation

Once every position has a value, the result of a game can be split into two parts:

Final result − starting value = net luck + net skill

The starting value is the equity or match winning chance at the beginning. The final result is what actually happened. The difference between them is explained by two things: how the dice treated each player, and how well each player used them.

Measuring luck

Before each roll, the program knows the value of the position averaged over all 36 possible rolls. After the roll, it knows the value of the position given the best play of that particular roll. The difference is the luck of that roll. Rolling a double that wins a race you were losing is very lucky; rolling a number that forces you to break your prime is unlucky. By definition, luck averages zero on each roll, so it also averages zero over a game or a match. You cannot be "owed" luck, but in a short session one player can easily receive much more of it.

Measuring skill

Skill is measured the other way around. For each decision, the program compares the equity of the move you chose with the equity of its best move. The gap is your error on that move. Adding up the errors of both players gives the net skill term. Cube decisions in match play are checked the same way: a wrong double, a wrong take or a wrong pass each cost a measurable amount.

What the reports show

• Error rate. The average equity lost per decision. GNU Backgammon shows it per move and per match; eXtreme Gammon summarizes it as a Performance Rating (PR), where a lower number is better.
• Luck rating. The total luck each player received, so you can see whether a loss came from the dice or from your moves.
• Luck-adjusted result. The actual result with the measured luck removed. Over a handful of matches, this is a far better guide to who played better than the raw score.

Limits to keep in mind

The method is only as good as the program's evaluations. Today's top programs are much stronger than human players, so their judgments are reliable in most positions, but they can still be slightly off in rare, complex ones. Analysis at deeper settings or with rollouts (playing a position out many times) reduces that error. Also remember that a single match is a small sample: even luck-adjusted numbers settle down only after many games.

How to use it yourself

Load your matches into an analysis program, look at the luck and error figures together, and then study the biggest errors first. That is the core of the study routine on our becoming a better player page. For the history of how programs became this strong, read computer backgammon.