Tabula
An arena where two language models play backgammon against each other, explaining every move, with Glicko-2 ratings and per-move performance metrics.
Details
- Started
- 28 Sep 2026
- Game
- Backgammon, full rules engine
- Ratings
- Glicko-2 with uncertainty
- Metrics
- Latency, tokens, notional cost, play quality
- Stack
- React, Vite, Node, SQLite
- Stage
- Prototype v0.1
- Last commit
- 29 Sep 2026
ConjectureLanguage models can play real backgammon well if the engine keeps the rules and they only choose the move.
FindingStill open.
The write-up
Pick a model for each side, press start, and watch two models play a real game on a realistic board. Each one gives a one-sentence reason for its move, typed out live in its panel.
The engine owns the rules
The most important design decision is that the model never makes a move directly. On each turn the engine rolls the dice and computes every legal play. The model is only asked to choose an index from that list and explain why. A model can therefore never make an illegal move however it formats its reply, which matters in a game as fiddly as backgammon.
The prompt shows the board from the model’s own side, with pip counts, the opponent’s blots and each legal play pre-scored for hits, resulting pip count and exposure. If a call fails, times out or cannot be parsed, the engine’s heuristic plays that move instead. It never hands the move to a different model, so no model is credited with another’s play.
Ratings, not just win counts
Finished games are stored in SQLite and every model earns a Glicko-2 rating with a ± uncertainty that shrinks as it plays. Beating a strong, well-established model counts for more than beating an uncertain or weaker one. A matchmaker suggests the most informative next pairing, so a growing roster gets rated fairly without every model playing every other.
More than who wins
Every decision records latency, tokens, a notional API cost, whether the model actually answered, and how often its pick matched the engine’s best play. Dice make results noisy, so the per-move quality signal builds up much faster than wins and losses do.