your report
demo player
1where you stand
your success rate on hard games next to every language model we tested and to the people who played this game in our study. llms are shown in their best condition (with extended reasoning when available).
success on hard games, averaged over game lengths. the optimal agent's 68% is the ceiling for this game. "other people" are simulated in this example.
2gathering evidence vs using it
we split the gap between you and the optimal agent in two. sampling loss: how much worse a perfect reasoner would do with the evidence you chose to collect. inference loss: how much worse your final pick is than that perfect reasoner's, given the same evidence. most people lose more at inference; reasoning llms lose more at sampling.
sampling loss = optimal success − map success on your samples; inference loss = map success on your samples − your success. lower is better on both axes.
3how closely you follow the evidence
for every choice we know how likely the button you picked was to be the biased one. an optimal player always picks the most likely button, so this curve climbs as evidence accumulates.
x axis: rounds before the final choice, pooled over game lengths. sampling: the posterior of the button you sampled, before seeing its color.
4your behavioral model
we fit the same behavioral model we used for humans and llms in the paper to your choices. each parameter describes one way of departing from a perfect bayesian reasoner.
your choices are being fitted with the paper's model. this takes a few minutes and runs in the background; this section fills in on its own once it's done.
memory \(\beta\) against inference sharpness \(\kappa_f\). in this example "your" parameters are the ones used to simulate the demo player.
5your games over time
top: success over a moving window of 20 games. bottom: median time per sampling round in each game.
example report · real reports live at /me for registered players with at least 100 hard games, computed from their own games.