your report

demo player

example: everything below is computed live from a simulated player's games, stored in the same format as real game logs. no real participant data is shown.
summary

1where you stand

your success rate on hard games next to every language model we tested and to the people who played this game in our study. llms are shown in their best condition (with extended reasoning when available).

figure 1 success rate: you, other people and llms

success on hard games, averaged over game lengths. the optimal agent's 68% is the ceiling for this game. "other people" are simulated in this example.

2gathering evidence vs using it

we split the gap between you and the optimal agent in two. sampling loss: how much worse a perfect reasoner would do with the evidence you chose to collect. inference loss: how much worse your final pick is than that perfect reasoner's, given the same evidence. most people lose more at inference; reasoning llms lose more at sampling.

figure 2 where your accuracy is lost
you other people llms optimal agent

sampling loss = optimal success − map success on your samples; inference loss = map success on your samples − your success. lower is better on both axes.

3how closely you follow the evidence

for every choice we know how likely the button you picked was to be the biased one. an optimal player always picks the most likely button, so this curve climbs as evidence accumulates.

figure 3 posterior of the button you chose
you people in our study optimal agent

x axis: rounds before the final choice, pooled over game lengths. sampling: the posterior of the button you sampled, before seeing its color.

4your behavioral model

we fit the same behavioral model we used for humans and llms in the paper to your choices. each parameter describes one way of departing from a perfect bayesian reasoner.

figure 4 your place in cognitive space
you other people llm llm, extended reasoning optimal agent

memory \(\beta\) against inference sharpness \(\kappa_f\). in this example "your" parameters are the ones used to simulate the demo player.

5your games over time

figure 5 success and response time across your games

top: success over a moving window of 20 games. bottom: median time per sampling round in each game.

example report · real reports live at /me for registered players with at least 100 hard games, computed from their own games.