What happened
Researchers introduced FM-Bench, a benchmark in which language-model agents manage a football club through 20 in-game years. The authors report that the benchmark evaluates 15 frontier models in solo and shared-world competitions, with a deterministic engine producing the final scores rather than an AI judge or human rater.
The supplied primary source is an arXiv page for a paper submitted on Aug. 19, 2026. The authors describe FM-Bench, or Football Management Benchmark, as an evaluation of long-horizon management by language-model agents. Each agent runs a club for 20 in-game years and makes roughly 340 to 400 decisions through 26 tools. The task includes drafting a squad, trading players, negotiating contracts, investing in facilities and youth development, setting lineups, and responding to a board that can dismiss the manager.
The benchmark has two reported tracks. In the solo track, each of 15 frontier models plays against a frozen scripted world. In the Arena, the same models and a scripted anchor compete in one shared 20-year world. The source says all models completed every horizon across three seeds, while blind scripted baselines failed to complete most of theirs. A deterministic game engine accumulates decisions into one final score, so the evaluation does not rely on an LLM judge or a human rater.
The authors report that claude-fable-5 ranked first on the solo leaderboard by mean score and also led the Arena, although the title rotated among ten models. They say rankings settled only late in the simulated horizon, and that the best first-play human participant placed at the bottom of the model leaderboard. The paper identifies six behavioral capabilities behind the overall score. According to the abstract, stronger performers reduced slow-payoff investment near the end, kept cash invested instead of idle, and began contract renewals well before deadlines.
The source also reports several weaknesses. The tested models did not learn hidden market prices despite hundreds of rejected bids. Self-managed memory failed in two opposing ways: some systems accumulated an ever-growing archive, while others rewrote their plans every season. The supplied source does not give the individual scores, full model list, game-engine details, prompt and tool specifications, or the statistical spread across seeds. It identifies the paper as an arXiv preprint; the supplied page does not establish independent peer review.
Read the primary source: arxiv.org ↗
Why it matters
The study targets a gap in current agent evaluations: whether systems can maintain effective decision-making across hundreds of linked choices. Its results suggest that long-horizon performance may depend more on managerial behavior—such as timing investments, managing cash, and renewing contracts—than on model scale, price, or token use. The evidence remains limited to the benchmark's simulated football environment and the authors' reported experiments.
FM-Bench addresses a practical limitation in many agent tests: bounded tasks can be completed one at a time, but a long sequence of choices can make early mistakes expensive and later decisions dependent on earlier ones. A 20-year simulation with hundreds of decision points creates a way to inspect those cumulative effects. The source's central claim is not that agents can manage real clubs, but that their ability to sustain coherent strategy can be measured more directly than in short, isolated tasks.
The reported results point toward behavior as an important source of performance differences. The authors say neither model scale, price, vendor, nor token spending predicted the ordering, while choices about investment timing, cash use, and contract renewals did. If reproduced, that finding would complicate the assumption that more computation or more generated text automatically produces better long-horizon control. It could also encourage developers and evaluators to measure planning discipline, deadline management, resource allocation, and memory use separately from a single end score.
The benchmark's strongest public value is its evaluation design. A deterministic engine can make outcomes more repeatable than subjective judging, and the shared Arena makes agents interact with the same evolving conditions rather than playing entirely separate games. Still, the evidence is bounded. Football management has a particular set of rules, incentives, and abstractions, and success in that environment does not independently demonstrate reliability in healthcare, finance, administration, or other high-consequence work. The supplied material contains no independent replication or real-world deployment evidence.
What to watch next
The main questions are whether FM-Bench can be independently reproduced, whether its rankings persist under different seeds and environments, and whether the measured behaviors transfer to real work. The supplied source does not provide detailed scores, model configurations, scoring rules, or evidence that the findings have been peer reviewed.
Reproducibility is the first test. The source says code is available, but the supplied text does not provide a repository, benchmark license, or enough implementation detail to reproduce the result. Readers should look for the complete scoring function, environment rules, tool definitions, prompts, model versions, sampling settings, seed-level results, and exact Arena protocol. Those details matter because small changes to the simulated market, board behavior, or budget constraints could alter the ranking.
The model leaderboard should also be treated as provisional. The abstract names claude-fable-5 as the top model but does not identify all 15 systems, report score margins, or explain how models were selected and accessed. It is therefore unknown whether the ordering reflects durable capabilities, prompt engineering, tool-use differences, model availability at the time of testing, or random variation. The claim that the title rotated among ten models suggests meaningful instability, but the supplied source does not quantify that instability.
A broader research question is whether the six reported behaviors generalize beyond football. Follow-up tests could vary the game, introduce different market structures, add adversarial or unexpected events, compare memory systems under controlled conditions, and use more human participants. The source also leaves open whether models can learn hidden prices with better feedback, whether end-of-horizon strategies are rational under the benchmark's incentives, and how much the scripted world shapes the outcome. Until those questions are answered, FM-Bench is evidence about a demanding simulation, not proof of general-purpose managerial competence.
