goal — models with a coherent intelligence: one that generalizes to genuinely unseen tasks

today — models only perform well on complex tasks that are close to the environments they were trained on

plan — train models on tasks built to reward real, transferable skills, beyond specific benchmarks

cognitive training — a scalable and principled way to make models produce high-quality RL environments for themselves

paradigm — a model improves on the tasks it discovers for itself, and gets better at discovering them as it goes

despite immense progress, models are very uneven: brilliant on the tasks they were trained on, unreliable on unseen ones

training is generally very costly — in both compute and human effort

and a training process made of numerous hand-made environments makes it hard to trust how a model will handle an unknown challenge

cognitive training takes a different stance: that AGI should follow from simple, sound principles

cognitive training is a self-improvement loop: a model grows its capabilities by creating challenges it then learns to tackle

at each step, the model creates a new task for itself to train on

the value of a task is measured by its transfer: how well it captures existing skills while opening up new ones

concretely, it runs on a specific space of tasks — the xent games — which combine concreteness, generality, and training performance

the value of each xent game is set by a specific meta-objective, identified through gauge-symmetry arguments

key idea: many questions that are purely functions of a model cannot be answered by the model itself

though it is probabilistic, a model does not know its own probability measure

it may not even know which prompt would lead it to a given answer

a most striking example of implicit knowledge is counterfactual thinking

the xent games are built from this idea: have a model play games about its own measures

a space of tasks engineered to hit a sweet spot:

rich enough to cover an immense space of learnable tasks

structured enough to keep the cognitive training loop tractable

expressible in a streamlined language, which makes their optimization a well-posed task

they extend basic tasks — prediction, completion, in-filling, reverse-prompting, jailbreaking

and reach sophisticated ones such as counterfactual thinking

they measurably improve models' reasoning across benchmarks

and they are amenable to new RL algorithms — see frost — beyond the Monte-Carlo paradigm

imagine an experiment with two copies of yourself: one receives a piece of information, the other does not

comparing how the two fare across situations gives you a measure of that information's value

humans can't run this experiment — life is lived once; for a language model it is trivial

similarly, given a document, we can ask: is there a piece of information that would change one's view of it?

surprise, logical consistency, and novelty can all be phrased through counterfactual thinking

these thought experiments are prime examples of xent games a model can learn to play to become more capable

the value of a xent game is computed by a meta-objective, in terms of transfer value:

externally — how much the game transfers to games and tasks we already care about

internally — how much it adds that those tasks don't already capture

using gauge-symmetry arguments, a unique formula for the meta-objective emerges — the only logically consistent way to combine transfer values into a measure of a game's usefulness

so it is natural to postulate that the path to AGI is to optimize that meta-objective

cognitive training is the ultimate game: the game of creating valuable games

each move in that game is the choice of a xent game of maximal possible value

the meta-objective measures the value of a game in terms of transfer learning

by iterating the loop, models keep discovering new valuable skills

and thanks to the depth of the implicit knowledge substrate, they keep growing — with no end in sight

frost is a new paradigm that lets models learn to play xent games faster than other methods

the idea: exploit the differentiable structure of xent games to learn faster than Monte-Carlo methods

for instance, models trained with Frost-GRPO learn to play moves they would once have assigned a very low probability

cognitive training focuses on a large, but specific space of tasks: the xent games

as a result, the process is a well-circumscribed optimization task

during the process, the model's abilities are transparently evaluated — and that evaluation is the basis for selecting the next xent game

models only ever play games of finite length to get generally better; they are never over-optimized on any single skill

the meta-objective stays fixed: capabilities keep growing and new skills appear, all within a closed framework

we deliberately avoid the open-ended process where an agent rewrites its own next version — and with it, the risk of alignment drift

Clément Hongler — website linkedin

Arthur Renard — website linkedin

Valentin Hartmann — linkedin

Franck Gabriel — website linkedin

contact — write to hello@xent.io

cross-entropy games — “Cross-Entropy Games for Language Models”

cognitive training — “Cognitive Training for Language Models”

frost training — “Cross-Entropy Games and Frost Training”

notes — Clément's notes on xent games at xent.blog