Self-Play Pretraining with Zero Data
Summary
Cowsik, Dolev, Li, and coauthors show self-play over a universal Turing machine can pretrain with zero natural data and predictable zero-shot scaling.
Two models co-evolve from random weights.
A generator proposes Brainfuck-like programs for a universal Turing machine; a learner trains on the emitted byte sequences by next-token prediction.
Learning-progress reward beats raw difficulty.
The generator is reinforced when learner gradients align with recent parameter movement, avoiding programs that are hard only because they inject noise.
Zero-shot loss scales as a power law in compute.
Across text, images, audio, music, DNA, and code, compute-optimal frontiers improve without any natural-data gradient steps.
Across several natural datasets, zero-shot loss exhibits predictable scaling in compute.
Adaptive search beats a fixed universal prior.
Sampling the same program space without adapting the curriculum scales much more slowly; self-play also discovers Fibonacci, geometric, and polynomial sequences far earlier.
access to a universal program space alone is not enough—self-play must learn where in that space to allocate training compute.
Universal structure is not contingent knowledge.
The authors treat the method as generating transferable predictive regularities and as a warm start that speeds later natural-data pretraining, not as a replacement for world-specific facts.
we do not view universal pretraining as a replacement for natural data
Read the original ↗Save to your Sponge
Summarized with Sponge. Quotes are the author’s own words.