← @bgSponge

Self-Play Pretraining with Zero Data

Summary

Cowsik, Dolev, Li, and coauthors show self-play over a universal Turing machine can pretrain with zero natural data and predictable zero-shot scaling.

  1. Two models co-evolve from random weights.

    A generator proposes Brainfuck-like programs for a universal Turing machine; a learner trains on the emitted byte sequences by next-token prediction.

  2. Learning-progress reward beats raw difficulty.

    The generator is reinforced when learner gradients align with recent parameter movement, avoiding programs that are hard only because they inject noise.

  3. Zero-shot loss scales as a power law in compute.

    Across text, images, audio, music, DNA, and code, compute-optimal frontiers improve without any natural-data gradient steps.

    Across several natural datasets, zero-shot loss exhibits predictable scaling in compute.

  4. Adaptive search beats a fixed universal prior.

    Sampling the same program space without adapting the curriculum scales much more slowly; self-play also discovers Fibonacci, geometric, and polynomial sequences far earlier.

    access to a universal program space alone is not enough—self-play must learn where in that space to allocate training compute.

  5. Universal structure is not contingent knowledge.

    The authors treat the method as generating transferable predictive regularities and as a warm start that speeds later natural-data pretraining, not as a replacement for world-specific facts.

    we do not view universal pretraining as a replacement for natural data

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →