← @bgSponge

Limite 1B

Summary

Paradigma releases Limite 1B - Violetto, a 1B dense transformer trained from scratch on under 300B curated tokens for high-throughput competition math, with 131k context, a training-time value model, and a custom vLLM plugin.

  1. Sample efficiency is the headline.

    Architecture ideas from pre-training speedruns plus curated synthetic, SFT, and RL data let a 1B model rival much larger math specialists.

    Limite is a 1-billion parameter dense autoregressive transformer, trained from scratch on a mixture of less than 300B highly curated tokens

  2. Compute Pareto is the chart.

    On AIME 2026 versus estimated FLOPs, Violetto reaches 94.01% at 1.71×10²¹ FLOPs, one to three orders below models that match its score.

  3. Persona is optional.

    Limite is deliberately lightly instruction-tuned for single-turn math rather than an assistant chat stack.

    as lightly instruction-tuned as possible, to challenge the assumption that models need to be embedded in an assistant persona

    In each case, Limite develops a substituted task rather than answering the question asked.

  4. Scope drift is a known failure.

    Ordinary prompts can be rewritten into mathematical tasks; the post documents photosynthesis, seasons, and metaphor examples.

  5. Release includes serving pieces.

    Weights, evaluations, a training value model, and a vLLM inference plugin ship together as the first of more science-oriented runs.

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →