Limite 1B
Summary
Paradigma releases Limite 1B - Violetto, a 1B dense transformer trained from scratch on under 300B curated tokens for high-throughput competition math, with 131k context, a training-time value model, and a custom vLLM plugin.
Sample efficiency is the headline.
Architecture ideas from pre-training speedruns plus curated synthetic, SFT, and RL data let a 1B model rival much larger math specialists.
Limite is a 1-billion parameter dense autoregressive transformer, trained from scratch on a mixture of less than 300B highly curated tokens
Compute Pareto is the chart.
On AIME 2026 versus estimated FLOPs, Violetto reaches 94.01% at 1.71×10²¹ FLOPs, one to three orders below models that match its score.
Persona is optional.
Limite is deliberately lightly instruction-tuned for single-turn math rather than an assistant chat stack.
as lightly instruction-tuned as possible, to challenge the assumption that models need to be embedded in an assistant persona
In each case, Limite develops a substituted task rather than answering the question asked.
Scope drift is a known failure.
Ordinary prompts can be rewritten into mathematical tasks; the post documents photosynthesis, seasons, and metaphor examples.
Release includes serving pieces.
Weights, evaluations, a training value model, and a vLLM inference plugin ship together as the first of more science-oriented runs.
Read the original ↗Save to your Sponge
Summarized with Sponge. Quotes are the author’s own words.