← @bgSponge

Free the models: harness design at the frontier

Summary

Replit Agent uses a composable harness to let frontier models like GPT-6 Astra decide their own subagent tiers, effort levels, and delegation timing, achieving Pareto-efficiency on DeepSWE and Terminal-Bench.

  1. Routers are fundamentally limited.

    Heuristic or small-model routers are always less capable than the primary model they select for, whereas Replit Agent lets the core loop decide.

    a router will always be less capable than the model it’s choosing for

  2. Composable primitives enable autonomy.

    The harness provides specialists, effort levels (small to large), and the ability to return to briefed subagents to manage context and cost.

    the harness offers specialists: read-only explorers, browser testers, reviewers, and a design subagent

  3. Dynamic effort preserves cache.

    Because GPT-6 Astra supports mid-turn effort changes without breaking the cache, the system uses an escalation system to match difficulty to task trajectory.

    changing effort mid-turn preserves the cache on some models

  4. Replit Agent beats sidekicks.

    On Terminal-Bench 4.0, Replit Agent scores 49% at $2.53, outperforming the sidekick architecture's 33% at $1.84.

    Replit Agent beats the sidekick architecture on both benchmarks, by 11 and 16 points

  5. Scaffolding follows the bitter lesson.

    Baking human knowledge into agents plateaus; letting models discover execution through computation scales better with model releases.

    Baking human knowledge into an agent helps in the short term, plateaus in the long run

Highlights

  • We’ve observed them naturally delegate to less costly subagents, keeping their own tokens for the decisions that need them.

  • No matter if based on advanced heuristics or a small model that reads each turn and picks which LLM to use, a router will always

  • Given that freedom, GPT-6 Astra hands routine implementation to less costly subagents and decides for itself where its tokens are worth spending.

  • Baking human knowledge into an agent helps in the short term, plateaus in the long run, and is eventually overtaken by general methods that scale

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →