← @bgSponge

tokens too cheap to meter

Summary

jyn argues AI token and task costs are falling by orders of magnitude through GPU efficiency, cheaper-per-task model frontiers, inference-engine gains, MoE and Mamba architectures, and specialized classifiers like Jev.

  1. Cost-per-task is collapsing even when frontier price-per-token is not.

    Smaller models may burn more tokens; the pareto frontier still moves left and up across 2025 into 2026.

    Extraordinary claims require extraordinary evidence, so I collected a whole bunch of evidence.

  2. Hardware, engines, and architectures stack.

    GPU efficiency roughly doubles every two years; serving stacks gain tens of percent yearly; MoE and Mamba cut compute or RAM for comparable quality.

  3. Specialized classifiers unlock another two orders.

    Jev-style yes/no models price input near $42 per billion tokens with free outputs, cheap enough for pipe tools like jgrep.

  4. When tokens undercut tools, models move into infrastructure.

    Adaptive schedulers and other once-expert systems become embeddable with general models.

  5. Cheap volume does not erase quality rents.

    Frontier labs can still monetize hard tasks while open weights and discounts absorb ordinary work under Jevons-driven demand.

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →