← @bgSponge

Laya — 33ms Multilingual System 1 Decision Engine

Summary

Convai Innovations ships Laya, an open Apache-2.0 family of non-autoregressive System 1 decision models that answer choice, score, and boolean questions over text or JSON in one forward pass (~33 ms on GPU).

  1. Reflex decisions do not need chatbots.

    Ticket routing, spam, jailbreak checks, and ordinal urgency are structured System 1 jobs; streaming LLMs add latency, cost, and fake confidence text.

    not every AI problem requires an autoregressive chatbot

  2. Three primitives, one pass.

    Choice, score, and noul return calibrated probabilities and numbers only, so the model never emits free text and cannot emit malformed schema JSON.

    the model's own confidence gives no warning when it cannot read the input script

    schema violations or malformed JSON are physically impossible

  3. Route before you trust confidence.

    English ModernBERT shreds non-Latin scripts while reporting ~0.9 confidence; script and stopword routing (sub-millisecond) picks the multilingual checkpoint first.

  4. Open weights beat a closed decision API on speed and cost.

    Routed Laya reports ~33 ms P50, ECE 0.081, and $0 self-host versus Jev's ~250 ms and metered API pricing on the team's head-to-heads.

  5. Ceilings are architectural.

    Choice quality falls past ~20 options, out-of-box zero-shot is near chance until fine-tuned, and temperature calibration on your domain is required for honest ECE.

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →