← @bgSponge

Programmable event loops for our distributed agent harness

Summary

Heyang Zhou shows that Comma's Salix harness uses agent-written eBPF programs, called Loops, to create an efficient outer loop that filters events before waking expensive large models.

  1. The inner loop fails at waiting.

    Standard agent loops must either poll or use cron schedules, both of which waste expensive model tokens on quiet ticks.

    That loop is good at doing work. It is bad at waiting for work.

  2. Loops use eBPF for density.

    By compiling agent-written C to eBPF, Salix runs millions of agents at over 100 agents per CPU core using minimal memory.

    An agent can't have a sandbox that stays on, so its watches can't either.

  3. Spinfoam ensures memory safety.

    The userspace runtime uses pointer cages, randomized guard pages, and machine-checked Lean proofs to isolate untrusted guest code.

    Each program's data sits inside a pointer cage

    The generated code is memory-safe.

  4. Small models filter expensive events.

    Loops call Jev to make structured decisions like 'notify' or 'defer', ensuring the large model only processes meaningful changes.

    a small model screens every event, and the large model only sees the ones that pass.

  5. Deduplication happens at admission.

    Salix deduplicates event IDs when they enter the mailbox, meaning retries must be managed by the guest Loop itself.

    spinfoam's mailbox deduplicates event IDs at admission, not at business completion.

Highlights

  • What an always-on agent actually needs is an outer loop: something that sits on the event stream, does the cheap filtering, and only escalates to

  • Running thousands of agent-written programs in one process is only reasonable if none of them can touch memory that isn't theirs.

  • A Loop is a single C file, written by the agent, compiled to eBPF, and run on the cluster next to the agent that owns

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →