← @bgSponge

EA Safety

Summary

Venkatesh Rao argues Effective Altruism has become an "EA formation"—a philanthropy-to-AI-safety complex whose abstract, low-empiricism moral machinery plus urge to act now threatens to be a second-order problem of its own.

  1. Name the formation, not the membership card.

    EA now spans GiveWell-style philanthropy, Singerite life-optimization, rationalism, longtermism, x-risk, and frontier-lab governance; porous borders make card-checks scholastic.

    My hypothesis is that these developments have a common cause.

  2. Friends with footnotes.

    Rao keeps personal friendship while refusing public fealty or obligatory debate—until the formation's power over AI moral framing forces a harder look.

    Friends with footnotes are great to have.

  3. Uncertainty inside the model, not about it.

    The style numbers doubt within a frame while insulating premises about value, futures, evidence, and who prioritizes.

  4. AI as eschatology.

    Extinction, transcendence, and a narrow window turn careers into vocation and alignment into a salvation genre dressed as secular decision theory.

  5. Now there are two problems.

    EA grew by idiosyncratically posing AI safety; success creates "EA safety"—how to constrain a formation sure it alone should govern the first.

    AI supplies the formation not merely with a problem but with an eschatology

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →