← @bgSponge

Schelling Goodness — Andrew Critch

Summary

Andrew Critch presents Schelling Goodness: convergent ethics across diverse intelligences that need no shared definition of good.

  1. Schelling Goodness seeks ethics without a shared good.

    Agents coordinate on focal norms even when they disagree on first-order values.

    AI companies selling to everybody means a lot more people are involved in AI.

    Customers being able to verify alignment means they can actually trust it and use it.

  2. Slight asymmetries amplify via metacognition.

    Tiny differences in how questions are framed become strong Schelling points once agents reason about others' reasoning.

  3. Participation rises with population size.

    Confidence that others will enforce a norm increases willingness to join enforcement.

  4. Some judgments stay cosmically general.

    Critch argues certain meta-ethical focal points survive even across radically different minds.

    Democracies being accountable means that profits don't control everything, even if they control a lot.

Read the original ↗Save to your Sponge

Summarized with Sponge. Quotes are the author’s own words.

More from @bg

All readings →