P(doom)
Summary
Armin Ronacher responds to Dario Amodei’s “pace the frontier” essay and viral P(doom) talk by agreeing on many risks—botnets, blind labs, regulatory failure—while opposing closed-lab pacing as the cure.
Doom as nuisance and human damage, not nukes.
Persistent botnets and what closed models do to ordinary people matter more to him than bioterror or culture-war China scenarios.
I almost exclusively worry about what this does to us as humans.
A powerful technology that is out there for everyone to use comes with built-in pacing.
Only two labs are being asked to pace.
Anthropic and OpenAI dominate; METR-tied evaluators and China-gap rhetoric look like concentrating control after training on the commons.
From where I sit, what we observe right now is a total regulatory failure everywhere.
Open weights pace by proliferation.
Mass access on more equal terms would cut the distorted economics that let closed labs burn millions and ignore crimes.
If it were not for Chinese labs distilling American models, we would be in a pretty awful situation right now
Regulation is already failing.
EU rules miss live problems; US turbo-capitalism plus sinophobia leaves scraping, opaque token markets, and training-data disputes uncontrolled.
Self-improvement may tax society first.
Even without extinction, recursive gains raise costs for software, research, and anyone forced to buy closed tokens to keep up.
Read the original ↗Save to your Sponge
Summarized with Sponge. Quotes are the author’s own words.