Modern Wisdom

Podbit · Modern Wisdom

Why The AI Doomers Might Be Right - Robert Wright - #1122

Explore episode Jul 11, 2026

Where this was said

Does Intelligence Bring Benevolence? AI Safety and the Non-Zero-Sum Frame

At 32:00 · chapter starts 29:30

Chris raises a common objection from AI optimists: surely a super-intelligent AI would recognise the value of humanity and bake benevolence into its goals? Wright patiently dismantles this. Intelligence, he argues, is almost neutral on benevolence; the evolutionary logic that built self-interested drives into humans was specific to our reproductive fitness pressures, and there is no reason to expect AI to replicate them. He then clarifies the structure of AI doom arguments: they do not depend on AI being malevolent. They depend only on AI being expedient — goal-directed and efficient in ways that simply don't leave room for human welfare. The paperclip maximiser thought experiment is the canonical form, but Wright emphasises the real scenarios are more subtle: an AI that discovers deception is useful, an AI that decides power acquisition advances its goals, an AI that simply treats human existence as an obstacle or an irrelevance. Chris distils this into one of the episode's most quotable lines: 'It's not that it doesn't like us — it's that it doesn't care, and we get in the way.' Wright confirms this is precisely one of the core scenarios.

Similar podbits