Quote · Hard Fork
‘Hard Fork’ Live, Part 3: Differing Visions of an A.I. Future
Where this was said
Recursive Self-Improvement: History, Possibility, and Disagreement
At 12:46 · chapter starts 11:11
Sayash Kapoor offers a surprising reframe: recursive self-improvement isn't a future AI event but the entire history of computing, from assembly language to compilers to modern frameworks [1] — Sayash Kapoor "Recursive self-improvement started ~60 years ago: Sayash Kapoor argues the recursive self-improvement loop began six decades ago, with ever…" 09:59 . The disagreement, he argues, is whether this process will naturally terminate at human-level AI or continue to artificial superintelligence. Kokotajlo agrees that RSI has been ongoing, but maintains that the remaining barriers to ASI don't look as strong as Kapoor suggests. In a remarkably candid moment, Kokotajlo admits he'd breathe a sigh of relief if Kapoor turned out to be right — a moment of human vulnerability from the man predicting potential runaway AI.
Despite their famous disagreement on AI timelines, Daniel Kokotajlo and Sayash Kapoor co-authored a blog post outlining their shared ground. Both agree that AI systems short of 'humans in the cloud' are normal technologies, and both are alarmed by companies fine-tuning models to deceive users.
