Speaker
Dwarkesh Patel
Appearances over time
6 episodes
Episodes
6
The next big breakthrough will be AIs learning on the job
The data black hole at the center of AI
‘Hard Fork’ Live, Part 3: Differing Visions of an A.I. Future
Ada Palmer – Machiavelli is the most misunderstood thinker of all time
Alex Imas and Phil Trammell – What remains scarce after AGI?
Eric Jang – Building AlphaGo from scratch
Podcasts
Quotes & moments
Frontier AI models are trained on tens to hundreds of trillions of tokens, versus roughly 200 million tokens a human sees from birth to adulthood — nearly a million-fold difference.
Despite every wave of automation since the Industrial Revolution, labor's share of GDP has remained above 60%, which economists call a Kaldor fact — almost suspiciously stable.
Labs spend 30 to 50% of their total compute on inference, yet this compute currently plays no productive role in improving the model.
A teenager can learn to drive a car with about 20 hours of practice, yet self-driving car models from Waymo and Tesla require 3–4 orders of magnitude more data.
Current AI models are one-millionth as sample efficient as humans during training, a deficit Dwarkesh previously explored in depth.
People often say employees aren't net productive until 6+ months on the job, illustrating why continual learning matters for competence.
Humans can learn to teleoperate any humanoid or robot arm within hours, but AI systems require millions of hours of demonstrations and still can't perform complex open-ended tasks.
Despite massive improvements in compute technology and supply, H100 GPU rental prices are higher than 3 years ago because smarter AI models raise the opportunity cost of compute.
AlphaGo Zero was trained on far more compute than any other AI model of its era — roughly 3e23 flops, comparable in order of magnitude to frontier LLMs.
Epoch AI reported that open models lag state-of-the-art frontier models by only 4 months, which Dwarkesh attributes to data being the real driver — easily distilled from public APIs.
The CursorTab model online-learns by predicting which edits users accept, processing over 400 million requests per day.
The human genome is only 3 gigabytes and only 1–2% is protein-coding, which Dwarkesh argues is not enough space to store pre-trained neural network weights from evolution.
On-policy self-distillation provides per-token supervision rather than a single end-of-trajectory reward, making it a much denser training signal than naive RL.
RL training changes very few model parameters per step, concentrating updates only on what is relevant to achieving the outcome — a crucial property for continual learning.
The human brain has approximately 100 trillion synapses, while frontier AI models currently have around 5 trillion parameters — a 20x gap.
Ricardo correctly predicted that industrial-era jobs would be automated. He completely failed to predict that new jobs would replace them, and that prime-age employment in 2026 would be near an all-time high. The lump-of-labor fallacy has fooled experts for two centuries — and may be fooling them again.
Moore's Law isn't just about supply — it's about demand evaporation. We create new uses for compute so fast that the value of each unit halves every 18 months. For the first time, AI may have broken this: H100 rental prices are higher now than three years ago despite massive supply expansion.
A Mongolian economist in 1400 predicting scarcity would have assumed people satiate in horses and yogurt and concentrate spending on singers. They would have been wrong — because wealth expansion always generates new varieties. The same failure awaits anyone predicting AGI scarcity by holding varieties fixed.
For AI to automate white-collar work without generating redistribution wealth, automation would have to be cheap enough to replace workers but not cheap enough to produce abundance. That's an implausibly narrow window — and history suggests technological frontiers always expand alongside job displacement.
For AGI to cause negative economic growth, rich capital owners would have to stop investing entirely — not just consume less, but refuse to build more data centers even as the technological frontier expands. Abundance causing recession requires conditions that have never existed in history.
O-ring theory explains why automating 9 out of 10 job tasks sometimes lowers output quality enough to make automation counterproductive. But this friction works symmetrically: once AIs are reliable enough, the same logic will make humans — who slow down AI production flows — impossible to integrate.
If even a small number of agents — whether AI firms, humans like Elon Musk, or von Neumann probes — have effectively unlimited demand for capital accumulation, they will compound faster than everyone else and gradually dominate the economy's preference structure. Labor share could go to zero through selection alone.
If AGI diffuses like electricity, every company captures it and index ownership works. If it diffuses like social media, platform rents stay concentrated and ordinary people miss the gains. Which model prevails depends largely on whether open models stay competitive and whether frontier labs go public.
The most robust strategy for developing countries facing AGI is to index into the returns — buy sovereign wealth exposure to the AI supply chain — rather than rely on retraining programs. If AGI diffuses like electricity, index ownership captures the gains. If it concentrates, retraining won't save you anyway.
In an incentive-compatible experiment, people paid significantly more for human-made art prints than AI-made ones — but the human premium collapsed when 500 copies existed, while AI art was already treated as a commodity with no such drop. This is the kind of data needed to validate the entire relational sector hypothesis.
UBI makes basic needs contingent on who holds political power — a dangerous dependency. Universal Basic Capital gives people ownership stakes with property rights, making them normal shareholders rather than welfare recipients. The tradeoff is targeting: which stocks do you put in people's portfolios?
Naive RL reinforces entire winning trajectories — but most of those moves were irrelevant. MCTS gives you a strictly better action label for every single move in every game. That's why AlphaGo learns far faster than an LLM-style reinforce loop.
A 19x19 Go board generates a game tree larger than atoms in the universe — 361 to the power of 300. This isn't just big, it's a fundamentally different class of problem from chess, which is why computer scientists thought Go was unsolvable this century.
MCTS doesn't build the whole tree — it builds it interactively while searching. Selection, expansion, evaluation, backup: these four steps run hundreds or thousands of times per move, concentrating compute on the branches that matter.
Scarcity after AGI will concentrate in goods where human involvement is intrinsically part of the value — not just because humans are capable, but because consumers actively prefer the human to be in the loop. The hypothesis only holds if willingness-to-pay survives replacement, and we currently lack the data to know.
Analysis
What they talk about
- Technology 66%
- Business 17%
- Science 10%
- Education 4%
- History 3%
Connections
Shows they appear on and people they share episodes with. Drag to explore.
