A teenager learns to drive in 20 hours; Waymo needs millions of hours of data — AI is up to a million times less sample-efficient than humans, and scaling alone can't close that gap.
Dwarkesh Podcast
A teenager learns to drive in 20 hours; Waymo needs millions of hours of data — AI is up to a million times less sample-efficient than humans, and scaling alone can't close that gap.
TL;DR
Dwarkesh Patel argues that the dominant driver of AI progress is not architectural innovation or better training tricks, but raw data — and that current AI models are orders of magnitude less sample-efficient than humans [1] — Dwarkesh Patel "AI capabilities look like a galaxy of stars, but at the center is an invisible black hole of data. The main way AIs have gotten better is n…" . A teenager learns to drive in 20 hours; Waymo needs millions of hours of data [2] — Dwarkesh Patel "Open models lag frontier by 4 months: Epoch AI reported that open models lag state-of-the-art frontier models by only 4 months, which Dwark…" 02:36 . He addresses common objections (evolution pre-training, multimodal data, scaling laws) and explains why this still matters for automating white-collar work and AI research itself [3] — Dwarkesh Patel "AIs can be wildly data-inefficient and still automate white-collar work, because common tasks are common enough to bring into the training …" 07:48 . The key takeaway: AIs can afford to be wildly data-inefficient because training costs are amortized across billions of sessions.
Dwarkesh Patel examines what is really driving AI progress, arguing that data volume and quality — not algorithms or architecture — is the central force. He compares human and AI sample efficiency through concrete examples, debunks common objections, and explores whether this gap matters for automating white-collar work and AI research.
Dwarkesh argues that AI's main progress driver is data volume and quality, not algorithmic breakthroughs. RL is framed as synthetic data generation. Expert data labeling is described as a billion-dollar industry. [1] — Dwarkesh Patel "AI capabilities look like a galaxy of stars, but at the center is an invisible black hole of data. The main way AIs have gotten better is n…"
Three objections addressed: (1) evolution pre-training humans — debunked via genome size; (2) multimodal sensory data — debunked via blind/deaf intelligence; and (3) scaling up models — addressed next. [1] — Dwarkesh Patel "The common objection — that billions of years of evolution pre-trained humans, making data comparisons unfair — doesn't hold up. The human …" 04:43 [2] — Dwarkesh Patel "If multimodal sensory data were the secret ingredient behind human intelligence, blind and deaf people would lack general intelligence. The…" 05:48
Dwarkesh argues inefficiency doesn't block white-collar automation because AI's training costs amortize across billions of sessions. [1] — Dwarkesh Patel "AIs can be wildly data-inefficient and still automate white-collar work, because common tasks are common enough to bring into the training …" 07:48 He bets on more human software engineers in 2027, and previews a future post on the intelligence explosion. [2] — Dwarkesh Patel "Current discourse on intelligence explosions is stuck between two bad takes: it's impossible, or a god emerges at the end. Neither is right…" 10:17
Chapter 1 · 00:00
Dwarkesh argues that AI's main progress driver is data volume and quality, not algorithmic breakthroughs. RL is framed as synthetic data generation. Expert data labeling is described as a billion-dollar industry. [1] — Dwarkesh Patel "AI capabilities look like a galaxy of stars, but at the center is an invisible black hole of data. The main way AIs have gotten better is n…"
AI capabilities look like a galaxy of stars, but at the center is an invisible black hole of data. The main way AIs have gotten better is not through better architectures or training tricks — it's by adding more and better data and scaling compute to generate it.
The industry producing expert labels and RL training environments is already earning billions per year in revenue, soon to be tens of billions.
AI models aren't learning the way humans do. They're more like Frankenstein's monster — stitched together from a billion carefully constructed example graphs. That's the uncomfortable truth behind what looks like fluid, general intelligence.
With GRPO, AI models generate hundreds to thousands of rollouts per task to solve the credit assignment problem, far more than the one or two times a human student might practice a problem.
Open models trail frontier models by only 4 months. The reason is that data — the actual driver of progress — can be distilled from public APIs. Hyperparameters and training tricks can't be. If architecture were the real edge, the gap would be far larger.
Epoch AI reported that open models lag state-of-the-art frontier models by only 4 months, which Dwarkesh attributes to data being the real driver — easily distilled from public APIs.
Chapter 2 · 03:11
Three objections addressed: (1) evolution pre-training humans — debunked via genome size; (2) multimodal sensory data — debunked via blind/deaf intelligence; and (3) scaling up models — addressed next. [1] — Dwarkesh Patel "The common objection — that billions of years of evolution pre-trained humans, making data comparisons unfair — doesn't hold up. The human …" 04:43 [2] — Dwarkesh Patel "If multimodal sensory data were the secret ingredient behind human intelligence, blind and deaf people would lack general intelligence. The…" 05:48
Frontier AI models are trained on tens to hundreds of trillions of tokens. A human sees about 200 million from birth to adulthood. That's close to a million-fold difference — and it reveals just how data-hungry these systems really are.
Frontier AI models are trained on tens to hundreds of trillions of tokens, versus roughly 200 million tokens a human sees from birth to adulthood — nearly a million-fold difference.
Humans can learn to teleoperate any humanoid or robot arm within hours, but AI systems require millions of hours of demonstrations and still can't perform complex open-ended tasks.
A teenager learns to drive in 20 hours. Even accounting for 16 years of growing up and building physical intuition, that's still 3–4 orders of magnitude less data than Waymo and Tesla use to train self-driving cars. This gap is the sample efficiency problem in concrete terms.
A teenager can learn to drive a car with about 20 hours of practice, yet self-driving car models from Waymo and Tesla require 3–4 orders of magnitude more data.
The common objection — that billions of years of evolution pre-trained humans, making data comparisons unfair — doesn't hold up. The human genome is only 3 GB, and 1–2% is protein-coding. That's nowhere near enough to store pre-trained neural network weights. Evolution found the right hyperparameters; it didn't train the weights.
The human genome is only 3 gigabytes and only 1–2% is protein-coding, which Dwarkesh argues is not enough space to store pre-trained neural network weights from evolution.
If multimodal sensory data were the secret ingredient behind human intelligence, blind and deaf people would lack general intelligence. They don't. This suggests billions of sensory tokens aren't the key — and may mean the human-AI data gap is even larger than estimated.
Deaf people who rely on sign language and reading likely consume far fewer than 200 million language tokens in a lifetime, yet still have general intelligence — suggesting sensory data volume isn't what makes humans smart.
The human brain has approximately 100 trillion synapses, while frontier AI models currently have around 5 trillion parameters — a 20x gap.
The Chinchilla scaling law shows that parameters and data contribute independently to loss. Even with infinite parameters, you'd only reduce data needs by 10x. But humans are thousands to millions of times more sample-efficient. Scaling current models is simply not the answer.
Using the Chinchilla scaling law constants, even an infinite number of model parameters would only decrease the required training data by a factor of 10 — while humans are thousands to millions of times more sample-efficient.
Humans are somewhere between thousands to millions of times more sample-efficient than current AI models, a gap that scaling model size cannot bridge.
AIs can be wildly data-inefficient and still automate white-collar work, because common tasks are common enough to bring into the training distribution. And unlike a human who can only work one job at a time, AI amortizes its training across billions of simultaneous sessions.
Chapter 3 · 08:46
Dwarkesh argues inefficiency doesn't block white-collar automation because AI's training costs amortize across billions of sessions. [1] — Dwarkesh Patel "AIs can be wildly data-inefficient and still automate white-collar work, because common tasks are common enough to bring into the training …" 07:48 He bets on more human software engineers in 2027, and previews a future post on the intelligence explosion. [2] — Dwarkesh Patel "Current discourse on intelligence explosions is stuck between two bad takes: it's impossible, or a god emerges at the end. Neither is right…" 10:17
Software engineering is supposed to be the first job AI takes. But Dwarkesh bets there will be more demand for human software engineers in 2027 than today — because AI is acting as a complementary input that expands the overall market for software.
Dwarkesh bets there will be overall more demand for human software engineers in 2027 than today, largely due to AI acting as a complementary input rather than a replacement.
Current discourse on intelligence explosions is stuck between two bad takes: it's impossible, or a god emerges at the end. Neither is right. The real question is what a period of faster-than-usual AI progress looks like when it's built atop the particular kind of intelligence LLMs represent.
No indexed bits in this chapter.
The Chinchilla scaling law shows that parameters and data contribute independently to loss. Even with infinite parameters, you'd only reduce data needs by 10x. But humans are thousands to millions of times more sample-efficient. Scaling current models is simply not the answer.
Frontier AI models are trained on tens to hundreds of trillions of tokens. A human sees about 200 million from birth to adulthood. That's close to a million-fold difference — and it reveals just how data-hungry these systems really are.
This episode
Factual claims made this episode, and whether a source was named.
Frontier AI models are trained on tens to hundreds of trillions of tokens, compared to roughly 200 million tokens a human sees from birth to adulthood — a nearly million-fold difference.
Open-source AI models lag state-of-the-art frontier models by approximately 4 months, according to Epoch AI.
A teenager can learn to drive a car with approximately 20 hours of practice, which is 3–4 orders of magnitude less data than Waymo and Tesla use to train self-driving models.
The human genome is only 3 gigabytes and only 1–2% is protein-coding, which is not enough to store pre-trained neural network weights.
The human brain has approximately 100 trillion synapses, while current frontier AI models have around 5 trillion parameters.
Using the Chinchilla scaling law constants, even an infinite number of model parameters would only decrease required training data by a factor of 10.
Humans are somewhere between thousands to millions of times more sample-efficient than current AI models.
With GRPO, AI models generate hundreds to thousands of rollouts per task, compared to a human student who might practice a problem once or twice.
The data labeling and RL environment industry is earning billions per year in revenue, projected to reach tens of billions soon.
Blind and deaf people retain general intelligence despite lacking vast sensory input, suggesting multimodal sensory data is not the primary driver of human intelligence.
If a human sees and hears on average 2,000 words per hour, they will have seen approximately 200 million tokens from birth to adulthood.
This episode
Cited as the source of the 'evolution pre-trained humans' objection to human-AI data comparisons, from a previous podcast appearance.
Sponsor of the episode; an AI-powered banking platform whose new 'Command' feature is demonstrated as the ad read.
Named as the FDIC-insured bank providing banking services through Mercury, mentioned in the sponsor disclaimer.
Cited for the finding that open-source models lag frontier models by approximately 4 months, used as evidence that data drives AI progress.
A data labeling company whose job listings are cited as evidence of how task-specific and bespoke AI training data must be.
Mentioned alongside Waymo as a self-driving car company requiring vastly more training data than humans to learn to drive.
Cited as an example of a self-driving car company requiring 3–4 orders of magnitude more data than a human teenager to learn to drive.
DeepMind's scaling law paper whose constants Dwarkesh uses to demonstrate that even infinite parameters would only reduce data needs by 10x.
Used as a metaphor for the vast data AI needs to become a competent software engineer, contrasted with how quickly humans learn the same skill.
Humanoid robot referenced to illustrate how a robotics revolution would be possible if AI could match human teleoperation learning speed.
Stats
We use essential and analytics cookies to run Vuci. To understand how the site is used: Privacy Policy.
Install Vuci on your phone
Add it to your home screen for a faster, app-like experience.
Install Vuci on your phone
Tap the Share button, then “Add to Home Screen”.
A new version is available
Reload to get the latest Vuci.