EfficientZero beats novice humans in 2h
EfficientZero, trained to be data-efficient, could beat a novice human at an unseen Atari game given only 2 hours of play time.
Snapshot · Dwarkesh Podcast
The next big breakthrough will be AIs learning on the job
Explore episode
Jun 26, 2026
Where this was said
Getting the learning back to the weights
At 14:22 · chapter starts 8:41
Dwarkesh explores on-policy self-distillation (OPSD) as the leading technique for continual learning, contrasting it with naive SFT and explaining why RL's sparse parameter updates are a feature, not a bug [1] — Dwarkesh Patel "OPSD trains the base model to match what a veteran 'teacher' model — loaded with a full session's context — would predict. It needs no oute…" 11:05 .
400M+ CursorTab requests/day
The CursorTab model online-learns by predicting which edits users accept, processing over 400 million requests per day.
On-Policy Self-Distillation (OPSD): Compressing Sessions Into Weights
OPSD trains the base model to match what a veteran 'teacher' model — loaded with a full session's context — would predict. It needs no outer-loop reward, provides denser per-token supervision than RL, and avoids overwriting existing knowledge. It's the most credible path to continual learning today.
OPSD requires no outer-loop reward
Unlike RL-VR, on-policy self-distillation doesn't require an outer-loop verifiable reward — it only requires a model that can learn the right things within the context window.
OPSD: denser supervision signal
On-policy self-distillation provides per-token supervision rather than a single end-of-trajectory reward, making it a much denser training signal than naive RL.
Why SFT Is a Bad Fit for On-the-Job Learning
Naively fine-tuning on session transcripts trains models to recall everything with perfect fidelity — but that's not how skill acquisition works. RL concentrates updates only where they matter and changes very few parameters, which is exactly what you want when you don't want to forget what you already know.
Very few parameters changed per RL step
RL training changes very few model parameters per step, concentrating updates only on what is relevant to achieving the outcome — a crucial property for continual learning.
Similar snapshots
MVP built in one week
Sam's initial MVP was coded in approximately one week using ChatGPT voice mode and copy-pasting code, with no prior technical experience.
Discord vs email for young users
Sam argues Discord is 10x better than email for building relationships with younger users who rarely check their inbox.
Monthly infra costs ~$780–$980
Sam's monthly operating costs include Cursor ($200), AI image generation ($100), AI video generation ($200), hosting ($100), email marketing ($80), and AI compute ($300–$500).
ChatGPT-powered pain point research
Sam recommends copying days of Discord chat history into ChatGPT and prompting it to list recurring pain points as a fast, free market research technique.
50 free tools built total
Bhanu and his team built approximately 50 free tools to attract search traffic, each linked back to SiteGPT.
New tool built in under 5 minutes
With AI coding tools like Cursor, Bhanu can now create a new free marketing tool in less than 5 minutes by referencing existing tools.
KD filter < 10 for rankable keywords
Bhanu filters Ahrefs keyword results to show only those with a keyword difficulty below 10, making them realistic ranking targets for any decent website.
Minimum 1,000 monthly search volume
Bhanu sets a minimum search volume of 1,000 monthly searches when selecting keywords to target with free tools.
20 downloads/day at launch
PropGPT averaged 20 downloads per day right after launching on the App Store through influencer marketing.
