EfficientZero beats novice humans in 2h
EfficientZero, trained to be data-efficient, could beat a novice human at an unseen Atari game given only 2 hours of play time.
Snapshot · Dwarkesh Podcast
The next big breakthrough will be AIs learning on the job
Explore episode
Jun 26, 2026
Where this was said
Getting the learning back to the weights
At 14:22 · chapter starts 8:41
Dwarkesh explores on-policy self-distillation (OPSD) as the leading technique for continual learning, contrasting it with naive SFT and explaining why RL's sparse parameter updates are a feature, not a bug [1] — Dwarkesh Patel "OPSD trains the base model to match what a veteran 'teacher' model — loaded with a full session's context — would predict. It needs no oute…" 11:05 .
400M+ CursorTab requests/day
The CursorTab model online-learns by predicting which edits users accept, processing over 400 million requests per day.
On-Policy Self-Distillation (OPSD): Compressing Sessions Into Weights
OPSD trains the base model to match what a veteran 'teacher' model — loaded with a full session's context — would predict. It needs no outer-loop reward, provides denser per-token supervision than RL, and avoids overwriting existing knowledge. It's the most credible path to continual learning today.
OPSD requires no outer-loop reward
Unlike RL-VR, on-policy self-distillation doesn't require an outer-loop verifiable reward — it only requires a model that can learn the right things within the context window.
OPSD: denser supervision signal
On-policy self-distillation provides per-token supervision rather than a single end-of-trajectory reward, making it a much denser training signal than naive RL.
Why SFT Is a Bad Fit for On-the-Job Learning
Naively fine-tuning on session transcripts trains models to recall everything with perfect fidelity — but that's not how skill acquisition works. RL concentrates updates only where they matter and changes very few parameters, which is exactly what you want when you don't want to forget what you already know.
Very few parameters changed per RL step
RL training changes very few model parameters per step, concentrating updates only on what is relevant to achieving the outcome — a crucial property for continual learning.
Similar snapshots
Ad monetization for games
Ad-based monetization works well for game apps where users spend extended time in-session, as seen with Grid and Wordle.
Ads don't work for tool apps
Tool-focused apps like PuffCount are poor candidates for ad monetization because users don't stay in-session long enough.
Hard paywall is unskippable
A hard paywall is a screen that blocks all app features unless the user pays or starts a free trial — it cannot be dismissed.
Two core monetization models
Mobile apps are primarily monetized through either ads (best for games) or in-app purchases/subscriptions (best for tools).
YouTube beats every other platform
According to the episode, YouTube outperforms every other social platform for building trust and driving SaaS conversions.
Most users came from YouTube
Vasco stated that the majority of his app's user base came directly from his YouTube channel.
Boost My Domain Rating button
SEO Bot features a 'Boost My Domain Rating' button that routes users directly to Listing Bot, an example of in-product cross-selling.
All products connected to AI
The founder's entire product portfolio is AI-related, making it easier to package products attractively for directories.
AI coding viability debate as hook
The founder attached their SaaS demo to the trending debate about whether AI coding is actually good enough to build a full SaaS product.
