Naive RL learns at the rate of the entropy of a binary random variable per sample, while supervised learning learns negative log(pass rate) bits — orders of magnitude more at low pass rates.
Where this was said
At 2:13:55 · chapter starts 2:08:56
Dwarkesh presents an information-theoretic argument: naive RL learns near-zero bits per sample at low pass rates, while supervised learning learns negative log(pass rate) bits. The gap is enormous. [1] — Dwarkesh Patel "With a 1-in-100K pass rate, supervised learning gives you negative log(1/100K) = ~17 bits per sample. Naive RL gives you essentially zero. …" 2:09:35
With a 1-in-100K pass rate, supervised learning gives you negative log(1/100K) = ~17 bits per sample. Naive RL gives you essentially zero. You spend almost all of early training in a regime where nothing is learned.
LLMs can autonomously run experiments, tune hyperparameters, and optimize code. What they can't do: decide which question to investigate next, or recognize when an entire research track is a dead end and pivot to something fundamentally different.
AlphaGo's elegance is that MCTS always produces a training signal — the policy is never stuck at zero pass rate waiting to stumble on a win.
Sam's initial MVP was coded in approximately one week using ChatGPT voice mode and copy-pasting code, with no prior technical experience.
Sam argues Discord is 10x better than email for building relationships with younger users who rarely check their inbox.
Sam's monthly operating costs include Cursor ($200), AI image generation ($100), AI video generation ($200), hosting ($100), email marketing ($80), and AI compute ($300–$500).
Sam recommends copying days of Discord chat history into ChatGPT and prompting it to list recurring pain points as a fast, free market research technique.
Bhanu and his team built approximately 50 free tools to attract search traffic, each linked back to SiteGPT.
With AI coding tools like Cursor, Bhanu can now create a new free marketing tool in less than 5 minutes by referencing existing tools.
Bhanu filters Ahrefs keyword results to show only those with a keyword difficulty below 10, making them realistic ranking targets for any decent website.
Bhanu sets a minimum search volume of 1,000 monthly searches when selecting keywords to target with free tools.
PropGPT averaged 20 downloads per day right after launching on the App Store through influencer marketing.
We use essential and analytics cookies to run Vuci. To understand how the site is used: Privacy Policy.
Install Vuci on your phone
Add it to your home screen for a faster, app-like experience.
Install Vuci on your phone
Tap the Share button, then “Add to Home Screen”.
A new version is available
Reload to get the latest Vuci.