9x9→19x19 transfer learning
Pre-training on 9x9 Go boards and then warm-starting a 19x19 model dramatically cuts the time needed to learn endgame value functions.
Where this was said
Why doesn't MCTS work for LLMs
At 1:59:20 · chapter starts 1:41:09
Eric and Dwarkesh discuss why MCTS fails for LLM reasoning: language's vast token space violates the discrete finite-action assumption, and value estimation is much harder than in Go. [1] — Eric Jang "Language has billions of possible next tokens — the PUCT exploration heuristic assumes you'll visit the same node multiple times, but an LL…" 1:44:55
Scaling Laws Require a Working System First
Scaling laws only emerge cleanly when your system already works, your data is good, and there are no bugs. Trying to extract scaling insights before you have a working baseline just gives you scaling laws on garbage.
MCTS: one better label per move
Unlike naive RL which must figure out which of 100K+ tokens caused a win, MCTS provides a strictly better action target for every single move in every game.
Why MCTS Doesn't Work for LLMs
Language has billions of possible next tokens — the PUCT exploration heuristic assumes you'll visit the same node multiple times, but an LLM will almost never generate the exact same token sequence twice. The discrete action assumption breaks.
A 10-Layer Network Solves What Seemed NP-Hard
A 10-layer neural network can compress what looks like an NP-class search problem into a single forward pass. This happened with Go, protein folding, and tensor decomposition. It might mean our understanding of computational hardness is fundamentally incomplete.
$10K total compute spend
Eric Jang trained a strong Go bot for roughly $10K of donated compute from Prime Intellect, spending about $3K on the final training run.
AlphaGo Zero: ~3e23 flops
AlphaGo Zero was trained on far more compute than any other AI model of its era — roughly 3e23 flops, comparable in order of magnitude to frontier LLMs.
Similar snapshots
MVP built in one week
Sam's initial MVP was coded in approximately one week using ChatGPT voice mode and copy-pasting code, with no prior technical experience.
Discord vs email for young users
Sam argues Discord is 10x better than email for building relationships with younger users who rarely check their inbox.
Monthly infra costs ~$780–$980
Sam's monthly operating costs include Cursor ($200), AI image generation ($100), AI video generation ($200), hosting ($100), email marketing ($80), and AI compute ($300–$500).
ChatGPT-powered pain point research
Sam recommends copying days of Discord chat history into ChatGPT and prompting it to list recurring pain points as a fast, free market research technique.
50 free tools built total
Bhanu and his team built approximately 50 free tools to attract search traffic, each linked back to SiteGPT.
New tool built in under 5 minutes
With AI coding tools like Cursor, Bhanu can now create a new free marketing tool in less than 5 minutes by referencing existing tools.
KD filter < 10 for rankable keywords
Bhanu filters Ahrefs keyword results to show only those with a keyword difficulty below 10, making them realistic ranking targets for any decent website.
Minimum 1,000 monthly search volume
Bhanu sets a minimum search volume of 1,000 monthly searches when selecting keywords to target with free tools.
20 downloads/day at launch
PropGPT averaged 20 downloads per day right after launching on the App Store through influencer marketing.
