Where this was said
Why doesn't MCTS work for LLMs
At 1:57:07 · chapter starts 1:41:09
Eric and Dwarkesh discuss why MCTS fails for LLM reasoning: language's vast token space violates the discrete finite-action assumption, and value estimation is much harder than in Go. [1] — Eric Jang "Language has billions of possible next tokens — the PUCT exploration heuristic assumes you'll visit the same node multiple times, but an LL…" 1:44:55
Scaling laws only emerge cleanly when your system already works, your data is good, and there are no bugs. Trying to extract scaling insights before you have a working baseline just gives you scaling laws on garbage.
Unlike naive RL which must figure out which of 100K+ tokens caused a win, MCTS provides a strictly better action target for every single move in every game.
Language has billions of possible next tokens — the PUCT exploration heuristic assumes you'll visit the same node multiple times, but an LLM will almost never generate the exact same token sequence twice. The discrete action assumption breaks.
A 10-layer neural network can compress what looks like an NP-class search problem into a single forward pass. This happened with Go, protein folding, and tensor decomposition. It might mean our understanding of computational hardness is fundamentally incomplete.
Eric Jang trained a strong Go bot for roughly $10K of donated compute from Prime Intellect, spending about $3K on the final training run.
AlphaGo Zero was trained on far more compute than any other AI model of its era — roughly 3e23 flops, comparable in order of magnitude to frontier LLMs.
Pre-training on 9x9 Go boards and then warm-starting a 19x19 model dramatically cuts the time needed to learn endgame value functions.
