Where this was said
Self-play
At 1:18:15 · chapter starts 1:00:33
Eric explains how MCTS produces a more confident action distribution than the raw policy, and how AlphaGo trains the policy to imitate that improved distribution — the core self-play loop. [1] — Eric Jang "MCTS doesn't just pick the best move — it produces a better probability distribution than the raw policy network. AlphaGo then trains the p…" 1:01:00
MCTS doesn't just pick the best move — it produces a better probability distribution than the raw policy network. AlphaGo then trains the policy network to imitate that improved distribution. Iterate, and you get exponential improvement.
Naive RL reinforces entire winning trajectories — but most of those moves were irrelevant. MCTS gives you a strictly better action label for every single move in every game. That's why AlphaGo learns far faster than an LLM-style reinforce loop.
