Where this was said
Off-policy training
At 2:03:20 · chapter starts 2:01:09
Eric explains the DAGGER-inspired view of replay buffers: off-policy data is helpful if it covers reachable states, harmful if it covers states the current policy would never visit. [1] — Eric Jang "Off-policy data is fine — and useful — if those states are ones your current policy might visit. It becomes actively harmful when you're tr…" 2:01:20
Off-policy data is fine — and useful — if those states are ones your current policy might visit. It becomes actively harmful when you're training on states your policy would never reach, wasting capacity on irrelevant corrections.
