Where this was said
Brains vs chips
At 1:16:50 · chapter starts 1:12:00
Compares brain and GPU architectures. Brain's slow clock is explained by batch size of 1 vs GPU's 1,000+. Slowing GPU to MHz reduces energy 1,000× but not energy efficiency per operation. [1] — Reiner Pope "A GPU runs at GHz clock speeds because it processes batch sizes of 1,000 simultaneously. The brain runs at a much slower clock because it o…" 1:15:00
A GPU runs at GHz clock speeds because it processes batch sizes of 1,000 simultaneously. The brain runs at a much slower clock because it only ever processes one instance of itself. Running a GPU at MHz instead of GHz would give roughly 1,000× less energy consumption — but not a 1,000× improvement in energy efficiency per operation.
A GPU is architecturally equivalent to many small TPUs tiled across the chip — each streaming multiprocessor (SM) contains its own Tensor Core (like a mini MXU) plus a vector unit and register file.
Most of a chip's energy is consumed by charging and discharging capacitors as bits toggle between 0 and 1, known as dynamic or switching power.
