Where this was said
Muxes and the cost of data movement
At 24:50 · chapter starts 16:31
Traces a CUDA core / CPU data path: register file → MUX → ALU → write back. Shows that 7/8 of circuit area is consumed by MUX data movement, not computation. [1] — Reiner Pope "In a classic CPU or CUDA core, 7/8 of the circuit area is consumed by MUX circuits just to read and write the register file — not by actual…" 20:20
In a classic CPU or CUDA core, 7/8 of the circuit area is consumed by MUX circuits just to read and write the register file — not by actual computation. The multiply-accumulate unit you care about is a tiny fraction of the silicon. This is the fundamental problem that Tensor Cores and systolic arrays were invented to solve.
In a classic CUDA core / CPU data path, about 7/8 of the circuit area is consumed by MUX circuits reading the register file, not by the actual multiply-accumulate logic.
A systolic array stores the weight matrix locally in registers right next to the compute units and reuses it across many input vectors. This means you get x×y compute operations for only x units of register file bandwidth — a linear-vs-quadratic advantage that is the entire reason Tensor Cores exist.
