Where this was said
How systolic arrays work
At 35:35 · chapter starts 26:10
Motivates systolic arrays as the solution to CUDA core inefficiency. Stores weight matrix locally; achieves x×y compute for only x bandwidth. Older TPUs used 128×128 arrays. [1] — Reiner Pope "A systolic array stores the weight matrix locally in registers right next to the compute units and reuses it across many input vectors. Thi…" 26:10
A systolic array stores the weight matrix locally in registers right next to the compute units and reuses it across many input vectors. This means you get x×y compute operations for only x units of register file bandwidth — a linear-vs-quadratic advantage that is the entire reason Tensor Cores exist.
A systolic array achieves quadratically more compute than a CUDA core while requiring only linearly more communication, by storing the weight matrix locally and reusing it across many input vectors.
Older Google TPU chips used a 128×128 systolic array, representing the most area-efficient known hardware circuit for matrix multiplication.
