Dwarkesh Podcast

Podbit · Dwarkesh Podcast

Reiner Pope – Chip design from the bottom up

Explore episode May 22, 2026
Technology
A GPU is just a bunch of tiny TPUs

Reiner Pope – Chip design from the bottom up · May 22, 2026 Technology

A TPU has a few large systolic arrays (MXUs) with a shared vector unit. A GPU is the same architecture tiled many times in miniature: each SM has a Tensor Core (mini-MXU) plus its own vector unit. The GPU trades larger systolic array amortization for more parallel data paths between vector and matrix units.

Where this was said

A GPU is just a bunch of tiny TPUs

At 1:18:20 · chapter starts 1:18:15

TPU has few large systolic arrays with shared vector units. GPU tiles many miniaturized versions across the chip — each SM is a tiny TPU. GPUs trade large-array amortization for more parallel data paths.

Similar podbits