In a classic CPU or CUDA core, 7/8 of the circuit area is consumed by MUX circuits just to read and write the register file — not by actual computation. The multiply-accumulate unit you care about is a tiny fraction of the silicon. This is the fundamental problem that Tensor Cores and systolic arrays were invented to solve.