Neural networks are, at their mathematical core, a long chain of matrix multiplications — grids of numbers being multiplied together millions of times per second. A regular CPU handles such operations one calculation at a time, which makes it painfully slow for AI at scale.

NVIDIA addressed this in 2017 by adding tensor cores to its Volta GPU architecture. A tensor core doesn't compute one multiplication per clock cycle — it performs an entire matrix multiply-accumulate operation in one shot. Where a standard core needs dozens of cycles to multiply two small matrices together, a tensor core does it in one.

The result is that a modern H100 GPU devotes nearly all of its silicon to matrix math — roughly 989 teraflops of it, versus about 60 teraflops for everything else. This is why AI chips look nothing like the processor in your laptop: they are purpose-built matrix multiplication engines, and every AI response you've ever received ran through billions of those operations.