A benchmark is a standardized test used to measure AI performance — think of it like a standardized exam for models. Benchmarks are crucial because they give researchers a common way to compare models objectively. But when models start scoring near-perfect on a benchmark, it becomes saturated.

Saturation is a serious problem. Once most leading models cluster near the top of a benchmark's score range, the test stops being useful — it can no longer distinguish between a great model and an exceptional one. Worse, labs sometimes overfit to popular benchmarks, tuning models to score well on the test without actually becoming more capable in the real world.

When a benchmark saturates, the AI community has to design harder, more nuanced replacements. This cycle — create benchmark, models ace it, replace benchmark — is a recurring challenge in AI research, and it signals just how fast frontier model capabilities are actually advancing.