SWE-Bench is a benchmark designed to test whether an AI agent can solve real software engineering tasks — specifically, fixing bugs in actual open-source GitHub repositories. Unlike tests that ask models to write code from scratch, SWE-Bench drops an agent into a messy, real-world codebase and asks it to resolve a genuine issue, the way a human developer would.

Labs care because it's one of the closest proxies we have for practical coding ability. Scores on SWE-Bench have become a standard bragging right when new coding models launch. A newer variant, Senior SWE-Bench, raises the bar further by modeling the complexity of tasks a senior engineer would handle, pushing agents beyond simple bug fixes into deeper architectural reasoning.