A Mixture of Experts (MoE) model is a neural network architecture where the model is divided into many specialized sub-networks, called experts. Instead of activating the entire network for every input, a learned router selects only a small subset of experts to handle each token or task.

This design means you can build a model with a very large total number of parameters — giving it broad knowledge and capability — while only activating a fraction of those parameters at any given moment. The result is a model that behaves as powerfully as its full size suggests, but runs with the compute cost of something much smaller.

Recent models like Google's Gemini and newer open-weight releases use MoE to reach massive scale efficiently. A model might have over a trillion total parameters but only activate 40–50 billion for any single forward pass, making MoE one of the most important architectural ideas in modern AI.