Model distillation is a technique where a smaller AI model is trained to mimic the behavior of a larger, more capable one. The big model (called the teacher) generates outputs — answers, reasoning steps, probability distributions — and the smaller model (called the student) learns from those outputs rather than raw data alone. The result is a compact model that punches above its weight.

Distillation matters because large frontier models are expensive and slow to run. A distilled model can deliver much of the same quality at a fraction of the compute cost, making AI faster and cheaper to deploy at scale.

It's in the news right now because of DeepSeek, the Chinese AI lab that released models trained by distilling reasoning traces from top-tier models like OpenAI's o1. Their results were surprisingly strong, sparking debate about how much capability can be transferred this way.