The mechanism that measures prediction error during training is called the loss function. The model makes a prediction; the loss function compares it to the correct answer and produces a single number — the loss — representing how wrong the model was. High loss means a bad prediction; zero means a perfect one.

That number flows backward through the model via backpropagation, nudging each parameter slightly in the direction that would have reduced the error. Repeat this across billions of examples and the model gradually improves.

The choice of loss function matters: different tasks need different definitions of "wrong." A model predicting the next word uses cross-entropy loss; one generating images might use a perceptual loss that compares visual features rather than raw pixels. In a real sense, the loss function is the model's only teacher — it is the entire definition of what the model is trying to get right.