Training
Training fits a model's parameters to data by minimizing a loss with gradient descent, producing the model later used at inference.
Training is the phase in which a machine learning model learns by adjusting its internal parameters to fit data. The model makes predictions, a loss function measures how wrong they are, and an optimization algorithm updates the parameters to reduce that loss across many examples.
How It Works
Training feeds batches of examples through the model, computes the loss, and uses backpropagation to calculate gradients of the loss with respect to each parameter. Gradient descent then nudges parameters in the direction that lowers loss. This repeats for many passes over the data, called epochs, until performance plateaus. For large language models, pretraining uses a self-supervised objective such as next-token prediction over huge corpora; later stages may add supervised fine-tuning and reinforcement learning from human feedback. Validation data is used to detect overfitting.
Why It Matters
Training determines what a model can do and is by far the most compute-intensive part of the lifecycle for large models. Choices about data quality, scale, objectives, and regularization shape accuracy, generalization, and bias. Because pretraining is expensive, most teams reuse pretrained models and adapt them with fine-tuning rather than training from scratch.
Related Terms
Training uses gradient-descent to fit parameters, guards against overfitting, precedes inference, and underlies both the large-language-model and later fine-tuning.