Foundation Model

A foundation model is a large machine learning model trained on broad, often unlabeled data at massive scale, designed to serve as a general base that can be adapted to many specific tasks. The term emphasizes a shift away from building a separate model per task toward reusing one capable model across applications.

How It Works

A foundation model is pretrained with a self-supervised objective, such as predicting masked or next tokens, over enormous and diverse datasets. This produces broad, general-purpose representations. The model is then adapted to particular needs through methods like fine-tuning, instruction tuning, prompting, or retrieval augmentation. Large language models are the most familiar foundation models, but the idea also covers vision, audio, and multimodal models that span several data types.

Why It Matters

Foundation models concentrate the expensive pretraining step into a reusable asset, so most teams adapt an existing model rather than train from scratch. This lowers the cost and time to build AI features and has driven rapid adoption across industries. It also raises governance considerations, since many applications inherit the capabilities, biases, and risks of a shared base model.

Related Terms

A foundation model is commonly an large-language-model built on the transformer architecture, created through large-scale training, adapted via fine-tuning, and may be multimodal.