Embedding

An embedding is a fixed-length vector of floating-point numbers that encodes the semantic content of an input such as a sentence, document, or image. Items with similar meaning produce vectors that are close together under a distance or similarity measure, which lets software reason about meaning numerically.

How It Works

An embedding model, typically a neural network, maps an input to a point in a high-dimensional space, often hundreds or thousands of dimensions. The model is trained so that related inputs land near one another, frequently using contrastive objectives that pull matching pairs together and push unrelated pairs apart. Similarity is then computed with measures such as cosine similarity or dot product. The resulting vectors are stored and queried for nearest neighbors.

Why It Matters

Embeddings turn unstructured content into a form that supports semantic search, clustering, classification, deduplication, and recommendation. They are the foundation of retrieval-augmented generation, where a query is embedded and matched against embedded document chunks to find relevant context. Because embeddings capture meaning rather than exact words, they retrieve relevant results even when wording differs.

Related Terms

Embeddings are queried through vector-search, supply context for retrieval-augmented-generation, and are produced from token sequences by models related to the large-language-model.