How to add semantic search with embeddings and a vector database

What semantic search is

Keyword search matches exact words. Semantic search matches meaning. It works by converting text into embeddings, numeric vectors where similar concepts sit close together, then finding the vectors nearest to a query. This lets users find "how to reset my password" even when a document says "recover account access."

Prerequisites

• Python 3.10 or later
• An embedding model (hosted or local)
• A vector store such as PostgreSQL with pgvector, or Redis with a vector index

Steps

1. Choose an embedding model

Pick a model whose dimension and quality fit your needs. Smaller models are faster and cheaper; larger ones capture nuance. Record the vector dimension; your index must match it.

2. Prepare and clean the corpus

Strip boilerplate, normalize whitespace, and split long documents into passages. Cleaner input produces sharper vectors.

3. Generate embeddings in batches

Batching reduces overhead and cost.

batch = passages[i:i+64]
vectors = embed_model.embed(batch)

4. Create a vector index

In PostgreSQL with pgvector:

CREATE TABLE items (id serial PRIMARY KEY, text text, embedding vector(768));
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

An HNSW or IVF index turns a slow exact scan into fast approximate nearest-neighbor (ANN) search.

5. Query with nearest-neighbor search

Embed the query with the same model, then order by distance.

SELECT text FROM items ORDER BY embedding <=> %s LIMIT 10;

The <=> operator computes cosine distance; smaller is closer.

6. Rank and filter results

Combine vector distance with metadata filters (date, category) and optionally a keyword signal for a hybrid search. Cap results and drop low-similarity matches below a threshold.

Verification

Query with a phrase that shares no exact words with a known relevant document. Confirm the document still ranks near the top. Compare against a plain keyword search to see the improvement.

Next Steps

Add hybrid search, cache frequent queries, and monitor recall by spot-checking results against human judgment.