Vector Database Platform on Kubernetes
A self-hosted, multi-tenant vector database platform on Kubernetes built on Qdrant with a shared embedding service. It powers semantic search and RAG retrieval with sharding, replication, tenant isolation, and recall/latency SLOs.
Overview
A vector database stores high-dimensional embeddings and finds the nearest matches to a query vector quickly. It is the retrieval engine behind semantic search, RAG, recommendations, and deduplication. This architecture self-hosts an open-source vector database on Kubernetes for teams that need data residency, cost control at high volume, or features a managed service does not offer. It serves many applications through one shared, multi-tenant platform.
Components
- Qdrant: the vector database, run as a sharded, replicated StatefulSet for storage and approximate nearest-neighbor search.
- Embedding service: a GPU-backed deployment that turns text or images into vectors, shared by ingestion and query paths.
- Envoy: ingress that routes tenants, terminates TLS, and applies rate limits.
- PostgreSQL: stores collection metadata, tenant config, and access policies.
- MinIO: object storage for snapshots and backups.
- Redis: caches frequent queries and embeddings.
- Prometheus: tracks query latency, recall, and resource use.
Data Flow
Ingestion sends documents to the embedding service, which returns vectors written to a tenant's Qdrant collection along with payload metadata for filtering. At query time, the query text is embedded, and Qdrant runs a filtered nearest-neighbor search, optionally combined with keyword filters from payload fields. Results return with scores and source IDs. Snapshots are taken on a schedule and stored in MinIO for recovery and migration.
Scaling and Resilience
Qdrant shards collections across nodes and replicates each shard, so the cluster scales horizontally and survives node loss. The embedding service scales on GPU utilization and request rate. Tune the index (HNSW parameters) to balance recall against latency and memory. Run anti-affinity so replicas land on different nodes and zones. Restore from MinIO snapshots for disaster recovery, and test restores regularly. Define and alert on recall and latency SLOs.
Security
Isolate tenants with separate collections, API keys, and network policies so one tenant cannot read another's vectors. Encrypt data at rest and in transit. Apply least-privilege access to the metadata database and object store. Because retrieved content can carry prompt-injection payloads downstream, tag and track provenance on every vector. Audit queries that scan large portions of a collection, which can indicate data exfiltration.
Trade-offs and Alternatives
Self-hosting avoids per-vector SaaS pricing and keeps data in your environment, but you own scaling, upgrades, and recall tuning. Managed services (Pinecone, managed Qdrant, OpenSearch, or pgvector on a managed Postgres) trade cost and control for less operational work. pgvector is the simplest choice when vector counts are modest and you already run Postgres. Dedicated vector databases win at large scale and high query rates where index performance and sharding matter most.