Choosing a vector database by the constraint that will actually bite

The usual sequence: someone reads a benchmark showing a dedicated vector engine doing 20,000 queries per second at 99 percent recall, picks it, and six months later the assistant serves forty queries a minute, the recall problem turns out to be a chunking problem, and the team is operating a stateful distributed system nobody on call understands.

Query throughput is almost never the constraint that bites. The constraints that bite are metadata filtering at scale, tenant isolation, update frequency, and the operational cost of one more component in a system that already has enough. Pick for those.

pgvector is the default, and it holds longer than you think

If your data already lives in PostgreSQL, start there. pgvector gives you HNSW and IVFFlat indexes, cosine and inner-product distance, and the thing dedicated engines make hard: your vectors sit in the same transaction as your business rows. You insert a document and its chunks atomically. You join a similarity result against the permissions table in one query. You back it up with the backup you already have.

The ceiling is real but higher than the folklore suggests. On a managed Postgres instance with enough memory to hold the HNSW index, single-digit millions of vectors with sub-100ms p95 is routine. The failure mode is not the vector count, it is memory: if the index does not fit in RAM you fall off a cliff, and an HNSW index over 1536-dimension vectors is roughly 6 to 8 KB per row once you count the graph edges. Do that multiplication before you commit to anything, because it is the calculation that actually predicts the pain.

One operational note: index builds compete with your OLTP traffic for the same buffer cache, so if that database also serves your checkout path, put the vectors elsewhere.

Metadata filtering is where architectures break

Every serious RAG system filters. By tenant, by permission group, by document type, by date. As covered in the four decisions that matter more than the model, that filter has to apply before ranking, not after, or you get empty result sets and leaked document existence.

This is where stores genuinely differ. Postgres can combine a WHERE clause with a vector index, but the planner may choose to filter first and then brute-force the survivors, which is fast for a narrow filter and terrible for a broad one. Qdrant and Weaviate implement filtered HNSW traversal, keeping recall while restricting the search. If your typical query filters to a small, unpredictable slice of a large corpus, that difference is worth a migration. If your filter is one tenant out of two hundred with 50,000 chunks each, partition the Postgres table by tenant and the problem disappears.

Multi-tenancy and update frequency decide more than volume

Two questions predict a dedicated engine better than row count.

How many tenants, and how isolated must they be? A handful: separate schemas or partitions in Postgres. Thousands of small tenants: you want a store with first-class collections or tenant-aware indexes, because thousands of Postgres partitions is a planner problem.

How often does the corpus change? A knowledge base reindexed nightly is easy anywhere. A system ingesting thousands of chunks an hour with deletes is where IVFFlat degrades, HNSW takes inserts fine but handles deletes only as tombstones, and engines designed around segment compaction earn their keep.

Hybrid search is not optional, and it changes the shortlist

Pure vector search fails predictably on exact identifiers, product codes, error strings and rare proper nouns: anything where the user typed the literal token they want. Lexical BM25 handles those and fails on paraphrase. You need both, with the two result lists fused, usually by reciprocal rank fusion.

Postgres does this natively: a tsvector GIN index alongside the vector column, two CTEs, fuse in SQL. Forty lines, and one of the largest quality jumps available. Some dedicated engines offer hybrid as a built-in; others expect you to run OpenSearch beside them, which means two systems anyway. Factor that in.

The managed services, and what they cost you in optionality

Every cloud has an answer: Amazon OpenSearch Serverless vector collections and Aurora with pgvector, Vertex AI Vector Search, Azure AI Search, plus Pinecone as the main independent. They remove real operational work.

The lock-in is not uniform, and that is the part to weigh. Aurora pgvector or Azure Database for PostgreSQL keeps you on the Postgres API, so leaving is a pg_dump. Vertex AI Vector Search and Azure AI Search have proprietary APIs and their own hybrid and filtering semantics, so leaving is a rewrite of the retrieval layer plus a full reindex. That may be a fine trade. Make it knowingly, and keep your chunking and embedding pipeline independent of the store so the reindex is a script, not a project.

Things people forget

• The index is not your source of truth. Keep the chunks, their text and their metadata in durable storage you control. You will rebuild the index more than once, and reconstructing chunks from the vector store is miserable.
• Changing embedding model means reindexing everything. Dimensions and geometry both change. This is the single most expensive thing you can decide badly, which is why it belongs with chunking and embeddings, not with the database choice.
• Backups and restore drills. Several dedicated engines treat snapshots as a paid or manual feature. Test a restore before you need one.
• Cost is memory-shaped, not query-shaped. Vector search is a RAM game. Serverless vector offerings often bill on stored size plus a compute floor, and the floor is what surprises the finance review.
• Latency includes the embedding call. Embedding the query is a round trip to a model, often 30 to 80ms. Your fast database is one part of a slower whole.

The right answer for most teams we work with on an applied AI project is: pgvector until you have a measured reason to leave, and write down in advance what that reason would be.

What to do this week

Take your current or planned corpus, multiply the chunk count by 7 KB, and compare it against the RAM of the Postgres instance you would use. Then write one sentence naming the constraint you expect to hit first: volume, filter selectivity, tenant count or update rate. That sentence, not a benchmark, is your selection criterion.