"Which vector database should we use" usually arrives already narrowed to four vendor names, and it is almost never the question that decides whether the system works. Teams spend six weeks on that comparison and then ship something that returns the wrong documents, because what made it return the wrong documents was chunking, or filter semantics, or the absence of a test set. None of those are properties of the database. This is the evaluation we run instead. It starts with arithmetic, ends with a measurement on your own data, and normally cuts the shortlist to two before anyone opens a pricing page.
You are probably here because
- Search worked fine until you scoped it. Add a tenant or a date filter and it comes back with three results, or none.
- Somebody typed a part number, an error code or a surname and got back something adjacent and confidently wrong.
- Nobody can tell you what recall the system is running at, because it has never been measured against exact search.
- The shortlist has been the same four vendor names for weeks, and no one can say which is better on your own data.
The filtering section and the recall section below address all four. They usually share one root cause: the engine got argued about before anyone measured recall on their own vectors against brute force.
Most teams do not have a vector database problem yet
Under a certain size the answer is boring and correct: put the vectors in the database you already run. A million 1536-dimension vectors fit comfortably in a PostgreSQL table with pgvector, which has had HNSW indexing since version 0.5.0. You get transactions, joins back to the source rows, a backup story, a permissions model, and an on-call rotation that already exists. A second stateful system costs a backup you have to verify, an upgrade path you have to test, and a new class of 3am page. None of that appears in a benchmark chart.
Four conditions move you off that answer. Each one is a measurement, not an opinion.
The index stops fitting next to your transactional working set. Vector indexes are memory-resident by default and compete with the buffer cache your application depends on. Once p99 on ordinary queries moves because of the vector index, the vectors need their own machine, and the argument for keeping them in the same engine gets much weaker.
Write volume competes with read latency. Graph index construction is expensive, and a steady stream of inserts into an HNSW index on the instance serving transactional traffic is eventually noticed by somebody whose report got slow.
You need filtered search over high-cardinality metadata. This decides most real cases, and it gets its own section below.
Tenancy is structural. Ten thousand customers whose data must never cross is a different problem from a corpus with a permissions column.
The number that decides the architecture
Everything downstream follows from one calculation, and it takes thirty seconds. A float32 vector of d dimensions occupies 4d bytes. At 1536 dimensions, which is what the most commonly used embedding endpoints return by default:
1 million vectors. 6.1 GB of raw float32. Fits on a laptop. Fits in the memory of any database instance you would run in production anyway.
10 million vectors. 61 GB. Now you are buying a memory-optimized instance and thinking about compression.
100 million vectors. 614 GB. Float32 in RAM is off the table, and the design is decided by which compression scheme you choose.
HNSW adds a navigation graph on top. At the common setting of M=16 each node stores up to 32 neighbor identifiers on the bottom layer and 16 above, at four bytes each, so budget 150 to 250 bytes per vector for graph structure. At 1536 dimensions that graph is a rounding error next to the vectors. At 128 dimensions it is most of the index. That one observation explains why a benchmark run at 768 dimensions tells you very little about a 3072-dimension corpus.
Three compression levers change the arithmetic, and they compose with each other.
Scalar quantization to int8. Four times smaller. Recall typically lands within a point or two of float32 once you rescore the top candidates at full precision. This is the default that should be on almost everywhere above a few million vectors.
Binary quantization. Thirty-two times smaller, and Hamming distance is fast enough that search stops being memory-bound. It works well at 1024 dimensions and above and badly below, because the discarded information is a larger fraction of a short vector.
Dimension truncation. Several current embedding models are trained so a prefix of the vector stays usable on its own, letting you cut 1536 dimensions to 512 for a three times reduction at a small, measurable quality cost. Measure it on your corpus rather than trusting the model card.
What makes all three usable is one pattern: search the compressed representation, over-fetch by three or four, then re-rank those few hundred candidates against full-precision vectors read from disk. End-to-end recall lands within noise of exact search on a fraction of the RAM.
Index families, and what each one actually costs
Five index shapes are in production use. Every product on your list implements some subset, and the product name matters far less than which shape you end up running.
| Index | Memory per 1M @ 1536d | Recall behavior | Build and update cost | Use when |
|---|---|---|---|---|
| Flat / exact | 6.1 GB | Perfect by definition | None; appends are free | Under roughly 100K vectors, and always as the ground truth you measure against |
| HNSW | 6.3 GB | High recall at low latency, tunable at query time | Expensive build; deletes are tombstones | The default for a working set that fits in RAM |
| IVF / IVF-PQ | 0.2–1.6 GB | Depends on probe count and cluster quality; degrades quietly as the corpus drifts | Needs a training pass on a sample; appends are cheap | Large corpora under a hard memory ceiling |
| DiskANN-style | ~0.5 GB RAM plus SSD | High recall, latency set by SSD random reads | Slow to build, cheap to keep | Corpora that will not fit in RAM at a price you will pay |
| Binary plus rescore | 0.2 GB plus disk | Within a point of float32 with 4x over-fetch | Cheap to build and to update | Very large corpora at 1024 dimensions or more |
HNSW is the default for good reasons: one query-time knob trades latency for recall without a rebuild, it handles arbitrary distance metrics, and it degrades gracefully. Its two weaknesses are build cost and deletion, and both bite later rather than sooner, which is why they surprise people.
IVF gets recommended for large corpora, usually without the caveat that matters. It partitions the space by clustering a training sample, so recall is a function of how well those centroids still describe your data. A corpus that grows in a new direction, a new product line or a new document type, degrades recall silently and nothing emits a warning. The defense is re-running the recall measurement on a schedule and retraining when it moves.
Filtering is where most of these systems break
Almost no production query is unfiltered. It is scoped to a tenant, a date range, a document type, a permission set, a project. There are three ways an engine can honor that, and they are not close to equivalent.
Post-filtering. Run the approximate search, then discard results that fail the predicate. Simple, fast, and catastrophically wrong when the filter is selective. Ask for the 10 nearest neighbors and filter to a tenant owning 0.1 percent of the corpus and you do not get a slow query, you get an empty one. Teams read that as a relevance problem and go looking for a better embedding model.
Pre-filtering. Compute the matching set first, then search exhaustively inside it. Correct in every case, and fine when the matching set is small. When the predicate matches 40 percent of a 50-million-row corpus you have just written a brute-force scan with extra steps.
Filtered traversal. Apply the predicate during traversal and keep walking until k candidates pass. This is the single largest behavioral difference between the products on your list. Qdrant builds extra graph links over payload-indexed fields so the graph stays connected under a filter. pgvector added iterative index scans in 0.8.0, fixing the case where a filtered HNSW query silently returned fewer rows than requested. Lucene-based engines including Elasticsearch and OpenSearch filter during kNN traversal rather than after it.
The test is three lines long and hardly anybody runs it. Issue a query with a filter matching half a percent of your corpus, ask for 20 results, count what comes back, then compare against brute force over the filtered subset. An engine that hands back six, or twenty that are not the right twenty, has told you the most useful thing in the evaluation, in an afternoon.
Recall is a property of your configuration, not of the vendor
Published benchmarks run on public datasets at dimensions and distributions that are not yours, at parameter settings chosen by whoever published them. They are useful for ruling things out and useless for choosing. The measurement you need is straightforward and you only have to build it once.
Take 500 to 1000 real queries and compute exact nearest neighbors by brute force, which for a few million vectors is a matrix multiply that runs in seconds to minutes on hardware you already have. That is ground truth. Measure what fraction of the true top-10 each configuration returns. That single number, recall@10, makes every later comparison honest. Comparing two engines at unknown recall compares nothing, because either can be made faster by being made worse.
Keep two measurements separate. Index recall against brute force says whether approximate search finds what exact search would. End-to-end answer quality against labeled examples says whether the system is useful. A stack can sit at 0.98 recall@10 and still answer badly, because the correct chunk was never a good chunk. Conflating the two is what sends teams to replace a database when the problem was a PDF parser.
How often each layer is the binding constraint
Our working priors from systems we have been brought in to fix, not a population statistic.
Pure vector search loses on the queries people actually type
Dense embeddings are excellent at meaning and poor at exact tokens. Part numbers, error codes, invoice identifiers, surnames and version strings are what users actually type into a search box, and they are where cosine similarity over a 1536-dimension embedding confidently returns something adjacent and wrong. BM25 has been good at those since the 1990s.
The fix is hybrid retrieval: run both, then fuse. Start with Reciprocal Rank Fusion, which combines ranked lists using positions alone, scoring each document as the sum of 1 divided by (60 plus its rank) across retrievers. It needs no score normalization, has one constant, and is hard to beat without tuning work you probably do not want to fund yet. Weighted score blending does better once you have an evaluation set, and worse until then.
This narrows the shortlist more than teams expect. An engine with no native lexical index means running a second system for BM25 or reimplementing it badly. PostgreSQL has full-text search built in, so one query does both. Elasticsearch and OpenSearch are lexical engines that grew dense vector support. Among purpose-built vector stores, sparse and lexical support varies enough to be worth checking rather than assuming.
Writes, deletes, and the freshness question
HNSW is a graph optimized for search, not mutation. Deletion is normally a tombstone: the vector stays in the graph, traversal still visits it, and the result is filtered out afterward. Memory does not come back and latency creeps up. A corpus with 5 percent weekly churn spends a real fraction of every search walking through graves, and the remedy is a rebuild.
Ask three questions of any candidate and get the answers in writing. How long after a write is that vector searchable, and is that a guarantee or a typical case. What does a delete do physically, and when does space come back. Can the index be rebuilt without downtime, and what does that cost in wall-clock time at your corpus size.
The fourth question is the one nobody asks: what happens when you change embedding models. Every hosted model eventually gets deprecated and every self-hosted one gets replaced by something better. That day you re-embed the corpus, rebuild the index and re-run the evaluation. It is a scheduled project if you prepared, and an outage if you did not. Preparation is three things: keep source text and chunk boundaries outside the vector store, record the model name and version on every row, and make ingest re-runnable by one command.
Version every embedding at write time
Store the model identifier, version, dimension and ingest timestamp beside every vector. A few bytes per row turns a model migration into a dual-write and a cutover: write new vectors under the new identifier, run both indexes side by side, compare recall on the same evaluation set, then switch the read path. Without those columns the migration is a full re-index with no way back.
Send it over and we will tell you what we would change.
Email your corpus size and vector dimension, the filter your queries run under, and ten queries that come back with the wrong documents, to contact@precisionfederal.com. You get back a short written note naming the three things we would change and why. One business day. No charge, no meeting, no deck.
contact@precisionfederal.comMulti-tenancy eliminates half the shortlist on its own
If you serve many customers and one customer's content must never surface in another's results, the selection criteria change entirely. Recall stops being the interesting number and isolation becomes it. Three shapes exist and each has a failure mode.
One collection or index per tenant. Isolation is structural, the strongest guarantee available and the reason regulated buyers ask for it. The cost is operational: most engines carry a per-collection memory floor, so ten thousand collections is a real load and a version upgrade now iterates ten thousand times. Fine into the low hundreds of tenants, painful above that.
One shared index with a mandatory tenant predicate. Efficient, scales to any tenant count, and depends entirely on filtered traversal being implemented correctly and on no engineer ever forgetting the predicate. This is the shape that ships most often and it is also the shape that produces the incident.
Hybrid. A shared index for the long tail of small tenants and dedicated indexes for the handful of large ones. More code, and it is what most mature systems converge to.
On the shared-index route, enforce the predicate below the application layer: a query builder that refuses to construct a search without a tenant scope, and a test asserting a cross-tenant query returns zero rows. An hour of work, and the difference between a control you can show a SOC 2 auditor and a convention you hope people follow.
Selection weights — production retrieval system
Weights sum to 100. Set them before anyone sees a score.
The engines, described honestly
| Engine | What it is | Where it is the right call | Where it stops being the right call |
|---|---|---|---|
| PostgreSQL + pgvector | An extension on a database you probably already run | Up to roughly 10M vectors, when vectors join relational rows, and when you want one backup and one on-call rotation | When the index competes with your transactional working set, or write volume starts moving read latency |
| Qdrant | Purpose-built vector engine with strong payload filtering | Filter-heavy workloads, per-tenant payload scoping, built-in quantization | When you do not want to operate a second stateful service for a corpus Postgres would hold |
| Milvus | Distributed vector database that separates storage from compute | Hundreds of millions of vectors, multiple index families, horizontal scale | Small deployments, where the component count is out of proportion to the problem |
| Elasticsearch / OpenSearch | A lexical search engine that grew dense vector support | You already run it, and hybrid lexical plus dense retrieval is the actual requirement | Very high dimensions at very large scale on a tight memory budget |
| Weaviate | Purpose-built store with an embedding and module ecosystem attached | You want ingestion, embedding and search in one place with less glue code | When you want to own the embedding pipeline and its versioning yourself |
| Managed vector services | The store as a hosted API, including the search products inside the major clouds | Small teams, spiky load, no appetite for stateful operations | When cost per stored vector at your scale passes a node you could run, or data locality is a requirement |
| FAISS, hnswlib, embedded stores | Libraries, not services | Batch jobs, embedded single-process apps, computing your ground truth | The moment you need concurrent writes, durability, filtering or tenancy |
Notice what that table does not contain: a winner. At the scale most companies operate, every engine listed returns approximately the same documents once tuned to the same recall on the same vectors. What differs is filtering behavior under selective predicates, the operational surface you take on, and what happens the day you need to change something. None of those appear in a throughput chart.
Cost, compared in a way that survives scrutiny
Self-hosted cost is a machine plus a fraction of a person, and memory dominates the machine. A 61 GB float32 index needs headroom above it for the operating system, the query heap and index construction, so you are shopping for 96 to 128 GB of RAM. Memory-optimized instances in that class run in the several-hundred-dollars-per-month range on demand, roughly half that on a one-year commitment, and you want two because one is not an availability story. Int8 quantization puts the same corpus on a node a quarter the size.
Managed services price on stored vectors, queries or provisioned capacity, and the break-even against self-hosting arrives sooner than most teams model. Run the comparison at three times your current corpus, because a decision made at one million vectors gets lived with at ten. Include engineer-days on both sides: managed removes node operations and backup verification, and removes none of the chunking, evaluation, filtering or re-index work.
The line item nobody models is embedding generation on re-index. Pushing ten million chunks through a hosted embedding endpoint is a real invoice, and it recurs every time you change models or chunking. Price it once and write the number down, so a model change is decided deliberately rather than discovered in a billing alert.
Where the engineering hours go on a production retrieval build
Typical shape on the retrieval builds we run. The five percent at the bottom is the part that gets six weeks of meetings.
A two-week selection that produces a defensible answer
Selection Sprint
Two weeks is enough because every expensive unknown here is measurable inside it. Whether an engine holds recall under your filters, whether quantization costs quality on your corpus, whether the index survives a month of churn. What is not measurable is which vendor is still independent in three years, which is why step six exists and why portability carries real weight in the rubric.
Patterns that produce a bad choice
- Choosing on a public benchmark run at 768 dimensions when your own embeddings are 3072, where the memory profile is entirely different.
- Post-filtering a selective predicate, getting an empty result, and reading it as an embedding-quality problem.
- Never computing brute-force ground truth, so recall was assumed rather than measured and every comparison since has been unfounded.
- Storing chunk text only inside the vector store, which converts any future model change into a data migration.
- Tuning search parameters until latency looks good without re-measuring recall, which is how you make a system faster by making it worse.
- Running 5 percent weekly churn against an HNSW index with no scheduled rebuild.
- Treating a hosted embedding model as a permanent fixture with no version column and no dual-write path.
Own these regardless of which engine wins
- An evaluation set of real queries with agreed correct answers, in your repository
- A brute-force scoring script that reports recall@k against exact search
- Source text and chunk boundaries stored outside the vector store
- Embedding model name, version and dimension recorded on every row
- An ingest path that re-runs from empty with one command and no manual steps
- A test asserting a cross-tenant query returns zero rows
- Index parameters in version control, never set by hand in a console
- A written re-index runbook with a duration you have actually measured once
Bottom line
The honest version of this decision is smaller than the discourse around it. Under ten million vectors, use the database you already operate and spend the six saved weeks on chunking and evaluation, where the quality actually lives. Above that, the choice is settled by how the engine filters under selective predicates, how much memory you will buy after quantization, and whether you serve one tenant or ten thousand. Measure recall on your own vectors against exact search first, because without that number you are comparing marketing, and marketing has never returned a correct document.
Frequently asked questions
Below roughly ten million vectors, usually not. A relational database with a vector extension gives you transactions, joins to the source rows and one operational footprint. Move when the index competes with your transactional working set, when writes start moving read latency, or when tenancy has to be structural.
Start with 4 bytes per dimension per vector. At 1536 dimensions that is 6.1 GB per million in float32, plus 150 to 250 bytes per vector of HNSW graph. Int8 cuts it by four, binary by thirty-two with rescoring, dimension truncation by whatever fraction you cut. Budget headroom above the index for the OS, the query heap and index construction.
Almost certainly post-filtering: the engine finds nearest neighbors first and applies the predicate afterward, so a selective filter removes all of them. The fix is predicate evaluation during graph traversal, or a pre-filter path for small matching sets. Test it by asking for 20 results under a filter matching half a percent of your data.
Two separate numbers. Recall@k against brute-force exact search says whether the index finds what exact search would. End-to-end answer quality on labeled real queries says whether the system is useful. Keep them apart: a system can hold high recall and still answer badly when the chunks were wrong.
You re-embed the corpus, rebuild the index and re-run the evaluation. That is a scheduled project if source text and chunk boundaries live outside the store, the model name and version are recorded per row, and ingest re-runs from empty with one command. Without those three it is an outage with a migration attached.