Choosing a Vector Store
In one line: the honest default is the database you already run, and a dedicated vector store earns its place at a scale and a query pattern you should be able to name.
The four options
| Option | Strength | Weakness |
|---|---|---|
| A library, in-process | No service to run; fastest possible path | You own persistence, replication and updates |
| Your existing relational database | One store, real transactions, joins with metadata | Weaker at very large scale and high write rates |
| Your existing search engine | Lexical and dense in one system — hybrid is native | Heavier to operate; vector support is newer |
| A dedicated vector database | Best scale, filtering and index tooling | Another system, another copy of the data, another failure mode |
Start with what you already run
The consideration that decides most real cases has nothing to do with recall benchmarks: a separate vector store means your vectors and your source data are in different systems.
That creates a synchronisation problem with no clean answer. A document updated in the primary database and not yet re-indexed is a stale search result. A failed write to one system and not the other is a permanent inconsistency. There is no transaction spanning both.
Keeping vectors in the primary database removes that entire class of problem. The vector is a column, the write is one transaction, filters are ordinary predicates evaluated by a planner that has statistics, and joining against permissions or metadata is a join rather than a two-phase application-level dance.
That last point is underrated. Filtering is the hard problem, and a relational database has spent decades on exactly the cardinality estimation that decides whether to scan or use an index.
What a dedicated store actually buys
Being fair to the other side, because dismissing it is as wrong as reaching for it first.
Scale. Purpose-built systems handle hundreds of millions to billions of vectors with sharding, replication and distributed search built in.
Filtering that was designed in. The best implementations integrate predicates into graph traversal rather than bolting them on — which, given the selectivity cliff, is a genuine engineering difference.
Index tooling. Multiple index types, quantisation options, tunable parameters, and the ability to rebuild or reconfigure without writing it yourself.
Operational features. Snapshots, live reindexing, multi-tenancy primitives, and dedicated observability for recall and index health.
That last one matters more than it sounds. Building tombstone-ratio monitoring, filtered-recall measurement and index-age tracking yourself is real work you will otherwise skip — and the operations lesson showed those are exactly the metrics whose absence causes silent decay.
The questions that decide it
Scale is the first question and not the only one. Four others change the answer:
Do you need hybrid search? If yes, a search engine that does both natively removes a whole fusion layer and a second sync path.
How selective are your filters, and how often do they run? Heavy filtering at low selectivity is where dedicated implementations are genuinely differentiated.
What is the write rate? A corpus rebuilt nightly is very different from one absorbing thousands of updates a second, and the second is where general-purpose stores struggle first.
Who operates it? A team of four should not be running a distributed vector database if a column in the existing database works. Operational capacity is a real constraint and saying so is a strength, not an admission.
The migration you should keep possible
Whatever you pick, keep the option to change it. That means keeping the source text and metadata in a system you control, treating the index as derived data that can be rebuilt from scratch, and not letting application logic depend on store-specific query semantics.
An index you can rebuild is a decision you can revisit. One holding the only copy of something is a permanent commitment made early, with the least information you will ever have.
Key takeaway
Default to the database you already run: the vector becomes a column, the write stays transactional with its source document, and filters become predicates evaluated by a planner with real statistics — which removes the sync problem that a separate store creates and cannot solve. A dedicated store earns its place on scale past roughly ten million vectors, heavy low-selectivity filtering, high write rates, or the operational tooling for recall and index health. Whichever you choose, keep the source text and treat the index as rebuildable derived data.
Next: the walkthrough, where all of this gets chosen live.