Best Vector Database for RAG: 8 Options + When to Add Graph

The best vector database for RAG is the one whose retrieval model matches your questions. For plain top-k similarity over chunks, pgvector, Pinecone, Milvus, Zilliz Cloud, Weaviate, Chroma and Qdrant all work. For RAG that has to follow relationships between entities, HelixDB runs graph, vector and BM25 search in one engine, so you skip the second graph database.

That is the whole post in two lines. The rest is the evidence: seven criteria for ranking a RAG vector store, a fair read of each option from its own documentation, and the point where similarity search stops being enough.

How we ranked them: seven criteria that decide a RAG vector store

A vector database in retrieval-augmented generation does one job: it stores chunk embeddings and returns the nearest ones to a query embedding, which you paste into the prompt. Everything that separates the options sits around that job. These are the seven criteria we use, in the order they usually bite.

1. Where the index lives. RAM-only indexes are fast but cap you at memory size. Disk and object-storage tiers give up a little latency for much larger corpora and lower cost.

2. Hybrid keyword plus vector. Pure embeddings miss exact terms like error codes, SKUs and names. BM25 (the standard keyword-ranking function) next to ANN (approximate nearest neighbour) search fixes that.

3. Metadata filtering, before or after ANN. A post-filter runs the vector search first and then discards rows, so a selective filter can return fewer results than you asked for. pgvector's own docs say this directly: with approximate indexes, filtered queries can return fewer results because filtering is applied after the index is scanned (pgvector).

4. Relationship-following retrieval. Can a query hop from a chunk to the entity it mentions, then to other documents tied to that entity? Plain similarity cannot.

5. Transactions. Do writes to your vectors and your other data commit together?

6. Licence. Apache-2.0 and similar permissive licences let you embed and ship. Others restrict it.

7. Operations. Embedded, self-hosted, or managed, and how much of that you want to run.

We have not benchmarked these engines against each other. Vendor benchmarks run each engine on its own settings, so the ranking below follows these criteria and each product's own documentation, not a speed chart.

The short list: pgvector, Pinecone, Milvus, Zilliz Cloud, Weaviate, Chroma, Qdrant

These seven names come up in nearly every comparison of popular vector databases for RAG, and we cover them from their own docs. HelixDB is the eighth entry and gets its own section later.

pgvector is open-source vector similarity search for Postgres. It supports Postgres 13 and later, does exact nearest-neighbour search by default, and adds HNSW and IVFFlat approximate indexes (pgvector). Its README covers filtering, multitenancy, iterative index scans, half-precision and binary vectors.

Qdrant documents a rich filtering model: must, should and must-not clauses, plus match, range, geo, nested-key, full-text and has-ID conditions that you apply when searching or retrieving points (Qdrant docs).

Chroma is an embedding database built to handle retrieval for AI applications. You can self-host it or use Chroma Cloud (Chroma docs).

Milvus is a vector database built for large-scale similarity search workloads. Zilliz Cloud is the managed offering of Milvus.

Weaviate is a vector database built with hybrid search capabilities out of the box.

Pinecone is the reference managed vector database, per the same comparison.

On the cloud side, Google's Vertex AI RAG Engine defaults to RagManagedDb, a managed vector database that needs no extra provisioning, and lists other supported vector databases you can plug in instead. If you are on AWS, Azure or GCP, check which stores your provider's managed RAG service supports before you pick one.

Where each one is genuinely the right pick for plain top-k RAG

For top-k similarity over chunks, every store above does the job, and the real differences are operational. Match the pick to what you already run.

pgvector fits when your application data already lives in Postgres and you want vectors in the same database, queried with the same SQL and the same backups. The cost of that convenience is the post-filter behaviour above, which you manage with iterative index scans.

Pinecone and Zilliz Cloud fit when you want someone else to run the index. Managed operations are a real win, and no embedded or self-hosted option competes with it on effort.

Milvus fits a large pure-similarity corpus where scale is the main constraint.

Weaviate fits when you want BM25 and vector search fused in one query out of the box.

Qdrant fits when your queries carry heavy metadata conditions and you want a documented filtering model to express them.

Chroma fits a prototype or an embedded start. It is simple to stand up, Apache-2.0, and has a cloud option when you outgrow local.

The ecosystem and managed-ops advantages of these stores are real, and for a corpus where the nearest few chunks answer each question, they decide the choice. They stop deciding it when the answer is not in the nearest chunks.

The wall: when RAG needs relationships, not just nearest chunks

A vector store fails RAG when the question depends on how things connect. Similarity ranks chunks by resemblance to the query and says nothing about how those chunks relate to each other.

Take a question like "which contracts involve vendors that were flagged in last quarter's audit?" The audit finding, the vendor and the contract are three different documents. Top-k retrieval returns the three chunks closest to the question's wording, probably audit-finding text, and never walks from the finding to the vendor to the contract. The connected context is missed even though it sits in your corpus.

The second symptom is filtered search returning too few results. You scope retrieval to one customer or project, the ANN index returns its nearest candidates across the whole corpus, the filter then removes most of them, and you are left with two chunks instead of ten.

The usual fix is to add a graph database next to the vector store and keep the two in sync. That means entity IDs duplicated across systems, two write paths, two bills, and a join you maintain in application code. That is the two-database tax. If you are weighing graph options, we compare them in Best Graph Database for a RAG Pipeline: 8 Options Compared, and When to Use a Knowledge Graph Instead of Embeddings covers when the graph is worth it at all.

HelixDB: graph, vector and BM25 in one engine, with pre-filtering scoped by the graph

HelixDB is an open-source graph-vector database written in Rust that runs graph traversal, approximate vector search and BM25 full-text search in one engine. Vectors are a property on nodes and edges, so a chunk, the entity it mentions and the edge between them live in the same store and the same transaction.

The feature that matters for RAG is vector pre-filtering. Vector search can be scoped to things that are related to each other, so similarity runs over a graph-filtered candidate set instead of the whole index. Entity-scoped retrieval becomes one query: walk from the vendor to its contracts, then rank only those chunks by similarity, optionally combined with BM25. That is the case where a pure vector store forces a second database. A multi-hop walk followed by a scoped ranking is the shape. We walk through it end to end in How to Build a GraphRAG Pipeline: From Documents to Scoped Retrieval and cover the keyword side in How to Combine BM25 and Vector Search in One Query for RAG.

On the other criteria: the index runs across memory, disk and object storage, so RAM does not cap you. Graph and vector writes commit or roll back together, and HelixDB Cloud runs serializable snapshot-isolation transactions. The core is Apache-2.0 at github.com/HelixDB/helix-db. You can run it embedded in your application, self-hosted in Docker (see Self-Hosted Vector Database With Graph Queries: HelixDB in Docker), or on HelixDB Cloud. You query it using native SDKs without needing to learn a separate query language. The HelixDB docs cover each SDK.

On raw similarity speed, HelixDB's own benchmarks report vector performance on par with Pinecone and Qdrant. That is a vendor benchmark, so treat it as one. The reason to choose HelixDB is not that it out-searches a dedicated vector store at plain top-k. It is that you stop needing the second database when retrieval has to follow relationships.

A decision table: which retrieval need maps to which store

Match your retrieval need to the store, using the criteria above.

Retrieval need	Good fit	Why
Plain top-k over chunks, data already in Postgres	pgvector	Same database, SQL, backups; manage post-filter recall with iterative scans
Plain top-k, no infrastructure to run	Pinecone, Zilliz Cloud	Managed operations
Very large pure-similarity corpus	Milvus	Scale-oriented engine
Hybrid BM25 plus vector, simple setup	Weaviate	Hybrid query built in
Heavy metadata conditions on similarity search	Qdrant	Documented must/should/must-not filtering
Prototype or embedded start	Chroma	Apache-2.0, simple start, cloud option
Multi-hop retrieval, entity-scoped search, graph plus vector plus BM25	HelixDB	One engine, pre-filtering scoped by the graph, no second database

If you can't name a question your users ask that needs a hop between documents, start with the simplest store that fits your stack. If you can, or a graph database is already creeping in, start from the engine that has both.

Pick by retrieval shape, then try the one engine that covers both

Plain top-k RAG has many good answers, and the stack you already run usually decides it. Relationship-aware RAG has a narrower answer, because bolting a graph database onto a vector store means two systems to sync. If your questions need multi-hop or entity-scoped retrieval, clone github.com/HelixDB/helix-db, run it in Docker or embedded, load one slice of your corpus as chunks and entities, and run a scoped query against your hardest question. Star the repo if it gets you there.