Crafting Excellence in Software
Let’s build something extraordinary together.
Rely on Lasting Dynamics for unparalleled software quality.
Nunzio Giugliano
Jul 16, 2026 • 14 min read
A team we advised last year had a RAG pilot that returned excellent answers in the demo and embarrassing ones in production. Same model, same prompts, same documents. The only thing that changed was scale: 4,000 documents in the demo, 900,000 in production. Their vector database (chosen in an afternoon because it was the default in a tutorial) was silently dropping recall as the index grew, and nobody was measuring it. The fix was not a better model but was a different vector database for RAG and a re-indexing strategy that respected how their corpus actually behaved.
That story is common enough to be a genre. The vector store is the least glamorous component of a retrieval-augmented generation system and one of the two or three that most reliably decide whether it works. This guide is written from the seat of an engineering team that has deployed Pinecone, Weaviate, Qdrant, Milvus, Chroma and pgvector in real client systems, not from a vendor's feature matrix. It is a companion to our enterprise RAG architecture guide; if you want the whole pipeline, start there. If you have already decided to build RAG and now need to pick where the embeddings live, start here.
A vector database for RAG has one job that a normal database cannot do well: given a query embedding, find the nearest neighbours among millions or billions of stored embeddings, fast, without scanning all of them. It does this with an approximate nearest-neighbour (ANN) index, usually HNSW, sometimes IVF or a disk-based variant. "Approximate" is the word that matters. Every vector database trades a small amount of recall for a large amount of speed, and where each one sits on that curve is the difference between a RAG system that finds the right chunk and one that confidently retrieves the wrong one.
The failure mode is insidious because it is invisible in a demo. At 5,000 vectors, nearly any configuration returns the correct neighbours, because there are so few candidates that even a crude index finds them. At ten million, the index parameters (the ef search width, the number of graph connections, the quantization settings) start to decide your recall. A system that "worked" for six months has usually been degrading the whole time; the team just never had an evaluation harness pointed at retrieval quality to notice.
So the choice is not really "which product has the nicest API." It is: which engine gives you the recall you need, at the query latency you can afford, at the throughput your traffic demands, at a cost that does not balloon when the corpus triples, and which one your team can actually operate. Those five constraints rarely point at the same product, which is why the honest answer to "what is the best vector database for RAG " is always a question in return.
It helps to stop thinking of these as six competing products and start thinking of them as five categories that solve different problems. Most bad decisions come from comparing a database in one tier against a database in another as if they were interchangeable.
The fully-managed serverless tier is Pinecone's home ground: you never touch an index parameter, never run a node, and you pay for that in per-query cost and in vendor lock-in. The open-source, self-hostable engine tier (Qdrant, Weaviate, Milvus) gives you control of the deployment and the best performance-per-dollar at scale, in exchange for owning the operations. The embedded / prototyping tier is Chroma: brilliant for getting a RAG loop working on your laptop in an afternoon, not designed to be the thing serving a million users.
The bolt-on to an existing database tier is pgvector, which adds vector search to PostgreSQL you already run, a genuinely good answer when your scale is moderate and your operational appetite is low. And the billion-scale distributed tier is Milvus's real territory, where the dataset is large enough that a distributed architecture stops being overkill and starts being necessary.
The reason chroma vector database is one of the highest-volume searches in this space is not that Chroma is the best production store, it is that it is the first vector database for RAG most engineers ever touch. That is a distinction worth holding onto as we go.
| Database | Model | Index | Hybrid search | Realistic ceiling | Best for |
|---|---|---|---|---|---|
| Pinecone | Managed serverless / pods | Proprietary (HNSW-family) | Sparse-dense supported | Very high (managed) | Teams with no infra appetite |
| Qdrant | Open source / managed cloud | HNSW + payload filtering | Native (dense + sparse) | High, tens of millions+ | Best perf-per-dollar self-hosted |
| Weaviate | Open source / managed cloud | HNSW | Native BM25 + vector fusion | High | Hybrid search + built-in modules |
| Milvus | Open source / Zilliz cloud | IVF, HNSW, DiskANN, GPU | Supported | Billion-scale distributed | Very large corpora, high QPS |
| Chroma | Embedded / lightweight server | HNSW | Sparse + dense (BM25, RRF) | Low–moderate | Prototyping, LangChain tutorials |
| pgvector | PostgreSQL extension | HNSW + IVFFlat | Via SQL + full-text | Moderate | Teams already on PostgreSQL |
Pinecone is what you choose when the most expensive resource on your team is engineering time, not cloud spend. Its serverless tier removed the old "how many pods do I provision" guessing game, and for a team shipping a first production RAG feature without a platform group behind it, that operational silence is worth real money. The trade is the obvious one: you cannot see inside the index, you cannot self-host it for a data-residency requirement, and at scale the per-query economics stop being charming. Pinecone is the right first production database and, for many teams, the wrong tenth one.
Let’s build something extraordinary together.
Rely on Lasting Dynamics for unparalleled software quality.
Qdrant is the database we reach for most often when a client can run their own infrastructure and cares about cost-per-query at scale. Written in Rust, it pairs an HNSW index with genuinely first-class payload filtering, the ability to say "nearest neighbours, but only among documents this user is allowed to see, in this jurisdiction, from this date range" without wrecking recall. That filtered-search quality is not a footnote; in enterprise RAG it is usually a hard requirement, and Qdrant handles it better than most. Its native support for sparse vectors alongside dense ones makes hybrid search a configuration choice rather than a second system to run.
Weaviate occupies adjacent ground with a different emphasis. Its hybrid search (fusing BM25 keyword scoring with vector similarity) is a first-class, well-documented path rather than something you assemble yourself, and its module ecosystem (built-in vectorizers, re-rankers, generative steps) appeals to teams that want more of the pipeline inside one system. Choosing between Qdrant and Weaviate is rarely a performance argument; it is a question of whether you want a lean, fast engine you compose yourself (Qdrant) or a more batteries-included platform (Weaviate). Both are open source, both self-host, both scale well past the point where most enterprise corpora actually live.
Milvus is the answer to a question most teams do not have yet. It is built for the billion-vector, high-QPS world, with a distributed architecture, multiple index types including GPU and disk-based options, and the operational surface area that implies. If your corpus is genuinely enormous, web-scale, multi-tenant at massive volume, or a foundation-model-adjacent retrieval workload, Milvus (or its managed form, Zilliz) earns its complexity. If your corpus is two million internal documents, Milvus is a data-center-grade tool for a workshop-grade job, and you will spend your time operating it rather than improving retrieval.
What is the difference between Pinecone, Weaviate, Qdrant, and Milvus? Pinecone is the only fully-managed, serverless option, zero ops overhead, highest cost at scale. Weaviate and Qdrant are open-source and self-hosted, offering best performance-per-dollar with hybrid search support. Milvus handles billion-scale datasets via distributed architecture. Chroma is the fastest to prototype with but not production-grade at scale.
Chroma vector database deserves an honest section precisely because it is so popular. It is the fastest way to get from zero to a working retrieval loop (pip install, add documents, query, done) which is exactly why it is the default in most LangChain tutorials and why so many people search for it by name. That accessibility is a real virtue for prototyping, teaching, and small internal tools.
Chroma has also closed the hybrid-search gap: it now ships first-class sparse vectors (BM25 and SPLADE) fused with dense similarity through reciprocal rank fusion — the same mechanism this guide praises in Weaviate and Qdrant. So the reason enterprises graduate away from Chroma is no longer retrieval capability; it is operational scale — multi-tenancy, high-concurrency serving, and access control under production traffic. Starting on Chroma is a good decision; the migration question is now about operations, not search.
pgvector is the pragmatist's choice and a genuinely underrated one. If your data already lives in PostgreSQL, adding the pgvector extension lets you do similarity search next to your relational data, inside transactions you already trust, with backups and access control you already operate. Recent versions support HNSW indexing, which closed most of the performance gap that used to disqualify it.
For a corpus in the low millions where you value operational simplicity over raw ANN throughput, pgvector is often the right answer and "we do not need a new piece of infrastructure", is a legitimate architectural win. Push into the tens of millions with high concurrency and a dedicated engine will pull ahead, but plenty of production RAG systems never reach that point.
The comparison that actually changes decisions is not feature-by-feature, it is total cost of ownership at a scale you can picture. Take a realistic mid-size RAG workload: ten million embedded chunks at 1,024 dimensions, moderate query traffic, hybrid search on.
On a fully-managed serverless database, that footprint carries a predictable monthly storage-and-query bill that rises with query volume and never includes an engineer's salary. On a self-hosted Qdrant or Weaviate cluster on the same cloud, the raw infrastructure cost for equivalent capacity is typically a fraction of the managed price (often several times cheaper per month at this scale) but you add the real, ongoing cost of operating it: provisioning, upgrades, monitoring, on-call, and the re-indexing runs that any serious system needs when it changes embedding models.
From idea to launch, we craft scalable software tailored to your business needs.
Partner with us to accelerate your growth.
The break-even is a staffing question disguised as an infrastructure one. If you already run a platform team that operates stateful services, self-hosting an open-source vector database for RAG is almost always cheaper in total, and the savings widen as query volume grows. If you do not and if standing up this system means one engineer learning to operate a new distributed database on nights and weekends, the managed option is frequently cheaper once you count the engineering time honestly.
We have watched teams "save money" by self-hosting and then spend three engineer-months on operations that would have paid for years of a managed tier. The right way to model this is the same discipline we apply to AI SaaS development cost: count the people, not just the invoices.
Pure vector search has a specific, repeatable weakness: it is bad at exact tokens. Part numbers, contract IDs, drug names, error codes, ICD-10 codes, an unusual surname (the rare, high-signal terms that enterprise users actually search for) get smoothed into semantic mush by dense embeddings. A user asks about "policy AX-4471-C" and vector search helpfully returns documents about policies in general.
Hybrid search fixes this by running a lexical index (BM25 or similar) alongside the vector index and fusing the results, usually with reciprocal rank fusion. The keyword side nails the rare exact terms; the vector side catches the semantic matches that keywords miss; the fusion gives you both. In our experience it is the single retrieval change that most often moves an enterprise RAG system from "impressive but unreliable" to "trusted." This is why native hybrid support, the way Weaviate and Qdrant provide it, is a real selection criterion and not a checkbox. A database that forces you to run and synchronise a separate keyword system is a database that will eventually drift out of sync at 2 a.m.
Public benchmarks are useful and routinely misused. ANN-Benchmarks remains the standard reference for the recall-versus-throughput trade-off across index implementations, and vendors including Qdrant publish their own reproducible suites. Read them, and then discount the leaderboard position, because published benchmarks run on curated datasets with tuned parameters that will not match your corpus, your dimensionality, your filter patterns, or your latency budget.
The number that matters is not "database X did 2,000 QPS in someone's benchmark." It is recall@k on your evaluation set, at your latency ceiling, with your metadata filters applied; because filtered search behaves very differently from unfiltered search, and most enterprise queries are filtered. The only benchmark that should decide your architecture is the one you run on a representative slice of your own data. Any team that picks a vector database for RAG from a leaderboard without a private evaluation harness is choosing with someone else's homework. Do not fabricate a QPS figure to justify a decision; measure the one that applies to you.
Strip away the vendor noise and the choice of a vector database for RAG reduces to a short sequence of honest questions.
Start with operations. If nobody on the team will own a stateful distributed service, choose managed (Pinecone, or the managed cloud of Qdrant/Weaviate/Zilliz) and stop optimising for a cost you will pay back in incidents. If you have a platform team that already runs databases, self-hosting open source vector database is on the table and usually cheaper.
Then ask about your existing stack. If your data lives in PostgreSQL and your scale is in the low millions, try pgvector first; the best new infrastructure is often no new infrastructure. Only reach past it when you have measured a real ceiling.
We design and build high-quality digital products that stand out.
Reliability, performance, and innovation at every step.
Then ask about scale and search shape. Billion-vector or very high QPS points at Milvus. Heavy filtered search and cost-sensitive scale points at Qdrant. Hybrid search as a first-class need, with more of the pipeline in one platform, points at Weaviate. A first production feature with no infra team points at Pinecone. And a laptop prototype you will replace points at Chroma; deliberately, knowing you will migrate.
Is Pinecone the best vector database? Pinecone is the easiest to set up and best for teams without infrastructure expertise. However, at enterprise scale (millions of embeddings, high QPS) Qdrant and Weaviate typically deliver better cost efficiency. Pinecone's serverless tier suits prototyping; dedicated pods suit production.
Notice that this framework never produces a single global winner, because there is not one. It produces the right database for a specific set of constraints, which is the only kind of "best" that means anything in production.
The mistakes repeat across teams and verticals, which is what makes them worth naming.
The first is choosing by demo. A database that shines at 5,000 vectors tells you nothing about its behaviour at ten million, and the tutorial default becomes a load-bearing production decision by accident. The second is ignoring filtered search.Teams benchmark unfiltered nearest-neighbour queries, ship, and then discover that adding the access-control and jurisdiction filters they legally require tanks recall, because they never tested the query shape they actually run.
The third is treating hybrid search as optional; it is not, for any corpus with codes, IDs, or proper nouns. The fourth is under-budgeting re-indexing: embedding models get upgraded, and a system with no plan to re-embed its corpus quietly rots. And the fifth, the most expensive, is choosing without an evaluation harness, shipping a system whose retrieval quality nobody measures, so that "it works" means "no one has complained loudly enough yet."
None of these are exotic. They are the predictable consequences of picking infrastructure before you have decided how you will know it is working. If you are earlier in the journey and still assembling the surrounding system, our notes on building AI-native applications and the fundamentals of how to build an AI cover the ground around this decision.
What is the best vector database for RAG in 2026? For most production RAG teams, the best choice depends on your deployment model: Pinecone for fully-managed serverless; Qdrant or Weaviate for self-hosted open-source; pgvector if you're already on PostgreSQL and don't need billion-scale. For enterprise RAG at scale with hybrid search, Weaviate and Qdrant lead on performance-per-dollar.
There is one more question people ask by name, so it is worth answering plainly.
What vector database does LangChain use? LangChain natively integrates with all major vector databases: Pinecone, Weaviate, Qdrant, Milvus, Chroma, pgvector, and FAISS. Chroma is the default in most LangChain tutorials due to zero setup. For production RAG, LangChain's integrations with Pinecone and Qdrant are the most battle-tested.
If you take one thing from this vector database comparison, make it this: the vector database is a constraint-satisfaction problem, not a popularity contest, and the constraint you underweight is almost always operations. Pick the engine that matches how your team works and how your corpus behaves, wire an evaluation harness to it before you ship, and re-check recall as the index grows. The database that best fits those realities is the best vector database for your RAG system, regardless of what tops a benchmark this quarter.
Building or repairing a production RAG system and weighing the infrastructure underneath it? Our enterprise RAG architecture guide covers the full pipeline: ingestion, chunking, retrieval, re-ranking and evaluation; that this database choice sits inside.
This vector database comparison was written by Nunzio Giugliano at Lasting Dynamics, an EU-based custom software development company. We are deliberately model- and vendor-agnostic: in any given quarter we ship production RAG systems on Pinecone, Qdrant, Weaviate, Milvus and pgvector, choosing each one against the client's real scale, budget and operational constraints rather than a vendor's pitch. Every trade-off above comes from those engagements, not from a datasheet.
A vector database stores embeddings and, given a query embedding, finds the nearest neighbours among millions or billions of them in milliseconds using an approximate nearest-neighbour (ANN) index such as HNSW. RAG needs one because retrieval quality, finding the right chunk to ground the model, is what separates a reliable system from one that confidently retrieves the wrong context, and a normal database cannot do fast similarity search at that scale.
It depends on staffing, not just invoices. A self-hosted open source vector database like Qdrant, Weaviate or Milvus is usually several times cheaper per month in raw infrastructure at scale, but only if you already have a platform team to operate it. If standing it up means one engineer learning a new distributed database on nights and weekends, a managed option (Pinecone, or managed Qdrant/Weaviate/Zilliz) is often cheaper once you count the engineering time honestly.
For many production RAG systems, yes. If your data already lives in PostgreSQL and your corpus is in the low millions, pgvector adds similarity search next to your relational data, with the backups and access control you already run, and recent HNSW support closed most of the old performance gap. Move to a dedicated engine only when you push into the tens of millions with high concurrency and have measured a real ceiling.
Hybrid search runs a lexical index (BM25) alongside the vector index and fuses the results, usually with reciprocal rank fusion. Pure vector search is weak on exact tokens (part numbers, IDs, drug names, error codes) which dense embeddings smooth into semantic mush. Hybrid fixes that: the keyword side nails rare exact terms while the vector side catches semantic matches. It is often the single change that moves an enterprise RAG system from unreliable to trusted, which is why native support in Weaviate and Qdrant is a real selection criterion.
It is rarely a performance argument. both are open source, both self-host, and both scale well past most enterprise corpora. Choose Qdrant if you want a lean, fast Rust engine with first-class payload filtering that you compose yourself and tune for cost-per-query. Choose Weaviate if you want a more batteries-included platform with first-class hybrid search and built-in modules (vectorizers, re-rankers and generative steps) inside one system.
Start with operations: if no one will own a stateful distributed service, go managed. Then your stack: if you are already on PostgreSQL at low-millions scale, try pgvector first. Then scale and search shape, billion-vector or very high QPS points at Milvus; heavy filtered, cost-sensitive scale at Qdrant; first-class hybrid search at Weaviate; a first production feature with no infra team at Pinecone; a throwaway laptop prototype at Chroma. There is no single winner, only the best vector database for RAG that fits your specific constraints.
Transform bold ideas into powerful applications.
Let’s create software that makes an impact together.
Nunzio Giugliano
Nunzio Giugliano is a Marketing Specialist at Lasting Dynamics, where he works on SEO strategy, technical content and B2B research across AI, enterprise software and digital transformation topics. His work focuses on making complex technology subjects clear, useful and accessible for CTOs, founders, decision makers and technical teams.
