Vector Databases, Explained Simply: Do You Need One, and Which Kind?

Part 3 of the RAG series. The database behind Part 2 has a frightening name and a simple job.

In the last part we turned documents into long lists of numbers, one list per chunk, each list capturing what that chunk means. The question is where those numbers live, and how the system finds the right ones so fast.

A normal database finds exact things

A normal database is a filing cabinet with perfect labels. You ask for invoice 1042, it gives you invoice 1042. You ask for every customer in Virginia, it walks the index and gives you exactly those rows. It is precise, fast and completely literal. If the words do not match, nothing comes back.

A vector database works differently. It does not store rows to look up exactly. It stores points in space, one point per chunk, placed so that chunks with similar meaning sit close to each other. Searching means dropping your question into that same space and asking a simple question: what is near this point?

Near means similar in meaning. That is why a question about staff leaving can find a paragraph about employee turnover. The words differ. The meaning does not, and meaning is what got stored.

Do you actually need a special database?

Here is the part vendors rarely say out loud: for many companies, no, not a separate one.

There are dedicated vector databases with names you will hear in sales calls: Pinecone, Qdrant, Weaviate, Milvus and others. They are good tools. If you are searching hundreds of millions of chunks, you may genuinely need one.

But there is a quieter option. Databases many companies already run have learned this trick. PostgreSQL does it with an extension called pgvector. SQL Server and the big cloud search services offer vector search as a feature. If your document pile is thousands or tens of thousands of files, which describes most mid sized companies, the database you already have may do the job well, inside the building, under the access rules you already trust.

How to choose, without overbuying

Ask four questions, in this order.

How big is the pile, honestly? Not where you dream it will be in five years. What you have now decides what you need now. Systems can grow later.

Who will look after it? Every new platform is a new thing to patch, back up and understand at 2 a.m. A feature inside a database your team already runs is a smaller promise to keep.

Where must the data live? Some companies cannot let documents, or even their numeric fingerprints, leave their own environment. That answer picks the tool for you, before price is even discussed.

What does it cost at your size, not at demo size? Per search pricing looks tiny until a whole company is asking questions all day. Do the arithmetic with your real number of users.

The rule we give clients

Start with the simplest thing that answers your questions correctly. For most companies we meet, that is vector search inside a database they already own, added to documents they already have. Buy the dedicated platform when your scale proves you need it, not when a brochure says you might.

A database is a means to an answer. The answer is the product. In the next three parts of this series we stop talking about machinery and look at what this does to real piles of documents, starting with contracts.