Is RAG Safe for Company Documents?

Part 8 of the RAG series. The question we hear most from owners, asked in almost the same words every time: what happens to my documents?

It is the right question, and it deserves a straighter answer than it usually gets. So, in plain terms, here is where your documents go in a RAG system, and the decisions that decide whether they stay yours.

Do my documents train the AI?

In a properly built business system, no. Your documents are stored, cut into pieces and turned into numbers for searching, as Part 2 described. When a question is asked, the relevant pieces are shown to the model so it can write the answer. That is a reading, not a training. The model does not keep your files, and the next company asking questions gets nothing of yours.

But “properly built” is doing work in that sentence. Consumer tools and business tools are different animals. The question to ask any vendor, in writing, is simple: are our documents and questions used to train models, yes or no, and how long are they kept?

Where do the documents live?

That is a design decision, not a law of nature, and it is yours to make. The documents and their numeric fingerprints can live in your own cloud account, or on your own servers, with the AI model reached through an enterprise connection that keeps your data out of training and out of long term storage on the other side. For genuinely sensitive material, the model itself can run inside your environment, so nothing leaves the building at all. Each step toward private costs more and runs a little less magically. Which step is right depends on what is actually in the pile, contracts and procedures are one thing, health records are another.

Who inside the company can ask what?

This is the part people forget. If one system can read every document, then every user of that system can effectively read every document, by asking questions. A serious implementation carries your existing permissions into the search: a person finds only what they were allowed to open in the first place. If a vendor's demo cannot explain how permissions work in their system, the demo is not finished, however pretty it looks.

Questions to ask before you sign anything

• Are our documents used for training, in any form, ever?
• Where are documents, embeddings and question logs stored, and for how long?
• Can the whole thing run inside our environment if we need it to?
• Do search results respect our document permissions per user?
• When the contract ends, what exactly is deleted, and how do we verify it?

A vendor who answers these calmly and specifically is telling you something about themselves. A vendor who waves them away is also telling you something.

Our position, since we sell this

We build these systems, so weigh our words accordingly. But our view is simple: your documents are your company's memory, and a system that treats them casually is not a bargain at any price. Safety in this technology is not a feature you switch on. It is a set of decisions, made early, by people who expect you to ask these questions.

Which brings us to the last part of the series. Enough explanation: go and ask some documents yourself.