AI
RAG without leaking your data: a checklist before you ship an AI assistant
By Zettops. Published · 3 min read
An assistant that answers from your documents can also expose them. Here is what to check at indexing, retrieval, generation, and logging before launch.
Retrieval-augmented generation, or RAG, lets an AI assistant answer from your own documents instead of guessing. That is exactly what makes it useful, and exactly what makes it risky: if the system can read a document, it can also repeat it to the wrong person.
Where RAG systems leak
Most data exposure in RAG does not come from the model itself. It comes from the plumbing around it:
- Retrieval that ignores permissions. Every document lands in one index, and anyone who can ask a question can surface any passage.
- Instructions hidden in documents. A file contains text written to steer the model, such as asking it to reveal other content or ignore its rules.
- Logs and traces. Full prompts, retrieved passages, and answers are stored in plain text, often with longer retention than the source data.
- Third-party providers. Data is sent to a model or embedding API without checking how that provider stores or uses it.
- Citations that overshare. The answer is fine, but the source link opens a document the user should never see.
Before indexing
Decide what goes in before anything is embedded.
- Classify the sources. List each document collection, who owns it, and who is allowed to read it.
- Exclude what does not belong. Credentials, personal records, and drafts rarely need to be searchable. Leave them out rather than relying on filters.
- Carry permissions with every chunk. Store the access rules alongside each indexed passage, so retrieval can enforce them.
- Plan for deletion. When a source document is removed or its permissions change, the index must follow.
At query time
Retrieval: filter first
Enforce access before passages reach the model, not after the answer is written.
- Filter search results by the current user's permissions as part of the query.
- Never ask the model to decide what a user may see. It is not an access control system.
- Check that citation links go through the same authorization as the rest of your app.
Tip: test with two accounts that have different access. Ask the same question from both and compare which sources appear.
Generation: treat documents as untrusted
Retrieved text is input, and input can be hostile.
- Keep system instructions separate from retrieved content, and tell the model plainly that retrieved text is reference material, not instructions.
- Limit what the assistant can do. If it can call tools or take actions, each one needs its own permission check on the server.
- Ground answers in sources and let the assistant decline when the sources do not cover the question, instead of filling gaps.
In our Lumen concept, a research assistant built around company documents, this is the core design rule: every answer is generated only from retrieved passages, each claim links to its source, and questions the sources cannot answer are declined.
Logging and providers
| Area | Question to answer before launch |
|---|---|
| Logs | Do we store full prompts and passages? For how long, and who can read them? |
| Redaction | Are personal details masked before anything is logged? |
| Model provider | What do the provider's terms say about retention and training on our data? |
| Embeddings | Where are vectors stored, and are they protected like the source data? |
| Region | Does data leave the region our customers expect? |
Vectors are derived from your text, so protect the vector store with the same care as the documents themselves.
Test it like an attacker
Quality evaluation for RAG usually focuses on whether answers are correct. Add a security set to the same evaluation:
- Questions a given user should not be able to answer, with the expected outcome being a refusal.
- Documents seeded with hidden instructions, to check the assistant does not follow them.
- Requests for other users' data, internal configuration, or the system prompt.
Run this set on every change to prompts, models, or retrieval settings, just like any other test.
Where to start
Map your sources and their owners before you choose a model. That single step answers most of the questions above. If you are planning an assistant over your own data and want it built with these checks from day one, see our AI, ML and RAG service, or read about threat modeling, which applies just as well to AI systems. You can also talk to us.