In brief
A RAG assistant or AI agent shows users whatever the retrieval layer hands it, so it inherits every overshared folder and stale permission. Before you index company data, enforce access inside the index query, classify documents before ingestion and record where every chunk came from.
Key takeaways
- OWASP Top 10 for LLM Applications 2026 (published 3 August 2026) tells you to authorize before retrieval, at document and chunk level, inside the index query.
- Similarity search does not know about access rights: a filter applied after the model has seen a chunk cannot take that chunk back.
- Classify before indexing: some documents go into a shared index with access tags, some into a separate index, some nowhere.
- Lineage for RAG means provenance per chunk (source, ingestion time, pipeline version) plus a log of what each query retrieved.
- AI Act Art. 10 data governance applies to Annex III high-risk systems from 2 December 2027 (as of 17 November 2026); GDPR Art. 5, 25 and 32 already apply to every RAG over personal data.
Data governance for AI starts with an uncomfortable fact: a retrieval-augmented generation (RAG) assistant or an AI agent sees what its retrieval layer gives it, not what the user is allowed to see. If a SharePoint site has been open to the whole company since 2019, the assistant will quote it. The OWASP Top 10 for LLM Applications 2026 names this leak path directly. Data governance for RAG is the set of access, classification, lineage and quality controls applied before company data reaches the retrieval index. This guide covers the four areas to check before you build software with RAG.
Why does a RAG assistant show data the user should not see?
Because similarity search ranks text by meaning, not by access rights. The OWASP 2026 entry on sensitive information disclosure (LLM02) puts it in one line: "Cosine similarity does not respect ACLs." If the HR salary file sits in the same index as the travel policy, a question about allowances can retrieve both.
The same entry lists the usual sources of the problem: "unscoped drives, legacy permissions, and knowledge bases feed RAG with sensitive data". The assistant does not create new leaks. It makes the old ones searchable in plain language, by people who would never have browsed to that folder.
Off-the-shelf assistants follow the same logic. Microsoft states that Microsoft 365 Copilot only surfaces data to which a user has at least view permissions, and tells customers to use the permission models of SharePoint and other services. If those permissions are too broad, Copilot is too. We described the employee side of this risk in shadow AI and data security.
How do you enforce permissions in the retrieval layer?
Check access before retrieval, not after generation. OWASP LLM02:2026 asks you to "enforce document- and chunk-level authorization inside the index query" and warns that "post-generation filtering cannot undo a chunk already supplied to the model". In practice:
- Carry the access list into the index. Every chunk stores the groups or users allowed to read its source document, and every query filters on the caller's identity on the server side.
- Separate what must never mix. OWASP LLM09:2026 recommends physically separate indexes per tenant or trust zone for high-sensitivity workloads, because a separate index removes the misconfiguration path that tags on a shared index leave open.
- Run agents as the user. OWASP LLM03:2026 (Excessive Agency) asks that actions on behalf of a user run on downstream systems in that user's context, not through one privileged service account.
- Keep authorization out of the prompt. OWASP LLM08:2026 says controls such as privilege separation must not be delegated to the LLM. "Only answer for the user's department" in a system prompt is not access control.
Platforms help, with limits.
In Azure AI Search, security filters on user or group strings are generally available; native permission inheritance from ADLS Gen2, SharePoint ACLs and Purview labels was in preview as of November 2026. Permission changes reach results only after the next sync, so an employee removed from a group may still retrieve its documents until the index catches up.
Other stacks leave more to you. The Databricks vector search documentation states that row- and column-level permissions are not supported on indexes and points to application-level ACLs through the filter API. Before you choose a vector store, ask exactly where the user's identity is checked. The groups themselves come from your identity system: see IAM as the foundation of security.
What should you classify before indexing?
Every source, before the first embedding is computed. An embedding is not an anonymized copy: OWASP LLM09:2026 warns that "stored embeddings can be inverted to recover source text". Treat the vector index with the same classification as the most sensitive document in it. A simple decision table per source works better than a policy nobody applies:
| Class | Example | Decision for RAG |
|---|---|---|
| Public | product sheets, published policies | shared index, no access filter needed |
| Internal | procedures, project wikis | shared index with group-based access filter |
| Confidential | contracts, HR files, board minutes | separate index per trust zone, user-level filter, retrieval log |
| Restricted or personal data without a purpose | health data, ID scans, old CV folders | not indexed; personal data removed at ingestion where the source must stay |
OWASP LLM02:2026 asks you to "scrub PII at ingest" and to govern the corpus with provenance, classification and deduplication. When a document is split into chunks, its label has to go with every chunk. The Azure AI Search documentation is explicit: without this projection, chunk-level references are not filtered.
How do you trace an AI answer back to its source?
With lineage on two levels: the pipeline that built the index, and the retrieval behind each answer. OWASP LLM09:2026 asks you to "track provenance for every embedding (source, ingestion time, trust tier, pipeline version)" and to keep immutable logs of retrieval activity with the query, the returned IDs and the similarity scores.
- Pipeline lineage: which system, table or folder a document came from and which job transformed it. Unity Catalog captures lineage automatically for queries run on Databricks, down to column level; OpenLineage is an open standard for collecting lineage across other tools.
- Chunk provenance: source ID, version, ingestion time and classification stored as metadata on every chunk, so the answer can cite them.
- Retrieval log: who asked, which chunks came back, which model version answered. Without it you cannot show whether a leak happened.
Lineage also limits poisoning. OWASP LLM05:2026 (Data and Model Poisoning) notes that a single crafted document can override accurate content in a retrieval corpus, and recommends strict validation of incoming data and dataset lineage. If you know which pipeline wrote a chunk, you can remove everything it wrote. The catalog side of this is covered in data lineage for compliance.
Which data quality checks matter for AI?
The ones that change answers. A dashboard tolerates an outdated row; an assistant repeats the outdated procedure with full confidence. For RAG, check four things per source:
- Current version only. Old copies of policies and price lists are removed or marked as superseded, or the model mixes them.
- Near-duplicates. Ten copies of one document take ten places in the top results and push out the one that answers the question.
- A named owner. Someone decides when a source is wrong and must be re-indexed.
- Freshness rules. A re-index schedule per source, tied to how often it changes.
For personal data this is not optional: GDPR Art. 5(1)(d) requires data to be accurate and, where necessary, kept up to date. Our data governance work starts with these owners and schedules. If you want a framework, the ISO/IEC 5259 series covers data quality for analytics and machine learning in five parts published in 2024 and 2025.
What do GDPR and the AI Act require for RAG data?
GDPR applies today to every RAG system that indexes personal data. Purpose limitation and minimization (Art. 5(1)(b) and (c)) decide what may be indexed at all. Art. 25(2) requires that, by default, personal data is not made accessible to an indefinite number of people.
Art. 32 asks for security appropriate to the risk, and Art. 35 for a DPIA where processing, in particular with new technologies, is likely to result in a high risk.
The AI Act is narrower. Art. 10 (data governance, origin of data, examination for bias) applies to high-risk AI systems. After the Digital Omnibus, Regulation (EU) 2026/1744, it applies to Annex III systems from 2 December 2027 (as of 17 November 2026). An internal knowledge assistant is usually not high-risk. Art. 4 on AI literacy has applied since 2 February 2025; our overview of AI Act obligations lists the rest.
If you process data in Poland, the Polish data protection authority (UODO) published checklists of questions on GDPR for AI tools on 6 August 2026. They are not binding and do not replace a DPIA, but they show what an inspector is likely to ask.
Data governance for AI: a checklist before you connect company data
- List the sources you want to index and name an owner for each.
- Review sharing on those sources: "everyone" links, legacy groups, former employees.
- Classify each source with the table above and decide: shared index, separate index or not at all.
- Make sure the access list travels with every chunk and the query filters on the caller's identity.
- Store provenance per chunk and log every retrieval.
- Remove outdated versions and near-duplicates; set a re-index schedule.
- Check GDPR: purpose, minimization, DPIA where needed.
- Test with real accounts: ask as an intern for what only the CFO may read.
Step 8 is the one that finds the problems. If the intern gets the board minutes, the fix is in the permissions, not in the prompt.
We build RAG on your own infrastructure with document access control as part of software development, and the governance side in data governance and metadata. If you want to go through your sources together, contact us.
Frequently asked questions
- What does data governance for AI cover in a RAG project?
In a RAG project, it is the set of rules and controls that decide which data an AI system may use, who may see the results, where the data came from and how good it is. For RAG it comes down to access control in the retrieval layer, classification before indexing, lineage and data quality.
- Does a RAG assistant respect SharePoint or file share permissions?
Only if the retrieval layer enforces them. Similarity search ignores access rights, so the access list must be stored with every chunk and checked inside the index query. Permission changes take effect only after the index is synchronized again.
- Can I filter sensitive content after the model answers?
Not reliably. OWASP LLM02:2026 notes that post-generation filtering cannot undo a chunk already supplied to the model. Filter before retrieval and treat output filters as a second layer.
- Do I need a DPIA for a RAG system?
If the index contains personal data and the processing is likely to result in high risk, GDPR Art. 35 requires a DPIA. Large HR, customer or health document sets usually meet that bar.
- Does AI Act Article 10 apply to my internal assistant?
Art. 10 applies to high-risk AI systems, for Annex III systems from 2 December 2027 (as of 17 November 2026). Most internal knowledge assistants are not high-risk, but GDPR applies to them in full.

