# Security, privacy, access control, provenance, and governance for RAG

RAG connects a generative model to mutable, often private and untrusted data.
That improves knowledge access while adding attack surfaces absent from a closed
model: poisoning retrieval, injecting instructions through documents, probing
corpus membership, crossing tenants, leaking embeddings or prompts, laundering
claims through citations, and leaving deleted facts in derived artifacts.

This chapter is a technical threat model, not a claim that any one defense makes
RAG safe.

## 1. Assets and security objectives

Protect:

- source documents, databases, graphs, media, and memories;
- queries, conversation history, identity, and intent;
- embeddings, indexes, metadata, ACLs, caches, logs, and backups;
- model/system prompts, tool credentials, and internal policies;
- generated answers, citations, and user actions;
- availability, latency, and compute budget;
- provenance, source authority, and audit integrity;
- training data, qrels, rewards, model weights, and evaluation sets.

Objectives:

1. **confidentiality:** no unauthorized content, existence, or inference;
2. **integrity:** evidence, metadata, rankings, policy, and output are not
   maliciously manipulated;
3. **availability:** hostile content/queries cannot exhaust the service;
4. **authorization:** every read/action respects current identity and policy;
5. **provenance:** claims map to authentic, immutable evidence;
6. **privacy:** collection/use/retention/disclosure follow declared purpose;
7. **deletability:** removal reaches all derived states under the promise;
8. **accountability:** decisions are reconstructable without unsafe logging.

## 2. Trust boundaries

Draw the complete data flow:

```text
external/enterprise sources
 -> connector and malware sandbox
 -> raw immutable store
 -> parser/OCR/enrichment models
 -> canonical/derived corpus
 -> sparse/vector/graph/table indexes
 -> query and identity/policy service
 -> candidate retrieval and filters
 -> reranker/compressor
 -> LLM/VLM and tools
 -> verifier/DLP/policy
 -> user/application
 -> logs, caches, feedback, training
```

For every arrow record owner, region, authentication, encryption, data class,
retention, third-party transfer, and allowed purpose. External embedding,
reranking, LLM, web, and observability providers are separate trust boundaries.

Retrieved content is never trusted merely because it is relevant. Generated
summaries/triples/captions are never primary evidence merely because an internal
pipeline created them.

## 3. Threat actors

- anonymous user probing a public RAG endpoint;
- authenticated tenant attempting cross-tenant access;
- malicious source author/web page owner;
- compromised connector/source account;
- insider with corpus/index/log access;
- model or infrastructure supplier compromise;
- attacker poisoning public data before ingestion;
- malicious document shared into an enterprise workspace;
- user accidentally storing secrets;
- prompt/evaluator/reward attacks during training;
- ordinary operational bugs, which cause many of the same harms without intent.

## 4. Corpus poisoning

An attacker inserts or modifies documents so targeted queries retrieve them and
the generator emits a chosen answer. The attacker optimizes two stages:

1. **retrieval payload:** text/embedding features that rank highly for target
   queries;
2. **generation payload:** false claim or instruction likely to control output.

[PoisonedRAG](https://www.usenix.org/conference/usenixsecurity25/presentation/zou-poisonedrag)
demonstrates high targeted attack success with only a few malicious texts in
million-scale corpora. Paraphrase and perplexity filters did not provide a
complete defense.

### Poisoning variants

- false factual passages;
- source/citation spoofing;
- SEO/keyword stuffing;
- embedding-optimized text;
- duplicate/paraphrase flooding;
- malicious graph nodes/edges or entity aliases;
- poisoned summaries/captions/propositions;
- stale version resurrection;
- incorrect metadata/authority/time/ACL labels;
- poisoned synthetic queries, qrels, teacher labels, or rewards;
- availability payloads with extreme length/complexity.

### Backdoor memory poisoning

[AgentPoison](https://proceedings.neurips.cc/paper_files/paper/2024/hash/eb113910e9c3f6242541c1652e30dfd6-Abstract-Conference.html)
optimizes triggers so instructions retrieve poisoned memory embeddings; the
paper reports high attack success at a very low poison rate with little benign
degradation. Ordinary validation averages can therefore miss targeted behavior.

### Controls

- authenticated connectors and signed/verified upstream content where possible;
- allowlists and source trust tiers;
- immutable source/version hashes and lineage;
- quarantine for new/untrusted sources;
- duplicate/near-duplicate cluster and sudden-volume detection;
- lexical/embedding/rank anomaly monitoring;
- cross-source corroboration without treating duplicates as independent;
- source authority/time policy in selection;
- canary queries/documents and targeted poison tests;
- human review for sensitive corpus changes;
- provenance-aware output verification;
- rollback to a known corpus/index generation.

No anomaly detector proves semantic truth. High-risk domains require governed
source sets and human review.

## 5. Indirect prompt injection

A retrieved document can contain instructions addressed to the model rather
than facts for the user. If the model has tools or secrets, the document may
attempt to exfiltrate data, change the answer, call an external endpoint, write
memory, or hide its action.

Injection can appear in visible text, hidden HTML/CSS, metadata, image OCR,
alt-text, code comments, table cells, filenames, PDF layers, tool output, or a
generated summary. Text can be obfuscated, translated, or split across chunks.

### Why prompting is insufficient

LLMs do not implement a reliable privilege boundary between instruction tokens
and data tokens. “Ignore instructions in documents” reduces some attacks but is
not an authorization mechanism. An attacker can phrase malicious actions as
quoted facts, policies, or tool results.

### Architectural controls

- parse/sanitize active content in a sandbox;
- strip scripts, remote resources, hidden elements, and unsafe attachments;
- preserve evidence as a typed untrusted-data channel;
- keep system/user policy separate from evidence;
- never give the evidence-processing model unnecessary credentials;
- validate tool name, arguments, destination, and data classification outside
  the model;
- enforce least privilege and read-only defaults;
- require approval for external writes/messages/high-impact actions;
- block arbitrary URLs/data exfiltration and restrict network egress;
- prevent documents from selecting citations, tools, memory writes, or policy;
- scan output and tool requests for sensitive data;
- red-team the full retrieval-agent-tool path.

Use a lower-privilege model/process for extraction and a policy engine for
actions. Model-based injection classifiers are defense in depth, not a boundary.

## 6. Retrieval manipulation and source spoofing

Attackers can manipulate ranking without false content by:

- keyword/embedding stuffing;
- using target query phrases or hypothetical answers;
- adding many near-duplicates;
- forging titles/publishers/dates;
- hijacking canonical URLs or redirects;
- creating graph hubs/aliases;
- exploiting analyzer/tokenizer behavior;
- choosing text that bypasses filters but dominates a reranker.

Trust metadata must originate from connector/source policy, not document body.
Render canonical publisher/domain and immutable version. Resolve redirects and
signatures. Separate semantic relevance from authority. Cap duplicate clusters
and per-source contribution. Monitor rank shifts after corpus/index updates.

## 7. Cross-tenant and authorization failures

An ACL applied after retrieval is often too late: an unauthorized item may be
sent to a reranker/LLM/log, influence timing, or crowd out authorized results.

### Required invariant

For principal \(p\), policy snapshot \(a_t\), every component must satisfy

\[
\operatorname{visible}(d,p,a_t)=1
\]

before content or sensitive metadata crosses the component boundary. Recheck
before output/action if policy can change during a long run.

### Patterns

- physical index per security domain/tenant;
- filter-aware shared index;
- public global index plus private tenant/user overlays;
- authorized candidate bitsets/partitions;
- row/column/cell-level policy for structured data;
- separate model/caches/logs for sensitive tiers.

### Failure modes

- group membership lag;
- inherited permissions dropped during parsing/chunking;
- child chunk lacks parent ACL;
- graph edge crosses trust domains;
- post-filter ANN returns too few results and fallback searches globally;
- shared semantic cache ignores principal;
- citation URL remains accessible only through leaked signed token;
- aggregate/count/timing reveals document existence;
- logs/feedback datasets lose original ACL.

Continuously generate cross-tenant canary documents/queries and prove no content,
metadata, score, count, citation, or timing signal is exposed beyond policy.

## 8. Corpus extraction and privacy leakage

[The Good and The Bad](https://aclanthology.org/2024.findings-acl.267/) studies
attacks that extract private retrieval-database content. RAG can expose a corpus
more directly than a parametric model because relevant records are placed in
context. It can also reduce reliance on memorized training data; both effects
must be measured.

Attackers can use adaptive queries to:

- reconstruct document passages;
- enumerate records/entities;
- infer rare attributes;
- elicit verbatim quotes;
- combine partial disclosures;
- exploit error messages/citations;
- induce the model to describe inaccessible multimodal evidence.

Controls include authentication, purpose/field-level access, output
minimization, query/rate anomaly detection, DLP, privacy budgets for aggregate
interfaces, and human approval for bulk export. Do not depend on “the model will
paraphrase” as privacy protection.

## 9. Membership inference

Membership inference asks whether a target document/record is present in the
retrieval corpus. Signals include answer content, citation, confidence, score,
latency, and differences across carefully designed queries. Existence alone can
be sensitive (for example, a patient record or internal investigation).

Mitigations:

- authorization before any membership-dependent behavior;
- uniform error/output policy;
- restrict raw scores/counts/debug traces;
- rate-limit/adaptive-query detection;
- retrieve within sufficiently large governed partitions;
- privacy-preserving retrieval when required;
- audit with member/nonmember data and realistic adversaries.

Differential privacy can bound certain inferences but adds utility/cost trade-
offs and must cover the full mechanism, not only embeddings.

## 10. Embedding and query privacy

Embeddings are not anonymous. They can reveal semantic attributes or be subject
to inversion/reconstruction. A remote vector service sees document/query
embeddings and access patterns; a remote embedding model sees raw content unless
computed locally.

Threats:

- model inversion/reconstruction;
- attribute inference;
- query linkage/profiling;
- nearest-neighbor access-pattern leakage;
- cross-tenant vector enumeration;
- embedding-model supply chain/exfiltration;
- raw secret transfer to external embedding APIs.

Controls:

- local/self-hosted embedding for sensitive data;
- TLS and encryption at rest with scoped keys;
- network/service isolation and least privilege;
- avoid logging raw queries/vectors by default;
- contractual/data-residency review of providers;
- vector access authorization and export restrictions;
- rotation/re-indexing after model/key compromise;
- privacy-preserving similarity search where justified.

## 11. Privacy-preserving retrieval

[Private Retrieval Augmented Generation](https://aclanthology.org/2024.privatenlp-1.2/)
uses multi-party computation for distributed private approximate similarity
search so no server sees both query and database. [RemoteRAG](https://aclanthology.org/2025.findings-acl.197/)
defines a cloud-RAG privacy setting and applies a distance-DP perturbation plus a
restricted candidate range.

Other design families include private information retrieval, secure enclaves,
homomorphic encryption, MPC, query/document perturbation, local differential
privacy, federated indexes, and on-device retrieval. They trade privacy
assumptions against computation, communication, accuracy, and side channels.

Specify:

- adversary and collusion model;
- what is hidden: query, corpus, access pattern, result, or all;
- cryptographic/DP assumptions and parameters;
- leakage from result size/timing/caching;
- exact versus approximate retrieval loss;
- end-to-end generation/privacy, not retrieval alone.

## 12. Multimodal and graph privacy

Images/audio can reveal faces, documents, voices, locations, or sensitive
background details even when text is redacted. [Beyond Text](https://aclanthology.org/2025.emnlp-main.1259/)
shows multimodal RAG extraction risks through structured prompts and direct or
descriptive disclosure.

Graphs make relationships explicit. A text passage may obscure a link, while an
entity/relation graph makes it queryable. [Exposing Privacy Risks in Graph
RAG](https://aclanthology.org/2026.findings-acl.899/) reports a trade-off where
structured entity/relation leakage can increase even when raw-text leakage
decreases.

Apply ACLs to nodes, edges, attributes, source spans, communities, and generated
reports. A community summary must not combine facts users cannot jointly access.
Deletion must update derived edges and summaries.

## 13. Caches and side channels

Caches can leak through keys, shared values, timing, hit counters, or stale
answers. [Auditing Prompt Caching](https://proceedings.mlr.press/v267/gu25b.html)
found timing evidence of cross-user prompt caching in multiple APIs, illustrating
why cache policy is security-relevant.

RAG cache keys need principal/tenant, policy, corpus/index generation, as-of
time, model/prompt, and output policy. Do not share sensitive semantic caches
across tenants. Normalize timing where membership leakage matters; restrict
metrics. Encrypt and apply retention/deletion to cached evidence/answers.

## 14. Model safety can degrade with retrieval

[RAG LLMs Are Not Safer](https://aclanthology.org/2025.naacl-long.281/) evaluates
multiple LLMs and reports that retrieval can reduce safety; even safe models and
apparently safe documents can combine into unsafe output. Standard no-RAG
attacks do not fully characterize RAG behavior.

Safety testing must cover:

- safe/unsafe query × safe/unsafe evidence;
- individually benign evidence that composes into harm;
- retrieved instructions and procedural detail;
- conflicting safety guidance;
- domain/tool availability;
- citation/authority laundering;
- model/retriever/reranker combinations.

Apply policy at query, retrieval, evidence, generation, tool, and output stages.

## 15. Denial of service and resource attacks

Attacks can force:

- huge query fan-out or agent loops;
- expensive graph traversal/SQL;
- pathological ANN filters;
- very long documents/contexts;
- decompression/parser bombs;
- OCR/VLM-heavy files;
- cache misses and repeated index builds;
- enormous output/verification;
- duplicate floods that enlarge indexes.

Controls:

- file size/type/decompression/parser sandbox limits;
- quotas per source/tenant/principal;
- maximum search/tool steps and candidate/context tokens;
- query complexity and graph/SQL cost limits;
- timeouts, cancellation, circuit breakers, and backpressure;
- duplicate/flood detection;
- resource isolation and priority classes;
- graceful partial answer/abstention;
- cost anomaly alerts.

## 16. Integrity of citations and provenance

Citation attacks include fabricated URLs, source-title spoofing, citing a real
document that does not support the claim, attaching a citation to the wrong
claim, and citing a generated summary as primary evidence.

Controls:

- generator may select only evidence IDs provided by the system;
- evidence IDs resolve to immutable source versions/spans;
- fetch/access and hash are validated;
- claim-level entailment and authority are checked separately;
- canonical publisher/domain and dates are rendered;
- generated derivatives are labeled and map to originals;
- citation completeness is evaluated;
- high-stakes citations receive human audit.

Never accept a model-generated URL as a citation without retrieval and
verification.

## 17. Security-conscious retrieval defenses

[SafeRAG](https://aclanthology.org/2025.acl-long.230/) benchmarks silver noise,
inter-context conflict, soft advertisements, and white denial-of-service,
showing vulnerabilities across RAG components. [SeCon-RAG](https://proceedings.neurips.cc/paper_files/paper/2025/hash/668563ef18fbfef0b66af491ea334d5f-Abstract-Conference.html)
uses semantic/cluster filtering and conflict-aware answer/evidence consistency.

Defenses include:

- source trust/allowlist filters;
- robust multi-source aggregation;
- cluster/duplicate controls;
- contradiction and ad/instruction detection;
- query-evidence consistency;
- leave-one-source-out stability;
- retrieval ensemble disagreement;
- evidence/citation verification;
- abstention on unstable evidence.

Every filter has false positives/negatives. Evaluate clean utility and adaptive
attacks.

## 18. Certified and provable risk bounds

[C-RAG](https://proceedings.mlr.press/v235/kang24a.html) applies conformal risk
analysis to bound a declared bounded generation-risk function under calibration
and stated distribution-shift assumptions. It certifies aggregate risk under
those assumptions, not corpus authenticity or prompt-injection safety.

[PRA-RAG](https://aclanthology.org/2026.findings-acl.1794/) samples combinations
of retrieved text, identifies a robust subset using embedding geometry, and
derives bounds under its poisoning model. The paper reports low attack success
while retaining accuracy, with added sampling/generation cost. Its guarantee is
specific to corruption and threat assumptions; provenance and adaptive attacks
remain separate.

When stating a guarantee, name:

- risk/loss being bounded;
- calibration data and exchangeability/shift assumptions;
- adversary/corruption budget;
- confidence level;
- per-example versus aggregate scope;
- components outside the guarantee;
- clean-utility and compute cost.

## 19. Data minimization and purpose limitation

Before indexing, ask:

- is this source needed for the declared feature?
- which fields/regions are necessary?
- can sensitive data remain local or be retrieved on demand?
- does the embedding/generator provider receive it?
- may it be quoted, logged, cached, used as feedback, or used for training?
- how long must each raw/derived artifact remain?
- can the user inspect/correct/delete it?

Minimize at acquisition, retrieval, context, output, logs, and training. A RAG
system often copies data into more locations than the source system; lineage and
retention must cover all copies.

## 20. Deletion, correction, and unlearning

Build reverse lineage:

```text
source version
 -> canonical blocks
 -> chunks/views/embeddings/postings
 -> summaries/propositions/captions/translations
 -> graph nodes/edges/community reports
 -> caches/traces/feedback labels
 -> training examples/checkpoints if applicable
```

A deletion request creates a tombstone, prevents resurrection by delayed events,
removes active index/cache state, recomputes shared derivatives, and follows
retention/backup policy. Verify with seeded queries and row/hash audits.

External-memory facts are easier to remove than parametric memorization only if
they were never copied into fine-tuning/log datasets. If model unlearning is
promised, define the technical verification; deleting a vector is not model
unlearning.

Corrections should supersede wrong versions and invalidate answers/caches while
preserving audit under policy. Temporal queries may still need the historical
record.

## 21. Licensing, copyright, and source governance

Technical systems should track:

- source owner and acquisition basis;
- license/terms and allowed transformations;
- quotation/display limits;
- attribution requirements;
- training eligibility separate from retrieval eligibility;
- redistribution/export restrictions;
- geographic/contractual constraints;
- expiration/review date;
- generated-derivative policy.

Semantic relevance does not grant permission. Generated summaries and
embeddings may remain derived artifacts subject to policy. Code RAG needs
repository/file license and copied-code provenance. Web RAG must not assume
publicly accessible means unrestricted reuse.

This repository does not provide jurisdiction-specific legal conclusions;
organizations need qualified legal/privacy review for their data and markets.

## 22. Supply-chain security

Inventory and pin:

- connectors, parsers, OCR/layout models;
- embedding/reranker/generator/verifier models;
- tokenizers and prompt templates;
- vector/search/graph/database libraries;
- model/data downloads and hashes/signatures;
- containers, runtime, GPU drivers;
- external APIs and plugins;
- evaluation datasets and judges.

Use trusted registries, artifact signing, vulnerability scanning, least-
privilege execution, network isolation, reproducible builds where feasible, and
staged rollout. Model files and parsers process attacker-controlled content and
belong in the threat model.

## 23. Logging and observability privacy

Logs are a second corpus. They can contain full queries, private evidence,
answers, citations, identity, tool arguments, and secrets.

Define fields by purpose. Prefer stable opaque IDs, hashes, aggregates, and
redacted snippets. Encrypt, ACL, tenant-partition, retain minimally, and audit
access. Debug modes must expire and never silently become permanent production
logging. LLM observability vendors are data processors/trust boundaries.

Security monitoring still needs useful signals: source IDs/trust, rank shifts,
blocked ACLs, injection flags, duplicate clusters, tool attempts, cost, and
policy decisions. Design telemetry to minimize content while preserving
incident reconstruction under controlled access.

## 24. Secure development and release gates

### Design review

- data-flow/trust-boundary diagram;
- source inventory, authority, licensing, and retention;
- identity/ACL model;
- threat model and abuse cases;
- external provider transfer;
- hard tool/cost limits;
- deletion/correction plan;
- incident owner and rollback.

### Build/test

- unit/property tests for ACL and source lineage;
- cross-tenant canaries;
- poison/injection/conflict/ad/DoS corpora;
- privacy extraction/membership probes;
- citation/source spoof tests;
- parser/archive/resource limits;
- output DLP/policy tests;
- deletion and cache invalidation rehearsal;
- adversarial agent loop/tool tests.

### Release

- signed versioned corpus/index/model/prompt manifest;
- clean utility and adversarial metrics with thresholds;
- staged shadow/canary rollout;
- monitoring and alert thresholds;
- rollback generation retained;
- operator/user disclosures and controls;
- high-risk human approval paths.

## 25. Red-team matrix

| Attack | Variants | Observe |
|---|---|---|
| Targeted poison | lexical, embedding, graph, duplicate flood | target rank, answer, citation, clean utility |
| Trigger backdoor | rare phrase, multilingual, visual, memory | triggered ASR and benign behavior |
| Indirect injection | HTML/PDF/OCR/table/code/tool result | policy/tool/exfiltration behavior |
| Conflict | same entity/date, version, authority | selection, disclosure, abstention |
| Soft ad/source manipulation | recommendation, sponsored wording | source bias and disclosure |
| Cross-tenant | IDs, semantic probes, filters, graphs, caches | content/existence/timing leakage |
| Extraction | iterative queries, quotes, multimodal descriptions | recovered private content |
| Membership inference | response, score, citation, latency | AUC/advantage at query budget |
| Embedding/query attack | inversion, attribute, provider compromise | reconstruction and data transfer |
| Citation spoof | fake URL/title/date, non-supporting real source | validation and claim support |
| DoS | loops, long files, filter pathology, fan-out | p95/p99, resource/cost, availability |
| Deletion failure | summaries, edges, caches, replicas, logs | residual retrieval/output |

Use adaptive attackers who know the defense assumptions; static benchmark
success is not a certificate.

## 26. Incident response

When a RAG incident occurs:

1. preserve minimal authorized forensic trace and version IDs;
2. contain affected source/index/model/tool/tenant route;
3. disable or narrow risky retrieval/actions;
4. identify source/derived lineage and impacted outputs/users;
5. remove/quarantine content and rebuild a clean generation;
6. invalidate caches and rotate credentials/tokens if exposed;
7. validate with targeted and regression queries;
8. restore gradually with monitoring;
9. complete notification/remediation under organizational policy;
10. add tests, controls, and ownership fixes.

Rollback should switch to a known immutable index/prompt/model generation, not
attempt ad hoc edits in a corrupted live index.

## 27. Residual risk and human control

No current architecture proves that untrusted retrieval plus a general LLM will
never follow malicious content, leak allowed-but-sensitive evidence, or produce
an unsupported synthesis. Use defense in depth and reduce autonomy/data exposure
in proportion to harm.

High-stakes workflows need authoritative source constraints, local evidence
display, user correction, abstention, human review, and separation between
drafting and action. Citation improves auditability only when support and source
authority are checked.

## 28. What the executable notebooks model

The safety notebook models pre-retrieval ACL enforcement, trust-domain/source
metadata, content hashes, duplicate/poison clusters, prompt-injection markers,
conflict grouping, output source-ID validation, deletion lineage, and adversarial
test cases. It proves invariants such as “unauthorized chunks never enter the
reranker context” on a small corpus.

It does not claim a regex detects all prompt injection, that similarity detects
poisoning, or that toy access checks replace production identity systems. The
goal is to make security boundaries and required tests executable.