Security, privacy, access control, provenance, and governance for RAG

RAG connects a generative model to mutable, often private and untrusted data. That improves knowledge access while adding attack surfaces absent from a closed model: poisoning retrieval, injecting instructions through documents, probing corpus membership, crossing tenants, leaking embeddings or prompts, laundering claims through citations, and leaving deleted facts in derived artifacts.

This chapter is a technical threat model, not a claim that any one defense makes RAG safe.

1. Assets and security objectives

Protect:

• source documents, databases, graphs, media, and memories;
• queries, conversation history, identity, and intent;
• embeddings, indexes, metadata, ACLs, caches, logs, and backups;
• model/system prompts, tool credentials, and internal policies;
• generated answers, citations, and user actions;
• availability, latency, and compute budget;
• provenance, source authority, and audit integrity;
• training data, qrels, rewards, model weights, and evaluation sets.

Objectives:

1. confidentiality: no unauthorized content, existence, or inference;
2. integrity: evidence, metadata, rankings, policy, and output are not maliciously manipulated;
3. availability: hostile content/queries cannot exhaust the service;
4. authorization: every read/action respects current identity and policy;
5. provenance: claims map to authentic, immutable evidence;
6. privacy: collection/use/retention/disclosure follow declared purpose;
7. deletability: removal reaches all derived states under the promise;
8. accountability: decisions are reconstructable without unsafe logging.

2. Trust boundaries

Draw the complete data flow:

external/enterprise sources
 -> connector and malware sandbox
 -> raw immutable store
 -> parser/OCR/enrichment models
 -> canonical/derived corpus
 -> sparse/vector/graph/table indexes
 -> query and identity/policy service
 -> candidate retrieval and filters
 -> reranker/compressor
 -> LLM/VLM and tools
 -> verifier/DLP/policy
 -> user/application
 -> logs, caches, feedback, training

For every arrow record owner, region, authentication, encryption, data class, retention, third-party transfer, and allowed purpose. External embedding, reranking, LLM, web, and observability providers are separate trust boundaries.

Retrieved content is never trusted merely because it is relevant. Generated summaries/triples/captions are never primary evidence merely because an internal pipeline created them.

3. Threat actors

• anonymous user probing a public RAG endpoint;
• authenticated tenant attempting cross-tenant access;
• malicious source author/web page owner;
• compromised connector/source account;
• insider with corpus/index/log access;
• model or infrastructure supplier compromise;
• attacker poisoning public data before ingestion;
• malicious document shared into an enterprise workspace;
• user accidentally storing secrets;
• prompt/evaluator/reward attacks during training;
• ordinary operational bugs, which cause many of the same harms without intent.

4. Corpus poisoning

An attacker inserts or modifies documents so targeted queries retrieve them and the generator emits a chosen answer. The attacker optimizes two stages:

1. retrieval payload: text/embedding features that rank highly for target queries;
2. generation payload: false claim or instruction likely to control output.

PoisonedRAG demonstrates high targeted attack success with only a few malicious texts in million-scale corpora. Paraphrase and perplexity filters did not provide a complete defense.

Poisoning variants

• false factual passages;
• source/citation spoofing;
• SEO/keyword stuffing;
• embedding-optimized text;
• duplicate/paraphrase flooding;
• malicious graph nodes/edges or entity aliases;
• poisoned summaries/captions/propositions;
• stale version resurrection;
• incorrect metadata/authority/time/ACL labels;
• poisoned synthetic queries, qrels, teacher labels, or rewards;
• availability payloads with extreme length/complexity.

Backdoor memory poisoning

AgentPoison optimizes triggers so instructions retrieve poisoned memory embeddings; the paper reports high attack success at a very low poison rate with little benign degradation. Ordinary validation averages can therefore miss targeted behavior.

Controls

• authenticated connectors and signed/verified upstream content where possible;
• allowlists and source trust tiers;
• immutable source/version hashes and lineage;
• quarantine for new/untrusted sources;
• duplicate/near-duplicate cluster and sudden-volume detection;
• lexical/embedding/rank anomaly monitoring;
• cross-source corroboration without treating duplicates as independent;
• source authority/time policy in selection;
• canary queries/documents and targeted poison tests;
• human review for sensitive corpus changes;
• provenance-aware output verification;
• rollback to a known corpus/index generation.

No anomaly detector proves semantic truth. High-risk domains require governed source sets and human review.

5. Indirect prompt injection

A retrieved document can contain instructions addressed to the model rather than facts for the user. If the model has tools or secrets, the document may attempt to exfiltrate data, change the answer, call an external endpoint, write memory, or hide its action.

Injection can appear in visible text, hidden HTML/CSS, metadata, image OCR, alt-text, code comments, table cells, filenames, PDF layers, tool output, or a generated summary. Text can be obfuscated, translated, or split across chunks.

Why prompting is insufficient

LLMs do not implement a reliable privilege boundary between instruction tokens and data tokens. “Ignore instructions in documents” reduces some attacks but is not an authorization mechanism. An attacker can phrase malicious actions as quoted facts, policies, or tool results.

Architectural controls

• parse/sanitize active content in a sandbox;
• strip scripts, remote resources, hidden elements, and unsafe attachments;
• preserve evidence as a typed untrusted-data channel;
• keep system/user policy separate from evidence;
• never give the evidence-processing model unnecessary credentials;
• validate tool name, arguments, destination, and data classification outside the model;
• enforce least privilege and read-only defaults;
• require approval for external writes/messages/high-impact actions;
• block arbitrary URLs/data exfiltration and restrict network egress;
• prevent documents from selecting citations, tools, memory writes, or policy;
• scan output and tool requests for sensitive data;
• red-team the full retrieval-agent-tool path.

Use a lower-privilege model/process for extraction and a policy engine for actions. Model-based injection classifiers are defense in depth, not a boundary.

6. Retrieval manipulation and source spoofing

Attackers can manipulate ranking without false content by:

• keyword/embedding stuffing;
• using target query phrases or hypothetical answers;
• adding many near-duplicates;
• forging titles/publishers/dates;
• hijacking canonical URLs or redirects;
• creating graph hubs/aliases;
• exploiting analyzer/tokenizer behavior;
• choosing text that bypasses filters but dominates a reranker.

Trust metadata must originate from connector/source policy, not document body. Render canonical publisher/domain and immutable version. Resolve redirects and signatures. Separate semantic relevance from authority. Cap duplicate clusters and per-source contribution. Monitor rank shifts after corpus/index updates.

7. Cross-tenant and authorization failures

An ACL applied after retrieval is often too late: an unauthorized item may be sent to a reranker/LLM/log, influence timing, or crowd out authorized results.

Required invariant

For principal \(p\), policy snapshot \(a_t\), every component must satisfy

before content or sensitive metadata crosses the component boundary. Recheck before output/action if policy can change during a long run.

Patterns

• physical index per security domain/tenant;
• filter-aware shared index;
• public global index plus private tenant/user overlays;
• authorized candidate bitsets/partitions;
• row/column/cell-level policy for structured data;
• separate model/caches/logs for sensitive tiers.

Failure modes

• group membership lag;
• inherited permissions dropped during parsing/chunking;
• child chunk lacks parent ACL;
• graph edge crosses trust domains;
• post-filter ANN returns too few results and fallback searches globally;
• shared semantic cache ignores principal;
• citation URL remains accessible only through leaked signed token;
• aggregate/count/timing reveals document existence;
• logs/feedback datasets lose original ACL.

Continuously generate cross-tenant canary documents/queries and prove no content, metadata, score, count, citation, or timing signal is exposed beyond policy.

8. Corpus extraction and privacy leakage

The Good and The Bad studies attacks that extract private retrieval-database content. RAG can expose a corpus more directly than a parametric model because relevant records are placed in context. It can also reduce reliance on memorized training data; both effects must be measured.

Attackers can use adaptive queries to:

• reconstruct document passages;
• enumerate records/entities;
• infer rare attributes;
• elicit verbatim quotes;
• combine partial disclosures;
• exploit error messages/citations;
• induce the model to describe inaccessible multimodal evidence.

Controls include authentication, purpose/field-level access, output minimization, query/rate anomaly detection, DLP, privacy budgets for aggregate interfaces, and human approval for bulk export. Do not depend on “the model will paraphrase” as privacy protection.

9. Membership inference

Membership inference asks whether a target document/record is present in the retrieval corpus. Signals include answer content, citation, confidence, score, latency, and differences across carefully designed queries. Existence alone can be sensitive (for example, a patient record or internal investigation).

Mitigations:

• authorization before any membership-dependent behavior;
• uniform error/output policy;
• restrict raw scores/counts/debug traces;
• rate-limit/adaptive-query detection;
• retrieve within sufficiently large governed partitions;
• privacy-preserving retrieval when required;
• audit with member/nonmember data and realistic adversaries.

Differential privacy can bound certain inferences but adds utility/cost trade- offs and must cover the full mechanism, not only embeddings.

10. Embedding and query privacy

Embeddings are not anonymous. They can reveal semantic attributes or be subject to inversion/reconstruction. A remote vector service sees document/query embeddings and access patterns; a remote embedding model sees raw content unless computed locally.

Threats:

• model inversion/reconstruction;
• attribute inference;
• query linkage/profiling;
• nearest-neighbor access-pattern leakage;
• cross-tenant vector enumeration;
• embedding-model supply chain/exfiltration;
• raw secret transfer to external embedding APIs.

Controls:

• local/self-hosted embedding for sensitive data;
• TLS and encryption at rest with scoped keys;
• network/service isolation and least privilege;
• avoid logging raw queries/vectors by default;
• contractual/data-residency review of providers;
• vector access authorization and export restrictions;
• rotation/re-indexing after model/key compromise;
• privacy-preserving similarity search where justified.

11. Privacy-preserving retrieval

Private Retrieval Augmented Generation uses multi-party computation for distributed private approximate similarity search so no server sees both query and database. RemoteRAG defines a cloud-RAG privacy setting and applies a distance-DP perturbation plus a restricted candidate range.

Other design families include private information retrieval, secure enclaves, homomorphic encryption, MPC, query/document perturbation, local differential privacy, federated indexes, and on-device retrieval. They trade privacy assumptions against computation, communication, accuracy, and side channels.

Specify:

• adversary and collusion model;
• what is hidden: query, corpus, access pattern, result, or all;
• cryptographic/DP assumptions and parameters;
• leakage from result size/timing/caching;
• exact versus approximate retrieval loss;
• end-to-end generation/privacy, not retrieval alone.

12. Multimodal and graph privacy

Images/audio can reveal faces, documents, voices, locations, or sensitive background details even when text is redacted. Beyond Text shows multimodal RAG extraction risks through structured prompts and direct or descriptive disclosure.

Graphs make relationships explicit. A text passage may obscure a link, while an entity/relation graph makes it queryable. Exposing Privacy Risks in Graph RAG reports a trade-off where structured entity/relation leakage can increase even when raw-text leakage decreases.

Apply ACLs to nodes, edges, attributes, source spans, communities, and generated reports. A community summary must not combine facts users cannot jointly access. Deletion must update derived edges and summaries.

13. Caches and side channels

Caches can leak through keys, shared values, timing, hit counters, or stale answers. Auditing Prompt Caching found timing evidence of cross-user prompt caching in multiple APIs, illustrating why cache policy is security-relevant.

RAG cache keys need principal/tenant, policy, corpus/index generation, as-of time, model/prompt, and output policy. Do not share sensitive semantic caches across tenants. Normalize timing where membership leakage matters; restrict metrics. Encrypt and apply retention/deletion to cached evidence/answers.

14. Model safety can degrade with retrieval

RAG LLMs Are Not Safer evaluates multiple LLMs and reports that retrieval can reduce safety; even safe models and apparently safe documents can combine into unsafe output. Standard no-RAG attacks do not fully characterize RAG behavior.

Safety testing must cover:

• safe/unsafe query × safe/unsafe evidence;
• individually benign evidence that composes into harm;
• retrieved instructions and procedural detail;
• conflicting safety guidance;
• domain/tool availability;
• citation/authority laundering;
• model/retriever/reranker combinations.

Apply policy at query, retrieval, evidence, generation, tool, and output stages.

15. Denial of service and resource attacks

Attacks can force:

• huge query fan-out or agent loops;
• expensive graph traversal/SQL;
• pathological ANN filters;
• very long documents/contexts;
• decompression/parser bombs;
• OCR/VLM-heavy files;
• cache misses and repeated index builds;
• enormous output/verification;
• duplicate floods that enlarge indexes.

Controls:

• file size/type/decompression/parser sandbox limits;
• quotas per source/tenant/principal;
• maximum search/tool steps and candidate/context tokens;
• query complexity and graph/SQL cost limits;
• timeouts, cancellation, circuit breakers, and backpressure;
• duplicate/flood detection;
• resource isolation and priority classes;
• graceful partial answer/abstention;
• cost anomaly alerts.

16. Integrity of citations and provenance

Citation attacks include fabricated URLs, source-title spoofing, citing a real document that does not support the claim, attaching a citation to the wrong claim, and citing a generated summary as primary evidence.

Controls:

• generator may select only evidence IDs provided by the system;
• evidence IDs resolve to immutable source versions/spans;
• fetch/access and hash are validated;
• claim-level entailment and authority are checked separately;
• canonical publisher/domain and dates are rendered;
• generated derivatives are labeled and map to originals;
• citation completeness is evaluated;
• high-stakes citations receive human audit.

Never accept a model-generated URL as a citation without retrieval and verification.

17. Security-conscious retrieval defenses

SafeRAG benchmarks silver noise, inter-context conflict, soft advertisements, and white denial-of-service, showing vulnerabilities across RAG components. SeCon-RAG uses semantic/cluster filtering and conflict-aware answer/evidence consistency.

Defenses include:

• source trust/allowlist filters;
• robust multi-source aggregation;
• cluster/duplicate controls;
• contradiction and ad/instruction detection;
• query-evidence consistency;
• leave-one-source-out stability;
• retrieval ensemble disagreement;
• evidence/citation verification;
• abstention on unstable evidence.

Every filter has false positives/negatives. Evaluate clean utility and adaptive attacks.

18. Certified and provable risk bounds

C-RAG applies conformal risk analysis to bound a declared bounded generation-risk function under calibration and stated distribution-shift assumptions. It certifies aggregate risk under those assumptions, not corpus authenticity or prompt-injection safety.

PRA-RAG samples combinations of retrieved text, identifies a robust subset using embedding geometry, and derives bounds under its poisoning model. The paper reports low attack success while retaining accuracy, with added sampling/generation cost. Its guarantee is specific to corruption and threat assumptions; provenance and adaptive attacks remain separate.

When stating a guarantee, name:

• risk/loss being bounded;
• calibration data and exchangeability/shift assumptions;
• adversary/corruption budget;
• confidence level;
• per-example versus aggregate scope;
• components outside the guarantee;
• clean-utility and compute cost.

19. Data minimization and purpose limitation

Before indexing, ask:

• is this source needed for the declared feature?
• which fields/regions are necessary?
• can sensitive data remain local or be retrieved on demand?
• does the embedding/generator provider receive it?
• may it be quoted, logged, cached, used as feedback, or used for training?
• how long must each raw/derived artifact remain?
• can the user inspect/correct/delete it?

Minimize at acquisition, retrieval, context, output, logs, and training. A RAG system often copies data into more locations than the source system; lineage and retention must cover all copies.

20. Deletion, correction, and unlearning

Build reverse lineage:

source version
 -> canonical blocks
 -> chunks/views/embeddings/postings
 -> summaries/propositions/captions/translations
 -> graph nodes/edges/community reports
 -> caches/traces/feedback labels
 -> training examples/checkpoints if applicable

A deletion request creates a tombstone, prevents resurrection by delayed events, removes active index/cache state, recomputes shared derivatives, and follows retention/backup policy. Verify with seeded queries and row/hash audits.

External-memory facts are easier to remove than parametric memorization only if they were never copied into fine-tuning/log datasets. If model unlearning is promised, define the technical verification; deleting a vector is not model unlearning.

Corrections should supersede wrong versions and invalidate answers/caches while preserving audit under policy. Temporal queries may still need the historical record.

21. Licensing, copyright, and source governance

Technical systems should track:

• source owner and acquisition basis;
• license/terms and allowed transformations;
• quotation/display limits;
• attribution requirements;
• training eligibility separate from retrieval eligibility;
• redistribution/export restrictions;
• geographic/contractual constraints;
• expiration/review date;
• generated-derivative policy.

Semantic relevance does not grant permission. Generated summaries and embeddings may remain derived artifacts subject to policy. Code RAG needs repository/file license and copied-code provenance. Web RAG must not assume publicly accessible means unrestricted reuse.

This repository does not provide jurisdiction-specific legal conclusions; organizations need qualified legal/privacy review for their data and markets.

22. Supply-chain security

Inventory and pin:

• connectors, parsers, OCR/layout models;
• embedding/reranker/generator/verifier models;
• tokenizers and prompt templates;
• vector/search/graph/database libraries;
• model/data downloads and hashes/signatures;
• containers, runtime, GPU drivers;
• external APIs and plugins;
• evaluation datasets and judges.

Use trusted registries, artifact signing, vulnerability scanning, least- privilege execution, network isolation, reproducible builds where feasible, and staged rollout. Model files and parsers process attacker-controlled content and belong in the threat model.

23. Logging and observability privacy

Logs are a second corpus. They can contain full queries, private evidence, answers, citations, identity, tool arguments, and secrets.

Define fields by purpose. Prefer stable opaque IDs, hashes, aggregates, and redacted snippets. Encrypt, ACL, tenant-partition, retain minimally, and audit access. Debug modes must expire and never silently become permanent production logging. LLM observability vendors are data processors/trust boundaries.

Security monitoring still needs useful signals: source IDs/trust, rank shifts, blocked ACLs, injection flags, duplicate clusters, tool attempts, cost, and policy decisions. Design telemetry to minimize content while preserving incident reconstruction under controlled access.

24. Secure development and release gates

Design review

• data-flow/trust-boundary diagram;
• source inventory, authority, licensing, and retention;
• identity/ACL model;
• threat model and abuse cases;
• external provider transfer;
• hard tool/cost limits;
• deletion/correction plan;
• incident owner and rollback.

Build/test

• unit/property tests for ACL and source lineage;
• cross-tenant canaries;
• poison/injection/conflict/ad/DoS corpora;
• privacy extraction/membership probes;
• citation/source spoof tests;
• parser/archive/resource limits;
• output DLP/policy tests;
• deletion and cache invalidation rehearsal;
• adversarial agent loop/tool tests.

Release

• signed versioned corpus/index/model/prompt manifest;
• clean utility and adversarial metrics with thresholds;
• staged shadow/canary rollout;
• monitoring and alert thresholds;
• rollback generation retained;
• operator/user disclosures and controls;
• high-risk human approval paths.

25. Red-team matrix

Attack	Variants	Observe
Targeted poison	lexical, embedding, graph, duplicate flood	target rank, answer, citation, clean utility
Trigger backdoor	rare phrase, multilingual, visual, memory	triggered ASR and benign behavior
Indirect injection	HTML/PDF/OCR/table/code/tool result	policy/tool/exfiltration behavior
Conflict	same entity/date, version, authority	selection, disclosure, abstention
Soft ad/source manipulation	recommendation, sponsored wording	source bias and disclosure
Cross-tenant	IDs, semantic probes, filters, graphs, caches	content/existence/timing leakage
Extraction	iterative queries, quotes, multimodal descriptions	recovered private content
Membership inference	response, score, citation, latency	AUC/advantage at query budget
Embedding/query attack	inversion, attribute, provider compromise	reconstruction and data transfer
Citation spoof	fake URL/title/date, non-supporting real source	validation and claim support
DoS	loops, long files, filter pathology, fan-out	p95/p99, resource/cost, availability
Deletion failure	summaries, edges, caches, replicas, logs	residual retrieval/output

Use adaptive attackers who know the defense assumptions; static benchmark success is not a certificate.

26. Incident response

When a RAG incident occurs:

1. preserve minimal authorized forensic trace and version IDs;
2. contain affected source/index/model/tool/tenant route;
3. disable or narrow risky retrieval/actions;
4. identify source/derived lineage and impacted outputs/users;
5. remove/quarantine content and rebuild a clean generation;
6. invalidate caches and rotate credentials/tokens if exposed;
7. validate with targeted and regression queries;
8. restore gradually with monitoring;
9. complete notification/remediation under organizational policy;
10. add tests, controls, and ownership fixes.

Rollback should switch to a known immutable index/prompt/model generation, not attempt ad hoc edits in a corrupted live index.

27. Residual risk and human control

No current architecture proves that untrusted retrieval plus a general LLM will never follow malicious content, leak allowed-but-sensitive evidence, or produce an unsupported synthesis. Use defense in depth and reduce autonomy/data exposure in proportion to harm.

High-stakes workflows need authoritative source constraints, local evidence display, user correction, abstention, human review, and separation between drafting and action. Citation improves auditability only when support and source authority are checked.

28. What the executable notebooks model

The safety notebook models pre-retrieval ACL enforcement, trust-domain/source metadata, content hashes, duplicate/poison clusters, prompt-injection markers, conflict grouping, output source-ID validation, deletion lineage, and adversarial test cases. It proves invariants such as “unauthorized chunks never enter the reranker context” on a small corpus.

It does not claim a regex detects all prompt injection, that similarity detects poisoning, or that toy access checks replace production identity systems. The goal is to make security boundaries and required tests executable.