The first retrieval-augmented systems were easy to draw: a question entered a retriever, a fixed number of passages came back, and a generator answered. That diagram is still a valuable control, but it no longer describes the most capable systems. A contemporary retrieval agent may decide that no search is needed, rewrite the question, split it into subquestions, alternate between text search and graph traversal, inspect a page image, read a whole document, compare conflicting editions, retrieve a memory from six months ago, and then stop because another call is unlikely to justify its cost. Every verb in that sentence is a policy decision. Every decision crosses a boundary involving money, latency, authority, privacy, or trust.
This changes the engineering question. The problem is not simply whether the nearest chunks are relevant. It is whether a stateful policy can acquire enough trustworthy evidence to answer while obeying hard limits that its own learned objective cannot guarantee. At step \(t\), it is useful to describe the state as
where \(x\) is the request, \(h_t\) the action history, \(Z_t\) the evidence gathered so far, \(M_t\) persistent memory, \(b_t\) the remaining token, call, time, and money budget, and \(p_t\) the current identity and policy snapshot. The action may be to search, read, traverse, verify, write memory, answer, abstain, or stop. A learned policy \(\pi\) seeks quality, but a deployable objective must also price delay, resource use, and harm:
The coefficients express product choices; they do not turn prohibitions into guarantees. A penalty for leaking another tenant's document is not an access-control system. A negative reward for an expensive loop is not a deterministic step limit. Hard permissions, network scopes, tool schemas, and deadlines belong to the runtime around the policy.
Search begins with the option not to search
Adaptive retrieval starts before the first query. Searching can hurt when a model already knows a stable fact, when the only available corpus is weak, when private text would cross an external-model boundary, or when the additional latency is disproportionate to the task. Not searching hurts on fresh, long-tail, private, exact-quotation, and citation-required questions. A useful router therefore asks several questions at once: Is external knowledge necessary? Which authority is required? How many inferential hops are likely? Is the fact time-sensitive? Must the answer cite an exact source? What is the cost and risk of each available path?
Adaptive-RAG made this budget choice explicit by training a classifier to route among no retrieval, one retrieval, and iterative retrieval. Its HotpotQA result is instructive rather than triumphant: with FLAN-T5-XL, the adaptive route used 3.55 steps and 5.99 seconds for EM/F1 42.0/53.82, while always using the multi-step path took 5.53 steps and 9.38 seconds for 44.6/56.54. That is a quality-latency frontier, not a universal win. It also reveals a common category error. Question complexity is not retrieval need: a difficult proof may be solved without a corpus, while “Who is the current CEO?” is linguistically simple and operationally dependent on fresh evidence.
Self-RAG moves control into the token stream, teaching a generator reflection tokens for retrieval, relevance, support, and utility. Corrective RAG grades initial evidence, filters or refines it, and switches to rewritten or web queries when it judges the local result poor. FLARE retrieves when tentative generation becomes uncertain. These approaches provide useful control signals, yet none turns self-assessment into proof. Model probability is not factual confidence; a confident hallucination may never trigger search, while an obscure proper noun may cause wasteful retrieval. A grader can discard the only useful passage or preserve a poisoned one. Web fallback also changes the trust domain, reproducibility, and privacy contract at the moment it is invoked.
The router's safest output is not merely a label but an auditable decision: route chosen, alternatives considered, calibrated score, applicable policy, maximum calls, and fallback. A legal assistant may route to an official jurisdictional source even when open-web similarity is higher. A medical workflow may allow retrieval but prohibit sending the resulting passage to an external model. Authority and permission are features of the action space, not decorations applied after ranking.
Planning is the management of unresolved evidence
Once search begins, query planning should be understood as maintaining an evidence ledger. The agent must know which claims remain unresolved, which entities and temporal constraints have been established, which sources disagree, which query forms have already failed, and how much budget remains. A good next query targets one gap. A bad next query paraphrases the whole prompt, repeatedly retrieving the same cluster of pages.
ReAct, Self-Ask, and IRCoT established the pattern of interleaving questions, actions, and observations. Their internal prose is not necessarily a faithful explanation of why an answer was produced. The inspectable artifact is the external trajectory: normalized query, source or tool, returned IDs and scores, selected spans, rejected evidence, and final citations. Search-state deduplication should catch repeated queries and no-new-evidence loops. The original user query should remain available as a fallback, because an early hallucinated entity can send every later rewrite in the wrong direction.
One can describe the ideal value of a search step as information gain,
where \(H\) is uncertainty over an answer hypothesis. In practice, systems approximate this with claim coverage, entailment, novelty, or a learned critic. Novel text is not necessarily new evidence, and a second copy of the same upstream press release is not independent corroboration. A planner needs source lineage and duplicate clusters as much as embeddings.
Search-R1, ReSearch, and StepSearch show why reinforcement learning is attractive here. Search-R1 uses outcome reward to train interleaved reasoning and search while masking retrieved tokens from policy loss; ReSearch learns a similar pattern with GRPO; StepSearch adds process rewards for information gain and redundancy. In 2026, GRIP expresses retrieval, intermediary reasoning, answers, and solved state as typed output tokens, while Q-RAG learns a value function over candidate chunks and a STOP action with the generator frozen. These works demonstrate that search behavior can be learned. They also expose three traps: an answer-only reward can bless a spurious trajectory, a process proxy can be gamed, and a policy trained against one retriever and snapshot may fail when either changes.
The stopping rule is therefore as important as query generation. Stop when all required claims have sufficiently authoritative support; when the expected marginal value of another action falls below its cost and risk; when repeated attempts add no evidence; when a hard budget is exhausted; or when the evidence remains irreconcilable and the correct answer is a disclosed conflict. GRIP's reported increase from three to ten permitted calls raised actual mean calls only from 1.24 to 1.62 and its mean score from 41.0 to 41.8, a concrete picture of diminishing returns. Learned stopping may improve efficiency, but deterministic ceilings must terminate loops even when the policy does not.
Representation is another routing decision
The agent does not merely choose whether to retrieve. It chooses what kind of world to retrieve from. Flat text chunks are efficient for local facts. Graphs are useful when paths, communities, and relationships carry the answer. Tables and SQL preserve typed operations that prose similarity destroys. Page images retain layout, charts, handwriting, and spatial association. Long context can preserve narrative and document-wide dependencies that top-\(k\) selection severs.
Graph RAG itself names several different mechanisms. Microsoft GraphRAG extracts entities, relations, and claims, partitions the graph into hierarchical communities, writes community reports, and maps and reduces those reports for global questions. Its original experiments favored graph variants over vector retrieval for comprehensiveness on generated global-sensemaking questions, while vector search was often more direct. HippoRAG instead seeds an entity-relation graph from the query and uses Personalized PageRank to recover associative multi-hop evidence. These are not interchangeable: one is corpus-wide synthesis, the other query-focused path discovery. Entity resolution errors can fabricate a bridge or sever a real one, and a high PageRank score is not logical support.
Dynamic graph construction avoids maintaining a global structure but shifts extraction cost into the request path. RouteRAG learns to choose between text and graph evidence as reasoning unfolds. The useful lesson is conditionality: a graph can help when relational structure is both real and accurately extracted; it can lose to ordinary RAG on flat lookup. Every graph node, edge, community report, and summary also needs source lineage, permissions, valid time, and deletion propagation.
The same discipline applies to multimodal retrieval. ColPali embeds page images into patch vectors and scores each query token against its best-matching patch. It reported average nDCG@5 of 81.3 on ViDoRe versus 67.0 for the strongest parsed-text pipeline in that comparison, but its index was about 257.5 KB per page versus 8.6 KB for BGE. Visual retrieval avoids some OCR and layout losses; it does not automatically yield grounded generation. A citation must still localize the relevant page region, distinguish text from inferred chart content, and preserve access controls for faces, signatures, or sensitive imagery. Graph and multimodal routes expand both representational power and the privacy surface.
Long context should be treated as one more expensive tool, not as the opposite of RAG. An EMNLP 2024 Industry Track comparison found full-context models ahead of a fixed top-five RAG baseline when the whole input fit, while a Self-Route strategy first tried RAG and escalated when context appeared insufficient. The revealing PassKey case was lexical: a keyword query gave RAG 80.34 versus long context 65.25, but paraphrasing collapsed RAG to 4.58 while long context held 69.32. The choice depends on document count, query phrasing, global dependencies, evidence localization, privacy, and current token economics. More context can increase recall and simultaneously increase distraction. Route under an equal budget, and require citations even when the whole document is visible.
Memory turns retrieval into governance over time
Persistent memory is not a vector index with a longer retention period. It is a set of policies for writing, representing, retrieving, consolidating, updating, forgetting, and using prior state. Working memory holds the current task; episodic memory preserves events and their source turns; semantic memory derives compact facts; procedural memory stores reusable workflows; profile memory stores explicit or inferred preferences. Latent neural memory can be efficient, but it is harder to inspect, cite, correct, and delete.
A write policy must decide whether a statement is durable, useful, novel, sufficiently supported, consensual, and safe to retain. It should record whether the item is a fact, preference, hypothesis, or instruction, along with owner, sensitivity, source, confidence, temporal scope, and expiration. Writing every turn converts transient guesses, secrets, and injected text into future retrieval candidates. A malicious page that says “remember this instruction” is still untrusted evidence; it cannot authorize a memory write.
Consolidation creates derived risk. Suppose three episodes say that a user lives in Amsterdam and a later correction says Tokyo. A summary that overwrites the old value loses history; a summary that keeps both without validity intervals creates conflict; a derived embedding with no source links makes deletion unverifiable. A useful claim record includes valid_from, valid_to, observed_at, source episode IDs, confidence, status, sensitivity, and owner. New evidence may correct a claim, supersede it, or merely apply in another scope. LongMemEval accordingly evaluates extraction, multi-session reasoning, updates, temporal reasoning, and abstention rather than ANN recall alone.
Time in an external corpus has the same multiplicity. Event time says when a fact was true; publication time says when a source reported it; observation time says when the system ingested it; index time says when it became retrievable. A filing published today may restate an earlier quarter. A correction changes the system's knowledge without changing the historical event. Temporal selection can be sketched as
but recency decay must be task-specific. “Latest policy” prefers the current version; “policy in 2021” must not. A current answer should carry an as-of time. A historical answer should retrieve the version valid then. Conflicting versions should be grouped by claim and surfaced when authority or chronology does not resolve them.
Freshness is operational rather than architectural. Web access does not prove freshness, and an updated source does not prove an updated index. Measure source-to-ingestion lag, ingestion-to-index lag, replica convergence, cache invalidation, stale-answer rate, current-version selection, and delete failures. Cache keys must include principal, policy, index generation, temporal scope, and model/prompt versions. Otherwise an adaptive agent can retrieve yesterday's answer faster—or another user's answer fastest of all.
Retrieval creates a hostile data plane
The moment an agent reads mutable external or enterprise content, relevance and trust diverge. A threat model begins with assets—documents, queries, memories, embeddings, indexes, ACLs, credentials, citations, caches, logs, budgets—and follows them across connectors, parsers, OCR, derived summaries, sparse and vector indexes, rerankers, models, tools, and observability systems. Each boundary needs an owner, identity, region, retention rule, and allowed purpose. Generated captions, triples, and community reports remain derivatives, not primary evidence.
Corpus poisoning attacks both ranking and generation. The attacker crafts text likely to rank for a target query and includes a false claim or instruction likely to steer the answer. PoisonedRAG demonstrated targeted attacks with only a few malicious texts among millions; paraphrase and perplexity filters were insufficient. AgentPoison placed trigger-linked instructions in an agent's memory neighborhood at a very low poison rate while preserving benign aggregate behavior. This is why average accuracy can look healthy while a targeted backdoor succeeds.
Controls begin upstream: authenticated connectors, immutable hashes and versions, source trust tiers, quarantine, near-duplicate and sudden-volume detection, per-source contribution caps, cross-source lineage, canary queries, and rollback to a known index generation. None proves semantic truth. For high-risk domains, authority must be governed, not inferred from embedding proximity.
Indirect prompt injection is distinct from false content. A retrieved PDF, hidden HTML element, OCR layer, code comment, table cell, filename, or tool response may instruct the model to reveal data, call a URL, alter policy, or write memory. Telling the model to “ignore document instructions” is useful defense in depth but not a privilege boundary. Evidence should enter a typed, untrusted-data channel. The evidence-processing model should lack unnecessary credentials. An external runtime must validate every tool name, argument, destination, data class, and write action; restrict network egress; default to read-only; and require approval for consequential external effects.
Authorization must precede content exposure. For principal \(u\) and policy snapshot \(a_t\), every component should enforce
before the document or sensitive metadata reaches an ANN candidate list, reranker, LLM, log, or cache. Post-filtering is too late: the item may already have crossed a tenant boundary or displaced authorized evidence. Child chunks inherit parent ACLs; graph edges and community summaries must not bridge security domains; semantic caches must include identity and policy; long-running agents should recheck permissions before output when membership can change.
Privacy extends beyond content disclosure. Adaptive probing can reveal whether a record exists through citations, scores, counts, latency, or answer confidence. Embeddings can leak semantic attributes and remote services can observe raw queries or access patterns. Multimodal evidence can expose faces, locations, signatures, and background documents; graphs make formerly implicit relationships directly queryable. Logs are a second corpus containing user intent, private passages, tool arguments, and sometimes secrets. Minimize content at acquisition, context, output, caching, telemetry, and training, while preserving opaque IDs and controlled forensic traces sufficient for incident reconstruction.
No. Its embedding neighborhood, score, presence, or effect on candidate competition may already have leaked; an external reranker may already have received it. Security is an end-to-end invariant, not a prompt convention.
The bounded agent contract
A safe retrieval agent is powerful precisely because its discretion is bounded. The model may propose queries, evidence, and actions. The runtime resolves identity, authorizes sources, limits calls and graph or SQL complexity, sanitizes active content, validates structured arguments, blocks arbitrary exfiltration, separates reading from writing, and records immutable version IDs. The model may propose a memory; a policy decides whether it is stored. The model may cite an evidence ID; the system verifies that the ID was actually retrieved, remains accessible to the viewer, resolves to the immutable version shown, and supports the adjacent claim. The model may judge evidence sufficient; hard deadlines and abstention policy still govern termination.
The operating trace should make this contract visible without indiscriminately copying private text. It records the policy snapshot, route, queries, tools, candidate IDs and index generations, selected and rejected evidence with reasons, memory reads and writes, conflicts, stop reason, remaining budget, claim-to-citation map, stage latency and cost, blocked injections, retries, and fallback. The evaluation then asks not only whether the answer was correct, but whether the route was justified, evidence gain exceeded repetition, the stop was timely, the version was valid, permissions held at every boundary, and deletion reached every derivative.
Not every frontier gain needs an agent. ReasonIR trains retrieval around reasoning utility rather than surface similarity. GritLM shares representation work between embedding and generation. The 2026 CompactDS result is a useful rebuke to architectural vanity: a broad, high-quality datastore and efficient exact/approximate search can make a simple pipeline better than an elaborate policy working over weak evidence. Parser quality, negatives, reranking, context selection, and a cleaner corpus may yield more supported utility than another search loop. Agentic machinery should enter only after its failure slice and marginal evidence gain are named.
The agentic frontier, then, is not an agent that searches more. It is an agent that can choose among search, structure, memory, and long context while remaining legible to a system that does not trust it. Retrieval becomes genuinely useful when the agent knows what remains unknown. It becomes deployable only when permission, provenance, time, and limits remain true even when the agent is wrong.