# Adaptive and agentic RAG, long-term memory, and temporal knowledge

Static RAG retrieves once from a fixed index. Adaptive systems decide whether,
where, when, and how often to retrieve; memory systems decide what to write,
consolidate, update, and forget; temporal systems decide which version was valid
for the requested time. These are control and state-management problems, not
just better similarity search.

## 1. Static, dynamic, adaptive, and agentic

Use precise terms:

- **static one-shot RAG:** fixed query, retriever, k, selector, and one generation;
- **dynamic retrieval:** retrieval occurs during generation based on a trigger;
- **adaptive RAG:** routes among no retrieval, retrievers, budgets, or methods;
- **iterative RAG:** later retrieval depends on earlier evidence or output;
- **corrective RAG:** evaluates evidence and retries/switches sources;
- **agentic RAG:** a policy plans and executes multiple search/read/tool/answer
  actions with state and a stopping rule;
- **memory-augmented agent:** reads and writes persistent state across sessions.

Calling a fixed chain “agentic” does not change its technical properties.

## 2. A control-theoretic frame

At step \(t\), state is

\[
s_t=(x,h_t,Z_t,M_t,b_t),
\]

where \(x\) is the request, \(h_t\) reasoning/action history, \(Z_t\) selected
evidence, \(M_t\) persistent memory state, and \(b_t\) remaining budget. Action

\[
a_t\in\{\text{no-search},\text{query source},\text{read},\text{select},
\text{verify},\text{write-memory},\text{answer},\text{abstain},\text{stop}\}.
\]

The objective is constrained expected utility:

\[
\max_\pi\mathbb E[Q(\tau)-\lambda C(\tau)-\rho R(\tau)]
\]

subject to per-request calls, latency, tokens, permissions, and safety. Hard
constraints must be enforced by an external runtime because average reward
penalties do not guarantee them.

## 3. Should retrieval happen?

Retrieval can harm when the model already knows a stable fact, evidence is
irrelevant/misleading, latency dominates, or sensitive data would cross a trust
boundary. No-retrieval can harm on fresh, long-tail, private, or cited facts.

Signals for a retrieve/no-retrieve router:

- task/intent and domain;
- time sensitivity and source requirement;
- named entities, identifiers, quoted text;
- model uncertainty or self-knowledge confidence;
- historical router outcomes;
- retrieval score/entropy from a cheap probe;
- question complexity/hop estimate;
- requirement for citations or exact source language;
- budget and policy.

Uncertainty alone is unreliable: models can be confident and wrong or uncertain
about simple phrasing. Evaluate over-search and under-search independently,
quality/cost curves, and calibration under model/corpus shifts.

## 4. Adaptive-RAG and complexity routing

[Adaptive-RAG](https://aclanthology.org/2024.naacl-long.389/) trains a classifier
to route questions among no retrieval, one-step retrieval, and iterative
multi-hop retrieval. It obtains an efficiency/quality trade-off but classifier
confusion can route simple questions to expensive search or complex questions
to insufficient retrieval.

Complexity is not the same as retrieval need. A complex math question may need
no corpus; a simple “current CEO?” question needs fresh evidence. Use a
multi-dimensional router: knowledge need, source type, hops, computation, and
risk.

## 5. Self-RAG

[Self-RAG](https://openreview.net/forum?id=hSyW5go0v8) trains a generator to emit
reflection/control tokens for retrieval, relevance, support, and utility. At
inference, weighted reflection probabilities guide generation/search.

Important distinctions:

- reflection labels came from a teacher/critic and can be wrong;
- emitting “supported” is not proof of support;
- retrieval is still bounded by the chosen retriever/corpus;
- decoding weights trade answer quality, citations, and retrieval frequency;
- training and benchmark snapshots constrain freshness claims.

It is influential because control is part of the model vocabulary, not because
self-critique solves grounding.

## 6. Corrective and evidence-grading RAG

[Corrective RAG](https://arxiv.org/abs/2401.15884) evaluates retrieval quality,
refines/filters evidence, and can use web search. A generic loop is:

1. initial retrieval;
2. grade each result and set sufficiency;
3. if sufficient, select/refine;
4. if ambiguous, broaden or combine;
5. if poor, rewrite/switch source/web;
6. generate and verify.

Grader false negatives delete useful evidence; false positives preserve poison.
Web fallback changes trust, freshness, and reproducibility. Evaluate grader
confusion and each transition, not only final answers.

## 7. Uncertainty-triggered active retrieval

FLARE generates a tentative continuation, detects low-confidence tokens, forms
a query, retrieves, and regenerates. Other dynamic systems use entropy, token
probability, hidden states, or learned gates.

Challenges:

- decoder probability is not factual confidence;
- low-confidence function words can trigger useless search;
- high-confidence hallucinations do not trigger;
- tentative text may contain false entities that misdirect retrieval;
- repeated generation/search increases latency;
- closed-model log probabilities may be unavailable or unstable.

Calibrate triggers on factual spans and impose cooldown/step limits.

## 8. ReAct, Self-Ask, and search reasoning

ReAct interleaves reasoning and actions/observations. Self-Ask decomposes a
question into follow-up questions, often using search. IRCoT interleaves
retrieval and chain-of-thought. These methods let intermediate entities bridge
multi-hop gaps.

Reasoning traces are operational plans, not necessarily faithful explanations.
Do not expose hidden reasoning as evidence. The auditable trace is actions,
queries, results, selected spans, and citations. Validate tool calls and keep
thought text outside security decisions.

## 9. Query generation and search state

At each step the planner should know:

- unresolved information needs;
- entities/relations/constraints already established;
- conflicting claims;
- source coverage and authority;
- previous queries/results and duplicate clusters;
- evidence gaps in the draft answer;
- remaining budget.

Generate targeted queries for unresolved needs, not paraphrases of everything.
Search-state deduplication should detect repeated normalized queries and no-new-
evidence loops. Preserve the original query as a fallback.

### Information gain

Reward a step when it reduces uncertainty or adds support for an uncovered need:

\[
IG_t=H(H\mid Z_{t-1})-H(H\mid Z_t).
\]

In practice, entropy is estimated by a model/coverage proxy. Novel text is not
necessarily useful information; penalize redundancy and unsupported expansions.

## 10. Stopping

Stop when one of these holds:

- all required claims/subquestions have sufficient authoritative evidence;
- a calibrated sufficiency threshold is met;
- the next action’s expected value is below cost/risk;
- no new evidence after a defined number of attempts;
- evidence is irreconcilably conflicting and the correct output is conflict;
- source/tool unavailable and safe partial answer/abstention is required;
- hard step/time/token/cost budget reached.

The stop model can be trained, but hard budgets are deterministic. Evaluate
premature stop, wasteful extra search, quality versus calls, and tail behavior.

## 11. Search-R1, ReSearch, and StepSearch

### Search-R1

Trains LLMs with outcome RL to interleave `<think>`, `<search>`, and retrieved
results, masking retrieved tokens from policy loss. Reported gains show that
small/open models can learn useful search behavior. Outcome correctness does not
prove intermediate evidence fidelity.

### ReSearch

Uses GRPO-style outcome training from scratch for reason-with-search behavior.
Strong gains again demonstrate learnable search, while reward attribution and
search trace faithfulness remain open.

### StepSearch

Adds stepwise process rewards for information gain and redundancy and uses PPO.
It improves search trajectories relative to outcome-only baselines, but process
proxies and synthetic subquestions can be gamed or domain-specific.

Compare methods under the same retriever, corpus snapshot, maximum calls,
generator, and evaluation. Percentage gains across incompatible setups are not
a leaderboard.

## 12. GRIP, Q-RAG, DeepRAG, and HiPRAG

### GRIP

[GRIP](https://aclanthology.org/2026.acl-long.196/) emits structured tokens such
as retrieval, intermediary, answer, and solved inside decoding. SFT teaches
typed trajectories; RL further optimizes them. In the paper’s ablation, most of
the reported gain over the no-RL variant comes from structured supervision,
illustrating the importance of trajectory representation before RL.

### Q-RAG

[Q-RAG](https://iclr.cc/virtual/2026/poster/10009944) freezes the LLM and learns
a value-based embedder that selects candidate chunks or STOP. State includes the
query and selected evidence; actions are remaining chunks. It separates policy
learning from generator fine-tuning but relies on support-fact supervision and
synthetic/long-context task structure.

### DeepRAG

Models decomposition and retrieve/reason decisions as an MDP. Its reported
accuracy improvement should be interpreted within its baselines/datasets, not
as a cross-paper universal percentage.

### HiPRAG

[HiPRAG](https://iclr.cc/virtual/2026/poster/10010451) adds hierarchical process
reward based on the fraction of optimal search/non-search steps, substantially
reducing over-search in its setup. Defining an “optimal” step requires labels or
an oracle and may not transfer across corpus/model changes.

## 13. Retriever and source routing

A router can choose BM25, dense, learned sparse, graph, visual, table/SQL, web,
or long context. A contextual bandit formulation chooses action \(r\) from
query features with reward final utility minus cost:

\[
r^*=\arg\max_r\mathbb E[Q\mid x,r]-\lambda C_r.
\]

Use offline logged-policy correction cautiously; unchosen retriever outcomes
are missing. A safe production router keeps exploration/minimum hybrid coverage
and fallback. R3AG/RouteRAG-style work learns routing, but policy quality is
conditional on its retriever pool.

Source routing also enforces authority/privacy. A legal query may require an
official jurisdictional source even if web search is semantically stronger.

## 14. Long context as an action

Treat full-context reading as one expensive tool. Route to it when:

- one/few documents fit and retrieval sufficiency is low;
- global/narrative dependencies are important;
- query paraphrase defeats lexical retrieval;
- document-wide contextualization is more reliable;
- privacy permits local full-context processing;
- latency/cost budget allows it.

Self-Route attempts RAG first and escalates when the model says context is
insufficient. Test the sufficiency judge and actual token/cost economics for the
deployed models. Long context still needs evidence localization and citations.

## 15. Persistent memory is more than a vector store

A memory system has six policies:

1. **write:** what becomes persistent;
2. **representation:** raw turn, fact, episode, summary, graph, latent vector;
3. **retrieve:** what is relevant now;
4. **consolidate:** merge/rewrite repeated memories;
5. **update/forget:** supersede, decay, delete, expire;
6. **use:** how memory influences response/actions.

Without write and update semantics, retrieval returns stale or contradictory
history more efficiently.

## 16. Memory types

### Working memory

Current task state and recent turns. Usually prompt/agent state with strict size
limits; not necessarily persistent.

### Episodic memory

Specific events/interactions with time and source turns. Useful for “what
happened last time?” and sequences.

### Semantic memory

Consolidated facts/preferences extracted from episodes. Compact but derived;
must link to episodes and support correction.

### Procedural memory

Learned workflows, successful plans, tool recipes, or demonstrations. Reusing
them can speed agents but can propagate obsolete or unsafe procedures.

### Profile/preferences

Explicit user settings and inferred preferences. Inference must be labeled,
editable, scoped, and sensitive to context/time.

### Latent neural memory

Persistent vectors or model-internal states such as MemoryLLM/M+. Efficient but
less interpretable, deletable, and citeable than explicit external records.

## 17. Memory write policy

Before writing, evaluate:

- user intent/consent and sensitivity;
- future utility;
- novelty versus existing memory;
- confidence and source;
- temporal scope/expiry;
- whether it is fact, preference, hypothesis, or instruction;
- tenant/user ownership and visibility;
- retention/deletion obligations.

Avoid writing transient guesses, model hallucinations, secrets, or malicious
document instructions. Prefer explicit confirmation for sensitive or durable
preferences. Store source turn and extraction model/version.

## 18. Consolidation and update

Consolidation reduces repeated memory and storage. A safe process:

1. retrieve related memories;
2. group by entity/attribute/scope;
3. detect agreement, update, or conflict;
4. create a derived summary/fact with source links;
5. mark superseded memories without destroying audit history;
6. re-evaluate on new evidence;
7. propagate deletion.

[RMM](https://aclanthology.org/2025.acl-long.413/) uses prospective reflection
at several granularities and retrospective RL to improve memory retrieval.
ComRAG clusters/consolidates historical QA. Summaries can drift, evaluator
rewards can leak target information, and deletion must reach consolidated
derivatives.

## 19. Memory conflicts and corrections

Represent claims with validity intervals and status:

```text
subject, predicate, value, confidence, valid_from, valid_to,
observed_at, source_episode_ids, status, sensitivity, owner
```

A new statement may correct, temporarily override, or apply in a different
scope. Do not overwrite blindly. Prefer most recent explicit user correction
for a current preference; retain old value as historical only if policy permits.
When uncertainty remains, ask or state the conflict.

## 20. LongMemEval and memory evaluation

[LongMemEval](https://openreview.net/forum?id=pZiyCaVuti) tests information
extraction, multi-session reasoning, knowledge updates, temporal reasoning, and
abstention over long interactions. Evaluate memory components separately:

- write precision/recall;
- retrieval recall/precision;
- update/supersession correctness;
- temporal order;
- answer use/faithfulness;
- abstention when memory absent;
- privacy/deletion;
- storage and latency over time.

The correct denominator is not “all past turns retrieved.” Most history should
not be injected.

## 21. Temporal RAG

Time has several axes:

- event/valid time: when a fact is true;
- publication time;
- source update time;
- ingestion/observed time;
- index availability time;
- query/answer time.

Store them separately. A news article published today may describe an event
from years ago; a filing may restate an earlier period; a correction changes
what the system knows without changing event time.

### Temporal retrieval score

One form is

\[
s(q,d)=s_{rel}(q,d)+\alpha s_{authority}(d)
+\beta s_{valid}(t_q,d)-\gamma s_{stale}(t_q,d).
\]

`s_valid` checks interval compatibility; `s_stale` is task-dependent. Do not
apply exponential recency decay to historical or evergreen questions.

### Version and contradiction grouping

Group documents/claims referring to the same fact and identify current,
superseded, corrected, or disputed versions. Select under the requested as-of
time and cite version/date. If no time is specified for dynamic facts, answer
with an explicit current-as-of timestamp.

## 22. Freshness operations

Retrieval makes updates possible; it does not make an index fresh. Define:

- source-to-ingestion lag;
- ingestion-to-index lag;
- replica convergence;
- cache invalidation lag;
- stale-answer rate;
- retrieval age distribution;
- current-version selection rate;
- update/delete failure rate.

Use connector CDC, reconciliation, version IDs, tombstones, blue/green indexes,
short/targeted caches, and snapshot replay. Monitor volatile domains separately.

[FreshQA/FreshPrompt](https://aclanthology.org/2024.findings-acl.813/) evaluates
dynamic and false-premise questions using live search organization. Dynamic
benchmarks drift; preserve query time, result pages, and answer snapshot.
[CRAG](https://proceedings.neurips.cc/paper_files/paper/2024/hash/1435d2d0fca85a84d83ddcb754f58c29-Abstract-Datasets_and_Benchmarks_Track.html)
includes facts with dynamism from years to seconds and shows current systems
remain weak on dynamic/long-tail/complex questions.

## 23. Caching in adaptive and temporal RAG

Cache keys must include:

- normalized original query and relevant conversation state;
- tenant/user/ACL policy scope;
- corpus/index generation;
- as-of time and freshness class;
- retriever/reranker/prompt/model versions;
- source/tool parameters;
- output policy/language.

Semantic caches risk returning another user’s or stale answer. Do not share
sensitive caches across tenants. Set TTL by domain and invalidate on source/
index updates. Prompt-cache timing can leak information; treat caching as part
of the privacy threat model.

## 24. Agent runtime safety

The runtime—not the LLM—enforces:

- source/tool allowlists and scopes;
- per-action argument schema;
- ACL and tenant context;
- maximum calls/tokens/time/cost;
- read-only versus write actions;
- sandbox for code/SQL;
- rate limits and concurrency;
- loop/duplicate-query detection;
- evidence/data instruction separation;
- output DLP and approval gates;
- immutable action/evidence audit.

Retrieved content cannot grant new permissions. A memory instruction cannot
override current user/system policy. Tool observations are data and may be
malicious.

## 25. Observability for agentic RAG

Each trace records:

```text
request and policy context
router decisions and calibrated scores
all planned/executed queries and tools
candidate IDs/scores/index generations
selected/dropped evidence with reasons
memory reads/writes/updates
stop reason and remaining budget
draft/final claims and citations
latency/token/cost per step
errors, retries, fallbacks, safety decisions
```

Aggregate:

- search/no-search, retriever, source, and long-context route rates;
- calls and unique evidence per query;
- over/under-search;
- loop/no-new-evidence rate;
- stop reasons and budget exhaustion;
- memory write/read/use/update/delete metrics;
- stale evidence/answer rates;
- quality/citation/risk versus calls, latency, and cost;
- policy violations and blocked injections.

Do not log sensitive evidence indiscriminately; apply access control, redaction,
retention, and audit to observability data.

## 26. Evaluation matrix

### Routing

Compare oracle route, learned/rule route, always-no-retrieval, always-one-shot,
always-iterative, and cost-matched baselines. Report route confusion and regret.

### Search trajectory

Evaluate query validity, evidence gain, redundancy, supporting-path completion,
calls, stop quality, and fabricated/invalid actions. Counterfactually swap or
remove evidence to test reliance.

### Memory

Test extraction, long-delay retrieval, update, conflict, temporal order,
abstention, deletion, adversarial memory, and privacy.

### Temporal

Freeze historical snapshots and replay queries at several as-of times. Test
latest, historical, correction, future/unavailable, and false-premise cases.

### Robustness

Vary retriever, corpus version, generator, model size, language, domain, and
budget. A learned search policy that works only with its training retriever is
not general retrieval intelligence.

## 27. Common anti-patterns

- “Agentic” means several chained LLM calls with no state/budget evaluation.
- Retrieval is triggered by raw model confidence without calibration.
- Search traces are treated as faithful reasoning.
- Outcome reward is used as evidence that retrieval improved.
- The agent can loop or call arbitrary tools.
- Memory writes every conversation turn.
- Consolidated summaries lose source/deletion lineage.
- Latest ingestion time is treated as fact validity.
- Recency decay is applied to all queries.
- Caches ignore tenant/index/as-of time.
- Freshness is claimed because web search exists.
- Long context is used as a fallback without equal-cost evaluation.

## 28. What the executable notebooks model

The agent/memory notebook implements a bounded state machine with route, query,
retrieve, inspect, stop, and abstain actions; hard budgets; evidence-gain and
duplicate-query diagnostics; explicit memory write/update/delete records; and
bitemporal selection. It compares fixed, adaptive, and iterative policies at
equal call budgets and emits a complete trace.

The policy is deterministic and heuristic. It demonstrates the state/action/
constraint contract that supervised or RL policies must obey, not a reproduction
of Search-R1, GRIP, or Q-RAG.