Adaptive and agentic RAG, long-term memory, and temporal knowledge

Static RAG retrieves once from a fixed index. Adaptive systems decide whether, where, when, and how often to retrieve; memory systems decide what to write, consolidate, update, and forget; temporal systems decide which version was valid for the requested time. These are control and state-management problems, not just better similarity search.

1. Static, dynamic, adaptive, and agentic

Use precise terms:

static one-shot RAG: fixed query, retriever, k, selector, and one generation;
dynamic retrieval: retrieval occurs during generation based on a trigger;
adaptive RAG: routes among no retrieval, retrievers, budgets, or methods;
iterative RAG: later retrieval depends on earlier evidence or output;
corrective RAG: evaluates evidence and retries/switches sources;
agentic RAG: a policy plans and executes multiple search/read/tool/answer actions with state and a stopping rule;
memory-augmented agent: reads and writes persistent state across sessions.

Calling a fixed chain “agentic” does not change its technical properties.

2. A control-theoretic frame

At step $t$ , state is

s_t=(x,h_t,Z_t,M_t,b_t),

where $x$ is the request, $h_t$ reasoning/action history, $Z_t$ selected evidence, $M_t$ persistent memory state, and $b_t$ remaining budget. Action

a_t\in\{\text{no-search},\text{query source},\text{read},\text{select}, \text{verify},\text{write-memory},\text{answer},\text{abstain},\text{stop}\}.

The objective is constrained expected utility:

\max_\pi\mathbb E[Q(\tau)-\lambda C(\tau)-\rho R(\tau)]

subject to per-request calls, latency, tokens, permissions, and safety. Hard constraints must be enforced by an external runtime because average reward penalties do not guarantee them.

3. Should retrieval happen?

Retrieval can harm when the model already knows a stable fact, evidence is irrelevant/misleading, latency dominates, or sensitive data would cross a trust boundary. No-retrieval can harm on fresh, long-tail, private, or cited facts.

Signals for a retrieve/no-retrieve router:

task/intent and domain;
time sensitivity and source requirement;
named entities, identifiers, quoted text;
model uncertainty or self-knowledge confidence;
historical router outcomes;
retrieval score/entropy from a cheap probe;
question complexity/hop estimate;
requirement for citations or exact source language;
budget and policy.

Uncertainty alone is unreliable: models can be confident and wrong or uncertain about simple phrasing. Evaluate over-search and under-search independently, quality/cost curves, and calibration under model/corpus shifts.

4. Adaptive-RAG and complexity routing

Adaptive-RAG trains a classifier to route questions among no retrieval, one-step retrieval, and iterative multi-hop retrieval. It obtains an efficiency/quality trade-off but classifier confusion can route simple questions to expensive search or complex questions to insufficient retrieval.

Complexity is not the same as retrieval need. A complex math question may need no corpus; a simple “current CEO?” question needs fresh evidence. Use a multi-dimensional router: knowledge need, source type, hops, computation, and risk.

5. Self-RAG

Self-RAG trains a generator to emit reflection/control tokens for retrieval, relevance, support, and utility. At inference, weighted reflection probabilities guide generation/search.

Important distinctions:

reflection labels came from a teacher/critic and can be wrong;
emitting “supported” is not proof of support;
retrieval is still bounded by the chosen retriever/corpus;
decoding weights trade answer quality, citations, and retrieval frequency;
training and benchmark snapshots constrain freshness claims.

It is influential because control is part of the model vocabulary, not because self-critique solves grounding.

6. Corrective and evidence-grading RAG

Corrective RAG evaluates retrieval quality, refines/filters evidence, and can use web search. A generic loop is:

initial retrieval;
grade each result and set sufficiency;
if sufficient, select/refine;
if ambiguous, broaden or combine;
if poor, rewrite/switch source/web;
generate and verify.

Grader false negatives delete useful evidence; false positives preserve poison. Web fallback changes trust, freshness, and reproducibility. Evaluate grader confusion and each transition, not only final answers.

7. Uncertainty-triggered active retrieval

FLARE generates a tentative continuation, detects low-confidence tokens, forms a query, retrieves, and regenerates. Other dynamic systems use entropy, token probability, hidden states, or learned gates.

Challenges:

decoder probability is not factual confidence;
low-confidence function words can trigger useless search;
high-confidence hallucinations do not trigger;
tentative text may contain false entities that misdirect retrieval;
repeated generation/search increases latency;
closed-model log probabilities may be unavailable or unstable.

Calibrate triggers on factual spans and impose cooldown/step limits.

8. ReAct, Self-Ask, and search reasoning

ReAct interleaves reasoning and actions/observations. Self-Ask decomposes a question into follow-up questions, often using search. IRCoT interleaves retrieval and chain-of-thought. These methods let intermediate entities bridge multi-hop gaps.

Reasoning traces are operational plans, not necessarily faithful explanations. Do not expose hidden reasoning as evidence. The auditable trace is actions, queries, results, selected spans, and citations. Validate tool calls and keep thought text outside security decisions.

9. Query generation and search state

At each step the planner should know:

unresolved information needs;
entities/relations/constraints already established;
conflicting claims;
source coverage and authority;
previous queries/results and duplicate clusters;
evidence gaps in the draft answer;
remaining budget.

Generate targeted queries for unresolved needs, not paraphrases of everything. Search-state deduplication should detect repeated normalized queries and no-new- evidence loops. Preserve the original query as a fallback.

Information gain

Reward a step when it reduces uncertainty or adds support for an uncovered need:

IG_t=H(H\mid Z_{t-1})-H(H\mid Z_t).

In practice, entropy is estimated by a model/coverage proxy. Novel text is not necessarily useful information; penalize redundancy and unsupported expansions.

10. Stopping

Stop when one of these holds:

all required claims/subquestions have sufficient authoritative evidence;
a calibrated sufficiency threshold is met;
the next action’s expected value is below cost/risk;
no new evidence after a defined number of attempts;
evidence is irreconcilably conflicting and the correct output is conflict;
source/tool unavailable and safe partial answer/abstention is required;
hard step/time/token/cost budget reached.

The stop model can be trained, but hard budgets are deterministic. Evaluate premature stop, wasteful extra search, quality versus calls, and tail behavior.

11. Search-R1, ReSearch, and StepSearch

Search-R1

Trains LLMs with outcome RL to interleave <think>, <search>, and retrieved results, masking retrieved tokens from policy loss. Reported gains show that small/open models can learn useful search behavior. Outcome correctness does not prove intermediate evidence fidelity.

ReSearch

Uses GRPO-style outcome training from scratch for reason-with-search behavior. Strong gains again demonstrate learnable search, while reward attribution and search trace faithfulness remain open.

StepSearch

Adds stepwise process rewards for information gain and redundancy and uses PPO. It improves search trajectories relative to outcome-only baselines, but process proxies and synthetic subquestions can be gamed or domain-specific.

Compare methods under the same retriever, corpus snapshot, maximum calls, generator, and evaluation. Percentage gains across incompatible setups are not a leaderboard.

12. GRIP, Q-RAG, DeepRAG, and HiPRAG

GRIP

GRIP emits structured tokens such as retrieval, intermediary, answer, and solved inside decoding. SFT teaches typed trajectories; RL further optimizes them. In the paper’s ablation, most of the reported gain over the no-RL variant comes from structured supervision, illustrating the importance of trajectory representation before RL.

Q-RAG

Q-RAG freezes the LLM and learns a value-based embedder that selects candidate chunks or STOP. State includes the query and selected evidence; actions are remaining chunks. It separates policy learning from generator fine-tuning but relies on support-fact supervision and synthetic/long-context task structure.

DeepRAG

Models decomposition and retrieve/reason decisions as an MDP. Its reported accuracy improvement should be interpreted within its baselines/datasets, not as a cross-paper universal percentage.

HiPRAG

HiPRAG adds hierarchical process reward based on the fraction of optimal search/non-search steps, substantially reducing over-search in its setup. Defining an “optimal” step requires labels or an oracle and may not transfer across corpus/model changes.

13. Retriever and source routing

A router can choose BM25, dense, learned sparse, graph, visual, table/SQL, web, or long context. A contextual bandit formulation chooses action $r$ from query features with reward final utility minus cost:

r^*=\arg\max_r\mathbb E[Q\mid x,r]-\lambda C_r.

Use offline logged-policy correction cautiously; unchosen retriever outcomes are missing. A safe production router keeps exploration/minimum hybrid coverage and fallback. R3AG/RouteRAG-style work learns routing, but policy quality is conditional on its retriever pool.

Source routing also enforces authority/privacy. A legal query may require an official jurisdictional source even if web search is semantically stronger.

14. Long context as an action

Treat full-context reading as one expensive tool. Route to it when:

one/few documents fit and retrieval sufficiency is low;
global/narrative dependencies are important;
query paraphrase defeats lexical retrieval;
document-wide contextualization is more reliable;
privacy permits local full-context processing;
latency/cost budget allows it.

Self-Route attempts RAG first and escalates when the model says context is insufficient. Test the sufficiency judge and actual token/cost economics for the deployed models. Long context still needs evidence localization and citations.

15. Persistent memory is more than a vector store

A memory system has six policies:

write: what becomes persistent;
representation: raw turn, fact, episode, summary, graph, latent vector;
retrieve: what is relevant now;
consolidate: merge/rewrite repeated memories;
update/forget: supersede, decay, delete, expire;
use: how memory influences response/actions.

Without write and update semantics, retrieval returns stale or contradictory history more efficiently.

16. Memory types

Working memory

Current task state and recent turns. Usually prompt/agent state with strict size limits; not necessarily persistent.

Episodic memory

Specific events/interactions with time and source turns. Useful for “what happened last time?” and sequences.

Semantic memory

Consolidated facts/preferences extracted from episodes. Compact but derived; must link to episodes and support correction.

Procedural memory

Learned workflows, successful plans, tool recipes, or demonstrations. Reusing them can speed agents but can propagate obsolete or unsafe procedures.

Profile/preferences

Explicit user settings and inferred preferences. Inference must be labeled, editable, scoped, and sensitive to context/time.

Latent neural memory

Persistent vectors or model-internal states such as MemoryLLM/M+. Efficient but less interpretable, deletable, and citeable than explicit external records.

17. Memory write policy

Before writing, evaluate:

user intent/consent and sensitivity;
future utility;
novelty versus existing memory;
confidence and source;
temporal scope/expiry;
whether it is fact, preference, hypothesis, or instruction;
tenant/user ownership and visibility;
retention/deletion obligations.

Avoid writing transient guesses, model hallucinations, secrets, or malicious document instructions. Prefer explicit confirmation for sensitive or durable preferences. Store source turn and extraction model/version.

18. Consolidation and update

Consolidation reduces repeated memory and storage. A safe process:

retrieve related memories;
group by entity/attribute/scope;
detect agreement, update, or conflict;
create a derived summary/fact with source links;
mark superseded memories without destroying audit history;
re-evaluate on new evidence;
propagate deletion.

RMM uses prospective reflection at several granularities and retrospective RL to improve memory retrieval. ComRAG clusters/consolidates historical QA. Summaries can drift, evaluator rewards can leak target information, and deletion must reach consolidated derivatives.

19. Memory conflicts and corrections

Represent claims with validity intervals and status:

subject, predicate, value, confidence, valid_from, valid_to,
observed_at, source_episode_ids, status, sensitivity, owner

A new statement may correct, temporarily override, or apply in a different scope. Do not overwrite blindly. Prefer most recent explicit user correction for a current preference; retain old value as historical only if policy permits. When uncertainty remains, ask or state the conflict.

20. LongMemEval and memory evaluation

LongMemEval tests information extraction, multi-session reasoning, knowledge updates, temporal reasoning, and abstention over long interactions. Evaluate memory components separately:

write precision/recall;
retrieval recall/precision;
update/supersession correctness;
temporal order;
answer use/faithfulness;
abstention when memory absent;
privacy/deletion;
storage and latency over time.

The correct denominator is not “all past turns retrieved.” Most history should not be injected.

21. Temporal RAG

Time has several axes:

event/valid time: when a fact is true;
publication time;
source update time;
ingestion/observed time;
index availability time;
query/answer time.

Store them separately. A news article published today may describe an event from years ago; a filing may restate an earlier period; a correction changes what the system knows without changing event time.

Temporal retrieval score

One form is

s(q,d)=s_{rel}(q,d)+\alpha s_{authority}(d) +\beta s_{valid}(t_q,d)-\gamma s_{stale}(t_q,d).

s_valid checks interval compatibility; s_stale is task-dependent. Do not apply exponential recency decay to historical or evergreen questions.

Version and contradiction grouping

Group documents/claims referring to the same fact and identify current, superseded, corrected, or disputed versions. Select under the requested as-of time and cite version/date. If no time is specified for dynamic facts, answer with an explicit current-as-of timestamp.

22. Freshness operations

Retrieval makes updates possible; it does not make an index fresh. Define:

source-to-ingestion lag;
ingestion-to-index lag;
replica convergence;
cache invalidation lag;
stale-answer rate;
retrieval age distribution;
current-version selection rate;
update/delete failure rate.

Use connector CDC, reconciliation, version IDs, tombstones, blue/green indexes, short/targeted caches, and snapshot replay. Monitor volatile domains separately.

FreshQA/FreshPrompt evaluates dynamic and false-premise questions using live search organization. Dynamic benchmarks drift; preserve query time, result pages, and answer snapshot. CRAG includes facts with dynamism from years to seconds and shows current systems remain weak on dynamic/long-tail/complex questions.

23. Caching in adaptive and temporal RAG

Cache keys must include:

normalized original query and relevant conversation state;
tenant/user/ACL policy scope;
corpus/index generation;
as-of time and freshness class;
retriever/reranker/prompt/model versions;
source/tool parameters;
output policy/language.

Semantic caches risk returning another user’s or stale answer. Do not share sensitive caches across tenants. Set TTL by domain and invalidate on source/ index updates. Prompt-cache timing can leak information; treat caching as part of the privacy threat model.

24. Agent runtime safety

The runtime—not the LLM—enforces:

source/tool allowlists and scopes;
per-action argument schema;
ACL and tenant context;
maximum calls/tokens/time/cost;
read-only versus write actions;
sandbox for code/SQL;
rate limits and concurrency;
loop/duplicate-query detection;
evidence/data instruction separation;
output DLP and approval gates;
immutable action/evidence audit.

Retrieved content cannot grant new permissions. A memory instruction cannot override current user/system policy. Tool observations are data and may be malicious.

25. Observability for agentic RAG

Each trace records:

request and policy context
router decisions and calibrated scores
all planned/executed queries and tools
candidate IDs/scores/index generations
selected/dropped evidence with reasons
memory reads/writes/updates
stop reason and remaining budget
draft/final claims and citations
latency/token/cost per step
errors, retries, fallbacks, safety decisions

Aggregate:

search/no-search, retriever, source, and long-context route rates;
calls and unique evidence per query;
over/under-search;
loop/no-new-evidence rate;
stop reasons and budget exhaustion;
memory write/read/use/update/delete metrics;
stale evidence/answer rates;
quality/citation/risk versus calls, latency, and cost;
policy violations and blocked injections.

Do not log sensitive evidence indiscriminately; apply access control, redaction, retention, and audit to observability data.

26. Evaluation matrix

Routing

Compare oracle route, learned/rule route, always-no-retrieval, always-one-shot, always-iterative, and cost-matched baselines. Report route confusion and regret.

Search trajectory

Evaluate query validity, evidence gain, redundancy, supporting-path completion, calls, stop quality, and fabricated/invalid actions. Counterfactually swap or remove evidence to test reliance.

Memory

Test extraction, long-delay retrieval, update, conflict, temporal order, abstention, deletion, adversarial memory, and privacy.

Temporal

Freeze historical snapshots and replay queries at several as-of times. Test latest, historical, correction, future/unavailable, and false-premise cases.

Robustness

Vary retriever, corpus version, generator, model size, language, domain, and budget. A learned search policy that works only with its training retriever is not general retrieval intelligence.

27. Common anti-patterns

“Agentic” means several chained LLM calls with no state/budget evaluation.
Retrieval is triggered by raw model confidence without calibration.
Search traces are treated as faithful reasoning.
Outcome reward is used as evidence that retrieval improved.
The agent can loop or call arbitrary tools.
Memory writes every conversation turn.
Consolidated summaries lose source/deletion lineage.
Latest ingestion time is treated as fact validity.
Recency decay is applied to all queries.
Caches ignore tenant/index/as-of time.
Freshness is claimed because web search exists.
Long context is used as a fallback without equal-cost evaluation.

28. What the executable notebooks model

The agent/memory notebook implements a bounded state machine with route, query, retrieve, inspect, stop, and abstain actions; hard budgets; evidence-gain and duplicate-query diagnostics; explicit memory write/update/delete records; and bitemporal selection. It compares fixed, adaptive, and iterative policies at equal call budgets and emits a complete trace.

The policy is deterministic and heuristic. It demonstrates the state/action/ constraint contract that supervised or RL policies must obey, not a reproduction of Search-R1, GRIP, or Q-RAG.