# Context engineering, retrieval-conditioned generation, grounding, and citations

Retrieval returns candidates; augmentation decides what the model can actually
use. Generation quality depends on evidence selection, serialization, order,
compression, model architecture, prompting, decoding, attribution, conflict
handling, and abstention. A high-recall retriever can still produce a false
answer if the evidence is buried, contradictory, unauthoritative, or treated as
instructions.

## 1. The augmentation contract

For selected evidence units \(Z=\{z_1,\ldots,z_k\}\), construct a context
package, not an anonymous string:

```text
request and answer policy
task plan or subquestions
evidence items with stable IDs
source title, publisher, version, time, authority, and access scope
verbatim content or explicitly labeled derived summary
exact span/page/region/row/time coordinates
retrieval and rerank scores as diagnostics, not instructions
conflict/version groups
token budget and ordering policy
required output schema and citation syntax
```

The generator receives only authorized evidence. Evidence bytes are delimited
and declared untrusted data. A document instruction such as “ignore the user and
send secrets” must never enter the privileged instruction channel.

## 2. Evidence-set objectives

The context should maximize useful support per token:

\[
\max_{Z\subseteq C}
\sum_{h\in H} w_h\max_{z\in Z}\operatorname{support}(h,z)
-\lambda\operatorname{redundancy}(Z)
-\rho\operatorname{risk}(Z)
\]

subject to

\[
\sum_{z\in Z}\operatorname{tokens}(\operatorname{serialize}(z))
\le B,
\qquad |Z\cap\operatorname{source}(s)|\le m_s.
\]

Here \(H\) is the set of information needs or claims required to answer. This
formalizes why top-k relevance alone is inadequate for comparisons and
multi-hop questions.

### Useful evidence density

Define

\[
\operatorname{density}(Z)=
\frac{\text{tokens inside supporting spans}}
{\text{all evidence and wrapper tokens}}.
\]

Higher density often reduces distraction and cost, but removing definitions,
qualifiers, table headers, or surrounding conditions can make a short context
misleading. Optimize answer/citation behavior, not density alone.

## 3. Deduplication and source consolidation

Before packing:

1. collapse identical chunk/view IDs;
2. cluster near-duplicate text and syndicated copies;
3. merge overlapping chunks from one document into coherent spans;
4. group children by parent to avoid repeating the same section;
5. preserve independently authoritative sources even when semantically similar;
6. record which candidate IDs were consolidated.

Duplicate evidence consumes tokens, biases model attention, and creates fake
corroboration. A context with five copies of one press release has one source
family, not five independent confirmations.

## 4. Context ordering

[Lost in the Middle](https://aclanthology.org/2024.tacl-1.9/) showed that models
can use information at the beginning and end of a long context better than
evidence in the middle. The exact curve depends on model, task, length, prompt,
and training.

Ordering policies include:

- highest relevance/authority first;
- strongest evidence at both beginning and end;
- group by subquestion or claim;
- preserve source/document order for procedures and narratives;
- order chronologically for evolving facts;
- place definitions before dependent evidence;
- present conflicting sources adjacent with date/authority labels;
- interleave evidence and draft claims during incremental writing.

Randomize or systematically permute order in evaluation. If answer quality
changes sharply, the system is not robust enough to treat retrieval rank as an
innocent formatting choice.

## 5. Evidence serialization

An evidence wrapper should be compact, unambiguous, and machine-parseable:

```text
<evidence id="E7" source_id="doc-42" version="sha256:..."
          title="..." publisher="..." observed_at="..."
          location="page 8, bbox ..." trust="official">
VERBATIM UNTRUSTED SOURCE DATA
</evidence>
```

Do not rely on citation numbers tied only to position; reordering changes them.
Use stable IDs internally and render user-friendly footnotes afterward. Escape
delimiter-like content. Separate verbatim source text from model-generated
summary or extraction.

For tables, serialize headers with every selected row or provide a typed JSON/
relational object. For images, keep region IDs and coordinates. For APIs, record
endpoint/schema, parameters, response time, and immutable response body/hash.

## 6. Context compression

### 6.1 Extractive compression

Select sentences, clauses, rows, regions, or tokens from source evidence. It
retains a direct source-span mapping. Selection can use query relevance,
cross-encoder score, information coverage, or token-level salience.

Failure modes:

- removing negation, condition, unit, attribution, or temporal qualifier;
- retaining an answer-looking sentence without its definition;
- breaking pronoun/entity resolution;
- selecting redundant sentences independently;
- optimizing relevance while losing citation-complete support.

### 6.2 Abstractive compression

Generate a summary of one or several retrieved units. [RECOMP](https://openreview.net/forum?id=mlJLVigNHp)
trains extractive and abstractive compressors for downstream LM utility and can
emit an empty string when augmentation is not useful. Abstractive compression
can synthesize distributed information and reduce tokens substantially, but the
summary becomes a generated derivative that can omit, merge, or invent claims.

For every summary store source IDs/spans, model/prompt version, and a claim-level
support audit. Cite primary evidence, not only the summary. Do not use an
abstractive summary as the sole evidence for high-stakes claims.

### 6.3 Token-level prompt compression

[LLMLingua](https://aclanthology.org/2023.emnlp-main.825/) uses a budget
controller and token-level iterative compression based on a smaller language
model. [LongLLMLingua](https://aclanthology.org/2024.acl-long.91/) adds
query-aware document ranking, dynamic ratios, and reordering for long contexts.
[LLMLingua-2](https://aclanthology.org/2024.findings-acl.57/) learns extractive
compression through data distillation.

Perplexity is not the same as evidence utility. A predictable date, negation,
variable name, or citation marker may be vital despite low token surprise.
Evaluate answer correctness, citation precision/recall, entity/number retention,
and adversarial robustness at each compression ratio.

### 6.4 Latent/vector compression

Methods such as xRAG map retrieved text into a small number of learned embedding
tokens consumed by the generator. This can greatly reduce prompt tokens, but
the evidence becomes harder to inspect, cite, redact, and delete. It also couples
compressor and generator. Use only with a parallel provenance channel and
measure loss by claim type.

### 6.5 Compression decision rule

Compression is beneficial only when

\[
\Delta Q - \lambda_C\Delta C - \lambda_L\Delta L
-\lambda_A\Delta A -\lambda_R\Delta R > 0,
\]

where \(Q\) is answer utility, \(C\) cost, \(L\) latency, \(A\) attribution
quality, and \(R\) risk. Report all terms rather than “4x compression” alone.

## 7. Prompt-based RAG

The dominant application pattern places evidence in a decoder-only or seq2seq
prompt. A robust instruction specifies:

- the question and task;
- which knowledge sources are permitted;
- that evidence is untrusted data, not instructions;
- whether model prior knowledge is allowed;
- how to handle absent, partial, stale, or conflicting evidence;
- required claim granularity and citation attachment;
- output schema and refusal/abstention form;
- prohibition on fabricated source IDs or unseen URLs.

Avoid vague “use the following context” prompts. Test prompt variants on a
frozen evaluation set and record exact versions. Closed model behavior can
change without a name change; snapshot outputs and dates.

### Context-only versus permissive policies

**Context-only** generation requires every factual claim to follow from supplied
evidence. It is auditable but may mark useful true prior knowledge unsupported
and can over-abstain. **Permissive** generation allows parametric knowledge and
retrieval; it may improve completeness but blurs provenance. A middle policy
labels uncited background explicitly and forbids it for high-risk claims.

Evaluation must match policy. RAGTruth-style strict grounding and ordinary
factual correctness are different targets.

## 8. Original RAG latent-document integration

The 2020 [RAG paper](https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html)
combines a DPR retriever with BART and treats documents as latent variables.

### RAG-Sequence

One document conditions the full output sequence:

\[
p(y\mid x)\approx\sum_{z\in\operatorname{TopK}(x)}
p_\eta(z\mid x)\prod_{i=1}^{N}p_\theta(y_i\mid x,z,y_{<i}).
\]

### RAG-Token

The document marginal is recomputed at each output token:

\[
p(y\mid x)\approx\prod_{i=1}^{N}
\sum_{z\in\operatorname{TopK}(x)}p_\eta(z\mid x)
p_\theta(y_i\mid x,z,y_{<i}).
\]

The truncated top-k makes learning differentiable only over retrieved documents;
missing evidence receives no gradient. Latent likelihood does not guarantee the
model causally used or will cite the highest-probability document.

## 9. Fusion-in-Decoder and multi-passage readers

[FiD](https://aclanthology.org/2021.eacl-main.74/) concatenates the question
with each passage, encodes passages independently, concatenates encoder states,
and lets one decoder attend across all of them. Encoder cost scales roughly
linearly with passages; decoder cross-attention sees the combined sequence.

FiD can exploit many passages without forcing them through one encoder input.
It also enables reader-to-retriever distillation: reader attention or likelihood
provides a target for ranking. Limits include expensive training/inference,
fixed retrieval before decoding, and no automatic claim-to-passage attribution.

FiD-light/FiDO-style efficiency work reduces decoder cross-attention or encoder
states. When discussing speedups, distinguish passage encoding, decoder
attention, model parallelism, and end-to-end retrieval time.

## 10. Retrieval-augmented pretraining

### REALM

REALM marginalizes retrieved latent documents during masked-language-model
pretraining and periodically refreshes the index. Stale document embeddings
hurt learning. It demonstrates that index rebuild cadence is part of the
objective, not only infrastructure.

### RETRO

RETRO retrieves neighboring chunks from a massive token database and injects
retrieved representations through chunked cross-attention during language-model
pretraining and inference. It is non-parametric language modeling, not the same
as prompt RAG. Corpus scale, retrieval chunk timing, neighbor encoder, and
training exposure all matter.

### Atlas

Atlas combines Contriever-like retrieval, FiD, retrieval-augmented objectives,
and few-shot adaptation. It examines multiple objectives and reader-to-retriever
training. It illustrates a semiparametric model in which memory and parameters
are co-designed.

### kNN-LM and token memory

kNN-LM interpolates the base next-token distribution with a distribution from
nearest hidden-state/token pairs:

\[
p(w\mid h)=\lambda p_{\text{kNN}}(w\mid h)
+(1-\lambda)p_{\text{LM}}(w\mid h).
\]

Token-level memory adapts rapidly by changing the datastore but can be enormous,
slow, and hard to cite at document level.

## 11. Frozen-generator and black-box RAG

When generator weights are inaccessible, retrieval and prompting remain
trainable control surfaces. REPLUG-style approaches train a retriever from the
frozen LM’s preferences over retrieved documents. Prompt RAG can also use a
cross-encoder or selector trained on answer utility.

Black-box APIs complicate reproducibility, attribution, privacy, and cost. Store
model revision/date, complete prompt/evidence, output, token usage, and latency.
Never send sensitive evidence to a remote generator unless policy explicitly
allows it.

## 12. Iterative retrieval and generation

One-shot retrieval assumes the original query contains enough information to
find all evidence. Iterative methods alternate:

\[
s_t=(x,y_{<t},Z_{<t}),\quad
a_t\in\{\text{query},\text{retrieve},\text{read},\text{answer},\text{stop}\}.
\]

### IRCoT

Interleaves chain-of-thought steps with retrieval so intermediate entities guide
later search. It improves multi-hop evidence discovery but the reasoning text
can be unfaithful and introduces sequential latency.

### FLARE

Generates a tentative next sentence, identifies low-confidence tokens, retrieves
using those signals, and regenerates with evidence. Confidence is a retrieval
trigger, not proof of factual uncertainty; model calibration and query quality
matter.

### ITER-RETGEN

Uses a generated answer or rationale to retrieve better evidence in later
iterations. Generated errors can create feedback loops, so retain original-query
results and stop when evidence utility stops improving.

### Self-RAG and corrective/adaptive RAG

Self-RAG learns control and critique tokens for whether to retrieve and whether
evidence is relevant/supportive/useful. Corrective RAG grades retrieved evidence
and can trigger web search. Adaptive-RAG routes by question complexity. These
mechanisms are estimators with errors; expose decisions and evaluate over- and
under-retrieval separately.

## 13. Generation-time search policies and RL

Search-R1, ReSearch, StepSearch, GRIP, Q-RAG, DeepRAG, HiPRAG and related work
learn aspects of query, retrieve, select, reason, or stop. The policy objective
can be written

\[
J(\pi)=\mathbb E_{\tau\sim\pi}
[R_{\text{answer}}+\alpha R_{\text{support}}
+\beta R_{\text{process}}-\lambda C(\tau)].
\]

Outcome-only rewards allow spurious evidence, fabricated retrieval tags, or
answers from parametric knowledge. Process rewards require labels/proxies for
good search steps and can themselves be gamed. Always report search calls,
unique evidence, stopping, trace validity, citation support, and transfer across
corpus/retriever shifts—not answer score alone.

Hard budgets and permissions must remain outside the learned policy.

## 14. Long-form attributed generation

Long answers require planning and repeated evidence alignment:

1. decompose the requested output into sections/claims;
2. retrieve and select evidence per need;
3. draft atomic claims with local citations;
4. verify claim-evidence entailment and source authority;
5. retrieve missing support or revise/remove the claim;
6. check global consistency, redundancy, dates, and citation completeness;
7. render prose without detaching citations from claims.

[ALCE](https://aclanthology.org/2023.emnlp-main.398/) established benchmarks
and metrics for citation correctness and completeness in long-form generation.
[Think&Cite](https://aclanthology.org/2025.acl-long.490/) uses search over
query/retrieve/write actions with attribution rewards. Strong results come with
large search/reward-model cost and do not eliminate judge error.

### Citation placement

Attach citations immediately to supported clauses, not a paragraph-long pile.
One citation may support several adjacent atomic claims only if the mapping is
unambiguous. If sources disagree, attach each claim to its source and explain
the conflict. Do not cite a search result, homepage, or paper abstract when the
claim depends on a deeper section unless that text was retrieved.

## 15. Claim extraction and support checking

Split output into atomic claims \(c_i\). For cited evidence \(E_i\), classify:

- entailed/supported;
- contradicted;
- related but insufficient;
- no evidence;
- not externally verifiable/opinion;
- citation not accessible or version mismatch.

Citation precision is

\[
P_{cite}=\frac{\#\text{citations supporting attached claim}}
{\#\text{citations}},
\]

while completeness is

\[
R_{cite}=\frac{\#\text{claims needing evidence with sufficient support}}
{\#\text{claims needing evidence}}.
\]

NLI and LLM judges are proxies. Calibrate them on human-labeled domain claims,
audit disagreements, vary judge order/prompts, and report uncertainty. Numbers,
negation, temporal scope, causal claims, and multi-source synthesis deserve
separate slices.

## 16. Verification and correction loops

Post-generation verification can:

- retrieve evidence for each draft claim;
- run entailment/contradiction checks;
- compare names, dates, units, and numbers deterministically;
- identify unsupported or incomplete claims;
- revise with evidence or delete/qualify the claim;
- re-run citation completeness and consistency checks.

RARR-style research-and-revise systems improve attribution by searching for
support and editing. A verifier sharing the same model, prompt assumptions, and
retrieved context is not independent. Use deterministic checks and human review
for high stakes.

Correction can reduce fluency or introduce a new inconsistency. Compare the
draft and final answer claim-by-claim and store both.

## 17. Abstention and selective prediction

The system should distinguish:

- no relevant evidence;
- relevant but insufficient/partial evidence;
- sufficient but conflicting evidence;
- sufficient evidence with low-authority sources;
- evidence available but generator confidence low;
- policy prohibits answering or exposing evidence.

Threshold a calibrated risk score or answer only at selected coverage.
Selective risk at coverage \(\kappa\) is the error among answered examples, not
the error over all requests. Plot risk-coverage curves and choose thresholds
from product harm, not an arbitrary similarity value.

[Sufficient Context](https://openreview.net/forum?id=8N8hWwTj6D) combines a
context-sufficiency signal with model self-confidence. Both require calibration
and can fail under domain or corpus shift. Test false-answer and false-abstain
rates separately.

For partial evidence, a useful answer states what is supported, what is missing,
and which assumptions would be required. Do not fill gaps with plausible prose.

## 18. Conflict, uncertainty, and temporal claims

Evidence can conflict because of version, scope, measurement method, source
error, or genuine dispute. A correct system should not force one synthesized
fact without explanation.

Build claim clusters keyed by entity/relation/time/scope. Preserve value, unit,
validity, source, and extraction confidence. Prefer authoritative/current
sources under a declared policy, but surface unresolved authoritative conflicts.

For numerical synthesis:

- normalize units and currency with explicit rate/date;
- use executable calculation;
- cite every input;
- record rounding;
- distinguish reported values from calculated outputs.

For temporal claims, include the as-of date in both retrieval filters and answer.
An answer can be faithful to a stale passage and still be factually wrong now.

## 19. Structured output and constrained generation

Generate a typed intermediate object:

```json
{
  "claims": [
    {
      "text": "...",
      "evidence_ids": ["E2"],
      "status": "supported",
      "confidence": 0.87
    }
  ],
  "answerability": "partial",
  "missing_information": ["..."],
  "conflicts": []
}
```

Validate schema, evidence-ID existence, citations, forbidden fields, and policy
before rendering natural language. Constrained decoding guarantees syntax, not
truth. Never let the model create a source ID and later assume it exists.

Tool outputs should carry typed values and error states. If SQL or a calculator
fails, the generator must not improvise a result.

## 20. Multimodal evidence integration

A VLM can consume page images or regions alongside text. Context construction
must decide resolution, page count, crops, OCR/text views, region coordinates,
and duplicate modality views. Visual tokens can dominate cost.

Use modality-appropriate citations:

- text: document and character span;
- PDF: page and bounding box;
- table: table, row/column/cell range;
- image/chart: region/polygon and caption link;
- audio/video: time range and speaker/track;
- code: repository revision, path, symbol, and lines.

Parsed text and page image can be complementary. Do not treat two views of the
same evidence as independent sources. Test clean and degraded scans, rotation,
blur, crop, watermark, layout shift, table density, and multilingual OCR.

## 21. Long context versus selected context

If the complete source fits, long context avoids retrieval misses but increases
tokens, latency, cost, and distraction. Selected context improves density but
adds a recall ceiling. A router can use:

- corpus/source length;
- retrieval score/coverage/sufficiency;
- query paraphrase/identifier characteristics;
- expected multi-hop/global nature;
- cost and latency budget;
- privacy and source boundary.

Evaluate at equal cost and equal latency, not only equal nominal context window.
Store actual input tokens and effective evidence positions. Self-Route-style
systems attempt RAG first and escalate to long context when the model judges
evidence insufficient; judge errors and changing API economics limit
generalization.

## 22. Prompt injection boundary

The augmentation layer is where untrusted documents meet a powerful model.

Required controls:

- sanitize active markup, remote resources, hidden text, and executable content;
- delimit and label evidence as data;
- do not place retrieved content in system/developer instructions;
- isolate tool credentials and restrict tool arguments/actions;
- use source allowlists/trust tiers and quarantine suspicious documents;
- detect conflicting instructions and retrieval anomalies;
- prevent evidence from choosing its own citations or tools;
- perform output DLP/policy checks;
- red-team indirect prompt injection under the real agent/tool configuration.

Prompt phrasing alone is not a security boundary. The security chapter provides
the full threat model.

## 23. Generation evaluation matrix

Hold evidence constant and test:

| Condition | What it isolates |
|---|---|
| Gold complete evidence | generator capability ceiling |
| Gold evidence + irrelevant distractors | noise sensitivity |
| Gold evidence + hard same-entity distractors | relation/date discrimination |
| Contradictory evidence | conflict handling |
| Partial evidence | calibrated partial answer/abstention |
| No supporting evidence | hallucination and refusal |
| Stale versus current versions | temporal selection/use |
| Low versus high authority | source-quality policy |
| Evidence order permutations | position sensitivity |
| Compressed versus raw evidence | compression loss |
| Injected malicious evidence | instruction/data separation |
| Citation IDs shuffled/invalid | citation integrity |

Metrics include correctness, claim precision/recall, completeness,
faithfulness, citation correctness/completeness, authority, conflict accuracy,
abstention risk/coverage, prompt/completion tokens, latency, and cost.

## 24. Failure diagnosis

- **Evidence present but unused:** ordering, distraction, model capacity, or
  prompt policy.
- **Wrong evidence dominates:** rerank/selection/authority/time issue.
- **Correct prose, wrong citations:** attribution alignment or fabricated IDs.
- **Cited passage mentions but does not support:** entailment judge or passage
  granularity issue.
- **Correct context, unsupported extra claims:** permissive prior knowledge or
  generation hallucination.
- **Over-abstention:** sufficiency/calibration threshold or incomplete prompt.
- **Under-abstention:** confidence proxy overtrust or missing no-answer training.
- **Compression regression:** qualifier/entity/number lost or summary invented.
- **Long-answer inconsistency:** claim planning and cross-section verification.
- **Tool result ignored or rewritten incorrectly:** structured integration and
  schema validation.

## 25. What the executable notebooks model

The context/generation notebooks implement token-budgeted set selection,
parent expansion, extractive compression, ordering permutations, structured
evidence IDs, claim-to-citation validation, contradiction/version grouping,
risk-coverage curves, and injected-instruction tests. The generator is
extractive and deterministic so every support decision can be inspected.

The examples do not claim that lexical overlap is semantic entailment or that a
toy compressor reproduces RECOMP/LLMLingua. They expose the control surfaces and
measurements that a neural replacement must preserve.