Retrieval, query transformation, fusion, reranking, and evidence selection

Retrieval is a cascade, not a model name. A complete system understands the query, applies policy and filters, generates candidates from one or more representations, approximates nearest neighbors under a resource budget, fuses heterogeneous results, reranks them, and selects a collectively useful evidence set. Each stage has a different objective and failure ceiling.

1. Retrieval as a constrained cascade

For query $q$ , authorized corpus $\mathcal C_a$ , retrievers $r_j$ , and generator utility $u$ , a general retrieval problem is

R_j = \operatorname{TopK}_{d\in\mathcal C_a} s_j(q_j,d), \qquad C=\bigcup_j R_j,

Z^*=\arg\max_{Z\subseteq C,\,\operatorname{cost}(Z)\le B} \mathbb E[u(y;q,Z)]-\lambda\operatorname{risk}(Z).

First-stage retrieval maximizes recall cheaply. Reranking estimates relevance or utility more accurately on the candidate set. Selection optimizes the set rather than each item independently. If relevant evidence never enters $C$ , later stages cannot recover it. If $C$ has high recall but the selector removes a necessary hop, generation still fails.

Report the candidate count at every transition:

query -> transformations -> per-retriever top-k -> union/dedup -> rerank depth
      -> selected chunks -> expanded parents -> packed tokens -> cited chunks

2. Query understanding before scoring

Create a query state rather than immediately embedding the raw string:

original text
conversation-resolved intent
language and requested answer language
entities, aliases, identifiers, quoted phrases
time/geography/product/version constraints
tenant and ACL scope
task type and expected evidence shape
answerability/complexity estimate
allowed sources/tools and budget
generated query variants with lineage

Do not let an LLM rewrite silently replace the user’s request. Preserve the original query, run exact-identifier retrieval against it, and record every transformation. Validate structured filters against a schema; generated filter values can be wrong even when the rewritten prose sounds reasonable.

Intent classes with retrieval consequences

navigational: find a known page or record; exact title/identifier is central;
informational fact: one supporting passage;
comparative/list: evidence coverage across several entities;
multi-hop: connected evidence and intermediate entities;
aggregate: a complete population or executable data query, not top-k similarity;
procedural: preserve order and prerequisites;
temporal: align evidence validity with requested time;
conversational: resolve history without importing stale topics;
exploratory/global: diversity and corpus-level themes;
unanswerable/false premise: sufficiency and abstention rather than nearest neighbor confidence.

3. Classical lexical retrieval

3.1 Boolean and phrase retrieval

An inverted index supports exact term, phrase, proximity, prefix, range, and field queries. Boolean filters are essential for identifiers, dates, product versions, jurisdictions, languages, tenants, and source classes. Dense similarity should not approximate a predicate such as effective_date <= t < expiry_date or tenant_id = A.

Positional postings allow phrase and distance constraints. Exact fields should use a keyword analyzer that preserves case/punctuation when relevant; full-text fields may use tokenization, stemming, or decompounding. Keep raw and analyzed views separate.

3.2 Query likelihood

The language-model retrieval view ranks a document by its probability of generating query terms. With Dirichlet smoothing,

\log p(q\mid d)= \sum_{t\in q} f(t,q) \log\frac{f(t,d)+\mu p(t\mid\mathcal C)}{|d|+\mu}.

The corpus model prevents zero probability and $\mu$ controls how much short documents are smoothed. Query likelihood and BM25 encode related but different assumptions; both remain valuable baselines.

3.3 BM25 and fielded BM25

For term $t$ ,

\operatorname{BM25}(q,d)=\sum_{t\in q} \operatorname{IDF}(t) \frac{(k_1+1)f(t,d)}{f(t,d)+k_1(1-b+b|d|/\operatorname{avgdl})}.

k1 controls saturation: repeating a term ten times should not produce ten times the relevance. b controls length normalization. Tune by retrieval-unit type; tables, titles, tickets, and long sections have different length distributions.

BM25F combines field-specific term frequencies and length normalization. A conceptual field weight is

\widetilde f(t,d)=\sum_f w_f \frac{f(t,d_f)}{1-b_f+b_f|d_f|/\operatorname{avgdl}_f},

followed by saturation. It is preferable to literally duplicating title text because title/body statistics and boosts remain explicit.

3.4 BM25 variants

BM25+, BM25L, pivoted length normalization, proximity features, and domain analyzers address specific biases such as over-penalizing long documents or underusing term proximity. Treat variants as tunable hypotheses, not a generic upgrade. Always retain a transparent ordinary BM25 result.

3.5 Pseudo-relevance feedback

Rocchio moves a query vector toward assumed relevant documents and away from assumed nonrelevant ones:

q'=\alpha q+\frac{\beta}{|D_r|}\sum_{d\in D_r}d -\frac{\gamma}{|D_n|}\sum_{d\in D_n}d.

RM3 estimates a relevance model from top documents and interpolates its terms with the original query. Feedback can bridge vocabulary mismatch using the actual corpus, but the assumption that top results are relevant causes query drift on ambiguous or low-recall initial searches. Limit expansion terms, retain original terms, and evaluate per-query regressions.

3.6 Efficient top-k query execution

A teaching implementation that loops over every document computes BM25 scores correctly but is not a search engine. Production inverted indexes exploit sorted postings, compressed gaps, score upper bounds, and skipping.

WAND identifies a pivot where the sum of term upper bounds can exceed the current top-k threshold. Documents before the pivot are skipped or advanced. Block-Max WAND stores tighter upper bounds per postings block. MaxScore separates essential and nonessential lists under a threshold. Impact-ordered indexes group postings by score contribution. Exact dynamic-pruning algorithms avoid scoring candidates that provably cannot enter top-k; approximate variants trade effectiveness for more skipping.

Measure postings decoded, candidates fully scored, p95 latency, and result identity against exhaustive scoring. Learned sparse systems can produce dense posting lists that erase sparse-serving advantages.

4. Neural sparse retrieval

Neural sparse methods keep a vocabulary-sized vector and dot-product scoring, so nonzero dimensions can be served with an inverted index.

4.1 Contextual term weighting: DeepCT

DeepCT predicts a contextual impact for each occurrence of a token, then aggregates/quantizes it into an indexable term weight. The same word can carry different importance in different passages. It improves weighting without semantic vocabulary expansion, so unseen synonym mismatch remains.

4.2 Document expansion: doc2query/docT5query

Train a sequence model $p(q\mid d)$ and generate queries that a document might answer. Append them to an index-only document view and use ordinary BM25. This moves model cost offline and lets generated vocabulary create lexical matches. Risks are index growth, generic generated queries, factual errors, and evaluation leakage if generation learned benchmark-like questions.

4.3 Learned impacts: DeepImpact and uniCOIL

DeepImpact combines document expansion with learned scalar impacts per unique token. COIL stores contextual vectors under exact lexical keys; uniCOIL reduces each contextual token to a scalar. These preserve exact-match routing while letting context determine strength. They cannot directly match a query term to a different vocabulary term unless expansion supplied it.

4.4 Vocabulary expansion: SPLADE

SPLADE maps contextual token logits into sparse vocabulary weights. A common aggregation is

w_j(x)=\max_{i\in x}\log(1+\operatorname{ReLU}(z_{ij})), \qquad s(q,d)=w(q)^\top w(d).

Masked-language-model heads activate terms not literally present, enabling learned expansion. FLOPS-style regularization penalizes average activation:

\mathcal L_{\text{FLOPS}}=\sum_j \left(\frac{1}{B}\sum_{i=1}^{B}w_j(x_i)\right)^2.

The retrieval loss, distillation teacher, hard negatives, query/document regularization, and initialization define distinct SPLADE variants. Compare effectiveness together with nonzeros/query, postings/doc, index bytes, and latency. Sparse in vector notation is not necessarily sparse in execution.

4.5 Learned-sparse selection criteria

Choose learned sparse when semantic expansion is useful but exact-term explanations, mutable inverted indexes, or lexical infrastructure matter. Keep BM25 for identifiers and audit learned expansions: a high-weight expansion can reveal why a result appeared, but it can also introduce a misleading concept.

5. Single-vector dense retrieval

5.1 Dual-encoder architecture

A query encoder $E_q$ and document encoder $E_d$ produce

u=E_q(q),\quad v=E_d(d),\quad s(q,d)=u^\top v

or cosine similarity. Towers may share weights or not. Pooling may use [CLS], mean pooling, weighted pooling, or a learned projection. Document vectors are precomputed; this is why a dual encoder scales and why it compresses all document relevance into one vector.

5.2 Training loss

With positive $d_i^+$ and candidate documents $D_i$ ,

\mathcal L_i=-\log \frac{\exp(s(q_i,d_i^+)/\tau)} {\sum_{d\in D_i}\exp(s(q_i,d)/\tau)}.

The negative distribution defines the task. Random negatives are too easy; in-batch negatives improve efficiency; BM25 negatives teach semantic disambiguation around lexical matches; ANN-mined negatives expose the current model’s confusions; teacher-denoised negatives reduce false-negative damage. See the training chapter for detailed curricula.

5.3 Representation and metric choices

Dot product uses both direction and norm; cosine removes norm.
L2-normalized cosine and dot product have identical ranking.
Euclidean distance on unit vectors is monotonic with cosine.
Maximum inner-product search may need different ANN treatment than metric L2 search.
Temperature affects contrastive gradients but not simple inference ranking.
Low precision, quantization, or Matryoshka truncation changes neighbor order; validate it.

5.4 Model families

DPR: supervised QA positives, in-batch negatives, and BM25 hard negatives;
ANCE: asynchronously mines global ANN negatives;
RocketQA: cross-batch negatives, teacher denoising, and data augmentation;
Condenser/coCondenser/RetroMAE/SimLM: retrieval-oriented representation pretraining;
Contriever: unsupervised contrastive dense retrieval;
GTR/E5/INSTRUCTOR/BGE/GritLM: broad weak supervision or instructions for general-purpose embeddings;
ReasonIR and 2026 utility-aware work: train toward reasoning/helpfulness, not only topical similarity.

Do not select an embedding by a global MTEB average. Match language, domain, unit length, query style, instruction format, dimension, license, latency, and index cost. Evaluate exact identifiers and out-of-domain slices because dense models often fail there.

5.5 Single-vector bottleneck

A long unit can contain many topics, entities, and relations, but one vector must represent them all. Relevant fine details can be averaged away. Smaller chunks, multi-vector retrieval, contextual/late chunking, sparse signals, or a cross-encoder can mitigate this; none removes the need to test evidence recall.

6. Multi-vector and late-interaction retrieval

6.1 ColBERT MaxSim

Encode query tokens $Q=(q_1,\ldots,q_m)$ and document tokens $D=(d_1,\ldots,d_n)$ :

s(Q,D)=\sum_{i=1}^{m}\max_{j\in[1,n]} q_i^\top d_j.

Each query token finds its strongest document-token match. This preserves fine-grained interaction while document vectors remain precomputable. It costs many vectors per document and MaxSim computation.

ColBERTv2 adds denoised supervision and residual compression. PLAID performs centroid interaction and staged pruning before decompression/exact scoring. XTR trains important document tokens to be directly retrievable and scores using retrieved tokens. CITADEL learns lexical routing keys so only compatible token vectors interact.

6.2 Deployment modes

use multi-vector search as first-stage candidate generation;
rerank BM25/dense candidates with late interaction;
route only difficult queries to it;
retain token vectors for selected high-value corpora;
compress/prune vectors and exact-rescore survivors.

Report vectors and bytes per unit, index build time, candidate-stage recall, MaxSim latency, and final utility. A small vector dimension is misleading if a page stores a thousand vectors.

6.3 Visual late interaction

ColPali/ColQwen-style visual retrievers compare text query tokens with page image-patch vectors. They can recover layout, tables, and figures without text-first parsing. The same MaxSim storage problem is larger, and page-level retrieval needs region localization before precise citation.

7. ANN is part of retrieval quality

Dense and multi-vector papers often report exact or paper-specific indexes, while production uses approximate search. For each index configuration report:

\operatorname{ANNRecall@k}= \frac{|\operatorname{ANN}_k(q)\cap\operatorname{Exact}_k(q)|}{k},

plus qrel recall and downstream answer effect. Tune HNSW efSearch, IVF nprobe, PQ code size, rerank depth, and filter handling on a representative query distribution. The exact nearest neighbor may itself be irrelevant; ANN recall is a systems diagnostic, not factuality.

High-selectivity metadata filters are a separate benchmark. Pre-filtering can fragment the graph/partitions; post-filtering can return fewer than k; iterated search can create unpredictable tail latency. Measure by selectivity and tenant.

8. Query transformation families

8.1 Correction and normalization

Correct obvious spelling and segmentation, but preserve identifiers and provide an original-query channel. Expand known acronyms/aliases from a governed dictionary. Detect dates, versions, units, and quoted phrases into structured constraints.

8.2 Corpus feedback

Rocchio/RM3 and related feedback use actual top documents. They are cheap and grounded in corpus vocabulary, but initial retrieval errors reinforce themselves. Gate on top-result confidence/diversity, and fuse original and feedback runs rather than committing completely to an expanded query.

8.3 Generated query expansion

Query2Doc generates a pseudo-document and concatenates it with the original query, retaining lexical anchors. Multi-query methods generate diverse paraphrases or perspectives and fuse result lists. Generation-Augmented Retrieval produces answer/title/sentence expansions. Control fan-out and measure unique relevant evidence per query, not only total retrieved chunks.

8.4 HyDE

HyDE asks an instruction model to generate a hypothetical answer document and embeds it with an unsupervised retriever. The dense encoder can act as an information bottleneck that maps the fictional document toward real neighbors. It is useful without relevance labels but can anchor search on fabricated entities or the wrong interpretation. Search the original query too and expose the hypothetical text in traces.

8.5 Conversational rewriting

A rewriter resolves pronouns, ellipsis, and prior entities into a standalone query. Alternatives directly encode dialogue context. Rewriting improves retrieval only if it preserves the current turn’s intent; topic changes and corrections are common failure cases. Evaluate by turn position, topic shift, answerability, and entity carryover.

8.6 Decomposition and multi-hop plans

Decompose comparison, multi-entity, or compositional questions into subquestions. Retrieve per subquestion, fuse candidates, and select for coverage. Later queries may depend on entities found in earlier evidence. Incorrect decomposition can make an answer impossible; retain a direct-query run and cap steps.

8.7 Step-back and abstraction

Generate a broader conceptual question to retrieve principles or definitions, then combine with specific evidence. This helps when the original query is too specific for corpus phrasing, but broad results can dominate. Tag evidence by which subgoal it serves.

8.8 Transformation evaluation

For each original query, store transformations and report:

change in relevant-document/claim recall;
unique useful evidence added;
relevant evidence lost due to drift;
number of retrieval calls and duplicate ratio;
latency/token/cost;
final answer and citation change;
regressions on exact identifiers, unanswerable queries, and ambiguity.

9. Hybrid retrieval and result fusion

Sparse and dense retrieval have complementary errors. Fusion begins by canonicalizing document/chunk identity and collapsing cross-view duplicates.

9.1 Reciprocal-rank fusion

\operatorname{RRF}(d)=\sum_r\frac{w_r}{K+\operatorname{rank}_r(d)}.

RRF ignores incomparable score scales and is robust. K, weights, and retrieval depths still matter. Missing a document from a shallow run differs from assigning it a low score, and one retriever can flood the union with near-duplicates.

9.2 Score normalization

Common per-run transforms are min-max, z-score, rank/quantile, softmax, and calibration to relevance probability. Min-max is unstable with outliers; z-score assumes a meaningful distribution; per-query softmax depends on temperature and candidate depth. Normalize over a declared pool and test drift.

9.3 CombSUM and CombMNZ

After normalization,

\operatorname{CombSUM}(d)=\sum_r s_r(d),\qquad \operatorname{CombMNZ}(d)=N_d\sum_r s_r(d),

where $N_d$ is the number of runs retrieving the document. CombMNZ rewards agreement, which helps independent signals but can overreward duplicated or correlated retrievers.

9.4 Linear and learned fusion

Calibrated interpolation uses

s(d\mid q)=\sum_r w_r(q)\,\widetilde s_r(q,d).

Weights may be global or query-dependent. Features can include query length, identifier rate, language, domain, score gaps, entropy, filter selectivity, retriever ranks/scores, authority, freshness, and duplicate cluster. Logistic regression, LambdaMART, or a router can learn fusion on judged data.

Guard against overfitting and retriever-version drift. A learned fusion model trained on one candidate distribution is not automatically valid after an embedding or chunking change.

9.5 Candidate-budget allocation

Equal top-k per retriever is arbitrary. Allocate depth by marginal useful recall, latency, and query class. Exact identifier queries may spend most budget on lexical retrieval; paraphrases on dense; table questions on schema/row retrievers. Preserve a minimum exploration budget so router errors are recoverable.

9.6 Fusion diagnostics

Measure each retriever’s relevant-only contribution, overlap matrix, oracle union recall, fused recall, rank displacement, duplicate rate, and latency. If the union recall improves but fused recall does not, fusion is the problem. If union recall does not improve, adding a correlated retriever only adds cost.

10. Reranking

10.1 Cross-encoder

Concatenate query and candidate and jointly encode them:

s_\phi(q,d)=w^\top h_{\text{CLS}}([q;d]).

Full token interaction improves semantic relevance and relationship/negation handling but costs a model forward pass per candidate. Batch by token length, truncate deliberately, and measure candidate depth against latency.

10.2 Pointwise loss

Binary cross-entropy treats each pair independently:

\mathcal L=-y\log\sigma(s)-(1-y)\log(1-\sigma(s)).

It yields calibratable relevance probabilities but does not directly optimize ordering or set coverage.

10.3 Pairwise loss

For positive $d^+$ and negative $d^-$ :

\mathcal L=\log(1+\exp[-(s(q,d^+)-s(q,d^-))])

or a margin hinge. Pairwise training focuses on relative order; pair sampling defines the learned distinctions.

10.4 Listwise loss

A softmax/listwise likelihood or LambdaRank-style gradient considers the whole candidate list and can weight errors by nDCG change. It better matches ranking metrics but requires list construction and careful handling of incomplete judgments.

10.5 Sequence-to-sequence rerankers

monoT5 prompts a T5 model to emit a relevance label and ranks by its token probability. RankT5 replaces label generation with direct scoring and ranking losses. These exploit generative pretraining but still need a stable scoring prompt/token and can be expensive.

10.6 LLM listwise reranking

RankGPT asks an LLM for a permutation, often using sliding windows. Listwise context exposes relationships among candidates, but results can be position-sensitive, inconsistent, costly, and hard to calibrate. Validate permutation completeness/uniqueness and retain scores or rationales only as diagnostics, not unquestioned truth.

10.7 Late-interaction reranking

ColBERT-like MaxSim can rerank a first-stage union without maintaining a full multi-vector ANN index. This gives fine token interaction at a narrower depth and may be an attractive storage/latency compromise.

10.8 Training-distribution mismatch

A reranker trained on BM25 candidates can fail on dense candidates and vice versa. HYRR trains with hybrid candidates to improve robustness. Include the intended first-stage retrievers, hard negatives, duplicate patterns, and unanswerable cases in training.

10.9 Reranker evaluation

Given candidates $C_k$ , compute:

oracle recall of $C_k$ ;
nDCG/MRR/claim recall before and after rerank;
reranker regret: relevant evidence present before but removed below the selection cutoff;
change by sparse-only, dense-only, both, and identifier slices;
latency by candidate count and token length;
calibration/reliability if scores drive abstention;
behavior under duplicates, conflicts, stale/low-authority evidence, and adversarial text.

11. Evidence selection is not just reranking

An individually ranked top-k can contain five redundant passages and omit a second required hop. Selection optimizes a set.

11.1 Maximal marginal relevance

d^*=\arg\max_{d\in C\setminus Z} \lambda\operatorname{rel}(q,d) -(1-\lambda)\max_{z\in Z}\operatorname{sim}(d,z).

MMR balances relevance and novelty. Its similarity model and $\lambda$ need tuning; novelty is not the same as covering a required subquestion.

11.2 Coverage and set selection

Infer information needs $H=\{h_1,\ldots,h_m\}$ and estimate support $a_{ij}$ from document $d_i$ to need $h_j$ . A budgeted coverage objective is

\max_Z\sum_j w_j\max_{d_i\in Z}a_{ij} -\rho\sum_{d_i,d_l\in Z}\operatorname{redundancy}(d_i,d_l).

SetR explicitly shifts from point ranking to selecting passages that jointly satisfy multi-hop information needs. Set supervision and need extraction add complexity, but the objective matches comparison and synthesis tasks better.

11.3 Source authority and independence

Semantic support is insufficient in high-stakes or contested domains. Score official/primary source, jurisdiction, evidence grade, publisher, retraction, and independence. Ten copied pages are one source family. Source diversity can improve robustness but should not force inclusion of lower-authority evidence.

11.4 Freshness and temporal consistency

Rank by compatibility between query time and evidence valid time, not simply publication recency. Group versions and conflicts. For “latest” questions, prefer current authoritative evidence; for historical questions, new documents can be wrong for the requested date.

11.5 Contradiction-aware selection

Detect mutually incompatible claims and retain representative evidence from each authoritative side rather than letting one arbitrary rank win. The generator can state the conflict or abstain. Contradiction classifiers are fallible, especially with dates, numbers, scope, and negation; preserve raw evidence and require human review where necessary.

11.6 Context budget and compression

Selection should account for tokens after parent expansion and formatting. Knapsack-style utility per token is more appropriate than fixed chunk count. Extractive sentence selection preserves spans; abstractive compression adds a new generated-evidence layer. See the context/generation chapter.

12. Retrieval calibration and sufficiency

Raw cosine, BM25, cross-encoder, and RRF scores are not comparable confidence. Calibrate on labeled examples with Platt/logistic scaling, isotonic regression, or conformal methods, and test reliability by slice. Distribution shifts in corpus, query style, model, and candidate depth invalidate calibration.

Sufficiency asks whether selected evidence can answer the question, which is different from whether top results are relevant. Train or prompt a sufficiency estimator on complete, partial, irrelevant, and conflicting contexts. Combine it with self-confidence cautiously; both can be overconfident. Abstention thresholds should optimize risk/coverage on the product loss.

13. Training data and qrels

Retrieval labels can be:

human-judged relevance/support;
explicit citations or clicked/used sources;
answer-containing heuristics;
BM25-selected positives;
teacher cross-encoder/LLM labels;
generated query-document pairs;
downstream reader attention/likelihood;
implicit behavior such as click, dwell, resolution, or edit.

Each is biased. Answer strings admit spurious passages. Clicks reflect position and presentation. Teacher labels inherit model preference. Generated questions reflect generator language. Incomplete qrels mark valid evidence as negative. Document label provenance and maintain a human-adjudicated validation set.

Pool candidates from diverse retrievers for judging; otherwise qrels favor the system that produced the pool. Include hard negatives that are topically similar, same-entity but wrong relation/date, conflicting, stale, and unauthorized. Mark rather than train on ambiguous cases when possible.

14. End-to-end retrieval experiment design

Factor grid

Vary separately:

unit construction and size;
sparse analyzer/model;
dense model/instruction/dimension;
exact versus ANN and ANN parameters;
metadata/time/ACL filters;
query transformation;
per-retriever candidate depth;
fusion;
reranker and depth;
selector and context budget;
generator held fixed for retrieval experiments.

Full factorial grids are expensive. Begin with one-factor ablations, then test interactions that matter: chunk size × top-k, retriever × reranker, filter selectivity × ANN, and evidence budget × generator.

Required baselines and ceilings

BM25;
exact dense;
sparse+dense union oracle;
simple RRF;
hybrid plus reranker;
oracle reranking of the candidate union;
gold/oracle evidence generation;
no-retrieval generation.

The union oracle says whether fusion can improve. Oracle reranking says whether first-stage recall is the ceiling. Gold-context generation says whether retrieval or the generator is limiting.

Metrics

Recall@k, precision@k, MRR, MAP, nDCG@k;
complete evidence-set recall and claim recall;
unique supporting sources and duplicate ratio;
reranker regret and evidence survival by stage;
exact-versus-ANN recall;
context tokens and useful-evidence density;
downstream correctness, completeness, faithfulness, citations, abstention;
build/update latency, p50/p95/p99 query latency, throughput, index bytes, model calls, tokens, and cost.

Slices

Exact identifier, paraphrase, rare/long-tail, popular, single-hop, multi-hop, comparison, aggregation, temporal, unanswerable, conversational turn, long document, table, visual, language, domain, tenant/filter selectivity, clean/noisy/conflicting/poisoned context.

15. Failure diagnosis decision tree

Gold evidence absent from source corpus: acquisition/parser/corpus issue.
Present in source but no retrievable unit contains it: chunking/extraction issue.
Unit exists but exact sparse/dense both miss: representation/query issue.
Exact dense hits but ANN misses: index approximation/filter issue.
A retriever hits but union identity drops it: dedup/ID issue.
Union contains it but fusion ranks it low: calibration/fusion issue.
Candidate contains it but reranker removes it: reranker distribution or objective issue.
Reranked list contains it but selector omits it: coverage/budget issue.
Packed context contains it but answer ignores it: generator utilization or position issue.
Answer uses it but citation is wrong: attribution/alignment issue.

Log enough state to place every failed query on this tree.

16. Technique-selection matrix

Requirement	Candidate generation	Precision/selection	Important test
Rare identifiers	BM25/keyword/phrase	fielded reranker	punctuation/case/version
Broad paraphrase	dense + BM25	cross-encoder	out-of-domain and long-tail
Low-latency mutable corpus	BM25/learned sparse	small reranker	update lag and WAND latency
Highest text recall	BM25 + dense + learned sparse	hybrid-aware reranker	union oracle and duplicate rate
Fine token matching	ColBERT/late interaction	MaxSim or cross-encoder	index bytes and latency
Multi-hop/comparison	decomposed hybrid	set coverage/MMR	complete evidence-set recall
Conversational	rewrite/context encoder + hybrid	history-aware reranker	topic shift and late turns
Temporal/legal	lexical+dense under temporal/authority filters	conflict/version selection	as-of replay and authority
Visual PDF	ColPali/text hybrid	VLM/region reranker	layout/OCR distortions
Tables	schema/table/row + BM25/dense/SQL	executable selection	table recall and calculation
Large filtered multitenant	partitioned/filter-aware ANN + BM25	ACL recheck	high-selectivity and leakage
Limited labels/domain shift	BM25 + general dense, GPL/teacher adaptation	calibrated cross-encoder	human domain holdout

17. What the executable notebooks model

The expanded notebooks implement:

an inverted index and exhaustive BM25 comparison;
pseudo-relevance feedback with drift inspection;
exact vector search and an IVF-style approximation with measured recall;
InfoNCE and hard-negative examples without a neural dependency;
RRF, score normalization, CombSUM, and query-dependent fusion;
pointwise/pairwise/listwise ranking-loss calculations;
MMR and subquestion set-cover selection;
stage-by-stage evidence survival and reranker regret.

The hashed vectors and heuristic scores remain transparent teaching proxies. They demonstrate algorithms, data contracts, and failure accounting, not neural paper leaderboards. The chapter links each interface to the primary literature needed for a real implementation.