# Retrieval, query transformation, fusion, reranking, and evidence selection Retrieval is a cascade, not a model name. A complete system understands the query, applies policy and filters, generates candidates from one or more representations, approximates nearest neighbors under a resource budget, fuses heterogeneous results, reranks them, and selects a collectively useful evidence set. Each stage has a different objective and failure ceiling. ## 1. Retrieval as a constrained cascade For query \(q\), authorized corpus \(\mathcal C_a\), retrievers \(r_j\), and generator utility \(u\), a general retrieval problem is \[ R_j = \operatorname{TopK}_{d\in\mathcal C_a} s_j(q_j,d), \qquad C=\bigcup_j R_j, \] \[ Z^*=\arg\max_{Z\subseteq C,\,\operatorname{cost}(Z)\le B} \mathbb E[u(y;q,Z)]-\lambda\operatorname{risk}(Z). \] First-stage retrieval maximizes recall cheaply. Reranking estimates relevance or utility more accurately on the candidate set. Selection optimizes the set rather than each item independently. If relevant evidence never enters \(C\), later stages cannot recover it. If \(C\) has high recall but the selector removes a necessary hop, generation still fails. Report the candidate count at every transition: ```text query -> transformations -> per-retriever top-k -> union/dedup -> rerank depth -> selected chunks -> expanded parents -> packed tokens -> cited chunks ``` ## 2. Query understanding before scoring Create a query state rather than immediately embedding the raw string: ```text original text conversation-resolved intent language and requested answer language entities, aliases, identifiers, quoted phrases time/geography/product/version constraints tenant and ACL scope task type and expected evidence shape answerability/complexity estimate allowed sources/tools and budget generated query variants with lineage ``` Do not let an LLM rewrite silently replace the user’s request. Preserve the original query, run exact-identifier retrieval against it, and record every transformation. Validate structured filters against a schema; generated filter values can be wrong even when the rewritten prose sounds reasonable. ### Intent classes with retrieval consequences - **navigational:** find a known page or record; exact title/identifier is central; - **informational fact:** one supporting passage; - **comparative/list:** evidence coverage across several entities; - **multi-hop:** connected evidence and intermediate entities; - **aggregate:** a complete population or executable data query, not top-k similarity; - **procedural:** preserve order and prerequisites; - **temporal:** align evidence validity with requested time; - **conversational:** resolve history without importing stale topics; - **exploratory/global:** diversity and corpus-level themes; - **unanswerable/false premise:** sufficiency and abstention rather than nearest neighbor confidence. ## 3. Classical lexical retrieval ### 3.1 Boolean and phrase retrieval An inverted index supports exact term, phrase, proximity, prefix, range, and field queries. Boolean filters are essential for identifiers, dates, product versions, jurisdictions, languages, tenants, and source classes. Dense similarity should not approximate a predicate such as `effective_date <= t < expiry_date` or `tenant_id = A`. Positional postings allow phrase and distance constraints. Exact fields should use a keyword analyzer that preserves case/punctuation when relevant; full-text fields may use tokenization, stemming, or decompounding. Keep raw and analyzed views separate. ### 3.2 Query likelihood The language-model retrieval view ranks a document by its probability of generating query terms. With Dirichlet smoothing, \[ \log p(q\mid d)= \sum_{t\in q} f(t,q) \log\frac{f(t,d)+\mu p(t\mid\mathcal C)}{|d|+\mu}. \] The corpus model prevents zero probability and \(\mu\) controls how much short documents are smoothed. Query likelihood and BM25 encode related but different assumptions; both remain valuable baselines. ### 3.3 BM25 and fielded BM25 For term \(t\), \[ \operatorname{BM25}(q,d)=\sum_{t\in q} \operatorname{IDF}(t) \frac{(k_1+1)f(t,d)}{f(t,d)+k_1(1-b+b|d|/\operatorname{avgdl})}. \] `k1` controls saturation: repeating a term ten times should not produce ten times the relevance. `b` controls length normalization. Tune by retrieval-unit type; tables, titles, tickets, and long sections have different length distributions. BM25F combines field-specific term frequencies and length normalization. A conceptual field weight is \[ \widetilde f(t,d)=\sum_f w_f \frac{f(t,d_f)}{1-b_f+b_f|d_f|/\operatorname{avgdl}_f}, \] followed by saturation. It is preferable to literally duplicating title text because title/body statistics and boosts remain explicit. ### 3.4 BM25 variants BM25+, BM25L, pivoted length normalization, proximity features, and domain analyzers address specific biases such as over-penalizing long documents or underusing term proximity. Treat variants as tunable hypotheses, not a generic upgrade. Always retain a transparent ordinary BM25 result. ### 3.5 Pseudo-relevance feedback Rocchio moves a query vector toward assumed relevant documents and away from assumed nonrelevant ones: \[ q'=\alpha q+\frac{\beta}{|D_r|}\sum_{d\in D_r}d -\frac{\gamma}{|D_n|}\sum_{d\in D_n}d. \] RM3 estimates a relevance model from top documents and interpolates its terms with the original query. Feedback can bridge vocabulary mismatch using the actual corpus, but the assumption that top results are relevant causes query drift on ambiguous or low-recall initial searches. Limit expansion terms, retain original terms, and evaluate per-query regressions. ### 3.6 Efficient top-k query execution A teaching implementation that loops over every document computes BM25 scores correctly but is not a search engine. Production inverted indexes exploit sorted postings, compressed gaps, score upper bounds, and skipping. **WAND** identifies a pivot where the sum of term upper bounds can exceed the current top-k threshold. Documents before the pivot are skipped or advanced. **Block-Max WAND** stores tighter upper bounds per postings block. **MaxScore** separates essential and nonessential lists under a threshold. Impact-ordered indexes group postings by score contribution. Exact dynamic-pruning algorithms avoid scoring candidates that provably cannot enter top-k; approximate variants trade effectiveness for more skipping. Measure postings decoded, candidates fully scored, p95 latency, and result identity against exhaustive scoring. Learned sparse systems can produce dense posting lists that erase sparse-serving advantages. ## 4. Neural sparse retrieval Neural sparse methods keep a vocabulary-sized vector and dot-product scoring, so nonzero dimensions can be served with an inverted index. ### 4.1 Contextual term weighting: DeepCT DeepCT predicts a contextual impact for each occurrence of a token, then aggregates/quantizes it into an indexable term weight. The same word can carry different importance in different passages. It improves weighting without semantic vocabulary expansion, so unseen synonym mismatch remains. ### 4.2 Document expansion: doc2query/docT5query Train a sequence model \(p(q\mid d)\) and generate queries that a document might answer. Append them to an index-only document view and use ordinary BM25. This moves model cost offline and lets generated vocabulary create lexical matches. Risks are index growth, generic generated queries, factual errors, and evaluation leakage if generation learned benchmark-like questions. ### 4.3 Learned impacts: DeepImpact and uniCOIL DeepImpact combines document expansion with learned scalar impacts per unique token. COIL stores contextual vectors under exact lexical keys; uniCOIL reduces each contextual token to a scalar. These preserve exact-match routing while letting context determine strength. They cannot directly match a query term to a different vocabulary term unless expansion supplied it. ### 4.4 Vocabulary expansion: SPLADE SPLADE maps contextual token logits into sparse vocabulary weights. A common aggregation is \[ w_j(x)=\max_{i\in x}\log(1+\operatorname{ReLU}(z_{ij})), \qquad s(q,d)=w(q)^\top w(d). \] Masked-language-model heads activate terms not literally present, enabling learned expansion. FLOPS-style regularization penalizes average activation: \[ \mathcal L_{\text{FLOPS}}=\sum_j \left(\frac{1}{B}\sum_{i=1}^{B}w_j(x_i)\right)^2. \] The retrieval loss, distillation teacher, hard negatives, query/document regularization, and initialization define distinct SPLADE variants. Compare effectiveness together with nonzeros/query, postings/doc, index bytes, and latency. Sparse in vector notation is not necessarily sparse in execution. ### 4.5 Learned-sparse selection criteria Choose learned sparse when semantic expansion is useful but exact-term explanations, mutable inverted indexes, or lexical infrastructure matter. Keep BM25 for identifiers and audit learned expansions: a high-weight expansion can reveal why a result appeared, but it can also introduce a misleading concept. ## 5. Single-vector dense retrieval ### 5.1 Dual-encoder architecture A query encoder \(E_q\) and document encoder \(E_d\) produce \[ u=E_q(q),\quad v=E_d(d),\quad s(q,d)=u^\top v \] or cosine similarity. Towers may share weights or not. Pooling may use `[CLS]`, mean pooling, weighted pooling, or a learned projection. Document vectors are precomputed; this is why a dual encoder scales and why it compresses all document relevance into one vector. ### 5.2 Training loss With positive \(d_i^+\) and candidate documents \(D_i\), \[ \mathcal L_i=-\log \frac{\exp(s(q_i,d_i^+)/\tau)} {\sum_{d\in D_i}\exp(s(q_i,d)/\tau)}. \] The negative distribution defines the task. Random negatives are too easy; in-batch negatives improve efficiency; BM25 negatives teach semantic disambiguation around lexical matches; ANN-mined negatives expose the current model’s confusions; teacher-denoised negatives reduce false-negative damage. See the training chapter for detailed curricula. ### 5.3 Representation and metric choices - Dot product uses both direction and norm; cosine removes norm. - L2-normalized cosine and dot product have identical ranking. - Euclidean distance on unit vectors is monotonic with cosine. - Maximum inner-product search may need different ANN treatment than metric L2 search. - Temperature affects contrastive gradients but not simple inference ranking. - Low precision, quantization, or Matryoshka truncation changes neighbor order; validate it. ### 5.4 Model families - **DPR:** supervised QA positives, in-batch negatives, and BM25 hard negatives; - **ANCE:** asynchronously mines global ANN negatives; - **RocketQA:** cross-batch negatives, teacher denoising, and data augmentation; - **Condenser/coCondenser/RetroMAE/SimLM:** retrieval-oriented representation pretraining; - **Contriever:** unsupervised contrastive dense retrieval; - **GTR/E5/INSTRUCTOR/BGE/GritLM:** broad weak supervision or instructions for general-purpose embeddings; - **ReasonIR and 2026 utility-aware work:** train toward reasoning/helpfulness, not only topical similarity. Do not select an embedding by a global MTEB average. Match language, domain, unit length, query style, instruction format, dimension, license, latency, and index cost. Evaluate exact identifiers and out-of-domain slices because dense models often fail there. ### 5.5 Single-vector bottleneck A long unit can contain many topics, entities, and relations, but one vector must represent them all. Relevant fine details can be averaged away. Smaller chunks, multi-vector retrieval, contextual/late chunking, sparse signals, or a cross-encoder can mitigate this; none removes the need to test evidence recall. ## 6. Multi-vector and late-interaction retrieval ### 6.1 ColBERT MaxSim Encode query tokens \(Q=(q_1,\ldots,q_m)\) and document tokens \(D=(d_1,\ldots,d_n)\): \[ s(Q,D)=\sum_{i=1}^{m}\max_{j\in[1,n]} q_i^\top d_j. \] Each query token finds its strongest document-token match. This preserves fine-grained interaction while document vectors remain precomputable. It costs many vectors per document and MaxSim computation. ColBERTv2 adds denoised supervision and residual compression. PLAID performs centroid interaction and staged pruning before decompression/exact scoring. XTR trains important document tokens to be directly retrievable and scores using retrieved tokens. CITADEL learns lexical routing keys so only compatible token vectors interact. ### 6.2 Deployment modes - use multi-vector search as first-stage candidate generation; - rerank BM25/dense candidates with late interaction; - route only difficult queries to it; - retain token vectors for selected high-value corpora; - compress/prune vectors and exact-rescore survivors. Report vectors and bytes per unit, index build time, candidate-stage recall, MaxSim latency, and final utility. A small vector dimension is misleading if a page stores a thousand vectors. ### 6.3 Visual late interaction ColPali/ColQwen-style visual retrievers compare text query tokens with page image-patch vectors. They can recover layout, tables, and figures without text-first parsing. The same MaxSim storage problem is larger, and page-level retrieval needs region localization before precise citation. ## 7. ANN is part of retrieval quality Dense and multi-vector papers often report exact or paper-specific indexes, while production uses approximate search. For each index configuration report: \[ \operatorname{ANNRecall@k}= \frac{|\operatorname{ANN}_k(q)\cap\operatorname{Exact}_k(q)|}{k}, \] plus qrel recall and downstream answer effect. Tune HNSW `efSearch`, IVF `nprobe`, PQ code size, rerank depth, and filter handling on a representative query distribution. The exact nearest neighbor may itself be irrelevant; ANN recall is a systems diagnostic, not factuality. High-selectivity metadata filters are a separate benchmark. Pre-filtering can fragment the graph/partitions; post-filtering can return fewer than k; iterated search can create unpredictable tail latency. Measure by selectivity and tenant. ## 8. Query transformation families ### 8.1 Correction and normalization Correct obvious spelling and segmentation, but preserve identifiers and provide an original-query channel. Expand known acronyms/aliases from a governed dictionary. Detect dates, versions, units, and quoted phrases into structured constraints. ### 8.2 Corpus feedback Rocchio/RM3 and related feedback use actual top documents. They are cheap and grounded in corpus vocabulary, but initial retrieval errors reinforce themselves. Gate on top-result confidence/diversity, and fuse original and feedback runs rather than committing completely to an expanded query. ### 8.3 Generated query expansion [Query2Doc](https://aclanthology.org/2023.emnlp-main.585/) generates a pseudo-document and concatenates it with the original query, retaining lexical anchors. Multi-query methods generate diverse paraphrases or perspectives and fuse result lists. Generation-Augmented Retrieval produces answer/title/sentence expansions. Control fan-out and measure unique relevant evidence per query, not only total retrieved chunks. ### 8.4 HyDE [HyDE](https://aclanthology.org/2023.acl-long.99/) asks an instruction model to generate a hypothetical answer document and embeds it with an unsupervised retriever. The dense encoder can act as an information bottleneck that maps the fictional document toward real neighbors. It is useful without relevance labels but can anchor search on fabricated entities or the wrong interpretation. Search the original query too and expose the hypothetical text in traces. ### 8.5 Conversational rewriting A rewriter resolves pronouns, ellipsis, and prior entities into a standalone query. Alternatives directly encode dialogue context. Rewriting improves retrieval only if it preserves the current turn’s intent; topic changes and corrections are common failure cases. Evaluate by turn position, topic shift, answerability, and entity carryover. ### 8.6 Decomposition and multi-hop plans Decompose comparison, multi-entity, or compositional questions into subquestions. Retrieve per subquestion, fuse candidates, and select for coverage. Later queries may depend on entities found in earlier evidence. Incorrect decomposition can make an answer impossible; retain a direct-query run and cap steps. ### 8.7 Step-back and abstraction Generate a broader conceptual question to retrieve principles or definitions, then combine with specific evidence. This helps when the original query is too specific for corpus phrasing, but broad results can dominate. Tag evidence by which subgoal it serves. ### 8.8 Transformation evaluation For each original query, store transformations and report: - change in relevant-document/claim recall; - unique useful evidence added; - relevant evidence lost due to drift; - number of retrieval calls and duplicate ratio; - latency/token/cost; - final answer and citation change; - regressions on exact identifiers, unanswerable queries, and ambiguity. ## 9. Hybrid retrieval and result fusion Sparse and dense retrieval have complementary errors. Fusion begins by canonicalizing document/chunk identity and collapsing cross-view duplicates. ### 9.1 Reciprocal-rank fusion \[ \operatorname{RRF}(d)=\sum_r\frac{w_r}{K+\operatorname{rank}_r(d)}. \] RRF ignores incomparable score scales and is robust. `K`, weights, and retrieval depths still matter. Missing a document from a shallow run differs from assigning it a low score, and one retriever can flood the union with near-duplicates. ### 9.2 Score normalization Common per-run transforms are min-max, z-score, rank/quantile, softmax, and calibration to relevance probability. Min-max is unstable with outliers; z-score assumes a meaningful distribution; per-query softmax depends on temperature and candidate depth. Normalize over a declared pool and test drift. ### 9.3 CombSUM and CombMNZ After normalization, \[ \operatorname{CombSUM}(d)=\sum_r s_r(d),\qquad \operatorname{CombMNZ}(d)=N_d\sum_r s_r(d), \] where \(N_d\) is the number of runs retrieving the document. CombMNZ rewards agreement, which helps independent signals but can overreward duplicated or correlated retrievers. ### 9.4 Linear and learned fusion Calibrated interpolation uses \[ s(d\mid q)=\sum_r w_r(q)\,\widetilde s_r(q,d). \] Weights may be global or query-dependent. Features can include query length, identifier rate, language, domain, score gaps, entropy, filter selectivity, retriever ranks/scores, authority, freshness, and duplicate cluster. Logistic regression, LambdaMART, or a router can learn fusion on judged data. Guard against overfitting and retriever-version drift. A learned fusion model trained on one candidate distribution is not automatically valid after an embedding or chunking change. ### 9.5 Candidate-budget allocation Equal top-k per retriever is arbitrary. Allocate depth by marginal useful recall, latency, and query class. Exact identifier queries may spend most budget on lexical retrieval; paraphrases on dense; table questions on schema/row retrievers. Preserve a minimum exploration budget so router errors are recoverable. ### 9.6 Fusion diagnostics Measure each retriever’s relevant-only contribution, overlap matrix, oracle union recall, fused recall, rank displacement, duplicate rate, and latency. If the union recall improves but fused recall does not, fusion is the problem. If union recall does not improve, adding a correlated retriever only adds cost. ## 10. Reranking ### 10.1 Cross-encoder Concatenate query and candidate and jointly encode them: \[ s_\phi(q,d)=w^\top h_{\text{CLS}}([q;d]). \] Full token interaction improves semantic relevance and relationship/negation handling but costs a model forward pass per candidate. Batch by token length, truncate deliberately, and measure candidate depth against latency. ### 10.2 Pointwise loss Binary cross-entropy treats each pair independently: \[ \mathcal L=-y\log\sigma(s)-(1-y)\log(1-\sigma(s)). \] It yields calibratable relevance probabilities but does not directly optimize ordering or set coverage. ### 10.3 Pairwise loss For positive \(d^+\) and negative \(d^-\): \[ \mathcal L=\log(1+\exp[-(s(q,d^+)-s(q,d^-))]) \] or a margin hinge. Pairwise training focuses on relative order; pair sampling defines the learned distinctions. ### 10.4 Listwise loss A softmax/listwise likelihood or LambdaRank-style gradient considers the whole candidate list and can weight errors by nDCG change. It better matches ranking metrics but requires list construction and careful handling of incomplete judgments. ### 10.5 Sequence-to-sequence rerankers [monoT5](https://aclanthology.org/2020.findings-emnlp.63/) prompts a T5 model to emit a relevance label and ranks by its token probability. RankT5 replaces label generation with direct scoring and ranking losses. These exploit generative pretraining but still need a stable scoring prompt/token and can be expensive. ### 10.6 LLM listwise reranking [RankGPT](https://aclanthology.org/2023.emnlp-main.923/) asks an LLM for a permutation, often using sliding windows. Listwise context exposes relationships among candidates, but results can be position-sensitive, inconsistent, costly, and hard to calibrate. Validate permutation completeness/uniqueness and retain scores or rationales only as diagnostics, not unquestioned truth. ### 10.7 Late-interaction reranking ColBERT-like MaxSim can rerank a first-stage union without maintaining a full multi-vector ANN index. This gives fine token interaction at a narrower depth and may be an attractive storage/latency compromise. ### 10.8 Training-distribution mismatch A reranker trained on BM25 candidates can fail on dense candidates and vice versa. [HYRR](https://aclanthology.org/2024.lrec-main.748/) trains with hybrid candidates to improve robustness. Include the intended first-stage retrievers, hard negatives, duplicate patterns, and unanswerable cases in training. ### 10.9 Reranker evaluation Given candidates \(C_k\), compute: - oracle recall of \(C_k\); - nDCG/MRR/claim recall before and after rerank; - **reranker regret:** relevant evidence present before but removed below the selection cutoff; - change by sparse-only, dense-only, both, and identifier slices; - latency by candidate count and token length; - calibration/reliability if scores drive abstention; - behavior under duplicates, conflicts, stale/low-authority evidence, and adversarial text. ## 11. Evidence selection is not just reranking An individually ranked top-k can contain five redundant passages and omit a second required hop. Selection optimizes a set. ### 11.1 Maximal marginal relevance \[ d^*=\arg\max_{d\in C\setminus Z} \lambda\operatorname{rel}(q,d) -(1-\lambda)\max_{z\in Z}\operatorname{sim}(d,z). \] MMR balances relevance and novelty. Its similarity model and \(\lambda\) need tuning; novelty is not the same as covering a required subquestion. ### 11.2 Coverage and set selection Infer information needs \(H=\{h_1,\ldots,h_m\}\) and estimate support \(a_{ij}\) from document \(d_i\) to need \(h_j\). A budgeted coverage objective is \[ \max_Z\sum_j w_j\max_{d_i\in Z}a_{ij} -\rho\sum_{d_i,d_l\in Z}\operatorname{redundancy}(d_i,d_l). \] [SetR](https://aclanthology.org/2025.acl-long.861/) explicitly shifts from point ranking to selecting passages that jointly satisfy multi-hop information needs. Set supervision and need extraction add complexity, but the objective matches comparison and synthesis tasks better. ### 11.3 Source authority and independence Semantic support is insufficient in high-stakes or contested domains. Score official/primary source, jurisdiction, evidence grade, publisher, retraction, and independence. Ten copied pages are one source family. Source diversity can improve robustness but should not force inclusion of lower-authority evidence. ### 11.4 Freshness and temporal consistency Rank by compatibility between query time and evidence valid time, not simply publication recency. Group versions and conflicts. For “latest” questions, prefer current authoritative evidence; for historical questions, new documents can be wrong for the requested date. ### 11.5 Contradiction-aware selection Detect mutually incompatible claims and retain representative evidence from each authoritative side rather than letting one arbitrary rank win. The generator can state the conflict or abstain. Contradiction classifiers are fallible, especially with dates, numbers, scope, and negation; preserve raw evidence and require human review where necessary. ### 11.6 Context budget and compression Selection should account for tokens after parent expansion and formatting. Knapsack-style utility per token is more appropriate than fixed chunk count. Extractive sentence selection preserves spans; abstractive compression adds a new generated-evidence layer. See the context/generation chapter. ## 12. Retrieval calibration and sufficiency Raw cosine, BM25, cross-encoder, and RRF scores are not comparable confidence. Calibrate on labeled examples with Platt/logistic scaling, isotonic regression, or conformal methods, and test reliability by slice. Distribution shifts in corpus, query style, model, and candidate depth invalidate calibration. Sufficiency asks whether selected evidence can answer the question, which is different from whether top results are relevant. Train or prompt a sufficiency estimator on complete, partial, irrelevant, and conflicting contexts. Combine it with self-confidence cautiously; both can be overconfident. Abstention thresholds should optimize risk/coverage on the product loss. ## 13. Training data and qrels Retrieval labels can be: - human-judged relevance/support; - explicit citations or clicked/used sources; - answer-containing heuristics; - BM25-selected positives; - teacher cross-encoder/LLM labels; - generated query-document pairs; - downstream reader attention/likelihood; - implicit behavior such as click, dwell, resolution, or edit. Each is biased. Answer strings admit spurious passages. Clicks reflect position and presentation. Teacher labels inherit model preference. Generated questions reflect generator language. Incomplete qrels mark valid evidence as negative. Document label provenance and maintain a human-adjudicated validation set. Pool candidates from diverse retrievers for judging; otherwise qrels favor the system that produced the pool. Include hard negatives that are topically similar, same-entity but wrong relation/date, conflicting, stale, and unauthorized. Mark rather than train on ambiguous cases when possible. ## 14. End-to-end retrieval experiment design ### Factor grid Vary separately: - unit construction and size; - sparse analyzer/model; - dense model/instruction/dimension; - exact versus ANN and ANN parameters; - metadata/time/ACL filters; - query transformation; - per-retriever candidate depth; - fusion; - reranker and depth; - selector and context budget; - generator held fixed for retrieval experiments. Full factorial grids are expensive. Begin with one-factor ablations, then test interactions that matter: chunk size × top-k, retriever × reranker, filter selectivity × ANN, and evidence budget × generator. ### Required baselines and ceilings 1. BM25; 2. exact dense; 3. sparse+dense union oracle; 4. simple RRF; 5. hybrid plus reranker; 6. oracle reranking of the candidate union; 7. gold/oracle evidence generation; 8. no-retrieval generation. The union oracle says whether fusion can improve. Oracle reranking says whether first-stage recall is the ceiling. Gold-context generation says whether retrieval or the generator is limiting. ### Metrics - Recall@k, precision@k, MRR, MAP, nDCG@k; - complete evidence-set recall and claim recall; - unique supporting sources and duplicate ratio; - reranker regret and evidence survival by stage; - exact-versus-ANN recall; - context tokens and useful-evidence density; - downstream correctness, completeness, faithfulness, citations, abstention; - build/update latency, p50/p95/p99 query latency, throughput, index bytes, model calls, tokens, and cost. ### Slices Exact identifier, paraphrase, rare/long-tail, popular, single-hop, multi-hop, comparison, aggregation, temporal, unanswerable, conversational turn, long document, table, visual, language, domain, tenant/filter selectivity, clean/noisy/conflicting/poisoned context. ## 15. Failure diagnosis decision tree 1. **Gold evidence absent from source corpus:** acquisition/parser/corpus issue. 2. **Present in source but no retrievable unit contains it:** chunking/extraction issue. 3. **Unit exists but exact sparse/dense both miss:** representation/query issue. 4. **Exact dense hits but ANN misses:** index approximation/filter issue. 5. **A retriever hits but union identity drops it:** dedup/ID issue. 6. **Union contains it but fusion ranks it low:** calibration/fusion issue. 7. **Candidate contains it but reranker removes it:** reranker distribution or objective issue. 8. **Reranked list contains it but selector omits it:** coverage/budget issue. 9. **Packed context contains it but answer ignores it:** generator utilization or position issue. 10. **Answer uses it but citation is wrong:** attribution/alignment issue. Log enough state to place every failed query on this tree. ## 16. Technique-selection matrix | Requirement | Candidate generation | Precision/selection | Important test | |---|---|---|---| | Rare identifiers | BM25/keyword/phrase | fielded reranker | punctuation/case/version | | Broad paraphrase | dense + BM25 | cross-encoder | out-of-domain and long-tail | | Low-latency mutable corpus | BM25/learned sparse | small reranker | update lag and WAND latency | | Highest text recall | BM25 + dense + learned sparse | hybrid-aware reranker | union oracle and duplicate rate | | Fine token matching | ColBERT/late interaction | MaxSim or cross-encoder | index bytes and latency | | Multi-hop/comparison | decomposed hybrid | set coverage/MMR | complete evidence-set recall | | Conversational | rewrite/context encoder + hybrid | history-aware reranker | topic shift and late turns | | Temporal/legal | lexical+dense under temporal/authority filters | conflict/version selection | as-of replay and authority | | Visual PDF | ColPali/text hybrid | VLM/region reranker | layout/OCR distortions | | Tables | schema/table/row + BM25/dense/SQL | executable selection | table recall and calculation | | Large filtered multitenant | partitioned/filter-aware ANN + BM25 | ACL recheck | high-selectivity and leakage | | Limited labels/domain shift | BM25 + general dense, GPL/teacher adaptation | calibrated cross-encoder | human domain holdout | ## 17. What the executable notebooks model The expanded notebooks implement: - an inverted index and exhaustive BM25 comparison; - pseudo-relevance feedback with drift inspection; - exact vector search and an IVF-style approximation with measured recall; - InfoNCE and hard-negative examples without a neural dependency; - RRF, score normalization, CombSUM, and query-dependent fusion; - pointwise/pairwise/listwise ranking-loss calculations; - MMR and subquestion set-cover selection; - stage-by-stage evidence survival and reranker regret. The hashed vectors and heuristic scores remain transparent teaching proxies. They demonstrate algorithms, data contracts, and failure accounting, not neural paper leaderboards. The chapter links each interface to the primary literature needed for a real implementation.