A chronological technical history of retrieval-augmented generation

Coverage: conceptual foundations through 2023, with 2024–2026 continued in the frontier review. Dates below use first public release; formal venues are shown separately.

Chronology at a glance

First public	Formal venue	Milestone	Architectural shift
1972–1976	journals	TF-IDF, vector-space retrieval, relevance weighting	corpus statistics and sparse lexical matching
1994/1995	TREC-3	Okapi BM25	probabilistic term weighting with saturation and length normalization
2014-10-15	ICLR 2015	Memory Networks	raw facts as addressable neural memory
2015-03-31	NeurIPS 2015	End-To-End Memory Networks	differentiable soft multi-hop reads from answer supervision
2017-03-31	ACL 2017	DrQA	Wikipedia-scale retrieve-then-read
2018	ICLR/EMNLP workshops	Wizard of Wikipedia; Retrieve and Refine	retrieval-conditioned dialogue generation
2019-06-01	ACL 2019	ORQA	latent dense retrieval with answer-only supervision
2019-11-01	ICLR 2020	kNN-LM	token-level non-parametric language-model memory
2020-02-10	ICML 2020	REALM	retrieval trained during language-model pretraining
2020-04-10	EMNLP 2020	DPR	simple supervised bi-encoder retrieval at Wikipedia scale
2020-05-22	NeurIPS 2020	RAG	latent-document retrieval plus pretrained seq2seq generation
2020-06-26	NeurIPS 2020	MARGE	retrieve-related-documents pretraining from scratch
2020-07-02	EACL 2021	FiD	independent passage encoders, joint decoder fusion
2020-12-08	ICLR 2021	FiD-KD	distill reader attention into the retriever
2021	TACL 2021	SPALM	learn a context-dependent parametric/memory gate
2021-02/03	NAACL 2021	KILT	shared snapshot and provenance-gated evaluation
2021-06-09	NeurIPS 2021	EMDR²	joint latent multi-document retriever-reader training
2021-09-20	—	SPLADEv2	learned sparse expansion in an inverted index
2021-12-08	ICML 2022	RETRO	chunk retrieval from a trillion-token datastore during LM training
2021-12-16	TMLR 2022	Contriever	unsupervised contrastive dense retrieval
2021-12-03	NAACL 2022	ColBERTv2	compressed token-level late interaction
2022-08-05	JMLR 2023	Atlas	few-shot retrieval-augmented pretraining and reader-to-retriever learning
2022-10-06	ICLR 2023	ReAct	reasoning interleaved with search/tool actions
2022-10-06	EMNLP 2022	MuRAG	end-to-end text-and-image retrieval augmentation
2022-11-22	— preprint	RA-CM3	retrieved multimodal documents for generative modeling
2022-12-20	ACL 2023	HyDE	generated hypothetical documents as zero-shot queries
2023-03-14	EMNLP 2023	Query2Doc	LLM pseudo-documents expand sparse and dense queries
2023-05-11	EMNLP 2023	FLARE	retrieve while generating when future tokens are uncertain
2023-05-23	EMNLP 2023	Rewrite–Retrieve–Read	train a query rewriter from downstream answer reward
2023-05-24	— preprint	ITER-RETGEN	alternate complete generations and retrieval
2023-07-06	TACL 2024	Lost in the Middle	demonstrate positional failure in long contexts
2023-10-17	ICLR 2024	Self-RAG	retrieval and evidence critique as generated reflection tokens

The table is not one lineage. Four branches converge around 2020:

1. sparse retrieve/read: BM25 → DrQA;
2. differentiable memory and latent evidence: Memory Networks → MemN2N → ORQA → REALM;
3. dense retrieval plus sequence generation: DPR → RAG / FiD → FiD-KD / EMDR² → Atlas;
4. non-parametric language modeling: kNN-LM → SPALM → RETRO.

The 2022–2023 systems add a fifth branch: inference-time retrieval control, from query generation and rewriting to uncertainty triggers, iteration, tools, and model-generated evidence judgments.

---

1. Before “RAG”: probabilistic information retrieval

1972–1976: term specificity, vector space, and relevance odds

Karen Spärck Jones's 1972 term-specificity paper formalized the intuition behind inverse document frequency: a term occurring in few documents is more discriminative than one occurring everywhere. Salton, Wong, and Yang's 1975 vector-space model represented queries and documents as weighted term vectors and ranked by vector similarity. Robertson and Spärck Jones's 1976 relevance weighting derived term weights from probabilistic relevance odds.

With \(N\) documents, \(n_t\) containing term \(t\), \(R\) judged relevant, and \(r_t\) relevant documents containing \(t\), the Robertson–Spärck Jones weight is

Without relevance judgments, it reduces to an IDF-like prior. The essential engineering invention was the inverted index: map a term to the documents and positions containing it, rather than scan every document.

1994–1995: Okapi BM25

Robertson, Walker, Jones, Hancock-Beaulieu, and Gatford described Okapi's TREC-3 experiments; the mature BM25 account was later consolidated by Robertson and Zaragoza. A common score is

• \(k_1\) controls term-frequency saturation: the tenth occurrence contributes less than the first.
• \(b\) controls document-length normalization.
• implementations may add query-term saturation, field weights, proximity, or a positive-IDF variant.

BM25's limitations—exact-term dependence, vocabulary mismatch, and no task objective—motivated dense retrieval. Its advantages never disappeared: rare identifiers survive, indexing and incremental updates are cheap, scores can be explained, and postings filters compose naturally with access control and metadata. DrQA used hashed TF-IDF; DPR used a BM25 hard negative; DPR itself lost to BM25 on SQuAD; modern high-recall stacks often fuse sparse and dense results.

Historical caveat. The original TREC-3 paper combined weighting, passage retrieval, expansion, and routing changes. It does not isolate a portable “BM25 gain,” so later leaderboard scores should not be retrospectively attributed to that experiment.

Primary sources: Spärck Jones 1972, vector-space model 1975, relevance weighting 1976, Okapi at TREC-3, and BM25 and Beyond.

---

2. Explicit neural memory and retrieve-then-read

2014: Memory Networks

Jason Weston, Sumit Chopra, and Antoine Bordes defined a system with input, memory-update, output, and response modules \((I,G,O,R)\). Statements occupy external memory slots. One or two supporting memories are selected with hard maximum-score reads:

Bilinear embedding scores learn the match. Separate margin-ranking objectives train each evidence hop and the final one-word response. The paper proposes an RNN response generator, but its main evaluated system ranks answer words; it is not a modern free-form RAG model.

The large experiment stored roughly 14 million ReVerb facts from ClueWeb09 plus QA and WikiAnswers data. Embedding-only F1 was 0.72; exact bag-of-words features raised it to 0.82. Cluster hashing reduced an average search from 14 million to 177,000 candidates at 0.80 F1; word hashing reduced it to 13,000 at 0.68. A time-aware two-hop model nearly solved synthetic reasoning tasks that RNN and LSTM baselines did not.

The system required supporting-fact labels; hard retrieval was non-differentiable; the best multi-hop evidence was synthetic. Its durable idea was raw, addressable, repeatedly readable external memory—a direct conceptual ancestor of RAG. Original paper.

2015: End-To-End Memory Networks

Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus replaced hard reads with soft attention. For memory item \(x_i\) and query \(q\),

Answer cross-entropy backpropagates through every read, eliminating explicit supporting-fact labels. Position and temporal encodings preserve word order and recency; weight tying controls parameters; multiple hops permit composition.

On bAbI, the narrative reports best mean error of 12.6% with 1,000 examples and 4.2% with 10,000, compared with the strongly supervised Memory Network's 6.7% and 3.2%. The main-table 12.4%/7.5% values describe particular joint variants, not the overall best. Language-model experiments were competitive with then current LSTMs, but memory covered only about 50 sentences or 100–200 words and softmax still scanned every slot. Ten random restarts and strong initialization sensitivity further limit the result. The influence is the differentiable, weakly supervised multi-hop read later paired with large ANN indexes. Official paper.

2017: DrQA operationalizes Wikipedia as memory

Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes built a two-stage open QA system over the 2016-12-21 English Wikipedia dump:

• 5,075,182 articles;
• hashed unigram/bigram TF-IDF using MurmurHash into \(2^{24}\) bins;
• an inverted index returning five articles;
• a three-layer bidirectional LSTM reader predicting answer-span start and end.

Retriever and reader were trained independently. For datasets without evidence, distant supervision kept retrieved paragraphs containing an answer string.

Top-five answer-string recall was 77.8% on SQuAD, 86.0% on CuratedTREC, 74.4% on WebQuestions, and 70.3% on WikiMovies. Full-Wikipedia top-one exact match was 29.8, 25.4, 20.7, and 36.5, respectively. On SQuAD, reader-only development EM was 69.5 but the full system reached 27.1, cleanly exposing the retrieval ceiling.

DrQA was lexical, extractive, paragraph-local, and pipelined. It nevertheless made large-scale retrieve-then-read standard and explicitly identified multi-passage aggregation and joint retriever-reader learning as future work. ACL paper.

2018: retrieval enters neural generation

Retrieve and Refine retrieved a training utterance with a Key-Value Memory Network, then conditioned an attentive LSTM on the dialogue and retrieved reply. On ConvAI2, the strongest variant's human engagingness was 3.80/5 versus 3.66 for retrieval and 2.70 for seq2seq. Yet standard retrieval barely moved perplexity, an early warning that likelihood may not reflect retrieval-conditioned generation quality. It retrieved response candidates rather than factual evidence and did not provide provenance. Paper.

Wizard of Wikipedia paired knowledge retrieval with knowledge-grounded conversation and human evaluation. These dialogue branches show why “RAG invented retrieve-then-generate in 2020” is historically wrong. What the 2020 RAG paper supplied was a general pretrained seq2seq latent-document formulation and a name that became standard. Wizard paper.

---

3. Latent dense retrieval and non-parametric language models

2019: ORQA learns dense retrieval from answer strings

Kenton Lee, Ming-Wei Chang, and Kristina Toutanova used separate BERT-base query and block encoders projected to 128 dimensions:

A BERT cross-encoder scored answer spans. Training marginalized probability over retrieved spans exactly matching any answer string. The Inverse Cloze Task provided a cold start: treat a sentence as a pseudo-query and its surrounding block as positive evidence.

The corpus was the 2018-12-20 English Wikipedia split into just over 13 million blocks of at most 288 wordpieces. A locality-sensitive-hashing MIPS index returned five blocks. Document embeddings stayed fixed during QA fine-tuning.

Test EM, BM25+BERT → ORQA:

• Natural Questions: 26.5 → 33.3;
• WebQuestions: 17.7 → 36.4;
• CuratedTREC: 21.3 → 30.1;
• TriviaQA: 47.1 → 45.0;
• SQuAD: 33.2 → 20.2.

Dense semantic retrieval helped genuine information-seeking questions but hurt datasets created while annotators knew the evidence. The 128-dimensional single-vector bottleneck, fixed document side, top-five truncation, and spurious answer-string matches constrained it. ORQA nevertheless established the dense MIPS + latent evidence blueprint. ACL paper.

2019: kNN-LM retrieves token contexts at inference

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis stored each training-token context representation as key and the following token as value:

At inference, nearest contexts define a token distribution:

No additional model training was required. FAISS searched quantized 1,024-dimensional keys, usually with \(k=1024\). On WikiText-103, base test perplexity 18.65 became 16.12 with kNN-LM; 15.79 required adding a separate continuous cache. Using a 3B-token datastore with a model trained on only 100M tokens produced perplexity 13.73, better than a model trained on all 3B at 15.17. Swapping a Books datastore into a Wikipedia-trained model reduced Books perplexity from 34.84 to 20.47.

An entry and lookup per token imposed large storage and latency; retrieval and the interpolation weight were not jointly learned; evidence was not document-level or provenance-bearing. Its influence is hot-swappable non-parametric memory for rare facts, leading to adaptive gating in SPALM and chunk retrieval in RETRO. ICLR paper.

2020: REALM makes retrieval part of pretraining

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang treated the document \(z\) as latent during masked-language-model pretraining:

A BERT bi-encoder retrieved; another BERT cross-encoded input and evidence. Documents received positive learning signal when they increased target likelihood. Salient entity/date masking, a null document, exclusion of the source document, ICT initialization, and periodic index refresh prevented easy shortcuts.

REALM used the same 13M-block 2018 Wikipedia corpus. It marginalized eight candidates in pretraining and five for QA. Cached embeddings and MIPS were rebuilt asynchronously about every 500 steps; downstream QA froze the document encoder/index while the query side changed.

Test EM on Natural Questions/WebQuestions/CuratedTREC was 39.2/40.2/46.8 for Wikipedia pretraining and 40.4/40.7/42.9 for CC-News pretraining, versus ORQA's 33.3/36.4/30.1. A 30-times staler index collapsed NQ development EM from 38.2 to 28.7. Random token masking produced 32.3 versus 38.2 for salient spans.

The engineering burden—64 TPUs in reported pretraining, index rebuilding, top-\(k\) approximation, frozen documents at QA time—and extractive output were substantial. REALM is the closest direct conceptual ancestor of RAG: retrieval participates in pretraining, fine-tuning, and inference, and learns from output likelihood. ICML paper.

2020: DPR makes dense retrieval simple and modular

Vladimir Karpukhin and colleagues trained independent BERT-base encoders by contrastive negative log likelihood:

The best recipe combined other questions' positives as in-batch negatives with one high-ranked BM25 passage lacking the answer. The 2018 Wikipedia snapshot was split into exactly 21,015,324 non-overlapping 100-word passages; 768-dimensional vectors were searched with FAISS/HNSW. A separate cross-attention reader remained extractive.

Single-dataset DPR versus BM25 top-20 answer-containing recall:

Dataset	DPR	BM25
Natural Questions	78.4	59.1
TriviaQA	79.4	66.9
WebQuestions	73.2	55.0
CuratedTREC	79.8	70.9
SQuAD	63.2	68.8

End-to-end EM was 41.5, 56.8, 34.6, 25.9, and 29.8 respectively. The paper's 995-query/s DPR versus 23.7-query/s-per-Lucene-thread comparison was specific to its 512GB CPU setup; passage encoding and HNSW construction each took roughly 8.5–8.8 hours while Lucene indexing took about 30 minutes.

Supervised/weak positives, answer-string false positives, full corpus re-encoding after encoder changes, and rare-term errors remain. DPR's minimal bi-encoder + in-batch negatives + lexical hard negative + FAISS recipe became the default retriever beneath RAG and FiD. EMNLP paper.

---

4. The 2020 generative convergence

RAG: latent documents plus pretrained seq2seq generation

Patrick Lewis and colleagues coupled DPR with BART-large. The retriever gives

RAG-Sequence uses one latent passage for the entire output:

while RAG-Token marginalizes a possibly different passage at every output token:

Negative marginal log likelihood fine-tuned BART and the question encoder. The document encoder and 21M-passage index remained frozen; the retriever inherited DPR supervision from Natural Questions and TriviaQA, so RAG was not retrieval-label-free.

RAG-Sequence test EM was 44.5 NQ, 56.8 standard TriviaQA, 45.2 WebQuestions, and 52.2 CuratedTREC, compared with cited DPR 41.5, 57.9, 41.1, and 50.6. It improved MS MARCO Rouge-L from BART's 38.2 to 40.8 and FEVER three-way accuracy from 64.0% to 72.5%, though specialized supervised FEVER pipelines remained stronger. Human raters on Jeopardy generations preferred RAG factuality in 42.7% of pairs versus BART in 7.1%.

On NQ, RAG still answered 11.8% correctly when no retrieved passage contained the answer: parametric memory can override or supplement retrieved evidence. Matched 2016/2018 indexes answered corresponding world-leader probes at 70%/68%, while mismatched indexes fell to 12%/4%, demonstrating hot-swappable knowledge but also temporal dependence.

Limitations were a frozen document side, discrete truncated retrieval, small top-\(k\), per-document decoding cost, Wikipedia-only memory, no abstention, and no guarantee that outputs were entailed by evidence. Its historical contribution was the general pretrained seq2seq latent-document formulation and the term “retrieval-augmented generation.” NeurIPS paper.

MARGE: pretraining by retrieving related documents

Mike Lewis and colleagues trained a multilingual autoencoder to retrieve related evidence documents and reconstruct a target. A shared encoder scored cosine relevance; the scores biased decoder cross-attention. Reconstruction likelihood jointly learned retrieval and generation from random initialization.

Training used 512-token multilingual CC-News/Wikipedia documents in 26 languages and metadata-defined shards such as same-date news or aligned Wikipedia. Reported zero-shot document translation reached BLEU 35.8 and unsupervised BUCC retrieval averaged 75.9. Candidate restriction to related metadata shards means it was not arbitrary global retrieval; its importance is retrieval-conditioned pretraining, anticipating Atlas. NeurIPS paper.

FiD: fuse many independently encoded passages in the decoder

Gautier Izacard and Edouard Grave's Fusion-in-Decoder encoded every [question; title; passage] independently with T5, concatenated the encoder states, and let a single decoder attend jointly across all evidence. Encoder self-attention scales linearly with passage count rather than quadratically over one raw concatenation.

T5-base/large used 220M/770M parameters and normally 100 passages truncated to 250 wordpieces. Test scores:

Model	NQ EM	TriviaQA open EM	SQuAD EM/F1
FiD-base	48.2	65.0	53.4 / 60.6
FiD-large	51.4	67.6	56.7 / 63.2

Increasing 10 to 100 passages improved NQ development EM by 3.5 and TriviaQA by about six points. Training with five passages and briefly fine-tuning on 100 nearly matched full 100-passage training while reducing NQ compute from 425 to 147 GPU-hours; full training used 64 V100s.

Retrieval was still separate and fixed, the decoder saw a huge representation, and training was expensive and QA-specific. FiD became the canonical multi-context reader underlying FiD-KD, EMDR², and Atlas. EACL paper.

---

5. Learning the retriever from the reader, scaling memory, and hybrid representation

2020/2021: FiD-KD

FiD-KD aggregated FiD decoder cross-attention over heads, layers, and passage tokens into teacher relevance scores. A BERT dual encoder minimized KL divergence to that distribution; reader and retriever training could then iterate. Selecting ten passages by reader attention from a 100-passage set kept 46.8 NQ EM versus 42.9 for DPR's top ten; all 100 scored 48.2.

The original ICLR paper reported T5-large NQ/TriviaQA test EM 54.4/72.5 and retrieval recall@20/100 of 84.3/89.3 on NQ. Later comparison tables sometimes quote 54.7/73.3, probably from another checkpoint; those are not the original table. Attention is a heuristic rather than causal attribution and training is multi-stage, but reader-to-retriever distillation became central to Atlas. ICLR paper.

2021: KILT demands shared snapshots and provenance

KILT mapped 11 datasets across fact checking, entity linking, slot filling, QA, and dialogue to one 2019-08-01 Wikipedia snapshot of 5.9M articles and roughly 3.2M instances. A KILT score gives downstream credit only when the output and the provenance page are both correct.

This exposed the cost of corpus mismatch but remained Wikipedia-only, page-level rather than claim-level, and all-or-nothing. Remapping discarded roughly 18% of development/test examples on average outside entity linking; provenance agreement was low for NQ and ELI5. Its influence was task-general retrieval, fixed corpus snapshots, and provenance-aware evaluation. NAACL paper.

2021: SPALM learns when to use memory

SPALM combined a Transformer's current context, a short-term hidden-state cache, and long-term nearest-neighbor token memory. A context-dependent gate replaced kNN-LM's globally tuned interpolation. This was still token-level language modeling with a large datastore, not document QA, but it introduced adaptive parametric/non-parametric fusion. TACL paper.

2021: EMDR² jointly trains a multi-document reader and retriever

EMDR² combined a DPR-like dual encoder with T5-base FiD and treated the top document set as latent. Exact set marginalization is combinatorial, so an EM-inspired objective used stop-gradient reader likelihood as retriever pseudo-supervision. A simplified term is

On a controlled development comparison, FiD scored 47.3/65.5/46.0 on NQ/TriviaQA/WebQuestions; EMDR² reached 50.4/71.1/49.9. The set posterior was approximate, reindexing costly, and answer likelihood could reward spurious evidence. The method demonstrated practical joint learning without passage labels. NeurIPS paper.

2021: SPLADEv2 keeps semantics inside an inverted index

SPLADE uses a masked-language-model vocabulary head to produce sparse term weights. SPLADEv2's max pooling can be written

A document receives learned expansion terms that need not appear literally, yet search remains an inverted-index dot product. Ranking loss is balanced by separate query/document FLOPS regularizers; DistilSPLADE uses hard negatives and cross-encoder distillation.

DistilSPLADE-max reported MS MARCO MRR@10 0.368 and recall@1,000 0.979; on the paper's BEIR subset it averaged nDCG@10 0.500. Latency and index size depend strongly on regularization, vocabulary and teacher choice. SPLADE illustrates that the history is not “sparse then dense”; learned sparse and dense retrieval are complementary. Paper.

2021/2022: ColBERTv2 preserves token-level interaction

ColBERTv2 stores token embeddings for every passage. Its late-interaction score is

Approximate centroid lists generate candidates and exact MaxSim reranks them. Cross-encoder-distilled training combines KL and in-batch cross entropy; centroid IDs plus quantized residuals reduce token storage roughly 6–10× from original ColBERT.

It reported MS MARCO development MRR@10 0.397, recall@50 0.868, and recall@1,000 0.984, with best results on 22 of 28 tested out-of-domain settings. The multi-vector index remains larger and more complex than single-vector ANN, but preserves fine-grained term alignment and later inspires visual page retrieval in ColPali. NAACL paper.

2021/2022: RETRO retrieves chunks from trillions of tokens

RETRO divided each 2,048-token training sequence into 64-token chunks. A frozen BERT embedding retrieves a neighbor chunk and its following 64-token continuation with SCaNN. A bidirectional neighbor encoder and Chunked Cross-Attention layers inject the retrieved 128-token value while maintaining causality: retrieval for a previous chunk informs current predictions.

MassiveText contained over 5T raw tokens; ordinary training retrieval used 600B, while evaluation used a 1.75T index—the rounded “2T” headline. MinHash removed documents with 13-gram Jaccard at least 0.8 to evaluation documents and held-out WikiText articles were removed from Wikipedia.

RETRO-7.5B was comparable to much larger GPT-3/Jurassic models on many Pile subsets, not all. NQ test EM was 45.5 with DPR passages, versus RAG 44.5 and FiD 51.4 in its comparison table. WikiText-103 perplexity was 18.97 with comparable Wikipedia retrieval; 3.92 with the 1.8T datastore was explicitly partly due to leakage. The Wikipedia index used 215GB versus a reported 15TB for kNN-LM; the MassiveText index used 93TB.

RETRO demonstrated parameter/memory scaling separation and chunk-level retrieval-augmented pretraining. A frozen similarity model, proprietary giant corpus, leakage/copying, privacy/licensing, enormous storage, and one-chunk causal delay constrain deployment. ICML paper.

2021/2022: Contriever learns dense retrieval without relevance labels

Contriever applies MoCo-style unsupervised contrastive learning to augmented text and average-pools BERT embeddings for dot-product search. On BEIR the unsupervised model beat BM25 in Recall@100 on 11 of 15 datasets, but lost on four; in-domain examples or MS MARCO fine-tuning improved it. It supplied Atlas with a general-purpose retriever while reinforcing two lessons: dense retrieval can be learned without QA pairs, and BM25 remains a necessary comparator. TMLR paper.

2022: Atlas unifies retrieval pretraining, FiD, and few-shot adaptation

Gautier Izacard and colleagues combined Contriever with T5 770M/3B/11B and FiD. Four retriever objectives were compared: Attention Distillation, EMDR², Likelihood Distillation, and leave-one-out likelihood. The selected target was

with a KL objective transferring reader preferences to the retriever. Masked language modeling with 15% masking and mean span length three was the selected pretraining objective.

Atlas indexed a 2021-12-20 Wikipedia with linearized lists and infoboxes as 37M section passages and a CCNet corpus of about 350M passages. Pretraining retrieved 100 stale candidates, re-embedded/reranked to 20, and refreshed every 2,500 steps. Query-side downstream tuning avoided full reindexing.

Atlas-11B NQ EM was 42.4 with 64 examples and 60.4 full data using the mixed index; temporally matched 2018 Wikipedia raised these to 45.1 and 64.0. TempLAMA-derived evaluation made index effects explicit: a 2017 model/index scored 57.7 on 2017 facts and 1.5 on 2020 facts; swapping only to a 2020 index changed those to 10.2 and 53.1. Its often-cited 42.4 versus PaLM-540B 39.6 is a 64-example fine-tuning versus prompting comparison, evidence of sample efficiency rather than a clean architecture comparison.

Atlas culminated the pre-2023 line—unsupervised dense initialization, FiD, generator-to-retriever distillation, joint retrieval-augmented pretraining, and index updates—at substantial memory and compute cost. JMLR paper.

---

6. 2022–2023: inference-time control becomes the research frontier

ReAct: reason, act, observe, repeat

ReAct alternates natural-language “thoughts,” tool actions such as search or lookup, and observations before answering. The headline QA experiments used hand-built few-shot trajectories; there was no learned retriever objective. It evaluated HotpotQA and FEVER plus ALFWorld and WebShop, reporting absolute success gains of 34 and 10 points on the two interactive tasks.

ReAct is the control-loop ancestor of agentic RAG, not a retrieval architecture. Tool and reasoning errors compound, trajectories add latency, and exemplars/tool schemas matter. Paper.

MuRAG and RA-CM3: retrieve across text and images

MuRAG trained a multimodal encoder to retrieve text or image-caption memories, then fused evidence for generation. Its combined autoregressive and in-batch contrastive losses used LAION-200M, Conceptual Captions, PAQ, and VQA data. Full-Wikipedia MultimodalQA EM was 51.4 versus cited AutoRouting 34.7. High pretraining cost, caption leakage, modality mismatch, counting and recognition errors constrain the result. EMNLP paper.

RA-CM3 retrieved mixed image-text documents with a frozen CLIP-based retriever and prepended zero to two of them to a CM3 sequence predicting text and image tokens. Without task fine-tuning, reported COCO image FID improved 29.5 → 15.7 and caption CIDEr 71.9 → 89.1. Because corpus and training domain overlapped, these gains do not establish fresh-knowledge grounding; FID/CIDEr do not test factuality. Preprint.

HyDE: search with a hypothetical answer

HyDE asks an instruction-tuned LM to generate a hypothetical answer document, embeds it with Contriever, and retrieves real documents by vector similarity. Several generated samples and the original query can be averaged. There is no HyDE-specific training: the generator expresses relevance intent; the dense encoder acts as an information bottleneck that may suppress invented details.

On TREC DL19, reported nDCG@10 increased from Contriever 44.5 to HyDE 61.3 and recall@1,000 from 74.6 to 88.0, with multilingual gains on Mr. TyDi. Generation cost, language coverage, intent drift, and hallucinated pseudo-document details remain. Crucially, the original paper evaluated retrieval, not answer faithfulness. ACL paper.

Query2Doc: preserve the original query while adding a pseudo-document

Query2Doc generates a pseudo-document few-shot, then repeats the original query and appends the expansion for sparse retrieval or encodes query+document for dense retrieval. In contrast to basic HyDE, it deliberately keeps lexical evidence. BM25 nDCG@10 rose 51.2 → 66.2 on TREC DL19 and 47.7 → 62.9 on DL20; gains were smaller for strong dense retrievers and some BEIR datasets regressed. Reported expansion latency exceeded two seconds and false details can enlarge or misdirect the query. Paper.

FLARE: retrieve when the next sentence looks uncertain

FLARE tentatively generates the next sentence. Low-probability tokens trigger retrieval; uncertain spans are masked or turned into questions; evidence is retrieved and the sentence regenerated. The original work was training-free and used text-davinci-003.

FLAREdirect reported 2WikiMultihopQA 51.0 EM/59.7 F1 versus 39.4 EM for one retrieval, and StrategyQA accuracy 77.3 versus 68.6. More retrieval did not help every task: Wizard of Wikipedia and ELI5 lacked significant gains, and over-retrieval could hurt StrategyQA. It requires token probabilities that are available and calibrated; repeated speculative generation is expensive; a wrong forecast creates a biased query. EMNLP paper.

Rewrite–Retrieve–Read: optimize the query for the final answer

A T5-large rewriter transforms the user query, Bing retrieves, and a frozen reader answers. Supervised warm-up uses pseudo-queries on which the reader succeeds; PPO reward combines answer EM, F1, and answer-string retrieval hit with a KL penalty.

HotpotQA EM/F1 rose from retrieve-read 30.47/41.34 to 34.38/45.97, with gains on AmbigNQ and MMLU. Answer-string reward permits shortcuts, Bing is mutable, PPO is costly, success filtering biases supervision, and a learned rewriter is not uniformly better than prompting. EMNLP paper.

ITER-RETGEN: let one generated answer guide the next retrieval

ITER-RETGEN retrieves from the question, generates a full chain-of-thought answer, concatenates that generation with the question for another retrieval, then regenerates. A teacher reranker sees generated answer+query while a student query encoder minimizes KL to its distribution.

Judged accuracy rose 64.8 → 71.2 on HotpotQA and 54.8 → 59.2 on Bamboogle but declined on MuSiQue and FEVEROUS; retriever distillation raised HotpotQA to about

regenerates a complete answer. Preprint.

1. Errors in one generation can poison the next query, and every iteration

Lost in the Middle: nominal capacity is not effective evidence use

Controlled experiments positioned one relevant paragraph among hard distractors. With 20 documents, GPT-3.5 answer accuracy was 75.8% when evidence was first, 53.8% in the middle, and 63.2% when last. Expanding 20 to 50 documents barely helped despite higher retrieval recall.

This separates context-window length from usable context: reranking, ordering, pruning, and diversity can matter more than maximizing \(k\). The result used a single answer document, answer-string scoring, and 2023 models, so exact numbers should not be generalized to every modern model. TACL paper.

Self-RAG: make retrieval and critique part of the output vocabulary

Self-RAG predicts reflection tokens for:

• whether to retrieve;
• whether a passage is relevant;
• whether a generated statement is supported;
• how useful the answer is.

At inference, segment-level beams combine language-model probability with configurable reflection scores. GPT-4 labeled reflection categories; a Llama-2 critic learned them and annotated roughly 150,000 instruction examples; a 7B/13B generator learned ordinary next-token likelihood over text and special tokens while retrieved passage tokens were masked from loss.

The 13B paper results included PopQA accuracy 55.8, TriviaQA 69.3, PubHealth 74.5, biography FactScore 80.2, and ASQA citation precision/recall 70.3/71.3. The approach permits controllable quality/cost trade-offs, but relies on a proprietary annotation teacher, special-token fine-tuning, multiple passage/beam generations, calibrated self-evaluation, and retriever recall. Some retrieval-heavy baselines retained higher citation recall. Generated judgments are not guarantees of entailment. ICLR paper.

---

7. What actually evolved

The chronology is clearer when decomposed by independent axes.

Axis	Early state	Intermediate state	By end of 2023
Retrieval unit	term/article/memory slot	100-word passage or token context	passage, token vectors, pseudo-document, multimodal document
Representation	TF-IDF/BM25	dense single vector	learned sparse, dense, late interaction, multimodal
Learning signal	corpus statistics/support labels	relevance pairs, ICT, MLM, answer likelihood	reader distillation, query reward, self-generated critique
Integration	pipeline or hard read	latent marginalization, concatenation	FiD, probability interpolation, chunk cross-attention, iterative retrieval
Timing	once before reading	pretraining + inference	conditional, forward-looking, iterative, tool-controlled
Knowledge lifecycle	fixed corpus	periodic embedding refresh	hot-swappable index, temporal experiments, mutable web search
Grounding	answer string/span	free-form answer with retrieved context	provenance benchmarks and citation metrics, still no guarantee

Three durable lessons before the 2024 frontier

1. Retrieval recall is a ceiling, not the objective. DrQA quantified the ceiling; reader-aware retrievers and FiD improved evidence use; Lost in the Middle showed that more recalled evidence can distract the generator.
2. Sparse versus dense is a false binary. Sparse preserves exact terms and cheap updates; dense handles semantic mismatch; learned sparse and late interaction fill different points. Hybrid candidate generation plus reranking is often the robust default.
3. Retrieved does not mean grounded. RAG answered some questions without an answer-bearing passage; reader attention is not causal attribution; Self-RAG's support tokens are learned predictions. Claim-level support, provenance, abstention, and human audit remain distinct requirements.

The next period therefore focuses less on inventing another vector retriever and more on policy, structure, verification, risk, and budget. Continue with The 2024–2026 frontier.