The Evidence Path

Production and optimization

12 of 12 · Production · 10 named systems

A production RAG system coordinates independently versioned data, retrieval, generation, control, and evaluation planes under quality and cost budgets.

Production and optimization: architecture comparisonEach lane reads from input to output.

Request lifecycle

  1. admitPolicy and identity
  2. retrieveCandidate services
  3. selectEvidence budget
  4. generateAnswer and citations
  5. recordTrace and metrics
  • Production reference architecture

Retrieval serving

  1. routeRetriever choice
  2. fanoutShards and replicas
  3. mergeCandidate fusion
  4. fallbackDegraded mode
  • Sparse retrieval service
  • Dense ANN service
  • Multi-vector service
  • Federated retrieval service
  • Semantic cache

Generation serving

  1. cacheReusable state
  2. prefillEvidence context
  3. speculateFuture retrieval
  4. decodeStructured output
  • RaLMSpec
  • FlashAttention
  • PagedAttention

Optimization and release

  1. searchPipeline configurations
  2. measureQuality, latency, cost
  3. selectPareto candidate
  4. shadowProduction replay
  5. releaseCanary and rollback
  • syftr

Read the full explanation