The Evidence Path

Corpus units and ANN

2 of 12 · Indexing · 20 named systems

Corpus construction fixes the evidence units and approximation boundary that every downstream retriever must accept.

Corpus units and ANN: architecture comparisonEach lane reads from input to output.

Evidence units

  1. inputCanonical document
  2. segmentStructural boundaries
  3. enrichContext and lineage
  4. outputRetrieval units
  • TextTiling
  • Dense X Retrieval
  • Contextual Retrieval
  • Late Chunking

Hierarchy

  1. indexChild units
  2. linkParent relations
  3. retrieveMatch children
  4. expandRestore context
  • Parent-child retrieval
  • Small-to-big retrieval
  • HiChunk

ANN search

  1. encodeDocument vectors
  2. buildGraph or partition
  3. probeBounded search
  4. outputApproximate neighbors
  • FAISS
  • LSH
  • IVF
  • PQ
  • OPQ
  • HNSW
  • DiskANN
  • Vamana
  • SPANN
  • ScaNN
  • AdANNS
  • CompactDS

Release audit

  1. baselineExact oracle
  2. measureANN recall
  3. validateEvidence recall
  4. publishIndex generation
  • Exact vector search

Read the full explanation