Skip to content

ADR-002: RAG architecture

Status: Accepted · Date: 2026-08-22

Context

We need a RAG pipeline that is production-credible yet fully explainable in an interview, runs offline for clone-and-run, and is easy to test at every stage.

Decision

  • Hand-rolled pipeline, not LangChain/LlamaIndex. Every stage (chunk → embed → store → retrieve → rerank → build context → generate → cite) is a small, typed, independently-testable unit. We can explain and unit-test each one.
  • pgvector as the primary vector store; InMemoryVectorStore for tests/quick demo; Qdrant left as a documented, bounded extension point (the VectorStore interface is four methods). One datastore for the happy path keeps Compose simple.
  • Deterministic hash embeddings by default (no downloads), fastembed (local ONNX) as an opt-in upgrade behind the same Embedder interface.
  • Lightweight lexical rerank blended with vector score. Deterministic and it measurably improves ordering on the demo corpus; a cross-encoder slots in at the same seam for production.
  • Citations are real: the context builder numbers blocks [N], the prompt asks the model to cite [source: N], and the pipeline parses those markers back to filenames.

Consequences

  • (+) Testable, offline, explainable; no heavyweight fast-moving dependency.
  • (+) Swapping store/embedder/LLM changes one factory call, not the pipeline.
  • (−) Hash embeddings are lexical, not semantic — fine for the demo corpus, but the README is explicit that fastembed/pgvector is the realistic config. This honesty is itself a talking point (knowing the limits of your test setup).