RAG Versioning

Outcome. Tie a retrieval to the exact corpus state, and catch a stale index before it answers anything.

Prerequisites. The rag extra, plus a rag-chroma, rag-pinecone, or rag-weaviate extra for a hosted store.

Packages. briefcase-ai

Languages. Python

RAG versioning records exactly which documents were embedded, with which model, at which source commit — so a retrieval can be tied back to the exact corpus state, and a stale index is caught before it answers anything.

rag For: governance & audit

When you’d reach for this

Your support-triage agent answers policy questions from a retrieval index built over a support_kb. The team revises a document and adds a new one, but the index isn’t rebuilt — so the agent keeps citing the old version and routes a ticket on outdated guidance. A versioned manifest makes the drift detectable: check_invalidation tells you precisely what changed before you trust a retrieval.

How it works

1. Build a manifest. create_embedding_batch then create_manifest fingerprint the indexed documents into an EmbeddingManifest.

2. Check invalidation. check_invalidation compares the current documents and model against the manifest to detect a stale index.

3. Rebuild. When sources changed, rebuild_index produces a fresh manifest so retrievals reflect the current corpus.

flowchart LR
    A["Documents"] --> B["create_embedding_batch()"]
    B --> C["create_manifest()"]
    C --> D["EmbeddingManifest"]
    E["Current documents + model"] --> F["check_invalidation()"]
    D --> F
    F --> G{"is_valid?"}
    G -- "no" --> H["rebuild_index()"]
    G -- "yes" --> I["Reuse index"]

Diagram description

Documents flow through create_embedding_batch() and create_manifest() to produce an EmbeddingManifest. check_invalidation() compares that manifest against the current documents and model: if it is no longer valid the index is rebuilt with rebuild_index(), otherwise the existing index is reused.

Install

pip install briefcase-ai[rag]

Note

The rag extra is pure Python and installs no third-party dependencies.

from briefcase.rag import VersionedEmbeddingPipeline, Document

Build an Index

A Document carries an id, content, and optional metadata. The pipeline’s embedding_model is any object with an embed(texts) -> list[list[float]] method; optional name and version attributes are recorded on the manifest.

from briefcase.rag import VersionedEmbeddingPipeline, Document


class HashingEmbedder:
    """Trivial deterministic embedder for the example."""

    name = "demo-embedder"
    version = "1.0"

    def embed(self, texts):
        return [[float(len(t) % 7), float(len(t) % 11)] for t in texts]


pipeline = VersionedEmbeddingPipeline(embedding_model=HashingEmbedder())

documents = [
    Document(id="kb-1", content="How to reset your password.", path="handbook/auth.md"),
    Document(id="kb-2", content="Refund policy for digital goods.", path="handbook/refunds.md"),
]

batch = pipeline.create_embedding_batch(documents)
manifest = pipeline.create_manifest("support_kb", [batch])

print(manifest.index_name)        # "support_kb"
print(manifest.document_count)    # 2
print(manifest.model)             # "demo-embedder"
print(manifest.status)            # "current"

rebuild_index() chains both steps when you just want a fresh manifest:

manifest = pipeline.rebuild_index("support_kb", documents)

Detect Staleness

check_invalidation() compares the latest manifest against the current document set and model, and returns an InvalidationReport describing what changed.

# A document changed and one was added since the manifest was built.
current = [
    Document(id="kb-1", content="How to reset your password (updated)."),
    Document(id="kb-2", content="Refund policy for digital goods."),
    Document(id="kb-3", content="Two-factor authentication setup."),
]

report = pipeline.check_invalidation("support_kb", current)

print(report.is_valid)            # False
print(report.status)              # "stale_documents"
print(report.added_documents)     # ["kb-3"]
print(report.changed_documents)   # ["kb-1"]
print(report.removed_documents)   # []
print(report.model_changed)       # False

status is one of: current, stale_documents, stale_model, stale_both, rebuilding. When the index is stale, rebuild it:

if not report.is_valid:
    manifest = pipeline.rebuild_index("support_kb", current)

EmbeddingManifest

The manifest is the versioning artifact. Persist it to compare future builds against it.

Field	Description	Why it matters
manifest_id	Unique id for this build	Names the corpus version a retrieval ran against.
index_name	Name of the index	Groups builds of the same index over time.
model / model_version	Model that produced the embeddings	A model change invalidates the index too.
source_commit	Source commit the documents came from	Ties the corpus to a versioned source.
document_count	Number of documents embedded	Quick sanity check on coverage.
document_hashes	doc_id -> content_hash at embed time	What check_invalidation compares against.
status	Current ManifestStatus value	current vs stale_*.
manifest_hash	Deterministic SHA-256 over the manifest content	Tamper-evident fingerprint of the whole build.

print(manifest.manifest_hash)     # integrity hash
serialized = manifest.to_json()

Instrument Retrieval

InstrumentedRetriever captures version provenance on each retrieved document. It is a reference implementation: the base retrieve() returns placeholder results (and emits a RuntimeWarning) so the provenance shape is clear. Override retrieve() with a real vector-store query — returning your own RetrievalResult objects — to silence the warning and use it for real.

from briefcase.rag import InstrumentedRetriever


class KbRetriever(InstrumentedRetriever):
    def retrieve(self, query, top_k=5, similarity_threshold=0.7):
        # Query your real vector store here, then wrap hits in RetrievalResult.
        return super().retrieve(query, top_k, similarity_threshold)


retriever = KbRetriever(
    vector_store=None,        # your vector store client
    lakefs_client=None,       # resolves document_version (commit SHA)
    repository="support_kb",
)

results = retriever.retrieve("how do I reset my password?", top_k=3)
for r in results:
    print(r.rank, r.document_id, r.score, r.document_version)

Each RetrievalResult carries document_id, content, score, rank, document_version (the commit SHA the document was read at), and metadata.

Use a version-aware vector store

VersionedChromaStore, VersionedPineconeStore, and VersionedWeaviateStore stamp vectors with the lakeFS repository and commit, then constrain queries to that same commit. Install only the matching rag-chroma, rag-pinecone, or rag-weaviate extra. See Data and VCS Adapters for provider versions and the shared provenance contract.

Limits

Manifests live on the pipeline instance. A new VersionedEmbeddingPipeline has no history, and check_invalidation against an index it has never seen returns is_valid=False with manifest_id="none" and every document listed under added_documents. “I have no manifest” and “the whole corpus is new” look identical, so persist the manifest (to_json()) and check manifest_id before acting on a report.

Invalidation compares content hashes, so metadata changes are invisible. Changing a document’s metadata leaves changed_documents empty and is_valid True. If metadata affects retrieval in your system, fold it into the content or hash it yourself.

The manifest hash is deterministic, and covers the manifest only. The same corpus and model produce the same manifest_hash in a different process, which is what makes it comparable. It does not cover the embedding vectors or the vector store, so it tells you the inputs matched, not that the index was rebuilt from them.

Nothing rebuilds automatically. check_invalidation reports; rebuild_index acts. Wiring the first to the second, and deciding what happens to in-flight retrievals during a rebuild, is yours.

InstrumentedRetriever.retrieve is a placeholder. The base implementation returns fabricated results and emits a RuntimeWarning. Override it with a real vector-store query before using it for anything real.

API reference

briefcase.rag has the pipeline, document, manifest, invalidation report, and retriever types.

Where this fits

Related: External Data Tracking The same versioning idea for non-RAG upstream values.

Related: Validation Engine Validate the content a versioned retrieval returns before a decision.

Guide: Reproducible RAG Wire manifests and snapshots into a reproducible pipeline.