briefcase.rag

pip install briefcase-ai[rag]

Versions an embedding index so it can be invalidated and rebuilt when documents or the embedding model change.

VersionedEmbeddingPipeline, Document

from briefcase.rag import VersionedEmbeddingPipeline, Document


class EmbeddingModel:
    def embed(self, texts):
        return [[0.1, 0.2, 0.3] for _ in texts]


pipeline = VersionedEmbeddingPipeline(embedding_model=EmbeddingModel())

documents = [
    Document(id="doc-1", content="Reset your password from settings.", metadata={"topic": "account"}),
]
print(documents[0].content_hash[:10])

batch = pipeline.create_embedding_batch(documents)
manifest = pipeline.create_manifest("faq-index", [batch])
report = pipeline.check_invalidation("faq-index", documents)
print(manifest.index_name, report.is_valid)

VersionedEmbeddingPipeline(embedding_model=None, lakefs_client=None,
                           repository=None, branch="main")
  .create_embedding_batch(documents, batch_id=None, source_commit=None)
  .create_manifest(index_name, batches, metadata=None)
  .check_invalidation(index_name, current_documents, ...)
  .rebuild_index(index_name, documents, source_commit=None, batch_id=None)
  .get_latest_manifest(index_name)
  .get_manifests(index_name, limit=None)

Document(id, content, metadata={}, path="")
  .content_hash

Version-aware vector stores

from briefcase.rag.vector_stores import (
    VersionedChromaStore,
    VersionedPineconeStore,
    VersionedWeaviateStore,
)

Each adapter takes a lakeFS repository and commit and constrains retrieval to that source version. Install rag-chroma, rag-pinecone, or rag-weaviate.