Skip to content
Briefcase AI

briefcase.replay

Terminal window
pip install briefcase-ai[replay]

Re-executes stored decisions against a backend. Valid modes are "strict" and "tolerant" (the default).

ReplayEngine

import briefcase
from briefcase import DecisionSnapshot, Input, Output
from briefcase.storage import SqliteBackend
from briefcase.replay import ReplayEngine


briefcase.init()
backend = SqliteBackend.in_memory()


decision = DecisionSnapshot("classify_ticket")
decision.add_input(Input("text", "reset my password", "string"))
decision.add_output(Output("category", "account_access", "string"))
decision_id = backend.save_decision(decision)


engine = ReplayEngine(backend)
engine.with_executor(lambda inputs: {"category": "account_access"})


result = engine.replay(decision_id, "strict")
print(result.status, result.outputs_match, result.execution_time_ms)


stats = engine.get_replay_stats([decision_id])
print(stats.total_replays, stats.success_rate)
ReplayEngine(storage)
  .with_executor(fn)                 # fn(inputs: dict) -> dict | value
  .replay(snapshot_id, mode)
  .replay_batch(snapshot_ids, mode, max_concurrent)
  .replay_with_policy(snapshot_id, policy, mode)
  .validate(snapshot_id, policy)
  .get_replay_stats(snapshot_ids)
  .default_mode

ReplayPolicy

from briefcase.replay import ReplayPolicy


policy = ReplayPolicy("output_match")
policy.with_exact_match("category")
policy.with_similarity_threshold("summary", 0.9)


result = engine.replay_with_policy(decision_id, policy, "strict")
print(result.status, result.policy_violations)

with_executor is what makes a replay re-execute. The callable receives the recorded inputs as a dict of name to value, and returns either a dict of output names to values or a single value recorded as result. An exception it raises propagates out of replay.

Without an executor, replay returns status "pending" with outputs_match False, and every policy rule reports actual: "not replayed": an unchecked replay never reads as a pass.

ReplayResult

Returned by replay / replay_with_policy. Attributes: status ("success", "failed", or "pending"), outputs_match, replay_output, original_snapshot, execution_time_ms, policy_violations, plus to_dict().

Each policy violation is a dict of rule_name, field, expected, actual, and message.

ReplayStats

Returned by get_replay_stats. Attributes: total_replays, successful_replays, failed_replays, exact_matches, mismatches, success_rate, average_execution_time_ms, total_execution_time_ms, plus to_dict().