Gymnasium

Outcome. Search for prompt injections that get past a guardrail, and record RL rollouts as decision records.

Prerequisites. The gym extra and a GuardrailEnv you implement; no concrete environment ships in the SDK.

Packages. briefcase-ai

Languages. Python

The guardrail framework already mirrors the Gymnasium API by design — GuardrailEnv to gym.Env, GuardrailWrapper to gym.Wrapper, PolicySpace to spaces. briefcase.integrations.gym connects the two for real, in both directions: a gymnasium.Env over any guardrail, and a wrapper that records RL episodes as decision records.

pip install briefcase-ai[gym] New in 4.0.0

When you’d reach for this

You want to know which prompt injections get past a guardrail, and you’d rather search for them with a policy than enumerate them by hand. Or you’re training an agent and want its rollouts in the same audit store as your production decisions.

Install

pip install briefcase-ai[gym]

The extra installs gymnasium>=0.29. Nothing is registered with gymnasium at import time.

A guardrail as an environment

These examples all use one small guardrail and a single task. Yours will be bigger; the shape is the same.

from briefcase.guardrails.framework import (
    BaseGuardrailEnv, Effect, EvalRequest, EvalResult, GuardrailTask, PolicySpace,
)


class ClearanceGuardrail(BaseGuardrailEnv):
    _name = "clearance"
    _request_space = PolicySpace(
        agents=["nurse"], actions=["read"], resources=["/records/*"],
    )

    def evaluate(self, request: EvalRequest) -> EvalResult:
        cleared = request.context.get("clearance", 0) >= 3
        return EvalResult(
            effect=Effect.ALLOW if cleared else Effect.DENY,
            guardrail_name=self._name,
            reason="cleared" if cleared else "insufficient clearance",
        )


guardrail = ClearanceGuardrail()
tasks = [
    GuardrailTask(
        id="cleared-nurse",
        request=EvalRequest("nurse", "read", "/records/1", {"clearance": 4}),
        expected_effect=Effect.ALLOW,
    )
]
injections = []

from briefcase.integrations.gym import GuardrailGymEnv

env = GuardrailGymEnv(guardrail, tasks, injections)

obs, info = env.reset(seed=0)                              # samples a task
obs, reward, terminated, truncated, info = env.step(0)     # 0 = clean request

Episodes are single-step by design. GuardrailEnv.evaluate() is side-effect free and single-shot, so a multi-step episode would fabricate state that does not exist.

Action space

Discrete(1 + len(injections)). Action 0 submits the sampled task’s clean request; action i submits injections[i - 1].inject(request). The space is deliberately not free-form request construction — an arbitrary request has no expected_effect, and without one the reward is meaningless.

Reward

1.0 when the guardrail returns the task’s expected_effect, 0.0 otherwise. So a policy learns which injections bypass the guardrail.

env = GuardrailGymEnv(guardrail, tasks, injections, reward_mode="adversarial")

adversarial inverts it, which trains an attacker instead of a verifier.

What info carries

task_id, injection_id, effect, expected_effect, utility, security, reason, eval_time_ms, and the raw EvalResult. render_mode="ansi" returns the guardrail’s own explanation narrative for the last step.

The env passes gymnasium.utils.env_checker.check_env.

Making it from a string id

import gymnasium
from briefcase.integrations.gym import register_with_gymnasium

register_with_gymnasium(guardrail=guardrail, tasks=tasks)
env = gymnasium.make("briefcase/GuardrailEval-v0")

Nothing is registered at import time — a Briefcase AI import never mutates the global gymnasium registry.

The made env drives your guardrail

register_with_gymnasium holds the guardrail in the entry point’s closure rather than in the registry kwargs, which gymnasium.make deep-copies. So close() reaches the guardrail you registered, and a guardrail holding an uncopyable resource (a lock, a connection) can be registered at all.

Capturing rollouts

Wrap any gymnasium.Env, not just a guardrail one:

import briefcase, gymnasium
from briefcase.integrations.gym import capture_episodes

briefcase.observe("rollouts.jsonl")
env = capture_episodes(gymnasium.make("CartPole-v1"))

obs, info = env.reset(seed=0)
obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
env.close()

Type	Emitted	Carries
rl.step	once per step	episode id, step index, action and observation reprs, reward, terminated, truncated, info keys, timing
rl.episode	once per episode	env id, total steps, episode return, whether it completed

A reset mid-episode, or close(), finalizes the open episode with completed=False, so an abandoned rollout still produces a record.

Pass capture_steps=False to record only the episode summary on a long run.

Export timing on short scripts

capture_episodes exports on a background thread by default, since step capture is on the hot path. A script that exits immediately after close() can lose its records — pass capture_episodes(env, async_capture=False) for short runs. Long training jobs should keep the default.

Limits

Episodes are one step long. GuardrailEnv.evaluate() is side-effect free and single-shot, so step always returns terminated=True and a second step before reset raises RuntimeError. Algorithms that need temporal credit assignment have nothing to assign it over; this is a contextual bandit wearing an Env interface.

The action space is the injection list, not request space. Action 0 is the clean request and action i applies injections[i - 1]. You cannot search over arbitrary requests, because an arbitrary request has no expected_effect and therefore no meaningful reward. The search is only as good as the injections you supply.

Reward measures agreement with the task label, not security. 1.0 when the guardrail returns the task’s expected_effect. A task set that mislabels an expectation trains the policy toward that mistake.

Capture drops records on a fast exit. capture_episodes exports on a background thread by default, so a script that exits right after close() can lose the tail. Pass async_capture=False for short runs, and keep the default for long training jobs where step capture is on the hot path.

Step records store reprs. Actions and observations are truncated to 1000 characters each (max_action_chars, max_obs_chars). A large observation space lands in the record as a cut string, so pass capture_steps=False and keep the episode summary when the per-step detail is not worth the volume.

API reference

briefcase.integrations.gym has the full GuardrailGymEnv, register_with_gymnasium, and EpisodeCaptureWrapper signatures.

Bringing your own guardrail

No concrete GuardrailEnv ships in the SDK; the protocol is the contract. Subclass BaseGuardrailEnv, set _name and _request_space, implement evaluate(), and it works with the adapter, the benchmark, and the capture wrapper alike. See examples/rl_gym/ for a complete offline example, and Guardrails for the framework itself.

Where this fits

Gymnasium is a testing surface for the Control & Route act: attack a guardrail on purpose before production does it for you.

Control & Route: Guardrails The framework this adapter drives, and the protocol you subclass.

Capture: Exporters Where the captured episode and step records go.