srb agent train — Train Agent

Start or Resume a Run

1. Choose an algorithm and environment.
2. Add environment overrides after options.
3. For replay prefill, see SKRL or Dreamer.
4. Optionally package a successful run (see Model Artifacts).

srb agent train --algo sbx_ppo --env landing env.num_envs=512 --hide_ui

srb agent train --algo sbx_ppo --env landing --continue_training
srb agent train --algo sbx_ppo --env landing \
  --model logs/landing/sbx_ppo/<run>/ckpt/<ckpt>

Explicit --model checkpoint paths resolve relative to the current working directory. Logs default to <repo>/logs in a checkout and <cwd>/logs when installed; use --logdir to change the root.

Run records

Each run has metadata.json and run.json; the record lists source and asset commits, simulator pair, image, host, GPU, driver, seed, algorithm, implementation, task, step budget, and hyperparameter overrides. Online training writes TensorBoard scalars to metrics.jsonl ({"tag", "step", "wall_time", "value"} per line); re-export interrupted runs with python -m srb.utils.metrics_jsonl <logdir>.

Source checkouts record Git commits; site-packages installs report null. Asset metadata records its submodule commit, pinned gitlink, and dirty flag (null if unknown). Metadata files and config snapshots redact secret-named values and flags, forwarded kit_args, URL credentials and query values, plus Authorization and Bearer values as <redacted>. SRB atomically reserves directories and suffixes same-second runs, such as -001.

Options | Shared Agent Options

Resume behavior by framework

Supported Algorithm Families

Registered labels include dreamer, tdmpc2, sbx_ppo, sb3_ppo, rsl_rl_ppo, skrl_ppo, skrl_sac, and skrl_ppo_rnn. SRB provides SKRL models and profiles for skrl_sac and skrl_ppo_rnn. See the Reinforcement Learning Workflow for Hydra overrides and Weights & Biases logging.

SKRL Demonstration Prefill

skrl_sac, skrl_ddpg, and skrl_td3 prefill replay from canonical SRB JSONL exports with metadata.demo_io_contract. The trainer validates observation projections and action coordinates, including runtime scales, against the live wrapper. It preserves explicit next observations and separate termination and truncation flags.

The cap applies to the whole source; exceeding it causes an error. The transition count must divide evenly by env.num_envs and fit the replay capacity; rows are neither padded nor dropped. The trainer checks admission before native allocation. Limits are 192 MiB for replay payloads and 256 MiB for serialized checkpoints. Oversized configurations report the maximum supported agent.memory.memory_size for the environment count and I/O dimensions. These limits do not bound peak process memory.

srb agent train --algo skrl_sac --env peg_in_hole \
  --demo-dataset /data/peg_in_hole_canonical \
  --demo-max-transitions 1000 \
  env.num_envs=1 agent.memory.memory_size=8192 --hide_ui

Resume requires both options and identical source bytes, I/O layout, and replay dimensions. Combined checkpoints restore replay, optimizer, and random state at the saved next timestep; the simulation resets. Resume rejects agent.agent.experiment.store_separately=true. Evaluation uses weights only.

DDPG and TD3 restore exploration-noise state and learning-rate schedules; TD3 also restores its delayed actor-update counter. Resume requires complete native modules and unchanged update, scheduler, sampling, and exploration settings, including native Gaussian or Ornstein-Uhlenbeck noise parameters and actor cadence. Custom noise implementations are rejected. Demo and online replay use the checkpoint-bound agent.agent.rewards_shaper_scale; arbitrary reward-shaping callbacks are unsupported.

Prefill is unsupported for other SKRL labels, SB3 (sb3_*), SBX (sbx_*), and RSL-RL (rsl_rl_ppo). srb dataset compatibility DATASET --purpose train reports the training mode and compatibility for each algorithm. For offline demonstrations, use Robomimic with srb agent learn.

Dreamer Checkpoints and Continuation

Dreamer resumable state requires the original agent.replay.save=true and agent.replay.save_wait=true settings; persistence is disabled by default. Weights-only checkpoints lack replay and cannot use --continue; import them into a new run. Dreamer training runs in worker processes and rejects --perf, --hide_ui, and --repl.

The authoritative manifest selects the committed agent, step, and replay generation together. In-run --model must identify that checkpoint; older generations are rejected. Use a new log directory to import weights from another generation.

Cleanup retains the current generation and its parent, preserves files with uncertain ownership, and waits for same-run readers. Snapshots pin saved chunks and exclude later in-memory updates. Failed cleanup may leave extra files without invalidating the committed checkpoint.

Dreamer Demonstration Prefill

dreamer uses paired options and canonical SRB JSONL. The task, observation terms, action layout, and runtime scales must match; actions must be continuous and within [-1, 1]. Each simulator worker needs matching projection metadata; missing or mismatched metadata causes rejection. The local route uses one environment; the process route uses one environment per worker.

Prefilled runs also require replicas: 1, positive replay_context, and the save settings above. Set replay.size and replay.chunksize for the source and observation/context dimensions; defaults of several million items may exceed admission limits. Replay arrays and staged rows, including recurrent context, are each capped at 256 MiB. Total process memory and model allocations are not bounded.

Dreamer adds a successor row per episode, retains terminal observations, and separates termination from timeout. Two 12-transition episodes yield 26 native rows and 18 windows at sequence length 9. Capacity counts windows and must fit the batch window; excess source transitions are rejected. Within each episode, projected next_obs must equal the next observation. Recurrent context uses the demonstrated previous actions.

Same-run resume requires an identical source path and bytes, projection, and bound learner/replay configuration. A mismatch is rejected before restore or config/metadata overwrite. Resume restores the agent and replay without reinserting demonstrations. Native replay omits the sampler cursor and random state, so batches are not bit-exact. Evaluation uses weights only.

Dreamer demonstration prefill has not been validated with GPU simulator training or on hardware. TD-MPC2 does not support it.

Model Artifacts

Package successful runs as portable SRB artifacts with --model-artifact-output. Record provenance with --model-artifact-dataset, or package the run afterward with srb model from-run.

For demo-prefilled runs, packaging retains the training-time manifest and member digests if the source moves or is removed. Explicit --model-artifact-dataset controls the declared dataset link; consumed demo binding stays in artifact metadata.