srb agent tune — Hyperparameter Optimization
Use srb agent tune with Optuna to tune reinforcement-learning hyperparameters. The Isaac-free study process launches training children. Install with uv sync --extra tune. See the Hyperparameter Tuning workflow for concepts, wall-clock estimates, and adoption guidance.
Verbs
srb agent tune check [options] # preflight without training (Isaac-free)
srb agent tune run [options] # run or resume a study
srb agent tune status --study S # summarize a study
srb agent tune validate --study S # rerun top trials on fresh seeds
srb agent tune export --study S # write a tuned config
srb agent tune repair --study S # fail stale RUNNING trials
srb agent tune baseline [options] # capture portfolio normalization baselines
run Options
| Option | Description | Default |
|---|---|---|
-e/--env/--task ENV | Single task | — |
--tasks T [T …] | Portfolio task set, trained in turn | — |
--algo ALGO | Algorithm label (e.g. skrl_ppo) | REQUIRED |
--obs {state,visual} | Observation modality; visual needs registered profiles | state |
--trial-steps N | Total environment steps per trial, per task segment (canonical budget) | REQUIRED |
--n-trials N | Target; terminal and RUNNING trials count | 20 |
--n-workers N | Concurrent trial slots | 1 |
--gpus 0,1 | GPU ids, round-robin; otherwise inherit framework default | unset |
--trial-num-envs N | Parallel envs per trial (pinned; drives budget conversion) | 1024 |
--seed-base N | Base seed; child seed = seed-base + trial*seeds-per-trial + replicate | 42 |
--seeds-per-trial K | Replicates averaged per trial; pruning uses replicate 0 | 1 |
--pin agent.X=Y … | Fixed, unsampled config values | — |
--objective {ep_return_mean,success_rate} | Tail-mean objective field | ep_return_mean |
--normalize {none,baseline,rank} | Portfolio score normalization | none |
--aggregate {median,mean,min} | Portfolio task aggregation | median |
--max-wall-hours H | Stop starting trials after this limit | — |
--trial-timeout S | Kill a trial after this many seconds | — |
--fps F | Measured env-steps/s for runtime estimate | — |
--n-startup N | TPE random-startup trials | 10; 15 portfolio |
--boot-stagger S | Serialize concurrent Isaac first-boots; stops after first child reports metrics | 30 if --n-workers > 1, else 0 |
--keep-artifacts | Keep PRUNED child logdirs (deleted after the prune by default); configs and metrics always survive | False |
--retry-failed N | Re-enqueue up to N failed sets at startup; skip reason=timeout, raise --n-trials to fund retries | 0 |
--version N | Version for incompatible reruns | 1 |
--logdir/--logs | Study artifact root | logs/tune |
--storage URL | PostgreSQL Optuna URL (postgresql:// or postgresql+psycopg2://) for shared workers; requires tune-rdb | Journal file |
All actions reject bare key=value and tokens after --; use --pin KEY=VALUE to fix values for every trial. --storage never falls back to a journal. Install both extras with uv sync --extra tune --extra tune-rdb. Credentials are redacted; unsupported schema changes fail closed without migration. Back up PostgreSQL before an operator-managed schema upgrade.
If a task does not emit the selected objective, e.g. success_rate, its trial fails early with success_rate_unavailable.
Averaging seeds during a study (--seeds-per-trial)
--seeds-per-trial K averages K distinct-seed children; failed replicates are excluded and survivors cost about K times more because pruning uses replicate 0. Resume rejects changes to search space, tasks, budget, objective, observation mode, ordered pins, resolved agent config, --trial-num-envs or --seeds-per-trial, and rejects legacy studies that lack these fields; use --version for an incompatible study. seed-base and Optuna-version changes are allowed. Use validate to rerank leaders on fresh seeds.
check Options
check runs trial config and budget checks in memory, starts no study or child, prints [OK], [WARN], or [FAIL], and exits nonzero on hard errors. It accepts config-shaping run options: -e/--env/--task, --tasks, --algo (REQUIRED), --obs, --trial-steps (REQUIRED), --pin, --trial-num-envs, --seed-base, --seeds-per-trial, --normalize, --aggregate, --objective, --fps, --n-trials, --n-workers, --version, and --logdir. Baseline-normalized portfolios also check each task’s baseline is present and fresh; --fps adds a runtime estimate.
srb agent tune check --env sample_collection --algo skrl_ppo --trial-steps 10_000_000 --fps 50000
srb agent tune check --tasks sample_collection peg_in_hole --algo skrl_ppo \
--trial-steps 10_000_000 --normalize baseline # also checks baselines.yaml
validate Options (noise arbiter)
validate reruns top completed trials with fresh seeds, using recorded configs and pins; only seeds change. Portfolio runs repeat every task segment with study normalization. It writes <study>/validation.json without changing trial records. Choose a --seed-base range disjoint from training seeds; candidates need at least --min-ok completed seeds (default: majority).
| Option | Description | Default |
|---|---|---|
--study NAME | Study to validate | REQUIRED |
--storage URL | Study PostgreSQL URL | journal default |
--top-k K | Top trials by training value to rerun | 3 |
--seeds S | Fresh seeds per candidate | 3 |
--seed-base B | Validation seed base; seeds are B through B+S-1 | 10000 |
--min-ok N | Completed seeds required for winner | majority of --seeds |
--validate-steps N | Env-steps per rerun; raise above the study budget for higher fidelity | study budget |
--gpus 0,1 | GPU ids, round-robin; otherwise inherit framework default | unset |
--n-workers N | Concurrent rerun slots | 1 |
--trial-timeout S | Kill a rerun after this many seconds | — |
--keep-artifacts | Keep rerun child logdirs | False |
--logdir/--logs | Study artifact root | logs/tune |
srb agent tune validate --study srb-skrl_ppo-state-sample_collection-v1 \
--top-k 5 --seeds 4 --gpus 0,1
The report ranks validated means and names the robust winner; if it differs from best_trial, it prints that trial’s export command.
status / export / repair Options
| Option | Applies to | Description | Default |
|---|---|---|---|
--study NAME | all | Study name (e.g. srb-skrl_ppo-state-pick-v1) | REQUIRED |
--storage URL | status, export, repair | PostgreSQL study URL | journal default |
--logdir/--logs PATH | status, export, repair | Study artifact root | logs/tune |
--trial N or best | export | Trial to export | best |
--to {task,global} | export | Task-specific (L1) or global (L0) slot; default depends on study | study-dependent |
--dry-run | export | Print diff without writing | False |
--force | export | Overwrite existing file | False |
--stale-hours H | repair | Fail RUNNING trials older than this | 24 |
Portfolio exports default to global; --to task is ambiguous and errors.
baseline Options
Capture floor/reference scores per task for --normalize baseline; entries go to hyperparams/tune/baselines.yaml.
| Option | Description | Default |
|---|---|---|
-e/--env/--task / --tasks | Task(s) to baseline | — |
--algo ALGO | Baseline algorithm label | REQUIRED |
--obs {state,visual} | Observation modality | state |
--budget/--ref-steps N | Reference env-steps (same as trial budget) | REQUIRED |
--floor-steps N | Random-rollout env-steps | 200000 |
--trial-num-envs N | Parallel envs (pinned; drives budget conversion) | 1024 |
--seed-base N | Base; task k uses seed-base + k | 42 |
--gpus 0,1 | GPU ids, round-robin | env / GPU 0 |
--objective {ep_return_mean,success_rate} | Measured floor/ref field | ep_return_mean |
--trial-timeout S | Kill a floor/ref run after this many seconds | — |
--force | Overwrite baseline entries | False |
--dry-run | Compute without writing baselines.yaml | False |
--logdir/--logs PATH | Study artifact root | logs/tune |
Baselines record measured scores and the source config’s git blob SHA. run and check warn on config changes.
Registered search spaces
Registered state-observation spaces (runtime support still depends on capability preflight):
| Framework | Labels |
|---|---|
| skrl | skrl_ppo; skrl_sac; skrl_ppo_rnn |
| stable-baselines3 | sb3_ppo, sb3_sac, sb3_tqc, sb3_td3, sb3_crossq |
| sbx (JAX SB3) | sbx_ppo, sbx_sac, sbx_tqc, sbx_td3, sbx_crossq |
| rsl_rl | rsl_rl_ppo |
| DreamerV3 | dreamer |
| TD-MPC2 | tdmpc2 |
Unknown --algo values list available spaces and exit; registered but unavailable labels fail preflight. Visual spaces are not yet registered.
DreamerV3 / TD-MPC2 caveats
Both integrations tune only global (L0) configs and ignore per-task blocks in dreamerv3.yaml / tdmpc2.yaml; a task-scoped export at hyperparams/task/<task>/dreamerv3.yaml or tdmpc2.yaml is not read yet. Plain-YAML frameworks support L1. --trial-steps sets native total-env-step budgets (run.steps for Dreamer, steps for TD-MPC2); export restores production values and --pin on these driver-owned keys errors. With num_envs > 1, framework and reporter step counts can differ by a constant factor; calibrate against a short reference run before estimating wall time.
Dreamer tunes run.train_ratio and model_size (RSSM/depth/units). TD-MPC2 tunes optimizer, MPPI planner, and loss coefficients; model_size sets architecture dimensions at load.
Portfolio (Multi-Task) Studies
--tasks A B C … trains one shared parameter set sequentially across tasks in fixed order, one srb agent train child per task in trials/NNN/seg-<task>/. Task scores normalize and aggregate into one objective; divergence prunes the trial, and intermediate aggregates let MedianPruner stop weak trials between tasks.
Raw returns (--normalize none, default) suit only tasks with shared scales; the driver warns otherwise. baseline normalizes (score − floor) / (ref − floor) from hyperparams/tune/baselines.yaml (missing fails fast; stale config SHA warns). rank uses per-task fractional ranks across completed trials; it needs no baseline but is non-stationary early. --aggregate median (default) resists one outlier; min makes the weakest task decisive; mean averages. Portfolio export defaults to global L0.
# 1. Capture floor/ref baselines for the task set.
srb agent tune baseline --tasks sample_collection peg_in_hole landing \
--algo skrl_ppo --budget 10_000_000
# 2. Run the portfolio study, normalized against those baselines.
srb agent tune run --tasks sample_collection peg_in_hole landing --algo skrl_ppo \
--trial-steps 10_000_000 --n-trials 30 --normalize baseline --aggregate median
# 3. Export the winner to the global per-algorithm config.
srb agent tune export --study srb-skrl_ppo-state-multi-<hash8>-v1 # --to global by default
Examples
Single-task smoke-budget study, two workers on two GPUs:
srb agent tune run --env sample_collection --algo skrl_ppo \
--trial-steps 20_000_000 --n-trials 40 --n-workers 2 --gpus 0,1
Inspect and export the best trial to the task-specific slot:
srb agent tune status --study srb-skrl_ppo-state-sample_collection-v1
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task --dry-run
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task
Pin a value outside the search and fix a task-side override:
srb agent tune run --env peg_in_hole --algo skrl_ppo --trial-steps 10_000_000 \
--pin agent.agent.grad_norm_clip=1.0 env.robot=franka
Output
Study artifacts live under logs/tune/<study-name>/:
logs/tune/srb-skrl_ppo-state-sample_collection-v1/
journal.log # Optuna JournalStorage (NFS-safe; shared by workers)
study.yaml # frozen manifest (search space, pins, resolved config, budgets)
trials/000/ # trial_cfg.yaml, metrics.jsonl, logs/
Exports write hyperparams/<framework>/<algo>.yaml (global) or hyperparams/task/<task>/<framework>/<algo>.yaml (task) with provenance; adopt through git review.
See Also
Hyperparameter Tuning Workflow · srb agent train · Agent Configuration