Hyperparameter Tuning

srb agent tune uses Optuna to optimize one shared hyperparameter set per algorithm, or an optional task-specific set. State and visual observations use separate configs.

Install the optional dependency first:

uv sync --extra tune

How it works

The Isaac-free driver launches each srb agent train trial in a separate process, reports smoothed returns to an Optuna pruner, and records the final-quarter mean. Workers can share a journal.

Config layering

For task-specific versus global precedence and framework exceptions, see Tuned configurations.

hyperparams/
  skrl/ppo.yaml                     # L0  — global, state observations (exists today)
  skrl/ppo_visual.yaml              # L0v — global, visual observations
  task/<task>/skrl/ppo.yaml         # L1  — task-specific, state
  task/<task>/skrl/ppo_visual.yaml  # L1v — task-specific, visual

--obs {state,visual} selects the state or visual config. For train and eval, it selects the matching log tree (…/<algo>/ or …/<algo>-visual/). Tune studies use a separate logs/tune/<study>/ path. No visual profile or search space is registered, so visual tuning currently fails.

Study modes

Use srb agent tune check to resolve the algorithm, search space, base config, and --pins without starting Isaac or creating a study. Omit run-only options such as --gpus.

Single-task (default) tunes one task and exports to its task-specific slot:

srb agent tune run --env sample_collection --algo skrl_ppo \
  --trial-steps 20_000_000 --n-trials 40 --n-workers 2 --gpus 0,1

Multi-task (portfolio) studies tune one hyperparameter set across a task set and export it to the global slot:

srb agent tune run --tasks sample_collection peg_in_hole landing --algo skrl_ppo \
  --trial-steps 10_000_000 --n-trials 30 --normalize baseline --aggregate median

Each trial trains sequentially on every task. A diverging segment prunes it; the aggregate can prune weak candidates between tasks.

Cross-task normalization

--normalize maps task scores onto a common scale before --aggregate (median, mean, or min) combines them:

• baseline — (score − floor) / (ref − floor), where floor is a random policy and ref a reference run. Capture both first with srb agent tune baseline --tasks … --algo … --budget <trial-steps>, which writes hyperparams/tune/baselines.yaml. A missing baseline fails fast; a stale one (the source config’s git blob SHA changed since capture) warns.
• rank — per-task fractional rank across completed trials; needs no baselines but is non-stationary early, so it is off by default.
• none (default) — aggregates raw returns; only meaningful when the tasks already share a scale, and the driver warns for a portfolio study otherwise.

Per-task raw and normalized scores are recorded in each trial’s user_attrs (seg/<task>/raw, seg/<task>/norm).

--trial-steps counts environment steps per task. --fps estimates runtime; --max-wall-hours stops starting trials at the limit. Start with a small study before spending a larger compute budget.

Inspecting and exporting

srb agent tune status --study srb-skrl_ppo-state-sample_collection-v1
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task --dry-run
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task

Trials use one seed by default. The command srb agent tune validate --study S --top-k 5 --seeds 4 reruns top trials on fresh seeds and writes validation.json; it does not change trial records. Export a selected trial with export --trial <winner>.

--seeds-per-trial K instead optimizes the mean over K seeds; pruning uses only the first replicate. Validation reranks surviving trials.

export writes parameters and study provenance to the task slot for a single-task study or global slot for a portfolio. It restores production budget and W&B settings. Existing files require --force; export warns if another config will shadow the result. --to task is ambiguous for portfolios.

Parallelism and resuming

• --n-workers N runs concurrent trials; --gpus 0,1 assigns GPUs round-robin.
• Multi-node workers can share logs/tune/<study>/journal.log; see the SLURM example.
• Repeating a command resumes a study. To change its search space, budget, objective, observation mode, or tasks, set a new --version.
• Remove interrupted trials with srb agent tune repair --study S --stale-hours H. Pruned logs are deleted unless --keep-artifacts is set.

Reproducibility

Trial directories preserve configs, seeds, and child metadata for reconstruction, but runs are not guaranteed identical across runtime or hardware changes. See Agent Configuration for tuned-file locations and precedence.