Integration — Reinforcement Learning

Start training with these steps:

1. Install SRB.
2. Complete Basic Usage.
3. Follow the RL Workflow to train and evaluate an agent.
4. Optionally, configure Parallelism for environment count, layout, and stacking constraints.

SRB’s Gymnasium environments support Reinforcement Learning (RL) policy prototyping, training, evaluation, and generalization studies across space robotics tasks with modern frameworks. SRB targets Isaac Sim 6.1.0 and Isaac Lab 3.0.0; see Requirements.

RL can develop autonomous behavior under communication delays or environmental uncertainty. SRB provides tasks from navigation to mobile manipulation, procedural generation and domain randomization, and consistent evaluation metrics. Batched data collection depends on task/provider support. The compatibility matrix covers supported robot categories; see Parallelism for environment-count and stacking constraints.

Available Integrations

Official framework integrations:

• Dreamer [JAX] — Model-based RL.
• Stable-Baselines3 (SB3) [PyTorch] — Popular RL algorithms.
• SBX [JAX] — JAX implementations of popular RL algorithms.
• skrl [PyTorch] - SRB’s adapter supports single-agent RL algorithms through the PyTorch API. It targets the skrl 2.1 API (skrl~=2.1, the Isaac Lab 3.0 profile); other major versions fail early with optional_dependency_unavailable, and checkpoints from skrl 1.x are not supported.
• TD-MPC2 [PyTorch] — Scalable, sample-efficient model-based RL.
• RSL-RL [PyTorch] — Fast on-policy PPO, widely used in legged locomotion research.

Unsupported Frameworks

ACME is unsupported: adapters share Isaac Sim’s Python process, and ACME’s dependencies (dm-reverb, dm-launchpad, and dm-acme[jax]’s jaxlib==0.4.3 pin) do not resolve with the SRB Docker runtime’s Python 3.12.

RL Games and RLinf are not SRB integrations.

Next: Reinforcement Learning Workflow