Space Robotics Bench (SRB) is a comprehensive collection of environments and tasks for robotics research in the challenging domain of space. It provides a unified framework for developing and validating autonomous systems under diverse extraterrestrial scenarios. At the same time, its design is flexible and extensible to accommodate a variety of development workflows and research directions beyond Earth.

Key Features

• Highly Parallelized Simulation via NVIDIA Isaac Sim: SRB supports thousands of parallel simulation instances to accelerate workflows such as online learning, synthetic dataset generation, parameter tuning, and validation.

• On-Demand Procedural Generation with SimForge: Automated procedural generation of simulation assets is leveraged to provide a unique scenario for each simulation instance, with the ultimate goal of developing autonomous systems that are both robust and adaptable to the unpredictable domain of space.

• Extensive Domain Randomization: All simulation instances can be further randomized to enhance the generalization of autonomous agents towards variable environment dynamics, visual appearance, illumination conditions, as well as sensor and actuation noise.

• Compatibility with Gymnasium API: All tasks are compatible with a standardized API to ensure seamless integration with a broad ecosystem of libraries and frameworks for robot learning research.

• Seamless Interface with ROS 2 & Space ROS: Simulation states, sensory outputs and actions of autonomous systems are available through ROS 2 middleware interface, enabling direct interoperability with the vast (Space) ROS ecosystem.

• Abstract Architecture: The architecture of SRB is designed to be modular and extensible, allowing for easy integration of new assets, robots, tasks and workflows

📑 If you have any questions or suggestions regarding this documentation, don't hesitate to reach out to us! More often than not, a lack of understanding is a result of poor documentation... Therefore, we are always looking to improve it.

Table of Contents (available in the left sidebar)

Overview

1. Environments
2. Robots
3. Integrations & Interfaces

Getting Started

4. System Requirements
5. Installation
   • Native
   • Docker (Recommended)
   • Apptainer/Singularity
6. Basic Usage
7. Workflows
   • ROS 2
   • Reinforcement Learning

Configuration

8. Environment Configuration
9. Agent Configuration

Reference

10. Command Line Interface (CLI)
11. Graphical User Interface (GUI)

Development

12. IDE Configuration
13. Dev Container
14. Testing
15. Documentation
16. Utilities

Contributing

17. New Assets
18. New Tasks

Miscellaneous

• Attributions
• Contributors
• Citation
• Community
• Troubleshooting

Environments

The Space Robotics Bench provides a variety of environments that simulate diverse scenarios in the challenging domain of space:

• Manipulation - Stationary manipulation skills and sequences in various contexts
• Mobile Robotics - Traversal and navigation under extraterrestrial conditions
• Mobile Manipulation - Complex combination of mobility and manipulation

Environments — Manipulation (Static)

The Space Robotics Bench provides a wide range of environments that focus on stationary manipulation across different application domains of space robotics.

Sample Collection (sample_collection)

Grasping and collection of samples on planetary surfaces

Sample Collection — Multi (sample_collection_multi)

A variation of sample collection with multiple samples

Debris Capture (debris_capture)

Capture and securement of floating debris in microgravity with a fixed-base manipulator

Peg-in-Hole Assembly (peg_in_hole_assembly)

Precision assembly task requiring the robot to insert a peg into a hole

Peg-in-Hole Assembly — Multi (peg_in_hole_assembly_multi)

A variation of peg-in-hole assembly with multiple pegs and holes

Solar Panel Assembly (solar_panel_assembly)

Complex assembly sequence involving solar panel components

Environments — Mobile Robotics

The mobile robotics environments of the Space Robotics Bench focus on tasks involving locomotion, navigation, and traversal across different terrains and conditions.

Ground Environments

Locomotion Velocity Tracking (locomotion_velocity_tracking)

Locomotion with legged robots and humanoids while tracking desired velocity commands

Aerial Environments

Orbital Environments

Landing (landing)

Landing of spacecraft on planetary surfaces with varying terrain and obstacles

Orbital Evasion (orbital_evasion)

Navigation of spacecraft in microgravity while avoiding obstacles

Environments — Mobile Manipulation

The mobile manipulation environments of the Space Robotics Bench represent a challenging combination of mobility and manipulation capabilities.

Ground-based Mobile Manipulation

Excavation (excavation)

Excavation of granular material (regolith) with a ground-based mobile manipulator

Aerial Mobile Manipulation

Orbital Mobile Manipulation

Mobile Debris Capture (mobile_debris_capture)

Capture of floating debris in microgravity with a floating-base mobile manipulator

Robots

The Space Robotics Bench provides a diverse set of robots for simulating various scenarios:

• Manipulators - Robotic arms
• Mobile Robots - Wheeled rovers, legged robots, aerial vehicles, spacecraft, ...
• Mobile Manipulators - All mobile robot + manipulator combinations, humanoids, ...

Manipulators

The Space Robotics Bench provides a variety of robotic arms for simulating manipulation tasks.

Robotic Arms

Canadarm3 (canadarm3)

Franka Emika (franka)

Kinova JACO2 — 6 DoF (kinova_j2n6s)

Kinova JACO2 — 7 DoF (kinova_j2n7s)

Kinova Gen3 — 7 DoF (kinova_gen3n7)

UR3 (ur3)

UR3e (ur3e)

UR5 (ur5)

UR5e (ur5e)

UR10 (ur10)

UR10e (ur10e)

UR16e (ur16e)

UR20 (ur20)

UR30 (ur30)

Random UR Manipulator (random_ur_manipulator)

Unitree Z1 (unitree_z1)

End-Effectors

Passive End-Effectors

Scoop (scoop)

Active End-Effectors

Allegro Hand (allegro_hand)

Franka Hand (franka_hand)

Kinova Jaco 2 3-Finger Gripper (kinova300)

Shadow Hand (shadow_hand)

Mobile Robots

The Space Robotics Bench provides various mobile robots, including legged robots, wheeled platforms, aerial vehicles, and spacecraft.

Wheeled Robots

Leo Rover (leo_rover)

Perseverance (perseverance)

Pragyan (pragyan)

Legged Robots

Bipeds

Cassie (cassie)

Quadrupeds

ANYmal C (anymal_c)

ANYmal D (anymal_d)

Random ANYmal Quadruped (random_anymal_quadruped)

Spot (spot)

Unitree A1 (unitree_a1)

Unitree Go1 (unitree_go1)

Unitree Go2 (unitree_go2)

Random Unitree Quadruped (random_unitree_quadruped)

Aerial Robots

Multicopters

Crazyflie (crazyflie)

Ingenuity (ingenuity)

Orbital Robots

Spacecraft

CubeSat (cubesat)

Venus Express (venus_express)

Lunar Gateway (gateway)

Lander

Apollo Lander (apollo_lander)

Peregrine Lander (peregrine_lander)

Vikram Lander (vikram_lander)

Random Lander (random_lander)

Mobile Manipulators

The Space Robotics Bench provides mobile manipulator robots that combine locomotion with manipulation capabilities.

Combined Systems

Aerial Manipulator (Combination)

Ground Manipulator (Combination)

Orbital Manipulator (Combination)

Humanoids

Humanoid 21 DOF (humanoid21)

Humanoid 28 DOF (humanoid28)

Unitree H1 (unitree_h1)

Unitree G1 (unitree_g1)

Integrations & Interfaces

The Space Robotics Bench has a modular design that supports integrations & interfaces with external frameworks and tools. This enables researchers and developers to leverage the high-fidelity environments while using their preferred toolchains for research, development, and validation.

At the moment, SRB natively supports the following categories of integrations/interfaces:

• ROS 2 & Space ROS - Interface with the wide ecosystem of robotics tools and libraries
• Reinforcement Learning (RL) - Train and evaluate policies using popular RL algorithms

Interface — ROS 2 & Space ROS

All environments and workflows in the Space Robotics Bench support the ROS 2 interface, which allows you to communicate with each environment instance using standard ROS 2 middleware interfaces. In doing so, you can collect sensor data, control robots, and interact with the simulation instances using the vast ecosystem of ROS tools and libraries.

Motivation

ROS 2 has become the de facto standard for developing robotic systems across various domains. The Space Robotics Bench provides a comprehensive ROS 2 interface that enables seamless integration between the simulation environment and the broader ROS ecosystem, offering several key benefits:

• Compatibility with existing tools - Leverage the rich ecosystem of ROS tools like RViz2, tf2, and rosbag2 for visualization, debugging, and data recording
• Workflow continuity - Develop algorithms and control systems that can transition smoothly from simulation to real hardware using the same interfaces
• Distributed architecture - Take advantage of ROS's node-based architecture to distribute computation and simulation across different processes or even machines
• Community standardization - Utilize standard message types (geometry_msgs, sensor_msgs, ...) that are widely understood and supported in the robotics community

Space ROS

Space ROS is an initiative aimed at extending the ROS ecosystem for applications beyond Earth. The ROS 2 interface of SRB is fully compatible with Space ROS, allowing you to develop and validate autonomous systems across a wide range of diverse extraterrestrial scenarios. In fact, the Space Robotics Bench is a spiritual continuation of the Parallel ProcGen Environments project developed during the NASA Space ROS Sim Summer Sprint Challenge.

You can integrate SRB with Space ROS in two ways:

• Direct: Run the ROS 2 interface of SRB using the Space ROS middleware stack alongside the rest of your Space ROS nodes, which can be achieved simply by sourcing your Space ROS environment before running SRB
• Indirect (recommended): Run the ROS 2 interface of SRB using the standard ROS 2 middleware stack and communicate with your Space ROS nodes using the standard ROS 2 interfaces (no additional setup required)

Implementation

The ROS 2 interface is implemented through a dynamically configured bridge node with the following key features:

• Dynamic ROS interfaces - All publishers, subscribers, and services are created automatically based on the selected simulation environment without any manual configuration
• Parallel environment support - Every parallel simulation instance can be managed through a separate ROS namespace for seamless parallelization, while global control is also available
• Standard message translation - Environment states, sensory outputs, and control commands are automatically translated from simulation tensors to appropriate ROS interfaces
• Complete access - All simulation entities (robots, sensors, objects, ...) are automatically detected, and their interfaces are exposed to ROS alongside task-specific MDP rewards and signals

How to Get Started?

If you want to start using the Space Robotics Bench with ROS 2, follow these steps:

1. Installation
2. Basic Usage
3. ROS 2 Workflow

Integration — Reinforcement Learning

The Space Robotics Bench is designed with Robot Learning research in mind, with a particular emphasis on Reinforcement Learning (RL). All environments follow the standard Gymnasium API, making them compatible with most modern RL frameworks. This architecture enables rapid prototyping, training, and evaluation of policies across diverse space robotics tasks, which makes SRB particularly suitable for comparing the generalization capabilities of novel RL algorithms.

Motivation

RL has emerged as a promising approach for developing autonomous behaviors in complex robotic systems, especially for space applications where manual control may be limited by communication delays or environmental uncertainties. SRB provides a comprehensive set of environments that are specifically designed to facilitate RL research in space robotics, offering several key features:

• Diverse Tasks - Collection of various space robotics tasks that range from simple navigation to complex mobile manipulation
• Parallel Training - All environments support parallel training across multiple instances for efficient data collection
• Procedural Generation & Domain Randomization - Generalization capabilities of RL algorithms can be put to the test using procedurally generated environments that are randomized across multiple dimensions
• Unified Benchmarking - RL algorithms can be compared across a set of reproducible tasks with consistent evaluation metrics

Available Integrations

We provide official integrations with the following RL frameworks:

• Dreamer [JAX] - Model-based RL algorithm
• Stable-Baselines3 (SB3) & SBX [PyTorch & JAX] - Popular implementation of RL algorithms
• skrl [PyTorch & JAX] - Implementation of single- and multi-agent RL algorithms

How to Get Started?

If you want to start using the Space Robotics Bench for Reinforcement Learning, follow these steps:

1. Installation
2. Basic Usage
3. RL Workflow

System Requirements

Hardware Requirements

The hardware requirements for Space Robotics Bench are inherited from the Isaac Sim requirements. With careful tuning, it is possible to run the included environments on lower-spec systems. However, the performance of some workflows might be limited. The bare minimum requirements are listed below:

Software Requirements

A Linux-based OS with an appropriate NVIDIA driver is required to use Space Robotics Bench. Other operating systems might be functional, but they are not officially supported. Please let us know if you confirm functionality on other non-listed systems.

NVIDIA Driver

Official instructions: Driver Installation Guide — Choose an Installation Method

Install the NVIDIA driver by following the official instructions above or through your distribution package manager. Although Isaac Sim specifies 535.129.03 as the recommended version, newer drivers should also be compatible.

... continue with Installation

Installation

Before proceeding, ensure your system meets the system requirements.

Installation Methods

SRB supports three installation methods, each with different trade-offs:

A. Native

• ✅ Full system integration
• ✅ Smooth development experience
• ❗ Complex setup process
• ❗ Potential dependency conflicts

B. Docker (Recommended)

• ✅ Simple installation & deployment
• ✅ Reproducible environment & easy to update
• ⚠️ Moderate development experience (via Dev Containers)
• ❗ Requires privileged access (not suitable for HPC)

C. Apptainer/Singularity

• ✅ Deployable to HPC clusters
• ❗ Uff...

Alternative — Temporary Setup (Quickstart)

For quick experimentation with SRB, you can use a temporary setup that downloads a pre-built Docker image and runs it in a pre-configured container. A single script accomplishes everything, which you can call directly via curl or wget (consider inspecting the script before executing it):

A. curl

ENSURE_DOCKER=true WITH_DEV_VOLUME=false WITH_HISTORY=false bash -c "$(curl -fsSL https://raw.githubusercontent.com/AndrejOrsula/space_robotics_bench/refs/heads/main/.docker/run.bash)" --

B. wget

ENSURE_DOCKER=true WITH_DEV_VOLUME=false WITH_HISTORY=false bash -c "$(wget -qO - https://raw.githubusercontent.com/AndrejOrsula/space_robotics_bench/refs/heads/main/.docker/run.bash)" --

Cleanup of Temporary Setup

If you do not wish to continue using SRB, you can remove the Docker container and its associated image by executing these commands:

docker rm -f space_robotics_bench
docker rmi andrejorsula/space_robotics_bench:latest

Installation — Native

This guide covers installing SRB natively on your system without containerization. Although this approach simplifies development, it requires more manual setup and decreases reproducibility.

1. Clone the Repository

First, clone the SRB repository with all submodules:

git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git

2. Install NVIDIA Isaac Sim 4.5

Official instructions: Isaac Sim — Workstation Installation

Install Isaac Sim either by following the official instructions above or using the provided convenience script:

./space_robotics_bench/scripts/install_isaacsim.bash "$HOME/isaac-sim"

ISAAC_SIM_PYTHON

It is highly recommended to make ISAAC_SIM_PYTHON point to the Python entrypoint of Isaac Sim in your shell configuration (the script above will prompt you to do so):

A. bash

echo "export ISAAC_SIM_PYTHON='$HOME/isaac-sim/python.sh'" >> ~/.bashrc
source ~/.bashrc

B. zsh

echo "export ISAAC_SIM_PYTHON='$HOME/isaac-sim/python.sh'" >> ~/.zshrc
source ~/.zshrc

C. fish

set -Ux ISAAC_SIM_PYTHON "$HOME/isaac-sim/python.sh"

3. Install NVIDIA Isaac Lab 2.0

Official instructions: Isaac Lab — Installation

Install Isaac Lab either by following the official instructions above or using the provided convenience script:

./space_robotics_bench/scripts/install_isaaclab.bash "$HOME/isaaclab"

4. Install Blender 4.3 with SimForge

Official instructions: Blender — Install from blender.org

1. Install Blender by following the official instructions for downloading and extracting its archive.
2. Ensure that the blender executable is accessible from your system's PATH.
3. Install SimForge with its assets within the Blender Python environment.

As an example, the commands below will install Blender in your home directory and create a symbolic link to $HOME/.local/bin/blender (assuming it is in your PATH):

export BLENDER_VERSION="4.3.2"
export BLENDER_VERSION_SHORT=$(echo $BLENDER_VERSION | sed 's/\.[^.]*$//')
mkdir -p $HOME/blender
curl -fsSL "https://download.blender.org/release/Blender$BLENDER_VERSION_SHORT/blender-$BLENDER_VERSION-linux-x64.tar.xz" | tar xJ -C $HOME/blender --strip-components=1
ln -sf $HOME/blender/blender $HOME/.local/bin/blender
"$HOME/blender/$BLENDER_VERSION_SHORT/python/bin/python3.11" -m pip install simforge[assets]

5. Install the Space Robotics Bench

Install the srb package in editable mode:

"$ISAAC_SIM_PYTHON" -m pip install --editable ./space_robotics_bench[all]

Note: The all extra installs optional dependencies to support all workflows and improve usability. Feel free to check pyproject.toml and adjust the extras to your needs.

Setup CLI

Make the srb CLI command available in your shell through the provided script:

./space_robotics_bench/scripts/setup_cli.bash

Note: In case the script fails, the srb CLI is still accessible via "$ISAAC_SIM_PYTHON" -m srb.

6. Verify Installation

After the installation, verify that everything works as expected.

Isaac Sim

Confirm that you can launch Isaac Sim:

"$HOME/isaac-sim/isaac-sim.sh"

Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.

Isaac Lab

Confirm that Isaac Lab is installed:

"$ISAAC_SIM_PYTHON" -m pip show isaaclab

Space Robotics Bench

Verify that the entrypoint script of SRB is available in the Python environment of Isaac Sim:

"$ISAAC_SIM_PYTHON" -m srb --help

Verify that the srb command is available:

srb --help

Verify that argument completion works:

srb <TAB> <TAB>

... continue with Basic Usage

Extras

Development

To improve your development experience, consider configuring your IDE (guide).

Installation — Docker (Recommended)

Using SRB inside Docker is recommended for most users, as it provides an isolated, reproducible environment that is fully pre-configured.

1. Clone the Repository

First, clone the SRB repository with all submodules:

git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git

2. Install Docker Engine & NVIDIA Container Toolkit

1. Official instructions: Install Docker Engine
2. Official instructions: Linux post-installation steps for Docker Engine
3. Official instructions: Installing the NVIDIA Container Toolkit

Install Docker Engine and NVIDIA Container Toolkit either by following the official instructions above or using the provided convenience script:

./space_robotics_bench/.docker/host/install_docker.bash

3. Run

Now, you can run the Docker container with the provided script. The first run will automatically pull the latest image from Docker Hub:

./space_robotics_bench/.docker/run.bash

4. Verify Installation

Once you enter the Docker container, verify that everything works as expected.

Isaac Sim

Confirm that you can launch Isaac Sim:

"$HOME/isaac-sim/isaac-sim.sh"

Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.

Space Robotics Bench

Verify that the srb command is available:

srb --help

... continue with Basic Usage

Extras

Build a New Docker Image

If you want to build a custom Docker image, you can use the provided script:

./space_robotics_bench/.docker/build.bash

Join a Running Container

To join a running container from another terminal, use the provided script:

./space_robotics_bench/.docker/join.bash

Development

The repository workspace is automatically mounted inside the Docker container, so you can edit the code either on the host or inside the container, and the changes will be persistently reflected in both environments.

To improve your development experience, you can open the project as a Dev Container (guide).

Installation — Apptainer/Singularity

You can use SRB on HPC clusters or systems where Docker is not available by leveraging Apptainer. Apptainer is a containerization tool similar to Docker but designed for environments with stricter security policies or without Docker support.

Setup (Local)

1. Clone the Repository

First, clone the SRB repository with all submodules:

git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git

2. Install Docker Engine & NVIDIA Container Toolkit

1. Official instructions: Install Docker Engine
2. Official instructions: Linux post-installation steps for Docker Engine
3. Official instructions: Installing the NVIDIA Container Toolkit

Install Docker Engine and NVIDIA Container Toolkit either by following the official instructions above or using the provided convenience script:

./space_robotics_bench/.docker/host/install_docker.bash

3. Install Apptainer

Official instructions: Installing Apptainer

Install Apptainer by following the official instructions above. For instance, you can use these commands on Ubuntu:

sudo add-apt-repository -y ppa:apptainer/ppa
sudo apt update
sudo apt install -y apptainer

4. Build the Apptainer Image

Now you can build the Apptainer image with the provided script:

./space_robotics_bench/.docker/hpc/build.bash

This will create an Apptainer image space_robotics_bench.sif inside the ./space_robotics_bench/.docker/hpc/images/ directory.

Deployment (HPC Cluster)

1. Transfer the Repository (with the Apptainer Image)

Transfer the local SRB repository to the HPC cluster using your preferred method (e.g., scp or rsync):

A. scp

scp -r ./space_robotics_bench user@hpc-cluster:/path/to/space_robotics_bench

B. rsync

rsync -avr ./space_robotics_bench user@hpc-cluster:/path/to/space_robotics_bench

2. SSH into the HPC Cluster

SSH into the HPC cluster:

ssh user@hpc-cluster

3. Run the Apptainer Image

Now you can run the Apptainer image with the provided script inside an interactive session (you might need make some adjustments based on your HPC environment):

# HPC interactive session
/path/to/space_robotics_bench/.docker/hpc/run.bash bash

4. Verify Installation

Once you enter the Apptainer container, verify that everything works as expected.

Isaac Sim

Confirm that you can launch Isaac Sim:

# Inside Apptainer container
"$HOME/isaac-sim/isaac-sim.sh"

Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.

Space Robotics Bench

Verify that the srb command is available:

# Inside Apptainer container
srb --help

... continue with Basic Usage

Extras

Schedule a SLURM Job

For long-running tasks or automated workflows, you can schedule a SLURM job that will automatically run the Apptainer image. It is highly recommended that you adjust the script to your needs before submitting the job:

# HPC login node
/path/to/space_robotics_bench/.docker/hpc/submit.bash [CMD]

Basic Usage

After successful installation, you are now ready to explore the Space Robotics Bench. This guide covers the essentials for getting started with the framework.

"$ISAAC_SIM_PYTHON" -m srb

1. List Registered Assets & Environments

Reference: srb ls — List Assets and Environments

As a first step, it is recommended that you list all registered assets, action groups, and tasks to get an overview of what SRB has to offer:

srb ls

After a while, you should see 3 tables printed in the terminal:

┏━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Name         ┃ Type    ┃ Subtype            ┃ Parent Class      ┃ Asset Config    ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ mars_surface │ scenery │ terrain            │ Terrain           │ AssetBaseCfg    │
│ sample_tube  │ object  │ common             │ Object            │ RigidObjectCfg  │
│ scoop        │ object  │ tool               │ Tool              │ RigidObjectCfg  │
│ shadow_hand  │ object  │ tool               │ ActiveTool        │ ArticulationCfg │
│ ur10         │ robot   │ manipulator        │ SerialManipulator │ ArticulationCfg │
│ cubesat      │ robot   │ mobile_robot       │ OrbitalRobot      │ RigidObjectCfg  │
│ ingenuity    │ robot   │ mobile_robot       │ Multicopter       │ ArticulationCfg │
│ perseverance │ robot   │ mobile_robot       │ WheeledRobot      │ ArticulationCfg │
│ unitree_g1   │ robot   │ mobile_manipulator │ Humanoid          │ ArticulationCfg │
│ ...          │ ...     │ ...                │ ...               │ ...             │
└──────────────┴─────────┴────────────────────┴───────────────────┴─────────────────┘

class MarsSurface(Terrain):
    ## Scenario - The asset is suitable for the Mars domain
    DOMAINS: ClassVar[Sequence[Domain]] = (Domain.MARS,)

    ## Model - Static asset
    asset_cfg: AssetBaseCfg = AssetBaseCfg(
        prim_path="{ENV_REGEX_NS}/mars_surface",
        ## Spawner procedurally generates SimForge models
        spawn=SimforgeAssetCfg(
            assets=[simforge_foundry.MarsSurface()],
            collision_props=CollisionPropertiesCfg(),
        ),
    )

class SampleTube(Object):
    ## Model - Rigid object affected by physics
    asset_cfg: RigidObjectCfg = RigidObjectCfg(
        prim_path="{ENV_REGEX_NS}/sample",
        ## Spawner loads a static USD file
        spawn=UsdFileCfg(
            usd_path=(
                SRB_ASSETS_DIR_SRB_OBJECT.joinpath("sample_tube.usdz").as_posix()
            ),
            collision_props=CollisionPropertiesCfg(),
            mesh_collision_props=MeshCollisionPropertiesCfg(
                mesh_approximation="convexDecomposition"
            ),
            rigid_props=RigidBodyPropertiesCfg(),
            mass_props=MassPropertiesCfg(density=1500.0),
        ),
    )

class Franka(SerialManipulator):
    ## Model - Articulation with several links connected by joints
    asset_cfg: ArticulationCfg = ArticulationCfg(
        prim_path="{ENV_REGEX_NS}/franka",
        ## Spawner loads a static USD file
        spawn=UsdFileCfg(
            usd_path=SRB_ASSETS_DIR_SRB_ROBOT.joinpath("manipulator")
            .joinpath("franka_arm.usdz")
            .as_posix(),
            ...
        ),
        ## Initial joint configuration of the robot
        init_state=ArticulationCfg.InitialStateCfg(
            joint_pos={
                "panda_joint1": 0.0,
                "panda_joint2": 0.0,
                "panda_joint3": 0.0,
                "panda_joint4": deg_to_rad(-90.0),
                "panda_joint5": 0.0,
                "panda_joint6": deg_to_rad(90.0),
                "panda_joint7": deg_to_rad(45.0),
            },
        ),
        ...
    )
    ## End effector - The default hand is separate to allow for easy replacement
    end_effector: Tool | None = FrankaHand()

    ## Actions - Inverse Kinematics action group that drives all joints
    actions: ActionGroup = InverseKinematicsActionGroup(
        DifferentialInverseKinematicsActionCfg(
            asset_name="robot",
            joint_names=["panda_joint[1-7]"],
            base_name="panda_link0",
            body_name="panda_link7",
            controller=DifferentialIKControllerCfg(
                command_type="pose",
                use_relative_mode=True,
                ik_method="svd",
            ),
            scale=0.1,
            body_offset=DifferentialInverseKinematicsActionCfg.OffsetCfg(),
        ),
    )

    ## Frames - Relevant frames for attaching the robot and mounting tool/sensors
    frame_base: Frame = Frame(prim_relpath="panda_link0")
    frame_flange: Frame = Frame(
        prim_relpath="panda_link7",
        offset=Transform(
            pos=(0.0, 0.0, 0.107),
            rot=rpy_to_quat(0.0, 0.0, -45.0),
        ),
    )
    frame_wrist_camera: Frame = Frame(
        prim_relpath="panda_link7/camera_wrist",
        offset=Transform(
            pos=(0.075, -0.075, 0.1),
            rot=rpy_to_quat(0.0, -80.0, 135.0),
        ),
    )

┏━━━━━━━━━━━━━━━━━━━━┓
┃ Name               ┃
┡━━━━━━━━━━━━━━━━━━━━┩
│ body_acceleration  │
│ joint_position     │
│ joint_velocity     │
│ joint_effort       │
│ inverse_kinematics │
│ ...                │
└────────────────────┘

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID                               ┃ Entrypoint                   ┃ Config                             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ _manipulation <template>         │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │
│ sample_collection                │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │
│ _aerial <template>               │ Task(AerialEnv)              │ TaskCfg(AerialEnvCfg)              │
│ _ground <template>               │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │
│ _orbital <template>              │ Task(OrbitalEnv)             │ TaskCfg(OrbitalEnvCfg)             │
│ locomotion_velocity_tracking     │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │
│ _aerial_manipulation <template>  │ Task(AerialManipulationEnv)  │ TaskCfg(AerialManipulationEnvCfg)  │
│ _ground_manipulation <template>  │ Task(GroundManipulationEnv)  │ TaskCfg(GroundManipulationEnvCfg)  │
│ _orbital_manipulation <template> │ Task(OrbitalManipulationEnv) │ TaskCfg(OrbitalManipulationEnvCfg) │
│ ...                              │ ...                          │ ...                                │
└──────────────────────────────────┴──────────────────────────────┴────────────────────────────────────┘

2. Teleoperate your 1^st Robot across Diverse Domains

Reference: srb agent teleop — Teleoperate Agent

Let's start with the sample_collection environment where you can manually control the Franka manipulator through your keyboard to collect samples on the Moon:

srb agent teleop --env sample_collection

Eventually, Isaac Sim will open with the selected environment, and you will be greeted in your terminal with a schematic of the teleoperation interface.

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃  Keyboard Scheme (focus the Isaac Sim window)  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Reset: [ L ]                                   │
│ Decrease Gain [ O ]   │ Increase Gain: [ P ]   │
│ Event: [ R / K ]                               │
├────────────────────────────────────────────────┤
│ Translation                                    │
│             [ W ] (+X)            [ Q ] (+Z)   │
│               ↑                     ↑          │
│               │                     │          │
│  (-Y) [ A ] ←─┼─→ [ D ] (+Y)        ┼          │
│               │                     │          │
│               ↓                     ↓          │
│             [ S ] (-X)            [ E ] (-Z)   │
├────────────────────────────────────────────────┤
│ Rotation                                       │
│       [ Z ] ←————————(±X)————————→ [ X ]       │
│                                                │
│       [ T ] ↻————————(±Y)————————↺ [ G ]       │
│                                                │
│       [ C ] ↺————————(±Z)————————↻ [ V ]       │
└────────────────────────────────────────────────┘

Note: Most tasks employ action spaces that support direct teleoperation (e.g. via Inverse Kinematics). However, some tasks such as locomotion_velocity_tracking rely on low-level control of individual joints. In this case, direct teleoperation is not supported, and you will need to provide a control policy that maps your teleoperation commands to low-level control signals. Further instructions are provided in the section for Teleoperation via Policy.

Move to a Different Domain

Reference: Environment Configuration
Reference: Environment Configuration — Domain

What if we want to collect samples on Mars instead? Luckily, you can easily configure many aspects of the environment through Hydra. For instance, you can adjust the domain and sample asset in order to simulate the Mars Sample Return mission:

srb agent teleop --env sample_collection env.domain=mars env.sample=sample_tube

3. Observe Random Agents in Action

Reference: srb agent rand — Random Agent

Now, let's observe an environment where agents act based on random actions sampled uniformly from their action space, which is particularly useful for verifying that environments function as intended. To demonstrate a random agent, we will use the locomotion_velocity_tracking task that uses the Spot quadruped by default:

srb agent rand --env locomotion_velocity_tracking

Hint: Use --hide_ui option to disable most of the Isaac Sim UI, as shown in the video above.

Change the Robot

Reference: Environment Configuration — Robot

Selecting a different robot for any environment is as simple as adjusting the env.robot parameter. This particular environment supports all legged robots, so let's try the Unitree G1 humanoid:

srb agent rand --env locomotion_velocity_tracking env.robot=unitree_g1

Simulate Multiple Robots

Reference: Environment Configuration — Parallelism

Many workflows benefit from running multiple parallel simulation instances. This can be achieved with the env.num_envs parameter. For instance, let's try 16 instances of the Cassie biped:

srb agent rand -e locomotion_velocity_tracking env.robot=cassie env.num_envs=16

Hint: As you can see in the video above, all 16 robots share the same terrain. This is the default behaviour of locomotion_velocity_tracking (the default behaviour is task-specific). However, you can easily create a unique terrain for each robot by setting env.stack=false. This will automatically trigger the generation of 16 unique assets with different geometry and materials. Here, we speed up the procedural generation by disabling texture baking with SF_BAKER=0:

SF_BAKER=0 srb agent rand -e locomotion_velocity_tracking env.stack=false env.num_envs=16

4. Explore & Experiment with Environment Templates

Reference: Robots
Reference: Environment Configuration — Robot

Both sample_collection and locomotion_velocity_tracking are examples of tasks that implement specific goal-oriented scenarios. However, SRB also provides a set of environment templates that can serve as a foundation for exploring and experimenting with custom scenarios.

In general, each robot category has its own template:

With this in mind, let's explore the _ground_manipulation template that combines the mobile Spot quadruped with Franka manipulator into an integrated mobile manipulation system:

srb agent rand -e _ground_manipulation

Diversify Robot Configurations

Robot configurations can also be randomized across multiple parallel instances. For example, you can combine a random Unitree quadruped (random_unitree_quadruped) with a random Universal Robots manipulator (random_ur_manipulator), ranging all the way from UR3 to UR30! For mobile manipulators, changing the mobile base and manipulator is separated into two parameters for more flexibility, namely env.robot.mobile_base and env.robot.manipulator:

srb agent rand -e _ground_manipulation env.robot.mobile_base=random_unitree_quadruped env.robot.manipulator=random_ur_manipulator env.num_envs=6 env.stack=true

Customize Payloads & End Effectors

Reference: srb agent zero — Zero Agent

Modifying only the robot might not be enough for your envisioned scenario. You might also want to customize the payload of mobile robots or the end effector of manipulators. Similar to previous examples, this can also be configured via env.robot for mobile robots and manipulators; or env.robot.mobile_base and env.robot.manipulator for mobile manipulators. The configuration is context-aware, and you can specify payloads and end effectors by separating them with a + sign, i.e. mobile_base+payload or manipulator+end_effector. For example, let's combine Unitree Z1 manipulator with Shadow Hand end effector on top of Anymal D quadruped with the Cargo Bay payload:

srb agent zero -e _ground_manipulation env.robot.mobile_base=anymal_d+cargo_bay env.robot.manipulator=unitree_z1+shadow_hand

Hint: If you only wish to change the payload or end effector but keep the rest of the robot configuration the same, +payload and +end_effector are also valid inputs. Similarly, mobile_robot+ and manipulator+ will maintain the original payload or end effector.

And while the results might look ridiculous, the same level of customization is available across the board for all domains. Furthermore, aspects such as the pose of sensors and dimensionality of action spaces are adjusted automatically.

srb agent zero -e _aerial_manipulation env.robot.mobile_base=ingenuity env.robot.manipulator=+scoop

srb agent zero -e _orbital_manipulation env.robot.mobile_base=gateway env.robot.manipulator=canadarm3+

5. Enable Visual Sensors

To maintain the performance of simulation, all visual sensors are disabled by default. However, you might have noticed that all listed environments also have their *_visual variant (e.g. _aerial -> _aerial_visual). This variant enables a number of pre-configured cameras that provide both RGB and depth images (available via observations and middleware communication).

Let's see a camera view of the Ingenuity helicopter on Mars:

srb agent teleop -e _aerial_visual

What's Next?

Everything you learned so far is just the tip of the iceberg, but it is applicable to all environments and workflows within the Space Robotics Bench. Yet, diving deeper into the codebase will allow you to customize and extend the environments further to suit your specific needs.

Depending on your interests, you are welcome to explore one or more of the following guides:

• Workflows: ROS 2
• Workflows: Reinforcement Learning
• Contributing — New Assets
• Contributing — New Tasks

Workflows

The Space Robotics Bench provides a variety of workflows that streamline the interaction with the simulation environments to accomplish specific goals like algorithm development, validation, or synthetic data generation.

Available Workflows

• ROS 2 - Interface with the wide ecosystem of robotics tools and libraries
• Reinforcement Learning - Train and evaluate policies using popular RL algorithms

ROS 2 Workflow

The ROS 2 interface of SRB allows you to interact with the simulation environment using standard ROS 2 middleware interfaces. This enables you to develop and validate autonomous systems across a wide range of diverse scenarios through seamless integration with the rest of your ROS 2 or Space ROS nodes.

There are two options for using the ROS 2 interface of SRB:

1. Active ROS 2 Agent: A specific srb agent ros workflow that enables full control over the simulation through the ROS 2 middleware, including actions of robots (control commands)
2. Passive ROS 2 Interface: Any workflow can be configured to use the ROS 2 interface for publishing all available data and interacting with the simulation through services, but the actions are controlled via the selected workflow (teleoperation, random agent, RL training and evaluation, ...)

1. Active ROS 2 Agent

This workflow is specifically designed for full control over the simulation, including the actions of robots (control commands). It is available via the srb agent ros command, and works with all SRB environments. Let's try it with the peg_in_hole_assembly_visual task:

srb agent ros -e peg_in_hole_assembly_visual

List Topics

Once the simulation is running, you can open another terminal and list all available ROS 2 topics:

ros2 topic list

Hint: For Docker installation, you can use .docker/join.bash to enter the running container if you don't have a local ROS 2 setup available.

/clock
/parameter_events
/rosout
/srb/env0/action/cmd_vel
/srb/env0/action/event
/srb/env0/cam_base/camera_info
/srb/env0/cam_base/image_depth
/srb/env0/cam_base/image_rgb
/srb/env0/cam_base/pointcloud
/srb/env0/cam_scene/camera_info
/srb/env0/cam_scene/image_depth
/srb/env0/cam_scene/image_rgb
/srb/env0/cam_scene/pointcloud
/srb/env0/cam_wrist/camera_info
/srb/env0/cam_wrist/image_depth
/srb/env0/cam_wrist/image_rgb
/srb/env0/cam_wrist/pointcloud
/srb/env0/end_effector/joint_pos
/srb/env0/end_effector/joint_states
/srb/env0/reward
/srb/env0/reward/penalty_action_rate
/srb/env0/reward/penalty_undesired_robot_contacts
/srb/env0/reward/reward_align_peg_to_hole_primary
/srb/env0/reward/reward_align_peg_to_hole_secondary
/srb/env0/reward/reward_distance_end_effector_to_obj
/srb/env0/reward/reward_distance_peg_to_hole_bottom
/srb/env0/reward/reward_distance_peg_to_hole_entrance
/srb/env0/reward/reward_grasp
/srb/env0/reward/reward_lift
/srb/env0/robot/delta_twist
/srb/env0/robot/joint_states
/srb/env0/terminated
/srb/env0/truncated
/srb/envs/action/cmd_vel
/srb/envs/action/event
/srb/envs/end_effector/joint_pos
/srb/envs/robot/delta_twist
/tf
/tf_static

Most topics fall under two namespaces:

• /srb/env[0-N]/...: Environment-specific topics that are unique to each environment instance
• /srb/envs/...: Global topics that apply to all environment instances

Let's break down the most important topics:

Subscribe to Topics

You can subscribe to topics to receive updates from the simulation. For example, you can echo the reward signal of the first environment instance:

ros2 topic echo /srb/env0/reward

data: 2.36441707611084
---
data: 2.3652756214141846
---
data: 2.36613130569458
---
data: 2.3669850826263428
---

Publish to Topics

You can also send control commands to the simulation by publishing to topics. For example, you can move robots in all environment instances by publishing a twist command:

ros2 topic pub --once /srb/envs/action/cmd_vel geometry_msgs/Twist "{linear: {z: -0.2}, angular: {z: 0.785}}"

2. Passive ROS 2 Interface

Alternatively, you can use the ROS 2 interface with any agent type by specifying the --interface ros flag. This allows you to collect sensor data and interact with the simulation instances while controlling the agent through the selected workflow. Let's try it with the rand agent in the sample_collection_multi_visual environment with 8 parallel instances:

srb agent rand -e sample_collection_multi_visual env.num_envs=8 --interface ros

Note: No Hydra overrides can be specified directly after the --interface X option because it accepts one or more interfaces.

Call Services

Upon listing the available services, you will see that each environment instance has a /srb/env{i}/reset service, and a global /srb/envs/reset service.

ros2 service list

You can call these services to reset a specific environment instance or all instances at once:

ros2 service call /srb/env0/reset std_srvs/srv/Empty

ros2 service call /srb/envs/reset std_srvs/srv/Empty

Reinforcement Learning Workflow

Reinforcement Learning (RL) is one of the primary focus areas of the Space Robotics Bench. While there are several RL frameworks with their unique peculiarities, SRB offers a unified interface for training and evaluating policies across a diverse set of space robotics tasks.

1. Train your 1^st RL Agent

Reference: srb agent train — Train Agent

The fastest way to get started with training an RL agent is by using the srb agent train command, which provides a streamlined interface for all integrated RL frameworks. In general, you want to specify the RL algorithm to use, the environment to train on, and the number of parallel environment instances used for rollout collection.

Let's start with a simple TODO_ENV environment using the TODO_ALGO algorithm. For now, omit the --headless flag so that you can observe the convergence in real time:

srb agent train --algo TODO_ALGO --env TODO_ENV env.num_envs=512

As you begin to observe the training process, you can also monitor the progress in your terminal. After about TODO_CONVERGENCE timesteps, you will see that the agent found a stable policy that successfully solves the task. Checkpoints are saved regularly, so you are free to stop the training process at any point by sending an interrupt signal (Ctrl+C in most terminals).

2. Evaluate your Agent

Reference: srb agent eval — Evaluate Agent

Once training is complete, you can evaluate your agent with the srb agent eval command:

srb agent eval --algo TODO_ALGO --env TODO_ENV env.num_envs=16

By default, the latest checkpoint from the training run is loaded for evaluation. However, you might want to run the evaluation for a checkpoint specified via --model:

srb agent eval --algo TODO_ALGO --env TODO_ENV env.num_envs=16 --model space_robotics_bench/logs/TODO_ENV/TODO_ALGO/TODO_CHECKPOINT

3. Try a Different Algorithm

SRB directly supports several popular RL algorithms from different frameworks:

This time, you can train another agent using an algorithm of your choice:

srb agent train --headless --algo <ALGO> --env TODO_ENV env.num_envs=1024

Hint: Use --headless mode with more parallel environments for faster convergence.

4. Monitor Training Progress

While training, you might be interested in monitoring the progress and comparing different runs through a visual interface. By default, TensorBoard logs are saved for all algorithms and environments in the space_robotics_bench/logs directory. You can start TensorBoard to visualize the training progress:

tensorboard --logdir ./space_robotics_bench/logs --bind_all

Furthermore, you can enable Weights & Biases (wandb) logging by passing framework-specific flags [subject to future standardization]:

• DreamerV3: srb agent train ... +agent.logger.outputs=wandb
• SB3 & SBX: srb agent train ... agent.track=true
• skrl: srb agent train ... +agent.experiment.wandb=true

Note: Logging to Weights & Biases requires an account and API key.

5. Configure Hyperparameters

Reference: Agent Configuration

The default hyperparameters for all algorithms and environments are available under the space_robotics_bench/hyperparams directory. Similar to the environment configuration, you can adjust the hyperparameters of the selected RL algorithm through Hydra. However, the available hyperparameters and their structure is specific to each framework and algorithm.

Here are some examples (consult hyperparameter configs for more details):

srb agent train --algo dreamer      agent.run.train_raio=128   ...
sed agent train --algo sb3_ppo      agent.gamma=0.99           ...
srb agent train --algo sbx_sac      agent.learning_rate=0.0002 ...
srb agent train --algo skrl_ppo_rnn agent.models.separate=True ...

Environment Configuration

The Space Robotics Bench provides a flexible configuration system for environments through Hydra. This document explains how to customize environment parameters across different domains and tasks.

How Does It Work?

Each SRB environment is registered alongside its Python configuration class, which defines the default parameters for that specific environment. All environment configuration classes are organized in a hierarchical structure via inheritance, where BaseEnvCfg does most of the heavy lifting. This design supports a modular and extensible configuration system that allows for easy customization of environment parameters.

Modifying Configurations

You can modify environment configurations in several ways:

1. Command-Line Overrides

The most direct way to modify environment parameters is through command-line overrides:

srb agent <WORKFLOW> --env <ENV> \
    env.domain=moon \
    env.robot=ur10 \
    env.scene.num_envs=4 \
    env.stack=false \
    ...

2. Configuration Files

Key Parameters

Below are the most important configuration parameters organized by category (several parameters have their own detailed documentation pages):

Scenario/Environment

• env.domain - Domain
• env.robot - Robot
• env.scene.num_envs, env.stack - Parallelism
• env._particles - Particles

Simulation

• env.sim - Low-level simulation parameters (physics, rendering, etc.)
• env.visuals - Visual appearance settings

Debugging

• env.debug_vis - Enables debug visualization features

Environment Configuration — Parallelism

One of Space Robotics Bench's most powerful features is parallel simulation, allowing you to run multiple simulation instances simultaneously. This capability is critical for:

• Reinforcement Learning: Collect experience at scale for faster training
• Parameter Tuning: Test multiple configurations simultaneously
• Monte Carlo Sampling: Evaluate robustness across varied scenarios
• Batch Processing: Process multiple scenarios in a single run

Number of Environments

The env.scene.num_envs parameter controls how many parallel simulation instances are created. This parameter is aliased as env.num_envs for brevity:

srb agent rand -e _manipulation env.scene.num_envs=16
srb agent zero -e _manipulation env.num_envs=128

Each environment is a fully independent physics simulation instance, with all cross-environment interactions disabled (filtered collisions). However, there is only one instance of the rendering engine, which means that a visual sensor in one environment will see entities from all environments. A simple workaround for this limitation is to increase the spacing between environments and clip the maximum sensor range.

Environment Spacing

The env.scene.env_spacing parameter controls the spatial distance between environments. This parameter is aliased as env.spacing for brevity:

srb agent rand -e _manipulation env.num_envs=64 env.scene.spacing=20.0
srb agent rand -e _manipulation env.num_envs=32 env.spacing=10.0

Environment Stacking

The env.stack parameter controls whether environments share assets or have independent assets:

Independent Environments (env.stack=false) [Default]

• ⚠️ Each environment has a unique scene and position
• ✅ Greater visual and physical diversity
• ✅ Supports tasks with visual sensors
• ❗ Slower to initialize
• ❗ Higher memory usage

srb agent rand -e _manipulation env.scene.num_envs=16 env.stack=false

Stacked Environments (env.stack=true)

• ⚠️ All environments share the same scenery and position
• ✅ Faster to initialize
• ✅ Lower memory usage
• ❗ Less environmental diversity
• ❗ Does not support tasks with visual sensors

srb agent rand -e _manipulation env.scene.num_envs=16 env.stack=true

Note: Non-visual locomotion_velocity_tracking task defaults to env.stack=true.

Environment Configuration — Domain

The env.domain parameter configures the planetary or orbital environment for your simulation. This affects gravity, lighting conditions, terrain type, atmospheric properties, and visual appearance.

Available Domains

Space Robotics Bench currently supports the following domains:

Usage

You can specify the domain via the command line:

# Run a simulation on Mars
srb agent teleop --env sample_collection env.domain=mars

Domain-Specific Effects

Each domain affects various aspects of the simulation:

Physical Properties

• Gravity magnitude: Affects object dynamics, robot mobility and manipulation requirements

Visual Properties

• Lighting: Light intensity, color temperature, and angular diameter
• Skydome: Background appearance and intensity
• Atmosphere: Fog intensity and color gradient (if applicable)

Asset Selection

Some assets are domain-specific and will only appear for compatible domains. When an asset is registered, it can specify the domains it supports through the DOMAINS class attribute.

Environment Configuration — Robot

The env.robot parameter allows you to specify which robot to use in your simulation. This is one of the most powerful configuration options, enabling you to change not just the robot model but also its components, payloads, and end effectors.

Robot Selection

The simplest way to select a robot is by specifying its name:

srb agent rand -e _manipulation env.robot=ur10

srb agent rand -e _ground env.robot=unitree_go1

srb agent rand -e _aerial env.robot=crazyflie

End Effector for Manipulators

You can attach specific end effectors to manipulators using the + syntax:

srb agent rand -e _manipulation env.robot=ur10+shadow_hand

To keep the default end effector while changing the base manipulator:

srb agent rand -e _manipulation env.robot=unitree_z1+

To only change the end effector:

srb agent rand -e _manipulation env.robot=+shadow_hand

Payload for Mobile Robots

Similarly, you can attach payloads to mobile robots:

srb agent rand -e _ground env.robot=spot+cargo_bay

To keep the default payload while changing the base mobile robot:

srb agent rand -e _ground env.robot=anymal_d+

To only change the payload:

srb agent rand -e _ground env.robot=+cargo_bay

Combine Mobile Manipulators

SRB supports combining any mobile robot with any manipulator for mobile manipulation environments via the env.robot.mobile_base and env.robot.manipulator parameters:

srb agent rand -e _ground_manipulation env.robot.mobile_base=unitree_a1 env.robot.manipulator=unitree_z1

Similarly, you can attach payloads to the mobile base and end effectors to the manipulator:

srb agent rand -e _ground_manipulation env.robot.mobile_base=anymal_d+cargo_bay env.robot.manipulator=ur10+shadow_hand

Change Action Groups

Each robot and active tool (end-effector) has its associated action group that is automatically manifested in the action space of the whole system. You can change the action group of the robot or active tool using the env.robot.actions parameter:

srb agent rand -e _manipulation env.robot.actions=joint_position_relative

Note: It is currently not possible to override both the robot and its action group via their registered names at the same time.

Environment Configuration — Particles

Space Robotics Bench can simulate liquid and granular materials like regolith, sand, and dust using particle-based physics. This is particularly relevant for space applications where interaction with loose granular material is common.

Enabling Particles

You can enable particles in any environment by setting the _particles parameter to true:

srb agent teleop -e _manipulation env._particles=true env.robot=+scoop

Particle Configuration Parameters

srb agent teleop -e _manipulation env._particles=true env._particles_size=0.01 env._particles_ratio=0.1

Particle Behavior

By default, the particle system uses a pyramid distribution to create natural-looking piles of granular material with higher density at the center. Particles interact with other objects through physical collision and settle over time due to gravity. Robots can push, scoop, or otherwise interact with particles.

Note: When particles are enabled, Fabric is disabled via env.sim.use_fabric=false.

Agent Configuration

The Space Robotics Bench leverages Hydra for managing agent configurations across different robot learning frameworks. This document provides an overview of the configuration structure and common parameters for training agents in SRB.

Default Hyperparameters

The default hyperparameters for all algorithms and environments are available under the space_robotics_bench/hyperparams directory. If you do not wish to use Hydra, you can directly modify these files before training your agent.

Note: The available hyperparameters and their structure are specific to each framework and algorithm.

Modifying Configurations

You can modify agent configurations in several ways:

1. Command-Line Overrides

srb agent <WORKFLOW> --algo <ALGO> --env <ENV> \
    agent.learning_rate=0.0001 \
    agent.batch_size=64 \
    ...

2. Configuration Files

Command Line Interface (CLI)

The Space Robotics Bench provides a comprehensive command-line interface that allows you to interact with most aspects of the framework through a unified interface. The CLI is structured as a set of subcommands, each with its own set of options.

srb <subcommand> [options]

If you need help with a specific subcommand, you can use the -h/--help flag:

srb --help
srb <subcommand> -h

You can also TAB your way through the available subcommands and options via autocompletion:

srb <TAB> <TAB>

Subcommands

The following subcommands are available in the CLI:

Agent Subcommands

The agent subcommand is further separated into agent-specific subcommands:

srb agent <agent_subcommand> [options]

Shared Agent Options

The following options are shared across all agent subcommands:

srb ls — List Assets and Environments

The srb ls command allows you to list all registered assets, action groups, and environments in Space Robotics Bench.

Usage

srb ls [options]

Options

Output

The command produces a tabular output with the requested registered entities.

Hint: In VSCode, the table is interactive and provides clickable links to the source code.

Example Output (Truncated)

                                                          Assets of the Space Robotics Bench
┏━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ # ┃ Name                        ┃ Type    ┃ Subtype            ┃ Parent Class          ┃ Asset Config    ┃ Path                             ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ mars_surface                │ scenery │ terrain            │ Terrain               │ AssetBaseCfg    │ planetary_surface.py             │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 2 │ cargo_bay                   │ object  │ payload            │ Payload               │ AssetBaseCfg    │ payload/cargo_bay.py             │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 3 │ scoop                       │ object  │ tool               │ Tool                  │ RigidObjectCfg  │ tool/scoop.py                    │
│ 4 │ shadow_hand                 │ object  │ tool               │ ActiveTool            │ ArticulationCfg │ tool/shadow_hand.py              │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 5 │ sample_tube                 │ object  │ common             │ Object                │ RigidObjectCfg  │ sample.py                        │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 6 │ franka                      │ robot   │ manipulator        │ SerialManipulator     │ ArticulationCfg │ manipulation/franka.py           │
│ 7 │ random_ur_manipulator       │ robot   │ manipulator        │ SerialManipulator     │ ArticulationCfg │ manipulation/universal_robots.py │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 8 │ random_unitree_quadruped    │ robot   │ mobile_robot       │ LeggedRobot           │ ArticulationCfg │ mobile/unitree.py                │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 9 │ unitree_g1                  │ robot   │ mobile_manipulator │ Humanoid              │ ArticulationCfg │ mobile_manipulation/unitree.py   │
│ . │ ...                         │ ...     │ ...                │ ...                   │ ...             │ ...                              │
└───┴─────────────────────────────┴─────────┴────────────────────┴───────────────────────┴─────────────────┴──────────────────────────────────┘

             Action Groups of the Space Robotics Bench
┏━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ # ┃ Name                          ┃ Path                       ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ body_acceleration             │ common/body.py             │
│ 2 │ body_acceleration_relative    │ common/body.py             │
│ 3 │ joint_position                │ common/joint.py            │
│ 4 │ joint_position_relative       │ common/joint.py            │
│ 5 │ joint_position_binary         │ common/joint.py            │
│ 6 │ joint_velocity                │ common/joint.py            │
│ 7 │ joint_velocity_binary         │ common/joint.py            │
│ 8 │ joint_effort                  │ common/joint.py            │
│ 9 │ inverse_kinematics            │ manipulation/task_space.py │
│ . │ ...                           │ ...                        │
└───┴───────────────────────────────┴────────────────────────────┘

                                                        Environments of the Space Robotics Bench
┏━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ # ┃ ID                               ┃ Entrypoint                   ┃ Config                             ┃ Path                                      ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ _manipulation <template>         │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │ manipulation/_manipulation                │
│ 2 │ sample_collection                │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │ manipulation/sample_collection            │
│ 3 │ _aerial <template>               │ Task(AerialEnv)              │ TaskCfg(AerialEnvCfg)              │ mobile/_aerial                            │
│ 4 │ _ground <template>               │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │ mobile/_ground                            │
│ 5 │ _orbital <template>              │ Task(OrbitalEnv)             │ TaskCfg(OrbitalEnvCfg)             │ mobile/_orbital                           │
│ 6 │ locomotion_velocity_tracking     │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │ mobile/locomotion_velocity_tracking       │
│ 7 │ _aerial_manipulation <template>  │ Task(AerialManipulationEnv)  │ TaskCfg(AerialManipulationEnvCfg)  │ mobile_manipulation/_aerial_manipulation  │
│ 8 │ _ground_manipulation <template>  │ Task(GroundManipulationEnv)  │ TaskCfg(GroundManipulationEnvCfg)  │ mobile_manipulation/_ground_manipulation  │
│ 9 │ _orbital_manipulation <template> │ Task(OrbitalManipulationEnv) │ TaskCfg(OrbitalManipulationEnvCfg) │ mobile_manipulation/_orbital_manipulation │
│ . │ ...                              │ ...                          │ ...                                │ ...                                       │
└───┴──────────────────────────────────┴──────────────────────────────┴────────────────────────────────────┴───────────────────────────────────────────┘

srb agent zero — Zero Agent

The srb agent zero command runs an environment with zero-valued actions. For action spaces with equilibrium at zero, the environment will evolve solely based on its internal dynamics. This is useful for testing the environment and observing the behavior of the agent without any control input.

Usage

srb agent zero --env ENV_ID [options]

Options | Shared Agent Options

No additional options are available for this subcommand.

Example

Run zero agent in the _ground template environment:

• Create 4 parallel environment instances
• Use the Cassie bipedal robot

srb agent zero -e _ground env.num_envs=4 env.robot=cassie

srb agent rand — Random Agent

The srb agent rand command runs an environment with randomly sampled actions. This is useful for testing the environment's response to diverse inputs, identifying edge cases, and ensuring robustness in the environment implementation.

Usage

srb agent rand --env ENV_ID [options]

Options | Shared Agent Options

No additional options are available for this subcommand.

Example

Run random agent in the _aerial template environment:

• Hide the simulation UI
• Use the body_acceleration action group

srb agent rand -e _aerial --hide_ui env.robot.actions=body_acceleration

srb agent teleop — Teleoperate Agent

The srb agent teleop command allows you to manually control robots in the environment using various input devices. This is useful for testing the environment and observing the behavior of the agent under specific control inputs.

Usage

srb agent teleop --env ENV_ID [options]

Options | Shared Agent Options

Teleoperation Modes

The teleop subcommand supports 2 modes of operation based on the environment action space:

• Direct Teleoperation
• Teleoperation via Policy

Direct Teleoperation

Direct teleoperation is available for environments with action spaces that can be mapped into high-level control commands, such as the desired end-effector pose or acceleration of a mobile robot.

Examples

Teleoperate a robot in the _manipulation template environment:

• Use the ur10 manipulator

srb agent teleop -e _manipulation env.robot=ur10

Teleoperate a robot in the _orbital template environment:

• Use the spacemouse input teleoperation device

srb agent teleop -e _orbital --teleop_device spacemouse

Teleoperation via Policy

Some action spaces do not support direct teleoperation due to their complexity or dimensionality, such as joint-space control of a humanoid robot. In such cases, you can still teleoperate the agent by providing a trained policy (only RL agents are currently supported). In this case, command-like observations are driven by the teleoperation device, and the policy generates the corresponding actions.

Examples

Reference: Reinforcement Learning Workflow
Reference: srb agent train — Train Agent

First, you need to train an RL agent using the srb agent train command. Let's demonstrate this with the locomotion_velocity_tracking environment using the dreamer algorithm:

srb agent train --headless --algo dreamer -e locomotion_velocity_tracking env.num_envs=256

After you have successfully trained a policy, you can teleoperate the agent:

srb agent teleop --algo dreamer -e locomotion_velocity_tracking

By default, the policy will be loaded from the latest checkpoint. However, you can specify a specific checkpoint using the --model option.

srb agent teleop --algo dreamer -e locomotion_velocity_tracking --model space_robotics_bench/logs/locomotion_velocity_tracking/dreamer/...

srb agent ros — ROS 2 Agent

The srb agent ros command enables full control over the simulation environment through ROS 2 middleware. This allows you to integrate the Space Robotics Bench with external ROS 2 nodes and systems, making it ideal for developing and validating autonomous systems across diverse scenarios.

Usage

srb agent ros --env ENV_ID [options]

Options | Shared Agent Options

No additional options are available for this subcommand.

Examples

See the ROS 2 Workflow guide for detailed examples.

srb agent train — Train Agent

The srb agent train command allows you to train Reinforcement Learning (RL) agents on Space Robotics Bench environments.

Usage

srb agent train --algo ALGO --env ENV_ID [options]

Options | Shared Agent Options

Examples

See the Reinforcement Learning Workflow guide for detailed examples.

srb agent eval — Evaluate Agent

The srb agent eval command allows you to evaluate trained Reinforcement Learning (RL) agents on Space Robotics Bench environments.

Usage

srb agent eval --algo ALGO --env ENV_ID [options]

Options | Shared Agent Options

Examples

See the Reinforcement Learning Workflow guide for detailed examples.

srb repl — Enter Python REPL

The srb repl command launches an interactive Python REPL (Read-Eval-Print Loop) with Isaac Sim initialized and srb module preloaded. This is useful for exploration, debugging, and ad-hoc experimentation with the simulation environment.

Usage

srb repl [options]

Options

Example

Enter the Python REPL with Isaac Sim initialized:

srb repl

Load the sample_collection environment with 4 parallel instances:

env_cfg = srb.tasks.manipulation.sample_collection.TaskCfg(num_envs=4)
env = srb.tasks.manipulation.sample_collection.Task(env_cfg)

Import PyTorch:

import torch

Step the environment 50 times:

env.reset()
for _ in range(50):
    env.step(action=torch.tensor(env.action_space.sample(), device=env.device))

Graphical User Interface (GUI)

SRB comes with a minimal GUI application that can serve as a more approachable demonstration for non-developers. The GUI is written in Rust using the egui framework, while the middleware between the GUI and the SRB framework is facilitated via the r2r ROS 2 bindings.

At the moment, only the srb teleop workflow is supported by the GUI. However, it can be easily extended to support other workflows if there is interest.

srb gui — Launch GUI

The srb gui command simplifies the launch of the GUI application via cargo run:

srb gui

Hint: This command supports forwarding of all arguments following --.

Example

Run the GUI with a release build:

srb gui -- --release

IDE Configuration

Just like any other software project, Space Robotics Bench development benefits from proper IDE setup.

VSCode Setup

Visual Studio Code (VS Code) is the recommended IDE for working with the Space Robotics Bench codebase. However, you are welcome to use any IDE of your choice.

Extensions

The Python source code of SRB is fully typed, and the recommended extensions for VSCode include:

• Python
• Pylance
• mypy
• Ruff
• Even Better TOML

code --reuse-window \
     --install-extension ms-python.python \
     --install-extension ms-python.vscode-pylance \
     --install-extension ms-python.mypy-type-checker \
     --install-extension charliermarsh.ruff \
     --install-extension tamasfe.even-better-toml

For Rust development, the recommended extensions include:

• Rust Analyzer
• Even Better TOML

code --reuse-window \
     --install-extension rust-lang.rust-analyzer \
     --install-extension tamasfe.even-better-toml

Workspace Settings

SRB comes with workspace settings that primarily configure Python paths for Isaac Sim/Lab modules. It does so with these assumptions (which you can adjust as needed):

• Isaac Sim is installed at ~/isaac-sim
• Isaac Lab is installed at ../isaaclab

Using the Dev Container

For a pre-configured development environment, consider using Dev Container, which automatically includes all necessary tools and extensions.

Dev Container

Space Robotics Bench includes a Dev Container configuration via devcontainer.json that you can customize for your development needs.

Open in VS Code

For convenience, Visual Studio Code (VS Code) users can open the repository as a Dev Container by running the following script:

./space_robotics_bench/.devcontainer/open.bash

Testing

Automated testing is an essential part of the development process to ensure the correctness and reliability of the Space Robotics Bench.

Scope

The SRB test suite focuses on integration tests of various workflows that are applied sequentially to all registered environments. As these tests require Isaac Sim to be running, NVIDIA GPU must be available on the system. For this reason, the test suite is not run automatically via CI/CD pipelines and must be executed manually.

srb test — Run Tests

The srb test command simplifies running the test suites of SRB via pytest and/or cargo test.

srb test [options]

Options

Hint: This command supports forwarding of all arguments following --.

Examples

Forward -vx arguments to Python test suite (pytest):

srb test -- -vx

Run tests for both Python and Rust:

srb test --lang python rust

Documentation

This documentation is built using mdBook, which creates modern online books from a collection of Markdown files.

srb docs — Local Preview

The srb docs is a convenience command that builds and serves the documentation locally via mdbook serve. However, you are welcome to use mdbook directly.

srb docs

Hint: This command supports forwarding of all arguments following --.

Automated Deployment

The documentation is automatically deployed to GitHub Pages via GitHub Actions. The deployment process is triggered by pushing to the main branch.

Contributing

Documentation is usually the weakest link in most open-source projects. We would greatly appreciate your help in improving this documentation. If you find any errors or have suggestions for improvements, don't hesitate to open an issue or a pull request. Thank you in advance!

Development Utilities

The Space Robotics Bench provides various utilities to streamline the development workflow and maintain the project.

• Update Assets
• Clean Cache

Development Utilities — Update Assets

You can update static assets distributed via Git submodules such as srb_assets by running the included script:

./space_robotics_bench/assets/update.bash

Development Utilities — Clean Cache

Reference: SimForge CLI — simforge clean

You can clean the cache of all procedurally generated SimForge assets by running the following command:

simforge clean

Contributing — New Assets

This guide explains how to contribute new assets to the Space Robotics Bench.

Asset Types Overview

The Space Robotics Bench categorizes assets into three main types:

Static Assets

All static assets used by the Space Robotics Bench are separated into the srb_assets repository to encourage their reuse.

If you wish to contribute your asset, please follow these guidelines:

• Simplify the mesh for efficient simulation
• Aim for watertight meshes with clean topology
• Bake materials into PBR textures for compatibility
• Export meshes in the USD format (.usd/.usda/.usdc/.usdz)
• For articulated assets, add relevant joints and APIs (you can use Isaac Sim for this)

Afterward, you can add the asset to your fork of srb_assets and submit a pull request. We greatly appreciate your contributions!

Procedural Assets with SimForge

Reference: SimForge

SimForge is a framework for creating diverse virtual environments through procedural generation. SRB leverages SimForge to generate all procedural assets. Below are some examples:

Please refer to the SimForge documentation if you wish to contribute new procedural assets, particularly the following sections:

• New Assets
  • Blender Generator: New Assets

We look forward to seeing your fantastic contributions. Many thanks in advance!

Contributing — New Tasks

The process of introducing a new environment into the Space Robotics Bench is intended to be straightforward, with a limited amount of boilerplate that you need to write yourself. This guide will walk you through the primary steps for creating a new SRB task.

1. Duplicate a Template

Navigate to the srb/tasks directory in your local repository. Then, duplicate one of the existing task templates and rename it to your desired task name. It is recommended that you keep your task in the same root directory as the template in order to simplify the registration process. You should select a template based on the type of task and scenario you wish to create:

2. Modify the Environment

Now, it is time to be creative and modify the environment to suit your needs. You can adjust the following components:

• Scene: Change the assets and layout of the environment to match your scenario.
• Robot: Specify a category of robots that are suitable for performing the task.
• Task: Define the task-specific logic and objectives.
  • Observation Space: Define the observation space for the task.
  • Reward Function: Define the reward function for the task.
  • Termination Condition: Define the termination condition for the task.

As this step is very task-specific, don't hesitate to reach out to the community for help or guidance.

3. Debug the Task

Reference: srb agent teleop — Teleoperate Agent
Reference: srb agent zero — Zero Agent
Reference: srb agent rand — Random Agent

While developing your task, it is essential to test it to ensure that it behaves as expected. Depending on the action space of your robot, you can either use teleop or zero/rand commands to control the robot. It is also recommended to enable debug visualizations to help you better understand the behavior of the task, and ensure that the environment works with parallel instances:

srb agent teleop -e <your_task_name> env.debug_vis=true env.num_envs=4

srb agent zero -e <your_task_name> env.debug_vis=true env.num_envs=4

srb agent rand -e <your_task_name> env.debug_vis=true env.num_envs=4

4. Use the Task in a Workflow

Reference: Workflows

Now, you are ready to use your task in your desired workflow. Whether you are experimenting with training RL agents or developing a future space mission, your task can be integrated into any workflow that the Space Robotics Bench provides.

Feel free to show & tell us about your task in the community. We are excited to see what you have created!

Attributions

This document lists the attributions for external assets used in the Space Robotics Bench.

Models

In addition to modifications listed below, all models have been converted to the Universal Scene Description (USD) format with non-destructive transformations.

1. Gateway Lunar Space Station by NASA's Gateway Program Office
   • Separated into individual assets (Gateway Core, Canadarm3)
2. Mars 2020 Sample Tube by JPL-Caltech
   • Added mesh of the sample tube cap
   • Added materials
3. Mars Ingenuity Helicopter by NASA/JPL-Caltech
4. Mars Perseverance Rover by NASA/JPL-Caltech
5. Starship by gallus-gallus
   • Recreated materials
6. Super Heavy by gallus-gallus
   • Changed number and location of engines
   • Recreated materials

Images

Where applicable, EXR images have been DWAA compressed.

1. Earth From Space by EB Adventure Photoscans
2. Low Lunar Orbit by NASA's Gateway Program Office
3. Sunflowers (Pure Sky) by Sergej Majboroda and Jarod Guest
4. The bright star foreground in celestial coordinates by NASA/Goddard Space Flight Center Scientific Visualization Studio
5. The Milky Way background in celestial coordinates by NASA/Goddard Space Flight Center Scientific Visualization Studio

Contributors

• Andrej Orsula

Citation

If you use this project in your research, please cite the following publication:

@inproceedings{orsula2024towards,
    title     = {{Towards Benchmarking Robotic Manipulation in Space}},
    author    = {Orsula, Andrej and Richard, Antoine and Geist, Matthieu and Olivares-Mendez, Miguel and Martinez, Carol},
    booktitle = {Conference on Robot Learning (CoRL) Workshop on Mastering Robot Manipulation in a World of Abundant Data},
    year      = {2024},
}

Community

We encourage you to join our Discord community to discuss the Space Robotics Bench, share your projects, ask questions, and collaborate with other researchers and developers. We are excited to see what you create with SRB!

Invite Link: https://discord.gg/p9gZAPWa65

Guidelines

Please adhere to the following guidelines when participating in the community:

• Be respectful and considerate to all members
• Keep your discussions relevant

We look forward to interacting with you in the community!

Troubleshooting

Runtime Errors

Driver Incompatibility

If you encounter the following error message:

[Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_INCOMPATIBLE_DRIVER

This indicates that your NVIDIA driver is incompatible with Omniverse. To resolve the issue, update your NVIDIA driver according to the Isaac Sim driver requirements.

Unexpected Behavior

Teleoperation Stuck

During teleoperation with the keyboard, if you change your window focus, Omniverse may fail to register a button release, causing the robot to move continuously in one direction. To fix this, press the L key to reset the environment.