Space Robotics Bench (SRB) is a comprehensive collection of environments and tasks for robotics research in the challenging domain of space. It provides a unified framework for developing and validating autonomous systems under diverse extraterrestrial scenarios. At the same time, its design is flexible and extensible to accommodate a variety of development workflows and research directions beyond Earth.
Key Features
-
Highly Parallelized Simulation via NVIDIA Isaac Sim: SRB supports thousands of parallel simulation instances to accelerate workflows such as online learning, synthetic dataset generation, parameter tuning, and validation.
-
On-Demand Procedural Generation with SimForge: Automated procedural generation of simulation assets is leveraged to provide a unique scenario for each simulation instance, with the ultimate goal of developing autonomous systems that are both robust and adaptable to the unpredictable domain of space.
-
Extensive Domain Randomization: All simulation instances can be further randomized to enhance the generalization of autonomous agents towards variable environment dynamics, visual appearance, illumination conditions, as well as sensor and actuation noise.
-
Compatibility with Gymnasium API: All tasks are compatible with a standardized API to ensure seamless integration with a broad ecosystem of libraries and frameworks for robot learning research.
-
Seamless Interface with ROS 2 & Space ROS: Simulation states, sensory outputs and actions of autonomous systems are available through ROS 2 middleware interface, enabling direct interoperability with the vast (Space) ROS ecosystem.
-
Abstract Architecture: The architecture of SRB is designed to be modular and extensible, allowing for easy integration of new assets, robots, tasks and workflows
๐ If you have any questions or suggestions regarding this documentation, don't hesitate to reach out to us! More often than not, a lack of understanding is a result of poor documentation... Therefore, we are always looking to improve it.
Table of Contents (available in the left sidebar)
Overview
Getting Started
Configuration
Reference
Development
Contributing
Miscellaneous
Environments
The Space Robotics Bench provides a variety of environments that simulate diverse scenarios in the challenging domain of space:
- Manipulation - Stationary manipulation skills and sequences in various contexts
- Mobile Robotics - Traversal and navigation under extraterrestrial conditions
- Mobile Manipulation - Complex combination of mobility and manipulation
Environments โ Manipulation (Static)
The Space Robotics Bench provides a wide range of environments that focus on stationary manipulation across different application domains of space robotics.
Sample Collection (sample_collection
)
Grasping and collection of samples on planetary surfaces
Sample Collection โ Multi (sample_collection_multi
)
A variation of sample collection with multiple samples
Debris Capture (debris_capture
)
Capture and securement of floating debris in microgravity with a fixed-base manipulator
Peg-in-Hole Assembly (peg_in_hole_assembly
)
Precision assembly task requiring the robot to insert a peg into a hole
Peg-in-Hole Assembly โ Multi (peg_in_hole_assembly_multi
)
A variation of peg-in-hole assembly with multiple pegs and holes
Solar Panel Assembly (solar_panel_assembly
)
Complex assembly sequence involving solar panel components
Environments โ Mobile Robotics
The mobile robotics environments of the Space Robotics Bench focus on tasks involving locomotion, navigation, and traversal across different terrains and conditions.
Ground Environments
Locomotion Velocity Tracking (locomotion_velocity_tracking
)
Locomotion with legged robots and humanoids while tracking desired velocity commands
Aerial Environments
Orbital Environments
Landing (landing
)
Landing of spacecraft on planetary surfaces with varying terrain and obstacles
Orbital Evasion (orbital_evasion
)
Navigation of spacecraft in microgravity while avoiding obstacles
Environments โ Mobile Manipulation
The mobile manipulation environments of the Space Robotics Bench represent a challenging combination of mobility and manipulation capabilities.
Ground-based Mobile Manipulation
Excavation (excavation
)
Excavation of granular material (regolith) with a ground-based mobile manipulator
Aerial Mobile Manipulation
Orbital Mobile Manipulation
Mobile Debris Capture (mobile_debris_capture
)
Capture of floating debris in microgravity with a floating-base mobile manipulator
Robots
The Space Robotics Bench provides a diverse set of robots for simulating various scenarios:
- Manipulators - Robotic arms
- Mobile Robots - Wheeled rovers, legged robots, aerial vehicles, spacecraft, ...
- Mobile Manipulators - All
mobile robot + manipulator
combinations, humanoids, ...
Manipulators
The Space Robotics Bench provides a variety of robotic arms for simulating manipulation tasks.
Robotic Arms
Canadarm3 (canadarm3
)
Franka Emika (franka
)
Kinova JACO2 โ 6 DoF (kinova_j2n6s
)
Kinova JACO2 โ 7 DoF (kinova_j2n7s
)
Kinova Gen3 โ 7 DoF (kinova_gen3n7
)
UR3 (ur3
)
UR3e (ur3e
)
UR5 (ur5
)
UR5e (ur5e
)
UR10 (ur10
)
UR10e (ur10e
)
UR16e (ur16e
)
UR20 (ur20
)
UR30 (ur30
)
Random UR Manipulator (random_ur_manipulator
)
Unitree Z1 (unitree_z1
)
End-Effectors
Passive End-Effectors
Scoop (scoop
)
Active End-Effectors
Allegro Hand (allegro_hand
)
Franka Hand (franka_hand
)
Kinova Jaco 2 3-Finger Gripper (kinova300
)
Shadow Hand (shadow_hand
)
Mobile Robots
The Space Robotics Bench provides various mobile robots, including legged robots, wheeled platforms, aerial vehicles, and spacecraft.
Wheeled Robots
Leo Rover (leo_rover
)
Perseverance (perseverance
)
Pragyan (pragyan
)
Legged Robots
Bipeds
Cassie (cassie
)
Quadrupeds
ANYmal C (anymal_c
)
ANYmal D (anymal_d
)
Random ANYmal Quadruped (random_anymal_quadruped
)
Spot (spot
)
Unitree A1 (unitree_a1
)
Unitree Go1 (unitree_go1
)
Unitree Go2 (unitree_go2
)
Random Unitree Quadruped (random_unitree_quadruped
)
Aerial Robots
Multicopters
Crazyflie (crazyflie
)
Ingenuity (ingenuity
)
Orbital Robots
Spacecraft
CubeSat (cubesat
)
Venus Express (venus_express
)
Lunar Gateway (gateway
)
Lander
Apollo Lander (apollo_lander
)
Peregrine Lander (peregrine_lander
)
Vikram Lander (vikram_lander
)
Random Lander (random_lander
)
Mobile Manipulators
The Space Robotics Bench provides mobile manipulator robots that combine locomotion with manipulation capabilities.
Combined Systems
Aerial Manipulator (Combination)
Ground Manipulator (Combination)
Orbital Manipulator (Combination)
Humanoids
Humanoid 21 DOF (humanoid21
)
Humanoid 28 DOF (humanoid28
)
Unitree H1 (unitree_h1
)
Unitree G1 (unitree_g1
)
Integrations & Interfaces
The Space Robotics Bench has a modular design that supports integrations & interfaces with external frameworks and tools. This enables researchers and developers to leverage the high-fidelity environments while using their preferred toolchains for research, development, and validation.
At the moment, SRB natively supports the following categories of integrations/interfaces:
- ROS 2 & Space ROS - Interface with the wide ecosystem of robotics tools and libraries
- Reinforcement Learning (RL) - Train and evaluate policies using popular RL algorithms
Interface โ ROS 2 & Space ROS
All environments and workflows in the Space Robotics Bench support the ROS 2 interface, which allows you to communicate with each environment instance using standard ROS 2 middleware interfaces. In doing so, you can collect sensor data, control robots, and interact with the simulation instances using the vast ecosystem of ROS tools and libraries.
Motivation
ROS 2 has become the de facto standard for developing robotic systems across various domains. The Space Robotics Bench provides a comprehensive ROS 2 interface that enables seamless integration between the simulation environment and the broader ROS ecosystem, offering several key benefits:
- Compatibility with existing tools - Leverage the rich ecosystem of ROS tools like RViz2, tf2, and rosbag2 for visualization, debugging, and data recording
- Workflow continuity - Develop algorithms and control systems that can transition smoothly from simulation to real hardware using the same interfaces
- Distributed architecture - Take advantage of ROS's node-based architecture to distribute computation and simulation across different processes or even machines
- Community standardization - Utilize standard message types (
geometry_msgs
,sensor_msgs
, ...) that are widely understood and supported in the robotics community
Space ROS
Space ROS is an initiative aimed at extending the ROS ecosystem for applications beyond Earth. The ROS 2 interface of SRB is fully compatible with Space ROS, allowing you to develop and validate autonomous systems across a wide range of diverse extraterrestrial scenarios. In fact, the Space Robotics Bench is a spiritual continuation of the Parallel ProcGen Environments project developed during the NASA Space ROS Sim Summer Sprint Challenge.
You can integrate SRB with Space ROS in two ways:
- Direct: Run the ROS 2 interface of SRB using the Space ROS middleware stack alongside the rest of your Space ROS nodes, which can be achieved simply by sourcing your Space ROS environment before running SRB
- Indirect (recommended): Run the ROS 2 interface of SRB using the standard ROS 2 middleware stack and communicate with your Space ROS nodes using the standard ROS 2 interfaces (no additional setup required)
Implementation
The ROS 2 interface is implemented through a dynamically configured bridge node with the following key features:
- Dynamic ROS interfaces - All publishers, subscribers, and services are created automatically based on the selected simulation environment without any manual configuration
- Parallel environment support - Every parallel simulation instance can be managed through a separate ROS namespace for seamless parallelization, while global control is also available
- Standard message translation - Environment states, sensory outputs, and control commands are automatically translated from simulation tensors to appropriate ROS interfaces
- Complete access - All simulation entities (robots, sensors, objects, ...) are automatically detected, and their interfaces are exposed to ROS alongside task-specific MDP rewards and signals
How to Get Started?
If you want to start using the Space Robotics Bench with ROS 2, follow these steps:
Integration โ Reinforcement Learning
The Space Robotics Bench is designed with Robot Learning research in mind, with a particular emphasis on Reinforcement Learning (RL). All environments follow the standard Gymnasium API, making them compatible with most modern RL frameworks. This architecture enables rapid prototyping, training, and evaluation of policies across diverse space robotics tasks, which makes SRB particularly suitable for comparing the generalization capabilities of novel RL algorithms.
Motivation
RL has emerged as a promising approach for developing autonomous behaviors in complex robotic systems, especially for space applications where manual control may be limited by communication delays or environmental uncertainties. SRB provides a comprehensive set of environments that are specifically designed to facilitate RL research in space robotics, offering several key features:
- Diverse Tasks - Collection of various space robotics tasks that range from simple navigation to complex mobile manipulation
- Parallel Training - All environments support parallel training across multiple instances for efficient data collection
- Procedural Generation & Domain Randomization - Generalization capabilities of RL algorithms can be put to the test using procedurally generated environments that are randomized across multiple dimensions
- Unified Benchmarking - RL algorithms can be compared across a set of reproducible tasks with consistent evaluation metrics
Available Integrations
We provide official integrations with the following RL frameworks:
- Dreamer [JAX] - Model-based RL algorithm
- Stable-Baselines3 (SB3) & SBX [PyTorch & JAX] - Popular implementation of RL algorithms
- skrl [PyTorch & JAX] - Implementation of single- and multi-agent RL algorithms
How to Get Started?
If you want to start using the Space Robotics Bench for Reinforcement Learning, follow these steps:
System Requirements
Hardware Requirements
The hardware requirements for Space Robotics Bench are inherited from the Isaac Sim requirements. With careful tuning, it is possible to run the included environments on lower-spec systems. However, the performance of some workflows might be limited. The bare minimum requirements are listed below:
Component | Requirement |
---|---|
CPU | x86_64 |
GPU | NVIDIA RTX |
RAM | 16 GB |
VRAM | 4 GB |
Disk Space | 32 GB |
This project requires a dedicated NVIDIA GPU with RT Cores (RTX series). Isaac Sim does not support GPUs from other vendors (AMD, Intel) or older NVIDIA GPUs without RT Cores.
Software Requirements
A Linux-based OS with an appropriate NVIDIA driver is required to use Space Robotics Bench. Other operating systems might be functional, but they are not officially supported. Please let us know if you confirm functionality on other non-listed systems.
Component | Requirement |
---|---|
OS (Native Installation) | Ubuntu 22.04 |
OS (Docker Installation) | Linux with X11 |
NVIDIA Driver | >=535,<560 |
NVIDIA Driver
Official instructions: Driver Installation Guide โ Choose an Installation Method
Install the NVIDIA driver by following the official instructions above or through your distribution package manager. Although Isaac Sim specifies 535.129.03
as the recommended version, newer drivers should also be compatible.
... continue with Installation
Installation
Before proceeding, ensure your system meets the system requirements.
Installation Methods
SRB supports three installation methods, each with different trade-offs:
A. Native
- โ Full system integration
- โ Smooth development experience
- โ Complex setup process
- โ Potential dependency conflicts
B. Docker (Recommended)
- โ Simple installation & deployment
- โ Reproducible environment & easy to update
- โ ๏ธ Moderate development experience (via Dev Containers)
- โ Requires privileged access (not suitable for HPC)
C. Apptainer/Singularity
- โ Deployable to HPC clusters
- โ Uff...
Alternative โ Temporary Setup (Quickstart)
For quick experimentation with SRB, you can use a temporary setup that downloads a pre-built Docker image and runs it in a pre-configured container. A single script accomplishes everything, which you can call directly via curl
or wget
(consider inspecting the script before executing it):
A. curl
ENSURE_DOCKER=true WITH_DEV_VOLUME=false WITH_HISTORY=false bash -c "$(curl -fsSL https://raw.githubusercontent.com/AndrejOrsula/space_robotics_bench/refs/heads/main/.docker/run.bash)" --
B. wget
ENSURE_DOCKER=true WITH_DEV_VOLUME=false WITH_HISTORY=false bash -c "$(wget -qO - https://raw.githubusercontent.com/AndrejOrsula/space_robotics_bench/refs/heads/main/.docker/run.bash)" --
The Docker container created by this setup is ephemeral (WITH_DEV_VOLUME=false
), and data does not persist between sessions. Any changes made inside the container will be lost when the container is removed.
Cleanup of Temporary Setup
If you do not wish to continue using SRB, you can remove the Docker container and its associated image by executing these commands:
docker rm -f space_robotics_bench
docker rmi andrejorsula/space_robotics_bench:latest
Installation โ Native
This guide covers installing SRB natively on your system without containerization. Although this approach simplifies development, it requires more manual setup and decreases reproducibility.
1. Clone the Repository
First, clone the SRB repository with all submodules:
git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git
2. Install NVIDIA Isaac Sim 4.5
Official instructions: Isaac Sim โ Workstation Installation
Install Isaac Sim either by following the official instructions above or using the provided convenience script:
./space_robotics_bench/scripts/install_isaacsim.bash "$HOME/isaac-sim"
ISAAC_SIM_PYTHON
It is highly recommended to make ISAAC_SIM_PYTHON
point to the Python entrypoint of Isaac Sim in your shell configuration (the script above will prompt you to do so):
A. bash
echo "export ISAAC_SIM_PYTHON='$HOME/isaac-sim/python.sh'" >> ~/.bashrc
source ~/.bashrc
B. zsh
echo "export ISAAC_SIM_PYTHON='$HOME/isaac-sim/python.sh'" >> ~/.zshrc
source ~/.zshrc
C. fish
set -Ux ISAAC_SIM_PYTHON "$HOME/isaac-sim/python.sh"
3. Install NVIDIA Isaac Lab 2.0
Official instructions: Isaac Lab โ Installation
Install Isaac Lab either by following the official instructions above or using the provided convenience script:
./space_robotics_bench/scripts/install_isaaclab.bash "$HOME/isaaclab"
4. Install Blender 4.3 with SimForge
Official instructions: Blender โ Install from blender.org
- Install Blender by following the official instructions for downloading and extracting its archive.
- Ensure that the
blender
executable is accessible from your system'sPATH
. - Install SimForge with its assets within the Blender Python environment.
As an example, the commands below will install Blender in your home directory and create a symbolic link to $HOME/.local/bin/blender
(assuming it is in your PATH
):
export BLENDER_VERSION="4.3.2"
export BLENDER_VERSION_SHORT=$(echo $BLENDER_VERSION | sed 's/\.[^.]*$//')
mkdir -p $HOME/blender
curl -fsSL "https://download.blender.org/release/Blender$BLENDER_VERSION_SHORT/blender-$BLENDER_VERSION-linux-x64.tar.xz" | tar xJ -C $HOME/blender --strip-components=1
ln -sf $HOME/blender/blender $HOME/.local/bin/blender
"$HOME/blender/$BLENDER_VERSION_SHORT/python/bin/python3.11" -m pip install simforge[assets]
Avoid installing Blender through Snap, as it would prevent the integration of the required Python dependencies within its environment.
5. Install the Space Robotics Bench
Install the srb
package in editable mode:
"$ISAAC_SIM_PYTHON" -m pip install --editable ./space_robotics_bench[all]
Note: The
all
extra installs optional dependencies to support all workflows and improve usability. Feel free to checkpyproject.toml
and adjust the extras to your needs.
Setup CLI
Make the srb
CLI command available in your shell through the provided script:
./space_robotics_bench/scripts/setup_cli.bash
Note: In case the script fails, the
srb
CLI is still accessible via"$ISAAC_SIM_PYTHON" -m srb
.
6. Verify Installation
After the installation, verify that everything works as expected.
Isaac Sim
Confirm that you can launch Isaac Sim:
"$HOME/isaac-sim/isaac-sim.sh"
Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.
Isaac Lab
Confirm that Isaac Lab is installed:
"$ISAAC_SIM_PYTHON" -m pip show isaaclab
Space Robotics Bench
Verify that the entrypoint script of SRB is available in the Python environment of Isaac Sim:
"$ISAAC_SIM_PYTHON" -m srb --help
Verify that the srb
command is available:
srb --help
Verify that argument completion works:
srb <TAB> <TAB>
... continue with Basic Usage
Extras
Development
To improve your development experience, consider configuring your IDE (guide).
Installation โ Docker (Recommended)
Using SRB inside Docker is recommended for most users, as it provides an isolated, reproducible environment that is fully pre-configured.
1. Clone the Repository
First, clone the SRB repository with all submodules:
git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git
2. Install Docker Engine & NVIDIA Container Toolkit
- Official instructions: Install Docker Engine
- Official instructions: Linux post-installation steps for Docker Engine
- Official instructions: Installing the NVIDIA Container Toolkit
Install Docker Engine and NVIDIA Container Toolkit either by following the official instructions above or using the provided convenience script:
./space_robotics_bench/.docker/host/install_docker.bash
3. Run
Now, you can run the Docker container with the provided script. The first run will automatically pull the latest image from Docker Hub:
./space_robotics_bench/.docker/run.bash
4. Verify Installation
Once you enter the Docker container, verify that everything works as expected.
Isaac Sim
Confirm that you can launch Isaac Sim:
"$HOME/isaac-sim/isaac-sim.sh"
Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.
Space Robotics Bench
Verify that the srb
command is available:
srb --help
... continue with Basic Usage
Extras
Build a New Docker Image
If you want to build a custom Docker image, you can use the provided script:
./space_robotics_bench/.docker/build.bash
Join a Running Container
To join a running container from another terminal, use the provided script:
./space_robotics_bench/.docker/join.bash
Development
The repository workspace is automatically mounted inside the Docker container, so you can edit the code either on the host or inside the container, and the changes will be persistently reflected in both environments.
To improve your development experience, you can open the project as a Dev Container (guide).
Installation โ Apptainer/Singularity
You can use SRB on HPC clusters or systems where Docker is not available by leveraging Apptainer. Apptainer is a containerization tool similar to Docker but designed for environments with stricter security policies or without Docker support.
Setup (Local)
1. Clone the Repository
First, clone the SRB repository with all submodules:
git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git
2. Install Docker Engine & NVIDIA Container Toolkit
- Official instructions: Install Docker Engine
- Official instructions: Linux post-installation steps for Docker Engine
- Official instructions: Installing the NVIDIA Container Toolkit
Install Docker Engine and NVIDIA Container Toolkit either by following the official instructions above or using the provided convenience script:
./space_robotics_bench/.docker/host/install_docker.bash
3. Install Apptainer
Official instructions: Installing Apptainer
Install Apptainer by following the official instructions above. For instance, you can use these commands on Ubuntu:
sudo add-apt-repository -y ppa:apptainer/ppa
sudo apt update
sudo apt install -y apptainer
4. Build the Apptainer Image
Now you can build the Apptainer image with the provided script:
./space_robotics_bench/.docker/hpc/build.bash
This will create an Apptainer image space_robotics_bench.sif
inside the ./space_robotics_bench/.docker/hpc/images/
directory.
Deployment (HPC Cluster)
1. Transfer the Repository (with the Apptainer Image)
Transfer the local SRB repository to the HPC cluster using your preferred method (e.g., scp
or rsync
):
A. scp
scp -r ./space_robotics_bench user@hpc-cluster:/path/to/space_robotics_bench
B. rsync
rsync -avr ./space_robotics_bench user@hpc-cluster:/path/to/space_robotics_bench
2. SSH into the HPC Cluster
SSH into the HPC cluster:
ssh user@hpc-cluster
3. Run the Apptainer Image
Now you can run the Apptainer image with the provided script inside an interactive session (you might need make some adjustments based on your HPC environment):
# HPC interactive session
/path/to/space_robotics_bench/.docker/hpc/run.bash bash
4. Verify Installation
Once you enter the Apptainer container, verify that everything works as expected.
Isaac Sim
Confirm that you can launch Isaac Sim:
# Inside Apptainer container
"$HOME/isaac-sim/isaac-sim.sh"
Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.
Space Robotics Bench
Verify that the srb
command is available:
# Inside Apptainer container
srb --help
... continue with Basic Usage
Extras
Schedule a SLURM Job
For long-running tasks or automated workflows, you can schedule a SLURM job that will automatically run the Apptainer image. It is highly recommended that you adjust the script to your needs before submitting the job:
# HPC login node
/path/to/space_robotics_bench/.docker/hpc/submit.bash [CMD]
Basic Usage
After successful installation, you are now ready to explore the Space Robotics Bench. This guide covers the essentials for getting started with the framework.
Native Installation โ If the srb
command is not available, you can use this syntax:
"$ISAAC_SIM_PYTHON" -m srb
1. List Registered Assets & Environments
Reference:
srb ls
โ List Assets and Environments
As a first step, it is recommended that you list all registered assets, action groups, and tasks to get an overview of what SRB has to offer:
srb ls
After a while, you should see 3 tables printed in the terminal:
1. Assets: Simulation assets categorized under sceneries, objects, and robots (click to expand)
- Sceneries - Terrains, space stations, ...
- Objects - Interactive objects, tools, ...
- Robots - Manipulators, mobile robots, ...
โโโโโโโโโโโโโโโโณโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโ
โ Name โ Type โ Subtype โ Parent Class โ Asset Config โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ mars_surface โ scenery โ terrain โ Terrain โ AssetBaseCfg โ
โ sample_tube โ object โ common โ Object โ RigidObjectCfg โ
โ scoop โ object โ tool โ Tool โ RigidObjectCfg โ
โ shadow_hand โ object โ tool โ ActiveTool โ ArticulationCfg โ
โ ur10 โ robot โ manipulator โ SerialManipulator โ ArticulationCfg โ
โ cubesat โ robot โ mobile_robot โ OrbitalRobot โ RigidObjectCfg โ
โ ingenuity โ robot โ mobile_robot โ Multicopter โ ArticulationCfg โ
โ perseverance โ robot โ mobile_robot โ WheeledRobot โ ArticulationCfg โ
โ unitree_g1 โ robot โ mobile_manipulator โ Humanoid โ ArticulationCfg โ
โ ... โ ... โ ... โ ... โ ... โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโ
Scenery asset (automatically registered as "mars_surface" scenery/terrain)
class MarsSurface(Terrain):
## Scenario - The asset is suitable for the Mars domain
DOMAINS: ClassVar[Sequence[Domain]] = (Domain.MARS,)
## Model - Static asset
asset_cfg: AssetBaseCfg = AssetBaseCfg(
prim_path="{ENV_REGEX_NS}/mars_surface",
## Spawner procedurally generates SimForge models
spawn=SimforgeAssetCfg(
assets=[simforge_foundry.MarsSurface()],
collision_props=CollisionPropertiesCfg(),
),
)
Object asset (automatically registered as "sample_tube" object/common)
class SampleTube(Object):
## Model - Rigid object affected by physics
asset_cfg: RigidObjectCfg = RigidObjectCfg(
prim_path="{ENV_REGEX_NS}/sample",
## Spawner loads a static USD file
spawn=UsdFileCfg(
usd_path=(
SRB_ASSETS_DIR_SRB_OBJECT.joinpath("sample_tube.usdz").as_posix()
),
collision_props=CollisionPropertiesCfg(),
mesh_collision_props=MeshCollisionPropertiesCfg(
mesh_approximation="convexDecomposition"
),
rigid_props=RigidBodyPropertiesCfg(),
mass_props=MassPropertiesCfg(density=1500.0),
),
)
Robot asset (automatically registered as "franka" robot/manipulator)
class Franka(SerialManipulator):
## Model - Articulation with several links connected by joints
asset_cfg: ArticulationCfg = ArticulationCfg(
prim_path="{ENV_REGEX_NS}/franka",
## Spawner loads a static USD file
spawn=UsdFileCfg(
usd_path=SRB_ASSETS_DIR_SRB_ROBOT.joinpath("manipulator")
.joinpath("franka_arm.usdz")
.as_posix(),
...
),
## Initial joint configuration of the robot
init_state=ArticulationCfg.InitialStateCfg(
joint_pos={
"panda_joint1": 0.0,
"panda_joint2": 0.0,
"panda_joint3": 0.0,
"panda_joint4": deg_to_rad(-90.0),
"panda_joint5": 0.0,
"panda_joint6": deg_to_rad(90.0),
"panda_joint7": deg_to_rad(45.0),
},
),
...
)
## End effector - The default hand is separate to allow for easy replacement
end_effector: Tool | None = FrankaHand()
## Actions - Inverse Kinematics action group that drives all joints
actions: ActionGroup = InverseKinematicsActionGroup(
DifferentialInverseKinematicsActionCfg(
asset_name="robot",
joint_names=["panda_joint[1-7]"],
base_name="panda_link0",
body_name="panda_link7",
controller=DifferentialIKControllerCfg(
command_type="pose",
use_relative_mode=True,
ik_method="svd",
),
scale=0.1,
body_offset=DifferentialInverseKinematicsActionCfg.OffsetCfg(),
),
)
## Frames - Relevant frames for attaching the robot and mounting tool/sensors
frame_base: Frame = Frame(prim_relpath="panda_link0")
frame_flange: Frame = Frame(
prim_relpath="panda_link7",
offset=Transform(
pos=(0.0, 0.0, 0.107),
rot=rpy_to_quat(0.0, 0.0, -45.0),
),
)
frame_wrist_camera: Frame = Frame(
prim_relpath="panda_link7/camera_wrist",
offset=Transform(
pos=(0.075, -0.075, 0.1),
rot=rpy_to_quat(0.0, -80.0, 135.0),
),
)
2. Action groups: Pre-configured action spaces for robots and active tools
- Actions for robots - Each robot (mobile or manipulator) has an action group
- Actions for active tools - Each active tool (e.g. gripper) has an action group
โโโโโโโโโโโโโโโโโโโโโโ
โ Name โ
โกโโโโโโโโโโโโโโโโโโโโโฉ
โ body_acceleration โ
โ joint_position โ
โ joint_velocity โ
โ joint_effort โ
โ inverse_kinematics โ
โ ... โ
โโโโโโโโโโโโโโโโโโโโโโ
3. Environments: Gymnasium environments for templates and tasks
- Templates - Barebones environments that can be used as a starting point
- Tasks - Goal-oriented environments that provide a specific scenario
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ ID โ Entrypoint โ Config โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ _manipulation <template> โ Task(ManipulationEnv) โ TaskCfg(ManipulationEnvCfg) โ
โ sample_collection โ Task(ManipulationEnv) โ TaskCfg(ManipulationEnvCfg) โ
โ _aerial <template> โ Task(AerialEnv) โ TaskCfg(AerialEnvCfg) โ
โ _ground <template> โ Task(GroundEnv) โ TaskCfg(GroundEnvCfg) โ
โ _orbital <template> โ Task(OrbitalEnv) โ TaskCfg(OrbitalEnvCfg) โ
โ locomotion_velocity_tracking โ Task(GroundEnv) โ TaskCfg(GroundEnvCfg) โ
โ _aerial_manipulation <template> โ Task(AerialManipulationEnv) โ TaskCfg(AerialManipulationEnvCfg) โ
โ _ground_manipulation <template> โ Task(GroundManipulationEnv) โ TaskCfg(GroundManipulationEnvCfg) โ
โ _orbital_manipulation <template> โ Task(OrbitalManipulationEnv) โ TaskCfg(OrbitalManipulationEnvCfg) โ
โ ... โ ... โ ... โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
2. Teleoperate your 1st Robot across Diverse Domains
Reference:
srb agent teleop
โ Teleoperate Agent
Let's start with the sample_collection
environment where you can manually control the Franka manipulator through your keyboard to collect samples on the Moon:
srb agent teleop --env sample_collection
Eventually, Isaac Sim will open with the selected environment, and you will be greeted in your terminal with a schematic of the teleoperation interface.
Teleoperation Interface (click to expand)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Keyboard Scheme (focus the Isaac Sim window) โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Reset: [ L ] โ
โ Decrease Gain [ O ] โ Increase Gain: [ P ] โ
โ Event: [ R / K ] โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Translation โ
โ [ W ] (+X) [ Q ] (+Z) โ
โ โ โ โ
โ โ โ โ
โ (-Y) [ A ] โโโผโโ [ D ] (+Y) โผ โ
โ โ โ โ
โ โ โ โ
โ [ S ] (-X) [ E ] (-Z) โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Rotation โ
โ [ Z ] โโโโโโโโโ(ยฑX)โโโโโโโโโ [ X ] โ
โ โ
โ [ T ] โปโโโโโโโโ(ยฑY)โโโโโโโโโบ [ G ] โ
โ โ
โ [ C ] โบโโโโโโโโ(ยฑZ)โโโโโโโโโป [ V ] โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Note: Most tasks employ action spaces that support direct teleoperation (e.g. via Inverse Kinematics). However, some tasks such as
locomotion_velocity_tracking
rely on low-level control of individual joints. In this case, direct teleoperation is not supported, and you will need to provide a control policy that maps your teleoperation commands to low-level control signals. Further instructions are provided in the section for Teleoperation via Policy.
Move to a Different Domain
Reference: Environment Configuration
Reference: Environment Configuration โ Domain
What if we want to collect samples on Mars instead? Luckily, you can easily configure many aspects of the environment through Hydra. For instance, you can adjust the domain and sample asset in order to simulate the Mars Sample Return mission:
srb agent teleop --env sample_collection env.domain=mars env.sample=sample_tube
3. Observe Random Agents in Action
Reference:
srb agent rand
โ Random Agent
Now, let's observe an environment where agents act based on random actions sampled uniformly from their action space, which is particularly useful for verifying that environments function as intended. To demonstrate a random agent, we will use the locomotion_velocity_tracking
task that uses the Spot quadruped by default:
srb agent rand --env locomotion_velocity_tracking
Hint: Use
--hide_ui
option to disable most of the Isaac Sim UI, as shown in the video above.
Change the Robot
Reference: Environment Configuration โ Robot
Selecting a different robot for any environment is as simple as adjusting the env.robot
parameter. This particular environment supports all legged robots, so let's try the Unitree G1 humanoid:
srb agent rand --env locomotion_velocity_tracking env.robot=unitree_g1
Simulate Multiple Robots
Reference: Environment Configuration โ Parallelism
Many workflows benefit from running multiple parallel simulation instances. This can be achieved with the env.num_envs
parameter. For instance, let's try 16 instances of the Cassie biped:
srb agent rand -e locomotion_velocity_tracking env.robot=cassie env.num_envs=16
Hint: As you can see in the video above, all 16 robots share the same terrain. This is the default behaviour of
locomotion_velocity_tracking
(the default behaviour is task-specific). However, you can easily create a unique terrain for each robot by settingenv.stack=false
. This will automatically trigger the generation of 16 unique assets with different geometry and materials. Here, we speed up the procedural generation by disabling texture baking withSF_BAKER=0
:SF_BAKER=0 srb agent rand -e locomotion_velocity_tracking env.stack=false env.num_envs=16
4. Explore & Experiment with Environment Templates
Reference: Robots
Reference: Environment Configuration โ Robot
Both sample_collection
and locomotion_velocity_tracking
are examples of tasks that implement specific goal-oriented scenarios. However, SRB also provides a set of environment templates that can serve as a foundation for exploring and experimenting with custom scenarios.
In general, each robot category has its own template:
Template | Description |
---|---|
_manipulation | Fixed-base manipulation with robotic arms |
_ground | Ground traversal on planetary surfaces |
_aerial | Aerial navigation above planetary surfaces |
_orbital | Spaceflight maneuvers |
_ground_manipulation | Mobile manipulation with ground-based robots |
_aerial_manipulation | Mobile manipulation with flying robots |
_orbital_manipulation | Mobile manipulation with spacecraft |
With this in mind, let's explore the _ground_manipulation
template that combines the mobile Spot quadruped with Franka manipulator into an integrated mobile manipulation system:
srb agent rand -e _ground_manipulation
Diversify Robot Configurations
Robot configurations can also be randomized across multiple parallel instances. For example, you can combine a random Unitree quadruped (random_unitree_quadruped
) with a random Universal Robots manipulator (random_ur_manipulator
), ranging all the way from UR3 to UR30! For mobile manipulators, changing the mobile base and manipulator is separated into two parameters for more flexibility, namely env.robot.mobile_base
and env.robot.manipulator
:
srb agent rand -e _ground_manipulation env.robot.mobile_base=random_unitree_quadruped env.robot.manipulator=random_ur_manipulator env.num_envs=6 env.stack=true
Customize Payloads & End Effectors
Reference:
srb agent zero
โ Zero Agent
Modifying only the robot might not be enough for your envisioned scenario. You might also want to customize the payload of mobile robots or the end effector of manipulators. Similar to previous examples, this can also be configured via env.robot
for mobile robots and manipulators; or env.robot.mobile_base
and env.robot.manipulator
for mobile manipulators. The configuration is context-aware, and you can specify payloads and end effectors by separating them with a +
sign, i.e. mobile_base+payload
or manipulator+end_effector
. For example, let's combine Unitree Z1 manipulator with Shadow Hand end effector on top of Anymal D quadruped with the Cargo Bay payload:
srb agent zero -e _ground_manipulation env.robot.mobile_base=anymal_d+cargo_bay env.robot.manipulator=unitree_z1+shadow_hand
Hint: If you only wish to change the payload or end effector but keep the rest of the robot configuration the same,
+payload
and+end_effector
are also valid inputs. Similarly,mobile_robot+
andmanipulator+
will maintain the original payload or end effector.
And while the results might look ridiculous, the same level of customization is available across the board for all domains. Furthermore, aspects such as the pose of sensors and dimensionality of action spaces are adjusted automatically.
srb agent zero -e _aerial_manipulation env.robot.mobile_base=ingenuity env.robot.manipulator=+scoop
srb agent zero -e _orbital_manipulation env.robot.mobile_base=gateway env.robot.manipulator=canadarm3+
5. Enable Visual Sensors
To maintain the performance of simulation, all visual sensors are disabled by default. However, you might have noticed that all listed environments also have their *_visual
variant (e.g. _aerial
-> _aerial_visual
). This variant enables a number of pre-configured cameras that provide both RGB and depth images (available via observations and middleware communication).
Let's see a camera view of the Ingenuity helicopter on Mars:
srb agent teleop -e _aerial_visual
What's Next?
Everything you learned so far is just the tip of the iceberg, but it is applicable to all environments and workflows within the Space Robotics Bench. Yet, diving deeper into the codebase will allow you to customize and extend the environments further to suit your specific needs.
Depending on your interests, you are welcome to explore one or more of the following guides:
- Workflows: ROS 2
- Workflows: Reinforcement Learning
- Contributing โ New Assets
- Contributing โ New Tasks
Workflows
The Space Robotics Bench provides a variety of workflows that streamline the interaction with the simulation environments to accomplish specific goals like algorithm development, validation, or synthetic data generation.
Available Workflows
- ROS 2 - Interface with the wide ecosystem of robotics tools and libraries
- Reinforcement Learning - Train and evaluate policies using popular RL algorithms
ROS 2 Workflow
The ROS 2 interface of SRB allows you to interact with the simulation environment using standard ROS 2 middleware interfaces. This enables you to develop and validate autonomous systems across a wide range of diverse scenarios through seamless integration with the rest of your ROS 2 or Space ROS nodes.
There are two options for using the ROS 2 interface of SRB:
- Active ROS 2 Agent: A specific
srb agent ros
workflow that enables full control over the simulation through the ROS 2 middleware, including actions of robots (control commands) - Passive ROS 2 Interface: Any workflow can be configured to use the ROS 2 interface for publishing all available data and interacting with the simulation through services, but the actions are controlled via the selected workflow (teleoperation, random agent, RL training and evaluation, ...)
1. Active ROS 2 Agent
This workflow is specifically designed for full control over the simulation, including the actions of robots (control commands). It is available via the srb agent ros
command, and works with all SRB environments. Let's try it with the peg_in_hole_assembly_visual
task:
srb agent ros -e peg_in_hole_assembly_visual
List Topics
Once the simulation is running, you can open another terminal and list all available ROS 2 topics:
ros2 topic list
Hint: For Docker installation, you can use
.docker/join.bash
to enter the running container if you don't have a local ROS 2 setup available.
You will see the following list of topics: (click to expand)
/clock
/parameter_events
/rosout
/srb/env0/action/cmd_vel
/srb/env0/action/event
/srb/env0/cam_base/camera_info
/srb/env0/cam_base/image_depth
/srb/env0/cam_base/image_rgb
/srb/env0/cam_base/pointcloud
/srb/env0/cam_scene/camera_info
/srb/env0/cam_scene/image_depth
/srb/env0/cam_scene/image_rgb
/srb/env0/cam_scene/pointcloud
/srb/env0/cam_wrist/camera_info
/srb/env0/cam_wrist/image_depth
/srb/env0/cam_wrist/image_rgb
/srb/env0/cam_wrist/pointcloud
/srb/env0/end_effector/joint_pos
/srb/env0/end_effector/joint_states
/srb/env0/reward
/srb/env0/reward/penalty_action_rate
/srb/env0/reward/penalty_undesired_robot_contacts
/srb/env0/reward/reward_align_peg_to_hole_primary
/srb/env0/reward/reward_align_peg_to_hole_secondary
/srb/env0/reward/reward_distance_end_effector_to_obj
/srb/env0/reward/reward_distance_peg_to_hole_bottom
/srb/env0/reward/reward_distance_peg_to_hole_entrance
/srb/env0/reward/reward_grasp
/srb/env0/reward/reward_lift
/srb/env0/robot/delta_twist
/srb/env0/robot/joint_states
/srb/env0/terminated
/srb/env0/truncated
/srb/envs/action/cmd_vel
/srb/envs/action/event
/srb/envs/end_effector/joint_pos
/srb/envs/robot/delta_twist
/tf
/tf_static
Most topics fall under two namespaces:
/srb/env[0-N]/...
: Environment-specific topics that are unique to each environment instance/srb/envs/...
: Global topics that apply to all environment instances
Let's break down the most important topics:
Topic | Message Type | Description |
---|---|---|
/clock | rosgraph_msgs/Clock | Simulation time |
/tf , /tf_static | tf2_msgs/TFMessage | Transformations for all scene entities |
Control | ||
**/{robot_name}/{action_name} | Robot-specific | Action-specific control |
**/action/cmd_vel | geometry_msgs/Twist | Universal command (task-specific mapping) |
**/action/event | std_msgs/Bool | Universal event (task-specific mapping) |
Perception | ||
**/[{sensor_name}]/{data} | Sensor-specific | Sensor data of all scene sensors |
**/{robot_name}/joint_states | sensor_msgs/JointState | Joint states of all scene articulations |
Environment (MDP) | ||
**/reward | std_msgs/Float32 | Total reward |
**/reward/{term_name} | std_msgs/Float32 | Reward components |
**/terminated | std_msgs/Bool | Episode termination flag |
**/truncated | std_msgs/Bool | Episode truncation flag |
Subscribe to Topics
You can subscribe to topics to receive updates from the simulation. For example, you can echo the reward signal of the first environment instance:
ros2 topic echo /srb/env0/reward
data: 2.36441707611084
---
data: 2.3652756214141846
---
data: 2.36613130569458
---
data: 2.3669850826263428
---
Publish to Topics
You can also send control commands to the simulation by publishing to topics. For example, you can move robots in all environment instances by publishing a twist command:
ros2 topic pub --once /srb/envs/action/cmd_vel geometry_msgs/Twist "{linear: {z: -0.2}, angular: {z: 0.785}}"
2. Passive ROS 2 Interface
Alternatively, you can use the ROS 2 interface with any agent type by specifying the --interface ros
flag. This allows you to collect sensor data and interact with the simulation instances while controlling the agent through the selected workflow. Let's try it with the rand
agent in the sample_collection_multi_visual
environment with 8 parallel instances:
srb agent rand -e sample_collection_multi_visual env.num_envs=8 --interface ros
Note: No Hydra overrides can be specified directly after the
--interface X
option because it accepts one or more interfaces.
Call Services
Upon listing the available services, you will see that each environment instance has a /srb/env{i}/reset
service, and a global /srb/envs/reset
service.
ros2 service list
You can call these services to reset a specific environment instance or all instances at once:
ros2 service call /srb/env0/reset std_srvs/srv/Empty
ros2 service call /srb/envs/reset std_srvs/srv/Empty
Reinforcement Learning Workflow
Reinforcement Learning (RL) is one of the primary focus areas of the Space Robotics Bench. While there are several RL frameworks with their unique peculiarities, SRB offers a unified interface for training and evaluating policies across a diverse set of space robotics tasks.
1. Train your 1st RL Agent
Reference:
srb agent train
โ Train Agent
The fastest way to get started with training an RL agent is by using the srb agent train
command, which provides a streamlined interface for all integrated RL frameworks. In general, you want to specify the RL algorithm to use, the environment to train on, and the number of parallel environment instances used for rollout collection.
Let's start with a simple TODO_ENV
environment using the TODO_ALGO
algorithm. For now, omit the --headless
flag so that you can observe the convergence in real time:
srb agent train --algo TODO_ALGO --env TODO_ENV env.num_envs=512
As you begin to observe the training process, you can also monitor the progress in your terminal. After about TODO_CONVERGENCE timesteps, you will see that the agent found a stable policy that successfully solves the task. Checkpoints are saved regularly, so you are free to stop the training process at any point by sending an interrupt signal (Ctrl+C in most terminals).
2. Evaluate your Agent
Reference:
srb agent eval
โ Evaluate Agent
Once training is complete, you can evaluate your agent with the srb agent eval
command:
srb agent eval --algo TODO_ALGO --env TODO_ENV env.num_envs=16
By default, the latest checkpoint from the training run is loaded for evaluation. However, you might want to run the evaluation for a checkpoint specified via --model
:
srb agent eval --algo TODO_ALGO --env TODO_ENV env.num_envs=16 --model space_robotics_bench/logs/TODO_ENV/TODO_ALGO/TODO_CHECKPOINT
3. Try a Different Algorithm
SRB directly supports several popular RL algorithms from different frameworks:
Algorithm Type | DreamerV3 | Stable-Baselines3 | SBX | skrl |
---|---|---|---|---|
Model-based | dreamer | |||
Value-based | sb3_dqn | sbx_dqn | skrl_dqn | |
sb3_qrdqn | skrl_ddqn | |||
sb3_crossq | sbx_crossq | |||
Policy Gradient | sb3_ppo | sbx_ppo | skrl_ppo | |
sb3_ppo_lstm | skrl_ppo_rnn | |||
sb3_trpo | skrl_trpo | |||
sb3_a2c | skrl_a2c | |||
skrl_rpo | ||||
Actor-Critic | sb3_ddpg | sbx_ddpg | skrl_ddpg | |
sb3_td3 | sbx_td3 | skrl_td3 | ||
sb3_sac | sbx_sac | skrl_sac | ||
sb3_tqc | sbx_tqc | |||
Evolutionary | sb3_ars | |||
skrl_cem | ||||
Imitation-based | skrl_amp | |||
Multi-agent | skrl_ippo | |||
skrl_mappo |
This time, you can train another agent using an algorithm of your choice:
srb agent train --headless --algo <ALGO> --env TODO_ENV env.num_envs=1024
Hint: Use
--headless
mode with more parallel environments for faster convergence.
4. Monitor Training Progress
While training, you might be interested in monitoring the progress and comparing different runs through a visual interface. By default, TensorBoard logs are saved for all algorithms and environments in the space_robotics_bench/logs
directory. You can start TensorBoard to visualize the training progress:
tensorboard --logdir ./space_robotics_bench/logs --bind_all
Furthermore, you can enable Weights & Biases (wandb
) logging by passing framework-specific flags [subject to future standardization]:
- DreamerV3:
srb agent train ... +agent.logger.outputs=wandb
- SB3 & SBX:
srb agent train ... agent.track=true
- skrl:
srb agent train ... +agent.experiment.wandb=true
Note: Logging to Weights & Biases requires an account and API key.
5. Configure Hyperparameters
Reference: Agent Configuration
The default hyperparameters for all algorithms and environments are available under the space_robotics_bench/hyperparams directory. Similar to the environment configuration, you can adjust the hyperparameters of the selected RL algorithm through Hydra. However, the available hyperparameters and their structure is specific to each framework and algorithm.
Here are some examples (consult hyperparameter configs for more details):
srb agent train --algo dreamer agent.run.train_raio=128 ...
sed agent train --algo sb3_ppo agent.gamma=0.99 ...
srb agent train --algo sbx_sac agent.learning_rate=0.0002 ...
srb agent train --algo skrl_ppo_rnn agent.models.separate=True ...
Environment Configuration
The Space Robotics Bench provides a flexible configuration system for environments through Hydra. This document explains how to customize environment parameters across different domains and tasks.
How Does It Work?
Each SRB environment is registered alongside its Python configuration class, which defines the default parameters for that specific environment. All environment configuration classes are organized in a hierarchical structure via inheritance, where BaseEnvCfg
does most of the heavy lifting. This design supports a modular and extensible configuration system that allows for easy customization of environment parameters.
Modifying Configurations
You can modify environment configurations in several ways:
1. Command-Line Overrides
The most direct way to modify environment parameters is through command-line overrides:
srb agent <WORKFLOW> --env <ENV> \
env.domain=moon \
env.robot=ur10 \
env.scene.num_envs=4 \
env.stack=false \
...
2. Configuration Files
Key Parameters
Below are the most important configuration parameters organized by category (several parameters have their own detailed documentation pages):
Scenario/Environment
env.domain
- Domainenv.robot
- Robotenv.scene.num_envs
,env.stack
- Parallelismenv._particles
- Particles
Simulation
env.sim
- Low-level simulation parameters (physics, rendering, etc.)env.visuals
- Visual appearance settings
Debugging
env.debug_vis
- Enables debug visualization features
Environment Configuration โ Parallelism
One of Space Robotics Bench's most powerful features is parallel simulation, allowing you to run multiple simulation instances simultaneously. This capability is critical for:
- Reinforcement Learning: Collect experience at scale for faster training
- Parameter Tuning: Test multiple configurations simultaneously
- Monte Carlo Sampling: Evaluate robustness across varied scenarios
- Batch Processing: Process multiple scenarios in a single run
Number of Environments
The env.scene.num_envs
parameter controls how many parallel simulation instances are created. This parameter is aliased as env.num_envs
for brevity:
srb agent rand -e _manipulation env.scene.num_envs=16
srb agent zero -e _manipulation env.num_envs=128
Each environment is a fully independent physics simulation instance, with all cross-environment interactions disabled (filtered collisions). However, there is only one instance of the rendering engine, which means that a visual sensor in one environment will see entities from all environments. A simple workaround for this limitation is to increase the spacing between environments and clip the maximum sensor range.
Environment Spacing
The env.scene.env_spacing
parameter controls the spatial distance between environments. This parameter is aliased as env.spacing
for brevity:
srb agent rand -e _manipulation env.num_envs=64 env.scene.spacing=20.0
srb agent rand -e _manipulation env.num_envs=32 env.spacing=10.0
Environment Stacking
The env.stack
parameter controls whether environments share assets or have independent assets:
Independent Environments (env.stack=false
) [Default]
- โ ๏ธ Each environment has a unique scene and position
- โ Greater visual and physical diversity
- โ Supports tasks with visual sensors
- โ Slower to initialize
- โ Higher memory usage
srb agent rand -e _manipulation env.scene.num_envs=16 env.stack=false
Stacked Environments (env.stack=true
)
- โ ๏ธ All environments share the same scenery and position
- โ Faster to initialize
- โ Lower memory usage
- โ Less environmental diversity
- โ Does not support tasks with visual sensors
srb agent rand -e _manipulation env.scene.num_envs=16 env.stack=true
Note: Non-visual
locomotion_velocity_tracking
task defaults toenv.stack=true
.
Environment Configuration โ Domain
The env.domain
parameter configures the planetary or orbital environment for your simulation. This affects gravity, lighting conditions, terrain type, atmospheric properties, and visual appearance.
Available Domains
Space Robotics Bench currently supports the following domains:
Domain | Description |
---|---|
moon [default] | Lunar surface |
mars | Martian surface |
earth | Earth surface |
asteroid | Low-gravity body with irregular terrain |
orbit | Orbital microgravity environment |
Usage
You can specify the domain via the command line:
# Run a simulation on Mars
srb agent teleop --env sample_collection env.domain=mars
Domain-Specific Effects
Each domain affects various aspects of the simulation:
Physical Properties
- Gravity magnitude: Affects object dynamics, robot mobility and manipulation requirements
Visual Properties
- Lighting: Light intensity, color temperature, and angular diameter
- Skydome: Background appearance and intensity
- Atmosphere: Fog intensity and color gradient (if applicable)
Asset Selection
Some assets are domain-specific and will only appear for compatible domains. When an asset is registered, it can specify the domains it supports through the DOMAINS
class attribute.
Environment Configuration โ Robot
The env.robot
parameter allows you to specify which robot to use in your simulation. This is one of the most powerful configuration options, enabling you to change not just the robot model but also its components, payloads, and end effectors.
Robot Selection
The simplest way to select a robot is by specifying its name:
srb agent rand -e _manipulation env.robot=ur10
srb agent rand -e _ground env.robot=unitree_go1
srb agent rand -e _aerial env.robot=crazyflie
End Effector for Manipulators
You can attach specific end effectors to manipulators using the +
syntax:
srb agent rand -e _manipulation env.robot=ur10+shadow_hand
To keep the default end effector while changing the base manipulator:
srb agent rand -e _manipulation env.robot=unitree_z1+
To only change the end effector:
srb agent rand -e _manipulation env.robot=+shadow_hand
Payload for Mobile Robots
Similarly, you can attach payloads to mobile robots:
srb agent rand -e _ground env.robot=spot+cargo_bay
To keep the default payload while changing the base mobile robot:
srb agent rand -e _ground env.robot=anymal_d+
To only change the payload:
srb agent rand -e _ground env.robot=+cargo_bay
Combine Mobile Manipulators
SRB supports combining any mobile robot with any manipulator for mobile manipulation environments via the env.robot.mobile_base
and env.robot.manipulator
parameters:
srb agent rand -e _ground_manipulation env.robot.mobile_base=unitree_a1 env.robot.manipulator=unitree_z1
Similarly, you can attach payloads to the mobile base and end effectors to the manipulator:
srb agent rand -e _ground_manipulation env.robot.mobile_base=anymal_d+cargo_bay env.robot.manipulator=ur10+shadow_hand
Change Action Groups
Each robot and active tool (end-effector) has its associated action group that is automatically manifested in the action space of the whole system. You can change the action group of the robot or active tool using the env.robot.actions
parameter:
srb agent rand -e _manipulation env.robot.actions=joint_position_relative
Note: It is currently not possible to override both the robot and its action group via their registered names at the same time.
Environment Configuration โ Particles
Space Robotics Bench can simulate liquid and granular materials like regolith, sand, and dust using particle-based physics. This is particularly relevant for space applications where interaction with loose granular material is common.
Enabling Particles
You can enable particles in any environment by setting the _particles
parameter to true
:
srb agent teleop -e _manipulation env._particles=true env.robot=+scoop
Particle Configuration Parameters
Parameter | Description | Default |
---|---|---|
env._particles | Enable/disable particles | false |
env._particles_size | Particle diameter (meters) | 0.025 |
env._particles_ratio | Particle density multiplier | 0.001 |
srb agent teleop -e _manipulation env._particles=true env._particles_size=0.01 env._particles_ratio=0.1
Particle Behavior
By default, the particle system uses a pyramid distribution to create natural-looking piles of granular material with higher density at the center. Particles interact with other objects through physical collision and settle over time due to gravity. Robots can push, scoop, or otherwise interact with particles.
Note: When particles are enabled, Fabric is disabled via
env.sim.use_fabric=false
.
Agent Configuration
The Space Robotics Bench leverages Hydra for managing agent configurations across different robot learning frameworks. This document provides an overview of the configuration structure and common parameters for training agents in SRB.
Default Hyperparameters
The default hyperparameters for all algorithms and environments are available under the space_robotics_bench/hyperparams directory. If you do not wish to use Hydra, you can directly modify these files before training your agent.
Note: The available hyperparameters and their structure are specific to each framework and algorithm.
Modifying Configurations
You can modify agent configurations in several ways:
1. Command-Line Overrides
srb agent <WORKFLOW> --algo <ALGO> --env <ENV> \
agent.learning_rate=0.0001 \
agent.batch_size=64 \
...
2. Configuration Files
Command Line Interface (CLI)
The Space Robotics Bench provides a comprehensive command-line interface that allows you to interact with most aspects of the framework through a unified interface. The CLI is structured as a set of subcommands, each with its own set of options.
srb <subcommand> [options]
If you need help with a specific subcommand, you can use the -h
/--help
flag:
srb --help
srb <subcommand> -h
You can also TAB
your way through the available subcommands and options via autocompletion:
srb <TAB> <TAB>
Subcommands
The following subcommands are available in the CLI:
Subcommand | Description |
---|---|
agent | Agent subcommands (listed below) |
ls | List registered assets, action groups and environments |
repl | Enter Python REPL with Isaac Sim initialized |
test | Run automated tests |
docs | Build documentation |
gui | Launch the Graphical User Interface |
Agent Subcommands
The agent
subcommand is further separated into agent-specific subcommands:
srb agent <agent_subcommand> [options]
Agent Subcommand | Description |
---|---|
zero | Run agent with zero-valued actions |
rand | Run agent with random actions |
teleop | Manually teleoperate the agent |
ros | Interface with ROS 2 or Space ROS |
train | Train Reinforcement Learning (RL) agents |
eval | Evaluate trained RL agents |
Shared Agent Options
The following options are shared across all agent
subcommands:
Argument | Description | Default |
---|---|---|
-e /--env /--task ENV | ID of the environment | REQUIRED |
--headless | Run simulation without display | False |
--hide_ui | Disable simulation UI | False |
--interface [IFACE ...] | Interfaces to enable | [] |
--logdir /--logs PATH | Path to logging directory | SRB_LOGS_DIR |
--video | Enable video recording | False |
--video_length | Duration of recorded videos | 1000 steps |
--video_interval | Interval of video recordings | 10000 steps |
srb ls
โ List Assets and Environments
The srb ls
command allows you to list all registered assets, action groups, and environments in Space Robotics Bench.
Usage
srb ls [options]
Options
Argument | Description | Default |
---|---|---|
*categories | Sequence of categories to list | [all] |
-a /--show_hidden | Show hidden entities (*_visual envs) | False |
Output
The command produces a tabular output with the requested registered entities.
Hint: In VSCode, the table is interactive and provides clickable links to the source code.
Example Output (Truncated)
Assets of the Space Robotics Bench
โโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ # โ Name โ Type โ Subtype โ Parent Class โ Asset Config โ Path โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ 1 โ mars_surface โ scenery โ terrain โ Terrain โ AssetBaseCfg โ planetary_surface.py โ
โโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 2 โ cargo_bay โ object โ payload โ Payload โ AssetBaseCfg โ payload/cargo_bay.py โ
โโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 3 โ scoop โ object โ tool โ Tool โ RigidObjectCfg โ tool/scoop.py โ
โ 4 โ shadow_hand โ object โ tool โ ActiveTool โ ArticulationCfg โ tool/shadow_hand.py โ
โโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 5 โ sample_tube โ object โ common โ Object โ RigidObjectCfg โ sample.py โ
โโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 6 โ franka โ robot โ manipulator โ SerialManipulator โ ArticulationCfg โ manipulation/franka.py โ
โ 7 โ random_ur_manipulator โ robot โ manipulator โ SerialManipulator โ ArticulationCfg โ manipulation/universal_robots.py โ
โโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 8 โ random_unitree_quadruped โ robot โ mobile_robot โ LeggedRobot โ ArticulationCfg โ mobile/unitree.py โ
โโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 9 โ unitree_g1 โ robot โ mobile_manipulator โ Humanoid โ ArticulationCfg โ mobile_manipulation/unitree.py โ
โ . โ ... โ ... โ ... โ ... โ ... โ ... โ
โโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Action Groups of the Space Robotics Bench
โโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ # โ Name โ Path โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ 1 โ body_acceleration โ common/body.py โ
โ 2 โ body_acceleration_relative โ common/body.py โ
โ 3 โ joint_position โ common/joint.py โ
โ 4 โ joint_position_relative โ common/joint.py โ
โ 5 โ joint_position_binary โ common/joint.py โ
โ 6 โ joint_velocity โ common/joint.py โ
โ 7 โ joint_velocity_binary โ common/joint.py โ
โ 8 โ joint_effort โ common/joint.py โ
โ 9 โ inverse_kinematics โ manipulation/task_space.py โ
โ . โ ... โ ... โ
โโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Environments of the Space Robotics Bench
โโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ # โ ID โ Entrypoint โ Config โ Path โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ 1 โ _manipulation <template> โ Task(ManipulationEnv) โ TaskCfg(ManipulationEnvCfg) โ manipulation/_manipulation โ
โ 2 โ sample_collection โ Task(ManipulationEnv) โ TaskCfg(ManipulationEnvCfg) โ manipulation/sample_collection โ
โ 3 โ _aerial <template> โ Task(AerialEnv) โ TaskCfg(AerialEnvCfg) โ mobile/_aerial โ
โ 4 โ _ground <template> โ Task(GroundEnv) โ TaskCfg(GroundEnvCfg) โ mobile/_ground โ
โ 5 โ _orbital <template> โ Task(OrbitalEnv) โ TaskCfg(OrbitalEnvCfg) โ mobile/_orbital โ
โ 6 โ locomotion_velocity_tracking โ Task(GroundEnv) โ TaskCfg(GroundEnvCfg) โ mobile/locomotion_velocity_tracking โ
โ 7 โ _aerial_manipulation <template> โ Task(AerialManipulationEnv) โ TaskCfg(AerialManipulationEnvCfg) โ mobile_manipulation/_aerial_manipulation โ
โ 8 โ _ground_manipulation <template> โ Task(GroundManipulationEnv) โ TaskCfg(GroundManipulationEnvCfg) โ mobile_manipulation/_ground_manipulation โ
โ 9 โ _orbital_manipulation <template> โ Task(OrbitalManipulationEnv) โ TaskCfg(OrbitalManipulationEnvCfg) โ mobile_manipulation/_orbital_manipulation โ
โ . โ ... โ ... โ ... โ ... โ
โโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
srb agent zero
โ Zero Agent
The srb agent zero
command runs an environment with zero-valued actions. For action spaces with equilibrium at zero, the environment will evolve solely based on its internal dynamics. This is useful for testing the environment and observing the behavior of the agent without any control input.
Usage
srb agent zero --env ENV_ID [options]
Options | Shared Agent Options
No additional options are available for this subcommand.
Example
Run zero agent in the _ground
template environment:
- Create 4 parallel environment instances
- Use the Cassie bipedal robot
srb agent zero -e _ground env.num_envs=4 env.robot=cassie
srb agent rand
โ Random Agent
The srb agent rand
command runs an environment with randomly sampled actions. This is useful for testing the environment's response to diverse inputs, identifying edge cases, and ensuring robustness in the environment implementation.
Usage
srb agent rand --env ENV_ID [options]
Options | Shared Agent Options
No additional options are available for this subcommand.
Example
Run random agent in the _aerial
template environment:
- Hide the simulation UI
- Use the
body_acceleration
action group
srb agent rand -e _aerial --hide_ui env.robot.actions=body_acceleration
srb agent teleop
โ Teleoperate Agent
The srb agent teleop
command allows you to manually control robots in the environment using various input devices. This is useful for testing the environment and observing the behavior of the agent under specific control inputs.
Usage
srb agent teleop --env ENV_ID [options]
Options | Shared Agent Options
Option | Description | Default |
---|---|---|
--teleop_device [DEV ...] | Interface (keyboard , spacemouse , ...) | [keyboard] |
--pos_sensitivity VAL | Translation sensitivity | 10.0 |
--rot_sensitivity VAL | Rotation sensitivity | 40.0 |
--invert_controls | Invert control schema | False |
Teleoperation via Policy | ||
--algo ALGO | RL algorithm of the policy | |
--model MODEL | Path to the policy checkpoint |
Teleoperation Modes
The teleop
subcommand supports 2 modes of operation based on the environment action space:
Direct Teleoperation
Direct teleoperation is available for environments with action spaces that can be mapped into high-level control commands, such as the desired end-effector pose or acceleration of a mobile robot.
Examples
Teleoperate a robot in the _manipulation
template environment:
- Use the
ur10
manipulator
srb agent teleop -e _manipulation env.robot=ur10
Teleoperate a robot in the _orbital
template environment:
- Use the
spacemouse
input teleoperation device
srb agent teleop -e _orbital --teleop_device spacemouse
Teleoperation via Policy
Some action spaces do not support direct teleoperation due to their complexity or dimensionality, such as joint-space control of a humanoid robot. In such cases, you can still teleoperate the agent by providing a trained policy (only RL agents are currently supported). In this case, command-like observations are driven by the teleoperation device, and the policy generates the corresponding actions.
Examples
Reference: Reinforcement Learning Workflow
Reference:srb agent train
โ Train Agent
First, you need to train an RL agent using the srb agent train
command. Let's demonstrate this with the locomotion_velocity_tracking
environment using the dreamer
algorithm:
srb agent train --headless --algo dreamer -e locomotion_velocity_tracking env.num_envs=256
After you have successfully trained a policy, you can teleoperate the agent:
srb agent teleop --algo dreamer -e locomotion_velocity_tracking
By default, the policy will be loaded from the latest checkpoint. However, you can specify a specific checkpoint using the --model
option.
srb agent teleop --algo dreamer -e locomotion_velocity_tracking --model space_robotics_bench/logs/locomotion_velocity_tracking/dreamer/...
srb agent ros
โ ROS 2 Agent
The srb agent ros
command enables full control over the simulation environment through ROS 2 middleware. This allows you to integrate the Space Robotics Bench with external ROS 2 nodes and systems, making it ideal for developing and validating autonomous systems across diverse scenarios.
Usage
srb agent ros --env ENV_ID [options]
Options | Shared Agent Options
No additional options are available for this subcommand.
Examples
See the ROS 2 Workflow guide for detailed examples.
srb agent train
โ Train Agent
The srb agent train
command allows you to train Reinforcement Learning (RL) agents on Space Robotics Bench environments.
Usage
srb agent train --algo ALGO --env ENV_ID [options]
Options | Shared Agent Options
Option | Description | Default |
---|---|---|
--algo ALGO | RL algorithm to use | REQUIRED |
--continue_training | Continue training from last checkpoint | False |
--model MODEL | Continue training from specific checkpoint |
Examples
See the Reinforcement Learning Workflow guide for detailed examples.
srb agent eval
โ Evaluate Agent
The srb agent eval
command allows you to evaluate trained Reinforcement Learning (RL) agents on Space Robotics Bench environments.
Usage
srb agent eval --algo ALGO --env ENV_ID [options]
Options | Shared Agent Options
Option | Description | Default |
---|---|---|
--algo ALGO | RL algorithm used for training the policy | REQUIRED |
--model MODEL | Path to a specific checkpoint |
Examples
See the Reinforcement Learning Workflow guide for detailed examples.
srb repl
โ Enter Python REPL
The srb repl
command launches an interactive Python REPL (Read-Eval-Print Loop) with Isaac Sim initialized and srb
module preloaded. This is useful for exploration, debugging, and ad-hoc experimentation with the simulation environment.
Usage
srb repl [options]
Options
Argument | Description | Default |
---|---|---|
--headless | Run simulation without display | False |
--hide_ui | Disable simulation UI | False |
Example
Enter the Python REPL with Isaac Sim initialized:
srb repl
Load the sample_collection
environment with 4 parallel instances:
env_cfg = srb.tasks.manipulation.sample_collection.TaskCfg(num_envs=4)
env = srb.tasks.manipulation.sample_collection.Task(env_cfg)
Import PyTorch:
import torch
Step the environment 50 times:
env.reset()
for _ in range(50):
env.step(action=torch.tensor(env.action_space.sample(), device=env.device))
Graphical User Interface (GUI)
SRB comes with a minimal GUI application that can serve as a more approachable demonstration for non-developers. The GUI is written in Rust using the egui framework, while the middleware between the GUI and the SRB framework is facilitated via the r2r ROS 2 bindings.
At the moment, only the srb teleop
workflow is supported by the GUI. However, it can be easily extended to support other workflows if there is interest.
srb gui
โ Launch GUI
The srb gui
command simplifies the launch of the GUI application via cargo run
:
srb gui
Hint: This command supports forwarding of all arguments following
--
.
Example
Run the GUI with a release build:
srb gui -- --release
IDE Configuration
Just like any other software project, Space Robotics Bench development benefits from proper IDE setup.
VSCode Setup
Visual Studio Code (VS Code) is the recommended IDE for working with the Space Robotics Bench codebase. However, you are welcome to use any IDE of your choice.
Extensions
The Python source code of SRB is fully typed, and the recommended extensions for VSCode include:
code --reuse-window \
--install-extension ms-python.python \
--install-extension ms-python.vscode-pylance \
--install-extension ms-python.mypy-type-checker \
--install-extension charliermarsh.ruff \
--install-extension tamasfe.even-better-toml
For Rust development, the recommended extensions include:
code --reuse-window \
--install-extension rust-lang.rust-analyzer \
--install-extension tamasfe.even-better-toml
Workspace Settings
SRB comes with workspace settings that primarily configure Python paths for Isaac Sim/Lab modules. It does so with these assumptions (which you can adjust as needed):
- Isaac Sim is installed at
~/isaac-sim
- Isaac Lab is installed at
../isaaclab
Using the Dev Container
For a pre-configured development environment, consider using Dev Container, which automatically includes all necessary tools and extensions.
Dev Container
Space Robotics Bench includes a Dev Container configuration via devcontainer.json
that you can customize for your development needs.
Open in VS Code
For convenience, Visual Studio Code (VS Code) users can open the repository as a Dev Container by running the following script:
./space_robotics_bench/.devcontainer/open.bash
Testing
Automated testing is an essential part of the development process to ensure the correctness and reliability of the Space Robotics Bench.
Scope
The SRB test suite focuses on integration tests of various workflows that are applied sequentially to all registered environments. As these tests require Isaac Sim to be running, NVIDIA GPU must be available on the system. For this reason, the test suite is not run automatically via CI/CD pipelines and must be executed manually.
srb test
โ Run Tests
The srb test
command simplifies running the test suites of SRB via pytest
and/or cargo test
.
srb test [options]
Options
Argument | Description | Default |
---|---|---|
-l /--language /--lang | Language suites to test | [python] |
Hint: This command supports forwarding of all arguments following
--
.
Examples
Forward -vx
arguments to Python test suite (pytest
):
srb test -- -vx
Run tests for both Python and Rust:
srb test --lang python rust
Documentation
This documentation is built using mdBook, which creates modern online books from a collection of Markdown files.
srb docs
โ Local Preview
The srb docs
is a convenience command that builds and serves the documentation locally via mdbook serve
. However, you are welcome to use mdbook
directly.
srb docs
Hint: This command supports forwarding of all arguments following
--
.
Automated Deployment
The documentation is automatically deployed to GitHub Pages via GitHub Actions. The deployment process is triggered by pushing to the main
branch.
Contributing
Documentation is usually the weakest link in most open-source projects. We would greatly appreciate your help in improving this documentation. If you find any errors or have suggestions for improvements, don't hesitate to open an issue or a pull request. Thank you in advance!
Development Utilities
The Space Robotics Bench provides various utilities to streamline the development workflow and maintain the project.
Development Utilities โ Update Assets
You can update static assets distributed via Git submodules such as srb_assets by running the included script:
./space_robotics_bench/assets/update.bash
Development Utilities โ Clean Cache
Reference: SimForge CLI โ
simforge clean
You can clean the cache of all procedurally generated SimForge assets by running the following command:
simforge clean
Contributing โ New Assets
This guide explains how to contribute new assets to the Space Robotics Bench.
Asset Types Overview
The Space Robotics Bench categorizes assets into three main types:
Asset Type | Description |
---|---|
Scenery | Environmental elements like terrain and structures |
Objects | Interactive objects, tools, and payloads |
Robots | Systems that are intended to be controlled |
Static Assets
All static assets used by the Space Robotics Bench are separated into the srb_assets repository to encourage their reuse.
If you wish to contribute your asset, please follow these guidelines:
- Simplify the mesh for efficient simulation
- Aim for watertight meshes with clean topology
- Bake materials into PBR textures for compatibility
- Export meshes in the USD format (
.usd
/.usda
/.usdc
/.usdz
) - For articulated assets, add relevant joints and APIs (you can use Isaac Sim for this)
Afterward, you can add the asset to your fork of srb_assets and submit a pull request. We greatly appreciate your contributions!
Procedural Assets with SimForge
Reference: SimForge
SimForge is a framework for creating diverse virtual environments through procedural generation. SRB leverages SimForge to generate all procedural assets. Below are some examples:
Please refer to the SimForge documentation if you wish to contribute new procedural assets, particularly the following sections:
We look forward to seeing your fantastic contributions. Many thanks in advance!
Contributing โ New Tasks
The process of introducing a new environment into the Space Robotics Bench is intended to be straightforward, with a limited amount of boilerplate that you need to write yourself. This guide will walk you through the primary steps for creating a new SRB task.
1. Duplicate a Template
Navigate to the srb/tasks directory in your local repository. Then, duplicate one of the existing task templates and rename it to your desired task name. It is recommended that you keep your task in the same root directory as the template in order to simplify the registration process. You should select a template based on the type of task and scenario you wish to create:
Template | Description |
---|---|
_manipulation | Fixed-base manipulation with robotic arms |
_ground | Ground traversal on planetary surfaces |
_aerial | Aerial navigation above planetary surfaces |
_orbital | Spaceflight maneuvers |
_ground_manipulation | Mobile manipulation with ground-based robots |
_aerial_manipulation | Mobile manipulation with flying robots |
_orbital_manipulation | Mobile manipulation with spacecraft |
2. Modify the Environment
Now, it is time to be creative and modify the environment to suit your needs. You can adjust the following components:
- Scene: Change the assets and layout of the environment to match your scenario.
- Robot: Specify a category of robots that are suitable for performing the task.
- Task: Define the task-specific logic and objectives.
- Observation Space: Define the observation space for the task.
- Reward Function: Define the reward function for the task.
- Termination Condition: Define the termination condition for the task.
As this step is very task-specific, don't hesitate to reach out to the community for help or guidance.
3. Debug the Task
Reference:
srb agent teleop
โ Teleoperate Agent
Reference:srb agent zero
โ Zero Agent
Reference:srb agent rand
โ Random Agent
While developing your task, it is essential to test it to ensure that it behaves as expected. Depending on the action space of your robot, you can either use teleop
or zero
/rand
commands to control the robot. It is also recommended to enable debug visualizations to help you better understand the behavior of the task, and ensure that the environment works with parallel instances:
srb agent teleop -e <your_task_name> env.debug_vis=true env.num_envs=4
srb agent zero -e <your_task_name> env.debug_vis=true env.num_envs=4
srb agent rand -e <your_task_name> env.debug_vis=true env.num_envs=4
4. Use the Task in a Workflow
Reference: Workflows
Now, you are ready to use your task in your desired workflow. Whether you are experimenting with training RL agents or developing a future space mission, your task can be integrated into any workflow that the Space Robotics Bench provides.
Feel free to show & tell us about your task in the community. We are excited to see what you have created!
Attributions
This document lists the attributions for external assets used in the Space Robotics Bench.
Models
In addition to modifications listed below, all models have been converted to the Universal Scene Description (USD) format with non-destructive transformations.
- Gateway Lunar Space Station by NASA's Gateway Program Office
- Separated into individual assets (Gateway Core, Canadarm3)
- Mars 2020 Sample Tube by JPL-Caltech
- Added mesh of the sample tube cap
- Added materials
- Mars Ingenuity Helicopter by NASA/JPL-Caltech
- Mars Perseverance Rover by NASA/JPL-Caltech
Images
Where applicable, EXR images have been DWAA compressed.
- Earth From Space by EB Adventure Photoscans
- Low Lunar Orbit by NASA's Gateway Program Office
- Sunflowers (Pure Sky) by Sergej Majboroda and Jarod Guest
- The bright star foreground in celestial coordinates by NASA/Goddard Space Flight Center Scientific Visualization Studio
- The Milky Way background in celestial coordinates by NASA/Goddard Space Flight Center Scientific Visualization Studio
Contributors
Citation
If you use this project in your research, please cite the following publication:
@inproceedings{orsula2024towards,
title = {{Towards Benchmarking Robotic Manipulation in Space}},
author = {Orsula, Andrej and Richard, Antoine and Geist, Matthieu and Olivares-Mendez, Miguel and Martinez, Carol},
booktitle = {Conference on Robot Learning (CoRL) Workshop on Mastering Robot Manipulation in a World of Abundant Data},
year = {2024},
}
Community 
We encourage you to join our Discord community to discuss the Space Robotics Bench, share your projects, ask questions, and collaborate with other researchers and developers. We are excited to see what you create with SRB!
Invite Link: https://discord.gg/p9gZAPWa65
Guidelines
Please adhere to the following guidelines when participating in the community:
- Be respectful and considerate to all members
- Keep your discussions relevant
We look forward to interacting with you in the community!
Troubleshooting
Runtime Errors
Driver Incompatibility
If you encounter the following error message:
[Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_INCOMPATIBLE_DRIVER
This indicates that your NVIDIA driver is incompatible with Omniverse. To resolve the issue, update your NVIDIA driver according to the Isaac Sim driver requirements.
Unexpected Behavior
Teleoperation Stuck
During teleoperation with the keyboard, if you change your window focus, Omniverse may fail to register a button release, causing the robot to move continuously in one direction. To fix this, press the L
key to reset the environment.