Basic Usage

After successful installation, you are now ready to explore the Space Robotics Bench. This guide covers the essentials for getting started with the framework.

Native Installation — If the srb command is not available, you can use this syntax:

"$ISAAC_SIM_PYTHON" -m srb

1. List Registered Assets & Environments

Reference: srb ls — List Assets and Environments

As a first step, it is recommended that you list all registered assets, action groups, and tasks to get an overview of what SRB has to offer:

srb ls

After a while, you should see 3 tables printed in the terminal:

1. Assets: Simulation assets categorized under sceneries, objects, and robots (click to expand)

Sceneries - Terrains, space stations, ...
Objects - Interactive objects, tools, ...
Robots - Manipulators, mobile robots, ...

┏━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Name         ┃ Type    ┃ Subtype            ┃ Parent Class      ┃ Asset Config    ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ mars_surface │ scenery │ terrain            │ Terrain           │ AssetBaseCfg    │
│ sample_tube  │ object  │ common             │ Object            │ RigidObjectCfg  │
│ scoop        │ object  │ tool               │ Tool              │ RigidObjectCfg  │
│ shadow_hand  │ object  │ tool               │ ActiveTool        │ ArticulationCfg │
│ ur10         │ robot   │ manipulator        │ SerialManipulator │ ArticulationCfg │
│ cubesat      │ robot   │ mobile_robot       │ OrbitalRobot      │ RigidObjectCfg  │
│ ingenuity    │ robot   │ mobile_robot       │ Multicopter       │ ArticulationCfg │
│ perseverance │ robot   │ mobile_robot       │ WheeledRobot      │ ArticulationCfg │
│ unitree_g1   │ robot   │ mobile_manipulator │ Humanoid          │ ArticulationCfg │
│ ...          │ ...     │ ...                │ ...               │ ...             │
└──────────────┴─────────┴────────────────────┴───────────────────┴─────────────────┘

Scenery asset (automatically registered as "mars_surface" scenery/terrain)

class MarsSurface(Terrain):
    ## Scenario - The asset is suitable for the Mars domain
    DOMAINS: ClassVar[Sequence[Domain]] = (Domain.MARS,)

    ## Model - Static asset
    asset_cfg: AssetBaseCfg = AssetBaseCfg(
        prim_path="{ENV_REGEX_NS}/mars_surface",
        ## Spawner procedurally generates SimForge models
        spawn=SimforgeAssetCfg(
            assets=[simforge_foundry.MarsSurface()],
            collision_props=CollisionPropertiesCfg(),
        ),
    )

Object asset (automatically registered as "sample_tube" object/common)

class SampleTube(Object):
    ## Model - Rigid object affected by physics
    asset_cfg: RigidObjectCfg = RigidObjectCfg(
        prim_path="{ENV_REGEX_NS}/sample",
        ## Spawner loads a static USD file
        spawn=UsdFileCfg(
            usd_path=(
                SRB_ASSETS_DIR_SRB_OBJECT.joinpath("sample_tube.usdz").as_posix()
            ),
            collision_props=CollisionPropertiesCfg(),
            mesh_collision_props=MeshCollisionPropertiesCfg(
                mesh_approximation="convexDecomposition"
            ),
            rigid_props=RigidBodyPropertiesCfg(),
            mass_props=MassPropertiesCfg(density=1500.0),
        ),
    )

Robot asset (automatically registered as "franka" robot/manipulator)

class Franka(SerialManipulator):
    ## Model - Articulation with several links connected by joints
    asset_cfg: ArticulationCfg = ArticulationCfg(
        prim_path="{ENV_REGEX_NS}/franka",
        ## Spawner loads a static USD file
        spawn=UsdFileCfg(
            usd_path=SRB_ASSETS_DIR_SRB_ROBOT.joinpath("manipulator")
            .joinpath("franka_arm.usdz")
            .as_posix(),
            ...
        ),
        ## Initial joint configuration of the robot
        init_state=ArticulationCfg.InitialStateCfg(
            joint_pos={
                "panda_joint1": 0.0,
                "panda_joint2": 0.0,
                "panda_joint3": 0.0,
                "panda_joint4": deg_to_rad(-90.0),
                "panda_joint5": 0.0,
                "panda_joint6": deg_to_rad(90.0),
                "panda_joint7": deg_to_rad(45.0),
            },
        ),
        ...
    )
    ## End effector - The default hand is separate to allow for easy replacement
    end_effector: Tool | None = FrankaHand()

    ## Actions - Inverse Kinematics action group that drives all joints
    actions: ActionGroup = InverseKinematicsActionGroup(
        DifferentialInverseKinematicsActionCfg(
            asset_name="robot",
            joint_names=["panda_joint[1-7]"],
            base_name="panda_link0",
            body_name="panda_link7",
            controller=DifferentialIKControllerCfg(
                command_type="pose",
                use_relative_mode=True,
                ik_method="svd",
            ),
            scale=0.1,
            body_offset=DifferentialInverseKinematicsActionCfg.OffsetCfg(),
        ),
    )

    ## Frames - Relevant frames for attaching the robot and mounting tool/sensors
    frame_base: Frame = Frame(prim_relpath="panda_link0")
    frame_flange: Frame = Frame(
        prim_relpath="panda_link7",
        offset=Transform(
            pos=(0.0, 0.0, 0.107),
            rot=rpy_to_quat(0.0, 0.0, -45.0),
        ),
    )
    frame_wrist_camera: Frame = Frame(
        prim_relpath="panda_link7/camera_wrist",
        offset=Transform(
            pos=(0.075, -0.075, 0.1),
            rot=rpy_to_quat(0.0, -80.0, 135.0),
        ),
    )

2. Action groups: Pre-configured action spaces for robots and active tools

Actions for robots - Each robot (mobile or manipulator) has an action group
Actions for active tools - Each active tool (e.g. gripper) has an action group

┏━━━━━━━━━━━━━━━━━━━━┓
┃ Name               ┃
┡━━━━━━━━━━━━━━━━━━━━┩
│ body_acceleration  │
│ joint_position     │
│ joint_velocity     │
│ joint_effort       │
│ inverse_kinematics │
│ ...                │
└────────────────────┘

3. Environments: Gymnasium environments for templates and tasks

Templates - Barebones environments that can be used as a starting point
Tasks - Goal-oriented environments that provide a specific scenario

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID                               ┃ Entrypoint                   ┃ Config                             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ _manipulation <template>         │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │
│ sample_collection                │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │
│ _aerial <template>               │ Task(AerialEnv)              │ TaskCfg(AerialEnvCfg)              │
│ _ground <template>               │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │
│ _orbital <template>              │ Task(OrbitalEnv)             │ TaskCfg(OrbitalEnvCfg)             │
│ locomotion_velocity_tracking     │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │
│ _aerial_manipulation <template>  │ Task(AerialManipulationEnv)  │ TaskCfg(AerialManipulationEnvCfg)  │
│ _ground_manipulation <template>  │ Task(GroundManipulationEnv)  │ TaskCfg(GroundManipulationEnvCfg)  │
│ _orbital_manipulation <template> │ Task(OrbitalManipulationEnv) │ TaskCfg(OrbitalManipulationEnvCfg) │
│ ...                              │ ...                          │ ...                                │
└──────────────────────────────────┴──────────────────────────────┴────────────────────────────────────┘

2. Teleoperate your 1^st Robot across Diverse Domains

Reference: srb agent teleop — Teleoperate Agent

Let's start with the sample_collection environment where you can manually control the Franka manipulator through your keyboard to collect samples on the Moon:

srb agent teleop --env sample_collection

Eventually, Isaac Sim will open with the selected environment, and you will be greeted in your terminal with a schematic of the teleoperation interface.

Teleoperation Interface (click to expand)

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃  Keyboard Scheme (focus the Isaac Sim window)  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Reset: [ L ]                                   │
│ Decrease Gain [ O ]   │ Increase Gain: [ P ]   │
│ Event: [ R / K ]                               │
├────────────────────────────────────────────────┤
│ Translation                                    │
│             [ W ] (+X)            [ Q ] (+Z)   │
│               ↑                     ↑          │
│               │                     │          │
│  (-Y) [ A ] ←─┼─→ [ D ] (+Y)        ┼          │
│               │                     │          │
│               ↓                     ↓          │
│             [ S ] (-X)            [ E ] (-Z)   │
├────────────────────────────────────────────────┤
│ Rotation                                       │
│       [ Z ] ←————————(±X)————————→ [ X ]       │
│                                                │
│       [ T ] ↻————————(±Y)————————↺ [ G ]       │
│                                                │
│       [ C ] ↺————————(±Z)————————↻ [ V ]       │
└────────────────────────────────────────────────┘

Note: Most tasks employ action spaces that support direct teleoperation (e.g. via Inverse Kinematics). However, some tasks such as locomotion_velocity_tracking rely on low-level control of individual joints. In this case, direct teleoperation is not supported, and you will need to provide a control policy that maps your teleoperation commands to low-level control signals. Further instructions are provided in the section for Teleoperation via Policy.

Move to a Different Domain

Reference: Environment Configuration
Reference: Environment Configuration — Domain

What if we want to collect samples on Mars instead? Luckily, you can easily configure many aspects of the environment through Hydra. For instance, you can adjust the domain and sample asset in order to simulate the Mars Sample Return mission:

srb agent teleop --env sample_collection env.domain=mars env.sample=sample_tube

3. Observe Random Agents in Action

Reference: srb agent rand — Random Agent

Now, let's observe an environment where agents act based on random actions sampled uniformly from their action space, which is particularly useful for verifying that environments function as intended. To demonstrate a random agent, we will use the locomotion_velocity_tracking task that uses the Spot quadruped by default:

srb agent rand --env locomotion_velocity_tracking

Hint: Use --hide_ui option to disable most of the Isaac Sim UI, as shown in the video above.

Change the Robot

Reference: Environment Configuration — Robot

Selecting a different robot for any environment is as simple as adjusting the env.robot parameter. This particular environment supports all legged robots, so let's try the Unitree G1 humanoid:

srb agent rand --env locomotion_velocity_tracking env.robot=unitree_g1

Simulate Multiple Robots

Reference: Environment Configuration — Parallelism

Many workflows benefit from running multiple parallel simulation instances. This can be achieved with the env.num_envs parameter. For instance, let's try 16 instances of the Cassie biped:

srb agent rand -e locomotion_velocity_tracking env.robot=cassie env.num_envs=16

Hint: As you can see in the video above, all 16 robots share the same terrain. This is the default behaviour of locomotion_velocity_tracking (the default behaviour is task-specific). However, you can easily create a unique terrain for each robot by setting env.stack=false. This will automatically trigger the generation of 16 unique assets with different geometry and materials. Here, we speed up the procedural generation by disabling texture baking with SF_BAKER=0:
SF_BAKER=0 srb agent rand -e locomotion_velocity_tracking env.stack=false env.num_envs=16

4. Explore & Experiment with Environment Templates

Reference: Robots
Reference: Environment Configuration — Robot

Both sample_collection and locomotion_velocity_tracking are examples of tasks that implement specific goal-oriented scenarios. However, SRB also provides a set of environment templates that can serve as a foundation for exploring and experimenting with custom scenarios.

In general, each robot category has its own template:

Template	Description
`_manipulation`	Fixed-base manipulation with robotic arms
`_ground`	Ground traversal on planetary surfaces
`_aerial`	Aerial navigation above planetary surfaces
`_orbital`	Spaceflight maneuvers
`_ground_manipulation`	Mobile manipulation with ground-based robots
`_aerial_manipulation`	Mobile manipulation with flying robots
`_orbital_manipulation`	Mobile manipulation with spacecraft

With this in mind, let's explore the _ground_manipulation template that combines the mobile Spot quadruped with Franka manipulator into an integrated mobile manipulation system:

srb agent rand -e _ground_manipulation

Diversify Robot Configurations

Robot configurations can also be randomized across multiple parallel instances. For example, you can combine a random Unitree quadruped (random_unitree_quadruped) with a random Universal Robots manipulator (random_ur_manipulator), ranging all the way from UR3 to UR30! For mobile manipulators, changing the mobile base and manipulator is separated into two parameters for more flexibility, namely env.robot.mobile_base and env.robot.manipulator:

srb agent rand -e _ground_manipulation env.robot.mobile_base=random_unitree_quadruped env.robot.manipulator=random_ur_manipulator env.num_envs=6 env.stack=true

Customize Payloads & End Effectors

Reference: srb agent zero — Zero Agent

Modifying only the robot might not be enough for your envisioned scenario. You might also want to customize the payload of mobile robots or the end effector of manipulators. Similar to previous examples, this can also be configured via env.robot for mobile robots and manipulators; or env.robot.mobile_base and env.robot.manipulator for mobile manipulators. The configuration is context-aware, and you can specify payloads and end effectors by separating them with a + sign, i.e. mobile_base+payload or manipulator+end_effector. For example, let's combine Unitree Z1 manipulator with Shadow Hand end effector on top of Anymal D quadruped with the Cargo Bay payload:

srb agent zero -e _ground_manipulation env.robot.mobile_base=anymal_d+cargo_bay env.robot.manipulator=unitree_z1+shadow_hand

Hint: If you only wish to change the payload or end effector but keep the rest of the robot configuration the same, +payload and +end_effector are also valid inputs. Similarly, mobile_robot+ and manipulator+ will maintain the original payload or end effector.

And while the results might look ridiculous, the same level of customization is available across the board for all domains. Furthermore, aspects such as the pose of sensors and dimensionality of action spaces are adjusted automatically.

srb agent zero -e _aerial_manipulation env.robot.mobile_base=ingenuity env.robot.manipulator=+scoop

srb agent zero -e _orbital_manipulation env.robot.mobile_base=gateway env.robot.manipulator=canadarm3+

5. Enable Visual Sensors

To maintain the performance of simulation, all visual sensors are disabled by default. However, you might have noticed that all listed environments also have their *_visual variant (e.g. _aerial -> _aerial_visual). This variant enables a number of pre-configured cameras that provide both RGB and depth images (available via observations and middleware communication).

Let's see a camera view of the Ingenuity helicopter on Mars:

srb agent teleop -e _aerial_visual

What's Next?

Everything you learned so far is just the tip of the iceberg, but it is applicable to all environments and workflows within the Space Robotics Bench. Yet, diving deeper into the codebase will allow you to customize and extend the environments further to suit your specific needs.

Depending on your interests, you are welcome to explore one or more of the following guides: